One of the things that’s held JavaScript back as a language has been the lack of dependency management and a module system. Unlike other languages, namespacing and modules aren’t something traditionally emphasized when people are learning JavaScript. Indeed, popular libraries like jQuery don’t enforce any application structure; there’s definitely an onus on the developer to resolve this himself. Too often, I see spaghetti-styled JavaScript, with a crazy amount of indentation and anonymous functions. Does this look familiar?
function() {
function() {
function() {
function() {
}
}
}
}The usage of modules and namespacing is one thing, but the lack of
native dependency systems is becoming an increasing concern when building
larger applications. For a long time, a script tag was deemed sufficient, as the amount of JavaScript present on
the page didn’t justify anything further. However, when you start writing
complex JavaScript applications, a dependency system is absolutely critical.
It’s completely impractical to keep
track of dependencies yourself by adding script tags to the page manually.
You’ll often end up with a mess like this:
<script src="jquery.js" type="text/javascript" charset="utf-8"></script> <script src="jquery.ui.js" type="text/javascript" charset="utf-8"></script> <script src="application.utils.js" type="text/javascript" charset="utf-8"></script> <script src="application.js" type="text/javascript" charset="utf-8"></script> <script src="models/asset.js" type="text/javascript" charset="utf-8"></script> <script src="models/activity.js" type="text/javascript" charset="utf-8"></script> <script src="states/loading.js" type="text/javascript" charset="utf-8"></script> <script src="states/search.js" type="text/javascript" charset="utf-8"></script> <!-- ... -->
It’s not just the practicalities of the situation that warrant a specific dependency management system—there are performance aspects as well. Your browser needs to make an HTTP request for each of these JavaScript files, and—although it can do this asynchronously—there’s a huge cost to making so many connections. Each connection has the overhead of HTTP headers, like cookies, and has to initiate another TCP handshake. The situation is exacerbated if your application is served using SSL.
As JavaScript has moved to the server side, several
proposals have been put forward for dependency management. SpiderMonkey and Rhino offer a load()
function, but they do not have any specific patterns for namespacing.
Node.js has the require() function for loading in extra source files, as well as its own module
system. The code wasn’t interchangeable, though, so what happens when you
want to run your Rhino code on Node.js?
It became obvious that a standard was needed to ensure code interoperability, which all JavaScript implementations could abide by, allowing us to use libraries across all the environments. Kevin Dangoor started the CommonJS initiative to do just that. It began with a blog post in which Kevin advocated a shared standard for JavaScript interpreters and for developers to band together and write some specs:
JavaScript needs a standard way to include other modules and for those modules to live in discreet namespaces. There are easy ways to do namespaces, but there’s no standard programmatic way to load a module (once!).
[This] is not a technical problem. It’s a matter of people getting together and making a decision to step forward and start building up something bigger and cooler together.
A mailing list was set up, and CommonJS was born. It quickly gathered momentum with support from the major players. It is now the de facto module format for JavaScript with a growing set of standards, including IO interfaces, Socket streams, and Unit tests.
Declaring a CommonJS module is fairly straightforward.
Namespacing is baked directly in; modules are separated into different
files, and they expose variables publicly by adding them to an
interpreter-defined exports
object:
// maths.js
exports.per = function(value, total) {
return( (value / total) * 100 );
};
// application.js
var Maths = require("./maths");
assertEqual( Maths.per(50, 100), 50 );To use any functions defined in a module, simply require() the file, saving the result in a
local variable. In the example above, any functions exported by
maths.js are available on the Maths variable. The key is that modules are
namespaced and will run on all CommonJS-compliant JavaScript
interpreters, such as Narwhal
and Node.js.
So, how does this relate to client-side JS development? Well, lots of developers saw the implications of using modules on the client side—namely, that the standard, as it currently stood, required CommonJS modules to be loaded in synchronously. This is fine for server-side JavaScript, but it can be very problematic in the browser because it locks up the UI and requires eval-based compilation of scripts (always something to be avoided). The CommonJS team developed a specification, the module transport format, to address this issue. This transport format wraps CommonJS modules with a callback to allow for asynchronous loading on clients.
Let’s take our module example above. We can wrap it in the transport format to allow asynchronous loading, making it palatable for the browser:
// maths.js
require.define("maths", function(require, exports){
exports.per = function(value, total) {
return( (value / total) * 100 );
};
});
// application.js
require.define("application", function(require, exports){
var per = require("./maths").per;
assertEqual( per(50, 100), 50 );
}, ["./maths"]); // List dependencies (maths.js)Our modules can then be required by a module loader library and executed in the browser. This is a really big deal. Not only have we split up our code into separate module components, which is the secret to good application design, but we’ve also got dependency management, scope isolation, and namespacing. Indeed, the same modules can be run on browsers, servers, in desktop apps, and in any other CommonJS-compliant environment. In other words, it’s now possible to share the same code between server and client!
To use CommonJS modules on the client side, we need to use a module loader library. There is a variety of options, each with its own strengths and weaknesses. I’ll cover the most popular ones and you can choose which one best suits your needs.
The CommonJS module format is still in flux, with various proposals under review. As it stands, there’s no officially blessed transport format, which unfortunately complicates things. The two main module implementations in the wild are Transport C and Transport D. If you use any of the wrapping tools mentioned in the sections below, you’ll have to make sure it generates wrapped modules in a format your loader supports. Fortunately, many module loaders also come with compatible wrapping tools, or they specify supported ones in their documentation.
Yabble is an excellent and lightweight module loader. You can
configure Yabble to either request modules with XHR or to use script
tags. The advantage to fetching modules with XHR is that they don’t need
wrapping in the transport format. However, the disadvantage is that
modules have to be executed using eval(), making debugging more difficult.
Additionally, there are cross-domain issues, especially if you’re using
a CDN. Ideally, you should only use the XHR option for quick and dirty
development, certainly not in production:
<script src="https://github.com/jbrantly/yabble/raw/master/lib/yabble.js"> </script>
<script>
require.setModuleRoot("javascripts");
// We can use script tags if the modules
// are wrapped in the transport format
require.useScriptTags();
require.ensure(["application", "utils"], function(require) {
// Application is loaded
});
</script>The above example will fetch our wrapped application module and then load its
dependencies, utils.js, before running the module.
We can load modules using the require() function:
<script>
require.ensure(["application", "utils"], function(require) {
var utils = require("utils");
assertEqual( utils.per( 50, 200 ), 25 );
});
</script>Although utils is required
twice—once by the inline require.ensure() function, and once by the
application module—our script is
clever enough to fetch the module only
once. Make sure any dependencies your module needs are listed
in the transport wrapping.
A great alternative to Yabble is RequireJS, one of the most popular loaders. RequireJS has a slightly different take on loading modules—it follows the Asynchronous Module Definition format, or AMD. The main difference you need to be concerned with is that the API evaluates dependencies eagerly, rather than lazily. In practice, RequireJS is completely compatible with CommonJS modules, requiring only different wrapping transport.
To load JavaScript files, just pass their paths to the require()
function, specifying a callback that will be invoked when the
dependencies are all loaded:
<script>
require(["lib/application", "lib/utils"], function(application, utils) {
// Loaded!
});
</script>As you can see in the example above, the application and utils modules are passed as arguments to the
callback; they don’t have to be fetched with the require() function.
It’s not just modules that you can require—RequireJS also supports ordinary JavaScript libraries as dependencies, specifically jQuery and Dojo. Other libraries will work, but they won’t be passed correctly as arguments to the required callback. However, any library that has dependencies is required to use the module format:
require(["lib/jquery.js"], function($) {
// jQuery loaded
$("#el").show();
});Paths given to require() are
relative to the current file or module, unless they begin with a
/. To help with optimization,
RequireJS encourages you to place your initial script loader in a
separate file. The library even provides shorthand to do this: the
data-main attribute:
<script data-main="lib/application" src="lib/require.js"></script>
Setting the data-main attribute
instructs RequireJS to treat the script tag like a require() call and load the attribute’s value.
In this case, it would load the lib/application.js script, which would in turn load
the rest of our application:
// Inside lib/application.js
require(["jquery", "models/asset", "models/user"], function($, Asset, User) {
//...
});So, we’ve covered requiring modules, but what about actually
defining them? Well, as stated previously, RequireJS uses a slightly
different syntax for modules. Rather than using require.define(), just use the plain define() function. As long as modules are in
different files, they don’t need explicit naming. Dependencies come
first, as an array of strings, and then comes a callback function
containing the actual module. As in the RequireJS require() function, dependencies are passed as
arguments to the callback function:
define(["underscore", "./utils"], function(_, Utils) {
return({
size: 10
})
});By default, there’s no exports
variable. To expose variables from inside the module, just return data
from the function. The benefit to RequireJS modules is that they’re
already wrapped up, so you don’t have to worry about transport formats
for the browser. However, the caveat to this API is that it’s not
compatible with CommonJS modules—i.e., you couldn’t share modules
between Node.js and the browser. All is not lost, though; RequireJS has
a compatibility layer for CommonJS modules—just wrap your existing
modules with the define()
function:
define(function(require, exports) {
var mod = require("./relative/name");
exports.value = "exposed";
});The arguments to the callbacks need to be exactly as shown
above—i.e., require and exports. Your modules can then carry on using
those variables as usual, without any alterations.
At this stage, we’ve got dependency management and namespacing, but there’s still the original problem: all those HTTP requests. Any module we depend on has to be loaded in remotely, and even though this happens asynchronously, it’s still a big performance overhead, slowing the startup of our application.
We’re also hand-wrapping our modules in the transport format which, while necessary for asynchronous loading, is fairly verbose. Let’s kill two birds with one stone by using a server-side step to concatenate the modules into one file. This means the browser has to fetch only one resource to load all the modules, which is much more efficient. The build tools available are intelligent, too—they don’t just bundle the modules arbitrarily, but statically analyze them to resolve their dependencies recursively. They’ll also take care of wrapping the modules up in the transport format, saving some typing.
In addition to concatenation, many module build tools also support minification, further reducing the request size. In fact, some tools—such as rack-modulr and Transporter—integrate with your web server, handling module processing automatically when they’re first requested.
For example, here’s a simple Rack CommonJS module server using rack-modulr:
require "rack/modulr"
use Rack::Modulr, :source => "lib", :hosted_at => "/lib"
run Rack::Directory.new("public")You can start the server with the rackup command. Any CommonJS modules contained inside the lib folder are now concatenated automatically with all
their dependencies and are wrapped in a transport callback. Our script
loader can then request modules when they’re needed, loading them into the
page:
>> curl "http://localhost:9292/lib/application.js"
require.define("maths"....If Ruby’s not your thing, there is a multitude of other options from which to choose. FlyScript is a CommonJS module wrapper written in PHP, Transporter is one for JSGI servers, and Stitch integrates with Node.js servers.
You may decide not to go the module route, perhaps because
you’ve already got a lot of existing code and libraries to support that
can’t be easily converted. Luckily, there are some great alternatives,
such as Sprockets. Sprockets adds synchronous require() support to your JavaScript. Comments
beginning with //= act as directives to
the Sprockets preprocessor. For example, the //=
require directive instructs Sprockets to look in its load path
for the library, fetch it, and include it inline:
//= require <jquery> //= require "./states"
In the example above, jquery.js is in Sprockets’ load path, and states.js is required relative to the current file. Sprockets is clever enough to include a library only once, regardless of the amount of time required. As with all the CommonJS module wrappers, Sprockets supports caching and minification. During development, your server can parse and concatenate files on demand in the course of the page load. When the site is live, the JavaScript files can be preconcatenated and served statically, increasing performance.
Although Sprockets is a command-line tool, there are some great integrations to Rack and Rails, such as rack-sprockets. There are even some PHP implementations. The downside to Sprockets—and indeed all these module wrappers—is that all your JavaScript files need to be preprocessed, either by the server or via the command-line tool.
LABjs is one of the simplest dependency management solutions out there. It doesn’t require any server-side involvement or CommonJS modules. Loading your scripts with LABjs reduces resource blocking during page load, which is an easy and effective way to optimize your site’s performance. By default, LABjs will load and execute scripts in parallel as fast as possible. However, you can easily specify the execution order if some scripts have dependencies:
<script>
$LAB
.script('/js/json2.js')
.script('/js/jquery.js').wait()
.script('/js/jquery-ui.js')
.script('/js/vapor.js');
</script>In the above example, all the scripts load in parallel, but LABjs ensures jquery.js is executed before jquery-ui.js and vapor.js. The API is incredibly simple and succinct, but if you want to learn about LABjs’ more advanced features, such as support for inline scripts, check out the documentation.
One thing to watch out for with any of these script loaders is that during the page load, users may see a flash of unbehaviored content (FUBC)—i.e., a glimpse at the raw page before any JavaScript is executed. This won’t be a problem if you’re not relying on JavaScript to style or manipulate the initial page. But if you are, address this issue by setting some initial styles in CSS, perhaps hiding a few elements, or by displaying a brief loading splash screen.