Chapter 17. Minification and Compression

Once you have your built files validated, concatenated, and baked, it’s time to make those files as small as possible. This step is accomplished with two processes: minification and compression. Minification is the process of eliminating unnecessary white space, removing comments, and performing some processing on the files to make them as small as possible. Compression uses a specific compression method, such as gzip, to shrink the file even further. The difference between a minified file and a compressed file is that minified files are still just plain text and can be edited and loaded as usual (albeit with a bit of trouble, because all formatting is removed), whereas compressed files are unreadable and must be decompressed to be usable in a web page. Today’s browsers automatically decompress any compressed files they receive with a Content-Encoding: gzip header in the response.

Minification

Minifying a JavaScript file isn’t very complicated, but mistakes or invalid syntax can result if you use an unsafe process. For this reason, it’s best to use a minifier that actually parses the JavaScript before making changes. Parsers know what valid syntax is and can more easily create valid syntax. The three most popular parsing minifiers are:

YUI Compressor

Often credited with popularizing parser-based minifiers instead of the regular expression-based minifiers. YUI Compressor was first written by Julien Lecomte (and is now maintained by the YUI team); it removes comments and extra white space and replaces local variable names with single- or double-character names to save even more space. YUI Compressor considers syntax and runtime safety as its highest priority, so it turns off variable replacement in cases in which errors might occur (such as using eval() or with). Download from http://yuilibrary.com/projects/yuicompressor/.

Closure Compiler

A parser-based minifier that tries to make your code as small as possible. The Closure Compiler is written and maintained by Google engineers; it removes comments and extra white space and performs variable replacement, but also inspects your code for ways to optimize. For instance, it can detect that a function isn’t used and simply remove it. It can also detect that a function is used only once and put it inline. For this reason, Closure Compiler works best when used on all of your JavaScript code at once. Download from http://code.google.com/closure/compiler/.

UglifyJS

Credited with being the first Node.js-based JavaScript minifier, UglifyJS is written in JavaScript using a JavaScript-based parser. Written by Mihai Bazon, UglifyJS removes comments and extra white space, replaces variable names, combines var statements, and performs other optimizations along the way. Download from https://github.com/mishoo/UglifyJS or install using npm.

Exactly which minifier to use is a matter of preference. Some prefer YUI Compressor because of its focus on ensuring that the resulting code doesn’t contain errors and its simple optimizations. Others prefer Closure Compiler because it tends to produce files that are smaller than YUI Compressor. Still others prefer UglifyJS because it doesn’t rely on Java and produces fewer syntax errors than Closure Compiler while also producing the smallest possible result.

Minifying with YUI Compressor

YUI Compressor ships as an executable JAR file that should be placed in your project’s dependencies folder (in this example, this folder is referred to as lib.dir). There are several command-line options, but the most important to know are:

--disable-optimizations

Turns off micro optimizations such as changing obj["prop"] to obj.prop

--line-break <column>

Specifies to break lines at the given column rather than creating a single line of output

--nomunge

Turns off local variable name replacement

--preserve-semi

Turns off removal of unnecessary semicolons

Once you decide which options to use, place them in a Java properties file, as in the following example:

src.dir = ./src
lib.dir = ./lib

yuicompressor = ${lib.dir}/yuicompressor.jar

yuicompressor.options = --preserve-semi

Next, create a target for minification. As with validation, it’s helpful to look at the command-line syntax first. To run YUI Compressor on the command line, type the following:

java -jar yuicompressor.jar [options] [file] -o [outputfile]

For example:

java -jar yuicompressor.jar --preserve-semi core/core.js -o core/core-min.js

This command runs YUI Compressor on core/core.js and outputs the result to core/core-min.js. Appending -min to the filename is a common practice when minifying files, so the target does that as well. The <apply> task is once again the one to use, and here’s the target:

<target name="minify">

    <apply executable="java" failonerror="true">

        <fileset dir="${build.dir}" includes="*.js"/>
        <mapper type="glob" from="*.js" to="${build.dir}/*-min.js"/>

        <arg line="-jar"/>
        <arg path="${yuicompressor}"/>
        <arg line="${yuicompressor.options}"/>
        <srcfile/>

        <arg line="-o"/>           
        <targetfile/>
    </apply>     

</target>

This target is very similar to the validate target from earlier in the book. It starts by specifying the executable as java and then indicates that the command should be run on all JavaScript files in the build directory. These files get ready one by one and are put in the place of <srcfile/>. The next line uses the <mapper> task to translate filenames. It takes any <srcfile/> and appends -min to the filename. So core.js becomes core-min.js, and this value is used in place of the <targetfile/> element. The rest of the <arg> elements are self-explanatory.

The minify task is designed to be used after you’ve built files into the build directory but can easily be modified to run in any directory.

Note

YUI Compressor also contains a CSS minifier. If you pass a CSS file to YUI Compressor, it automatically switches into CSS mode.

Buildr has a <yuicompressor> task that encapsulates all of this functionality. There are attributes to enable each command-line option (the attribute name is the same as the command-line option without the leading --) plus a required outputdir attribute that indicates where the minified files should be placed. Here’s an example:

<target name="minify">
    <yuicompressor outputdir="${build.dir}" preserve-semi="true">
        <fileset dir="${build.dir}" includes="*.js" />
    </yuicompressor>
</target>

The <yuicompressor> task automatically adds the -min suffix to all files it creates. You can include one or more elements to minify everything at once.

Minifying with Closure Compiler

The Closure Compiler is also an executable JAR file that needs to be placed in the dependencies folder. The Closure Compiler has significantly more command-line options than the YUI Compressor, many of which are used only by those working directly with the Closure JavaScript library. The most important option is --compilation_level, which determines how much processing is done on the JavaScript file. The options are:

WHITESPACE_ONLY

Removes only unnecessary white space and comments. Other optimizations are turned off.

SIMPLE_OPTIMIZATIONS

The default setting for Closure Compiler. This setting removes unnecessary white space and comments while also renaming local variables to shorter names. The renaming happens even in the presence of eval() and with, so it may cause runtime errors if either is present.

ADVANCED_OPTIMIZATIONS

Every optimization possible is done on the code. Use with caution, as this can introduce runtime or syntax errors.

Once you decide which options to use, place them in a Java properties file, such as:

src.dir = ./src
lib.dir = ./lib

closure = ${lib.dir}/compiler.jar

closure.options = --compilation_level SIMPLE_OPTIMIZATIONS

To run Closure Compiler on the command line, type the following:

java -jar compiler.jar [options] --js [file] --js_output_file [outputfile]

For example:

java -jar compiler.jar --compilation_level SIMPLE_OPTIMIZATIONS --js core/core.js 
    --js_output_file core/core-min.js

This command runs Closure Compiler on core/core.js and outputs the result to core/core-min.js. The Ant target is basically the same as the one used with YUI Compressor:

<target name="minify">

    <apply executable="java" failonerror="true">

        <fileset dir="${build.dir}" includes="*.js"/>
        <mapper type="glob" from="*.js" to="${build.dir}/*-min.js"/>

        <arg line="-jar"/>
        <arg path="${closure}"/>
        <arg line="${closure.options}"/>

        <arg line="--js"/>
        <srcfile/>

        <arg line="--js_output_file"/>            
        <targetfile/>
    </apply>     

</target>

Aside from changing the JAR file path and command-line options, this target is virtually identical to the minify target. The end result is the same: every JavaScript file in the build directory is minified and output into a second file with the -min suffix.

Buildr has a <closure> task that encapsulates all of this functionality. You can set the compilation level using the compilation-level attribute. As with the <yuicompressor> task, the outputdir attribute is required and indicates where the minified files should be placed. Here’s an example:

<target name="minify">
    <closure outputdir="${build.dir}" compilation-level="SIMPLE_OPTIMIZATIONS">
        <fileset dir="${build.dir}" includes="*.js" />
    </closure>
</target>

The <closure> task automatically adds the -min suffix to all files it creates. You can include one or more elements to minify everything at once.

Minifying with UglifyJS

UglifyJS is most commonly used on the command line through npm, the Node.js package manager. You must have both Node.js and npm installed first before you can install UglifyJS, which is done with this command:

sudo npm install -g uglify-js

The UglifyJS command-line interface also has a large number of options. However, the most commonly used are:

--beautify

Beautifies the code instead of minifying it

--no-mangle

Turns off function and variable name replacement

--no-mangle-functions

Turns off only function name replacement

--no-dead-code

Enables removal of unreachable code

The basic format of UglifyJS on the command line is:

uglifyjs [options] -o [outputfile] [file]

For example:

uglifyjs --no-mangle-functions -o core/core-min.js core/core.js

This command runs UglifyJS on core/core.js and outputs the result to core/core-min.js. It’s important that the original file comes after all of the other options.

As with the other minifiers, it’s best to put your preferred options into a properties file, as in:

src.dir = ./src
lib.dir = ./lib

uglifyjs = uglifyjs

uglifyjs.options = --no-mangle-functions

The Ant target for UglifyJS is a bit easier than the others, because it is a standalone executable. Instead of setting the executable attribute of <apply> to Java, set it to uglifyjs. Then pass the additional information as usual:

<target name="minify">

    <apply executable="uglifyjs" failonerror="true">

        <fileset dir="${build.dir}" includes="*.js"/>
        <mapper type="glob" from="*.js" to="${build.dir}/*-min.js"/>

        <arg line="${ugilfyjs.options}"/>

        <arg line="-o"/>           
        <targetfile/>
        <srcfile/>
    </apply>     

</target>

The minify target is the same basic format as the other targets in this chapter and produces a similar result. All of the files in the build directory are minified and placed into files with the -min suffix.

Buildr has an <uglifyjs> task that makes using UglifyJS easier. Each of the command-line options is available as an attribute (without the leading --), and as with the other minification tasks, the outputdir attribute is required. Here’s an example:

<target name="minify">
    <uglifyjs outputdir="${build.dir}" no-mangle-functions="true">
        <fileset dir="${build.dir}" includes="*.js" />
    </uglifyjs>
</target>

The <uglifyjs> task works in the same way as the others: -min is automatically added to minified filenames, and you can specify as many <fileset> elements as you’d like.

Compression

Minification of JavaScript files is the first step before deployment. The second is to compress the files to be as small as possible during transmission. The minifiers mentioned in this chapter don’t perform compression on JavaScript (even YUI Compressor only performs minification). Compression usually happens later in the process, either at runtime using HTTP compression on the web server or during build time.

Runtime Compression

Most web servers are capable of performing runtime compression of files. In practice, such compression is typically done only for text-based files such as JavaScript, HTML, and CSS. Modern browsers all support HTTP compression and will send an HTTP header as part of a request indicating the types of compression supported. For example:

Accept-Encoding: gzip, deflate

When the server sees this HTTP header in a request, it knows the browser is capable of decompressing files that are compressed using either gzip or deflate. When the server sends the response, it sets a header indicating the type of compression used, such as:

Content-Encoding: gzip

This header tells the browser that the body of the response is gzipped and must be uncompressed before use.

Apache 2, one of the most popular web servers, has HTTP compression built in as the mod_deflate module. This module is enabled by default and automatically compresses JavaScript, HTML, CSS, and XML files. If you’re using Apache 2, then you don’t need to do any further configuration for compressing JavaScript files.

Nginx, another popular web server, also has HTTP compression built in using gzip. Compression is also enabled by default for JavaScript, HTML, CSS, and XML files. If you’re using Nginx, then no further configuration is needed to enable compression for JavaScript.

Note

Internet Explorer 6 and earlier had problems with HTTP compression in certain situations. Apache 2 and Nginx allow you to turn off HTTP compression for these browsers if necessary. Since Microsoft began auto-upgrading everyone with Internet Explorer 6 to Internet Explorer 8 in 2012, this should no longer be much of a problem.

Build-Time Compression

You may choose to compress files during build time if you want to distribute the compressed file yourself without any server intervention. jQuery builds a gzipped version of the main JavaScript file and makes it available for download from http://jquery.com. Gzipping is the easiest way to perform compression at build time due to Ant’s <gzip> task.

The <gzip> task works only on a single file. You specify the src attribute as the filename to compress and the destfile attribute as the output file. For example:

<gzip src="${build.dir}/build.js" destfile="${build.dir}/build.js.gz"/>

This task gzips the build.js file and outputs the result to build.js.gz. You can certainly use <gzip> in this manner if you only ever have one file to gzip. For example:

<target name="compress">

    <gzip src="${build.dir}/build.js" destfile="${build.dir}/build.js.gz"/>

</target>

In this case, the filenames should be stored in properties so they can be easily changed later.

A little bit of creativity is required to gzip multiple files without explicitly setting their filenames. Ant doesn’t have a native way of looping over a list of files. However, it’s possible to use JavaScript inside of Ant to provide this behavior.

The Ant <script> task allows you to write scripts in a number of languages, of which JavaScript is just one. To specify JavaScript as the language you’re using, set the language attribute to "javascript". After that, enclose the contents of the <script> element with CData delimiters such as this:

<script language="javascript"><![CDATA[

    // code here

]]></script>

Adding the CData delimiters ensures that you needn’t worry about escaping characters within the script.

JavaScript inside of the <script> task executes in an environment similar to the default environment in Rhino. You have access to Java objects and can import more by using the importPackage() function. There is also a project object that refers to the overall project represented in build.xml. You can read properties using project.getProperty() and create new tasks using project.createTask(). Putting these pieces together, you can create a compress target that compresses all files in the build directory:

<target name="compress">

    <!-- store filenames in a property delimited by ; -->
    <pathconvert pathsep=";" property="compress.jsfiles">
        <fileset dir="${build.dir}" includes="*.js"/>
    </pathconvert>    

    <script language="javascript"><![CDATA[

        importPackage(java.io);

        <!-- get the property and convert to an array-->
        var files = project.getProperty("compress.jsfiles").split(";"),
            gzip,
            i,
            len;

        for (i=0, len=files.length; i < len; i++) {

            // create new gzip task
            gzip = project.createTask("gzip");
            gzip.setSrc(new File(files[i]));
            gzip.setDestfile(new File(files[i].replace(".js", ".js.gz")));
            gzip.perform();
        }

    ]]> </script>    
</target>

The first part of the compress target converts a <fileset> into a property. The property compress.jsfiles is filled with a string in which the filenames are separated by semicolons. Inside the <script> task, the first line imports the java.io package so that the File class is available. Next, the compress.jsfiles property is read and split with semicolons so that files is an array of filenames.

After that, a for loop is used to iterate over the filenames. For each filename, a new <gzip> task is created using project.createTask("gzip"). The gzip variable then contains a Java object representing the task. Each attribute has a method for setting its value and a method for getting its value, so setSrc() is used to set the src attribute and setDestfile() is used to set the destfile attribute. Both attributes represent files, so it’s necessary to pass an instance of File instead of the filename. The output file is set to have a .js.gz extension by using the JavaScript replace() method on the filename. The last step is to call perform(), which actually executes the task.

This version of the compress target doesn’t rely on knowing the filename ahead of time and is therefore better suited for general usage.

There is a Buildr <gzipall> task that allows you to compress multiple files:

<target name="compress">
    <gzipall>
        <fileset dir="${build.dir}" includes="*-min.js" />
    </gzipall>
</target>

Using <gzipall>, a gzipped file is created with a .gz appended to the filename.

Note

If you intend to serve the compressed file yourself, you’ll still need to configure the web server to send the Content-Encoding: gzip header so that the browser can use the file correctly.