Chapter 16. Concatenation and Baking

If you’ve properly set up your JavaScript files to contain one object per file, then it’s likely you have dozens of JavaScript files. Before deploying to production, it’s best to concatenate the files so there are fewer HTTP requests on the page. Exactly how many files and which files should be concatenated with which is a project-specific decision. In any case, Ant provides an easy way to concatenate multiple files.

The Task

The <concat> task is one of the simplest in Ant. You simply specify a destfile attribute containing the destination filename and then include as many <fileset> and <filelist> elements as you want. At it’s simplest, you can have a target such as:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js">
        <fileset dir="${src.dir}" includes="**/*.js" />
    </concat>

</target>

This target concatenates all JavaScript files in the source directory into a single file called build.js in the build directory. Keep in mind that the files are concatenated in the order in which they appear on the filesystem (alphabetically). If you want a more specific ordering, you’ll need to specify it explicitly, such as:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js">
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
    </concat>

</target>

This version of the target ensures that first.js is the first file added to the final file and second.js comes right after that. Remember, you can use any number of <fileset> and <filelist> elements, so you can concatenate in any order you can imagine.

Note

Even though it’s possible to create complex concatenation schemes using Ant, it’s best to limit the special cases. Keeping filenames out of your build script is a good idea for maintainability. Try to use <fileset> elements whenever possible.

Line Endings

Concatenating files together comes with a series of challenges. One of the trickiest issues is dealing with the last line of a file. If a file doesn’t have a newline character on its last line, then concatenating that file with another may result in broken syntax. By setting fixlastline attribute to "yes", the <concat> task will automatically add a newline character to the last line if one doesn’t already exist:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js" fixlastline="yes">
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
    </concat>

</target>

It’s a good idea to always set fixlastline to "yes" for JavaScript, as newlines are valid end-of-statement tokens.

Fixing the last line is useful, but how do you know which end-of-line marker is used? If your source files are being edited by people on different operating systems, you may want to ensure consistency in the built files. The <concat> task has an optional eol attribute that specifies which end-of-line markers to use. The default value is the default for the system ("crlf" for Windows, "lf" for Unix, and "cr" for Mac OS X). You can choose any one of these values, and all of the line endings in the concatenated file will automatically be switched to that format:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js" fixlastline="yes" eol="lf">
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
    </concat>

</target>

This target changes all end-of-line markers to Unix style, which is the recommended format for JavaScript files, as it has the greatest cross-platform compatibility (and because most web servers run on a Unix variant).

Headers and Footers

The <concat> task also has the handy ability to prepend and append plain text to the resulting file. This capability allows you to insert pieces of information into the file that you might otherwise not have available. For instance, I tend to insert the build time into files so I can more easily track down errors. To do so, I start by defining a <tstamp> element at the top of the file:

<tstamp>
  <format property="build.time"
          pattern="MMMM d, yyyy hh:mm:ss"
          locale="en,US"/>
</tstamp>

This code creates a new timestamp when the build.xml file is executed by Ant. The resulting date string is stored in a property named build.time. The pattern attribute is a date-time formatting string. I can then use the <header> element to add this information into the built file:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js" fixlastline="yes" eol="lf">
        <header>/* Build Time: ${build.time} */</header>
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
    </concat>

</target>

This new version of the concatenate target inserts a comment at the top of the file containing the build time. The resulting first line of the file has this format:

/* Build Time: May 25, 2012 03:20:45 */

There is also a <footer> element that can be used to add additional text at the bottom of the file. For example, this version of concatenate puts the build time at the bottom:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js" fixlastline="yes" eol="lf">
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
        <footer>/* Build Time: ${build.time} */</footer>            
    </concat>

</target>

You can use both <header> and <footer> at the same time or just one at a time.

Baking Files

Baking refers to the final touches you put into files before considering them ready for deployment. A lot of times, this step involves either adding additional text into a file or replacing existing text with something else. Inserting the build time, as in the previous example, is a type of baking. Other common tasks are automatically including license information and inserting version information. Both can be done very easily using Ant.

Many projects have a license file included somewhere in source control. The license file is separate because it may change independently of the code. It’s useful to automatically insert the license information at the top of files before pushing them to production. You could potentially insert the license file using a <filelist> element, but that would mean that the license file must be in a property comment format for JavaScript. It’s much easier to let the license file be plain text and add the comments around it. You can load text from any file using the <loadfile> task:

<loadfile property="license" srcfile="license.txt" />

This code loads text from license.txt and stores it in a property named license. Once it’s in a property, you can use the <header> element to insert the text as a comment:

<target name="concatenate">

    <loadfile property="license" srcfile="license.txt" />

    <concat destfile="${build.dir}/build.js" fixlastline="yes" eol="lf">
        <header trimleading="yes">/*!
        ${license}
        */
        /* Build time: ${build.time} */
        </header>
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
    </concat>

</target>

The license is inserted with a multiline JavaScript comment that has an exclamation point as the first character, which tells code minifiers (see Chapter 17) that the comment is important and should not be removed. The <header> element also has trimleading set to "yes". This attribute specifies that leading white space on each line inside of <header> should be removed. That way, all text is aligned at the first column in the final file.

The other part of baking, replacing some text within files, is accomplished quite easily using the <replaceregexp> task. This task systematically goes through any number of files and uses regular expressions to replace values. As an example, I tend to use the token @VERSION@ in my source files to indicate where the version number should be inserted. For instance, you might have this in a JavaScript file:

var MyProject = {
    version: "@VERSION@"
};

You can replace @VERSION@ with an actual version number using the following:

<replaceregexp match="@VERSION@" replace="${version}" flags="g" byline="true">
    <fileset dir="${build.dir}" includes="**/*"/>
</replaceregexp>

The <replaceregexp> task takes the regular expression from the match attribute and replaces it with the text in the replace attribute. Regular expression flags such as g, i, and m are specified using the flags attribute, and byline indicates whether the regular expression should match just a single line. You then specify any number of files in which this replacement should be made.

Because <replaceregexp> doesn’t create new files, be sure to run it on the built files, as in the following example:

<target name="concatenate">

    <concat destfile="${build.dir}/build.js" fixlastline="yes" eol="lf">
        <filelist dir="${src.dir}" files="first.js,second.js" />
        <fileset dir="${src.dir}" includes="**/*.js" excludes="first.js,second.js"/>
        <footer>/* Build Time: ${build.time} */</footer>            
    </concat>

    <replaceregexp match="@VERSION@" replace="${version}" flags="g" byline="true">
        <fileset dir="${build.dir}" includes="**/*"/>
    </replaceregexp>

</target>

This code replaces all instances of @VERSION@ in all built files with the version property. Although the replacement takes place in the concatenate target here, you may also want to have a separate target for baking, such as:

<target name="bake">

    <replaceregexp match="@VERSION@" replace="${version}" flags="g" byline="true">
        <fileset dir="${build.dir}" includes="**/*"/>
    </replaceregexp>

</target>

Separating out the baking step makes sense when it doesn’t involve the <concat> task and may not always be done as part of the build process.