Table of Contents for

Web Development with Node and Express

Broadly speaking, your two options for sending client data to the server are the querystring and the request body. Normally, if you’re using the querystring, you’re making a GET request, and if you’re using the request body, you’re using a POST request (the HTTP protocol doesn’t prevent you from doing it the other way around, but there’s no point to it: best to stick to standard practice here).

It is a common misperception that POST is secure and GET is not: in reality, both are secure if you use HTTPS, and neither is secure if you don’t. If you’re not using HTTPS, an intruder can look at the body data for a POST just as easily as the querystring of a GET request. However, if you’re using GET requests, your users will see all of their input (including hidden fields) in the querystring, which is ugly and messy. Also, browsers often place limits on querystring length (there is no such restriction for body length). For these reasons, I generally recommend using POST for form submission.

This book is focusing on the server side, but it’s important to understand some basics about constructing HTML forms. Here’s a simple example:

<form action="/process" method="POST">
    <input type="hidden" name="hush" val="hidden, but not secret!">
    <div>
        <label for="fieldColor">Your favorite color: </label>
        <input type="text" id="fieldColor" name="color">
    </div>
    <div>
        <button type="submit">Submit</button>
    </div>
</form>

Notice the method is specified explicitly as POST in the <form> tag; if you don’t do this, it defaults to GET. The action attribute specifies the URL that will receive the form when it’s posted. If you omit this field, the form will be submitted to the same URL the form was loaded from. I recommend that you always provide a valid action, even if you’re using AJAX (this is to prevent you from losing data; see Chapter 22 for more information).

From the server’s perspective, the important attribute in the <input> fields are the name attributes: that’s how the server identifies the field. It’s important to understand that the name attribute is distinct from the id attribute, which should be used for styling and frontend functionality only (it is not passed to the server).

Note the hidden field: this will not render in the user’s browser. However, you should not use it for secret or sensitive information: all the user has to do is examine the page source, and the hidden field will be exposed.

HTML does not restrict you from having multiple forms on the same page (this was an unfortunate restriction of some early server frameworks; ASP, I’m looking at you).^[8] I recommend keeping your forms logically consistent: a form should contain all the fields you would like submitted (optional/empty fields are okay), and none that you don’t. If you have two different actions on a page, use two different forms. An example of this would be to have a form for a site search and a separate form for signing up for an email newsletter. It is possible to use one large form and figure out what action to take based on what button a person clicked, but it is a headache, and often not friendly for people with disabilities (because of the way accessibility browsers render forms).

When the user submits the form, the /process URL will be invoked, and the field values will be transmitted to the server in the request body.

When the form is submitted (either by the browser or via AJAX), it must be encoded somehow. If you don’t explicitly specify an encoding, it defaults to application/x-www-form-urlencoded (this is just a lengthy media type for “URL encoded”). This is a basic, easy-to-use encoding that’s supported by Express out of the box.

If you need to upload files, things get more complicated. There’s no easy way to send files using URL encoding, so you’re forced to use the multipart/form-data encoding type, which is and is not handled directly by Express (actually, Express still supports this encoding, but it will be removed in the next version of Express, and its use is not recommended: we will be discussing an alternative shortly).

If you’re not using AJAX, your only option is to submit the form through the browser, which will reload the page. However, how the page is reloaded is up to you. There are two things to consider when processing forms: what path handles the form (the action), and what response is sent to the browser.

If your form uses method="POST" (which is recommended), it is quite common to use the same path for displaying the form and processing the form: these can be distinguished because the former is a GET request, and the latter is a POST request. If you take this approach, you can omit the action attribute on the form.

The other option is to use a separate path to process the form. For example, if your contact page uses the path /contact, you might use the path /process-contact to process the form (by specifying action="/process-contact"). If you use this approach, you have the option of submitting the form via GET (which I do not recommend; it needlessly exposes your form fields on the URL). This approach might be preferred if you have multiple URLs that use the same submission mechanism (for example, you might have an email sign-up box on multiple pages on the site).

Whatever path you use to process the form, you have to decide what response to send back to the browser. Here are your options:

Since the recommendation is that you respond to a form submission with a 303 redirect, the next question is “Where does the redirection point to?” The answer to that is up to you. Here are the most common approaches:

If you are using AJAX, I recommend a dedicated URL. It’s tempting to start AJAX handlers with a prefix (for example, /ajax/enter), but I discourage this approach: it’s attaching implementation details to a URL. Also, as we’ll see shortly, your AJAX handler should handle regular browser submissions as a failsafe.

If you’re using GET for your form handling, your fields will be available on the req.query object. For example, if you have an HTML input field with a name attribute of email, its value will be passed to the handler as req.query.email. There’s really not much more that needs to be said about this approach: it’s just that simple.

If you’re using POST (which I recommend), you’ll have to link in middleware to parse the URL-encoded body. First, install the body-parser middleware (npm install --save body-parser), then link it in:

app.use(require('body-parser')());

Once you’ve linked in body-parser, you’ll find that req.body now becomes available for you, and that’s where all of your form fields will be made available. Note that req.body doesn’t prevent you from using the querystring. Let’s go ahead and add a form to Meadowlark Travel that lets the user sign up for a mailing list. For demonstration’s sake, we’ll use the querystring, a hidden field, and visible fields in /views/newsletter.handlebars:

<h2>Sign up for our newsletter to receive news and specials!</h2>
<form class="form-horizontal" role="form"
        action="/process?form=newsletter" method="POST">
    <input type="hidden" name="_csrf" value="{{csrf}}">
    <div class="form-group">
        <label for="fieldName" class="col-sm-2 control-label">Name</label>
        <div class="col-sm-4">
            <input type="text" class="form-control"
            id="fieldName" name="name">
        </div>
    </div>
    <div class="form-group">
        <label for="fieldEmail" class="col-sm-2 control-label">Email</label>
        <div class="col-sm-4">
            <input type="email" class="form-control" required
                id="fieldName" name="email">
        </div>
    </div>
    <div class="form-group">
        <div class="col-sm-offset-2 col-sm-4">
            <button type="submit" class="btn btn-default">Register</button>
        </div>
    </div>
</form>

Note we are using Twitter Bootstrap styles, as we will be throughout the rest of the book. If you are unfamiliar with Bootstrap, you may want to refer to the Twitter Bootstrap documentation. Then see Example 8-1.

app.use(require('body-parser')());

app.get('/newsletter', function(req, res){
    // we will learn about CSRF later...for now, we just
    // provide a dummy value
    res.render('newsletter', { csrf: 'CSRF token goes here' });
});

app.post('/process', function(req, res){
    console.log('Form (from querystring): ' + req.query.form);
    console.log('CSRF token (from hidden form field): ' + req.body._csrf);
    console.log('Name (from visible form field): ' + req.body.name);
    console.log('Email (from visible form field): ' + req.body.email);
    res.redirect(303, '/thank-you');
});

That’s all there is to it. Note that in our handler, we’re redirecting to a “thank you” view. We could render a view here, but if we did, the URL field in the visitor’s browser would remain /process, which could be confusing: issuing a redirect solves that problem.

Handling AJAX forms is very easy in Express; it’s even easy to use the same handler for AJAX and regular browser fallbacks. Consider Examples 8-2 and 8-3.

<div class="formContainer">
    <form class="form-horizontal newsletterForm" role="form"
            action="/process?form=newsletter" method="POST">
        <input type="hidden" name="_csrf" value="{{csrf}}">
        <div class="form-group">
            <label for="fieldName" class="col-sm-2 control-label">Name</label>
            <div class="col-sm-4">
                <input type="text" class="form-control"
                id="fieldName" name="name">
            </div>
        </div>
        <div class="form-group">
            <label for="fieldEmail" class="col-sm-2 control-label">Email</label>
            <div class="col-sm-4">
                <input type="email" class="form-control" required
                    id="fieldName" name="email">
            </div>
        </div>
        <div class="form-group">
            <div class="col-sm-offset-2 col-sm-4">
                <button type="submit" class="btn btn-default">Register</button>
            </div>
        </div>
    </form>
</div>
{{#section 'jquery'}}
    <script>
        $(document).ready(function(){
            $('.newsletterForm').on('submit', function(evt){
                evt.preventDefault();
                var action = $(this).attr('action');
                var $container = $(this).closest('.formContainer');
                $.ajax({
                    url: action,
                    type: 'POST',
                    success: function(data){
                        if(data.success){
                            $container.html('<h2>Thank you!</h2>');
                        } else {
                            $container.html('There was a problem.');
                        }
                    },
                    error: function(){
                        $container.html('There was a problem.');
                    }
                });
            });
        });
    </script>
{{/section}}

app.post('/process', function(req, res){
    if(req.xhr || req.accepts('json,html')==='json'){
        // if there were an error, we would send { error: 'error description' }
        res.send({ success: true });
    } else {
        // if there were an error, we would redirect to an error page
        res.redirect(303, '/thank-you');
    }
});

Express provides us with a couple of convenience properties, req.xhr and req.accepts. req.xhr will be true if the request is an AJAX request (XHR is short for XML HTTP Request, which is what AJAX relies on). req.accepts will try to determine the most appropriate response type to return. In our case, req.accepts('json,html') is asking if the best format to return is JSON or HTML: this is inferred from the Accepts HTTP header, which is an ordered list of acceptable response types provided by the browser. If the request is an AJAX request, or if the user agent has specifically requested that JSON is better than HTML, appropriate JSON will be returned; otherwise, a redirect would be returned.

We can do whatever processing we need in this function: usually we would be saving the data to the database. If there are problems, we send back a JSON object with an err property (instead of success), or redirect to an error page (if it’s not an AJAX request).

We’ve already mentioned that file uploads bring a raft of complications. Fortunately, there are some great projects that help make file handling a snap.

Currently, file uploads can be handled with Connect’s built-in multipart middleware; however, that middleware has already been removed from Connect, and as soon as Express updates its dependency on Connect, it will vanish from Express as well, so I strongly recommend that you do not use that middleware.

There are two popular and robust options for multipart form processing: Busboy and Formidable. I find Formidable to be slightly easier, because it has a convenience callback that provides objects containing the fields and the files, whereas with Busboy, you must listen for each field and file event. We’ll be using Formidable for this reason.

Let’s create a file upload form for a Meadowlark Travel vacation photo contest (views/contest/vacation-photo.handlebars):

<form class="form-horizontal" role="form"
        enctype="multipart/form-data" method="POST"
        action="/contest/vacation-photo/{year}/{month}">
    <div class="form-group">
        <label for="fieldName" class="col-sm-2 control-label">Name</label>
        <div class="col-sm-4">
            <input type="text" class="form-control"
            id="fieldName" name="name">
        </div>
    </div>
    <div class="form-group">
        <label for="fieldEmail" class="col-sm-2 control-label">Email</label>
        <div class="col-sm-4">
            <input type="email" class="form-control" required
                id="fieldName" name="email">
        </div>
    </div>
    <div class="form-group">
        <label for="fieldPhoto" class="col-sm-2 control-label">Vacation photo
        </label>
        <div class="col-sm-4">
            <input type="file" class="form-control" required accept="image/*"
                id="fieldPhoto" name="photo">
        </div>
    </div>
    <div class="form-group">
        <div class="col-sm-offset-2 col-sm-4">
            <button type="submit" class="btn btn-primary">Submit</button>
        </div>
    </div>
</form>

Note that we must specify enctype="multipart/form-data" to enable file uploads. We’re also restricting the type of files that can be uploaded by using the accept attribute (which is optional).

Now install Formidable (npm install --save formidable) and create the following route handlers:

var formidable = require('formidable');

app.get('/contest/vacation-photo',function(req,res){
    var now = new Date();
    res.render('contest/vacation-photo',{
        year: now.getFullYear(),month: now.getMont()
    });
});

app.post('/contest/vacation-photo/:year/:month', function(req, res){
    var form = new formidable.IncomingForm();
    form.parse(req, function(err, fields, files){
        if(err) return res.redirect(303, '/error');
        console.log('received fields:');
        console.log(fields);
        console.log('received files:');
        console.log(files);
        res.redirect(303, '/thank-you');
    });
});

(Year and month are being specified as route parameters, which you’ll learn about in Chapter 14.) Go ahead and run this and examine the console log. You’ll see that your form fields come across as you would expect: as an object with properties corresponding to your field names. The files object contains more data, but it’s relatively straightforward. For each file uploaded, you’ll see there are properties for size, the path it was uploaded to (usually a random name in a temporary directory), and the original name of the file that the user uploaded (just the filename, not the whole path, for security and privacy reasons).

What you do with this file is now up to you: you can store it in a database, copy it to a more permanent location, or upload it to a cloud-based file storage system. Remember that if you’re relying on local storage for saving files, your application won’t scale well, making this a poor choice for cloud-based hosting. We will be revisiting this example in Chapter 13.

If you want to offer really fancy file uploads to your users—with the ability to drag and drop, see thumbnails of the uploaded files, and see progress bars—then I recommend Sebastian Tschan’s jQuery File Upload.

Setting up jQuery File Upload is not a walk in the park. Fortunately, there’s an npm package to help you with the server-side intricacies. The frontend scripting is another matter. The jQuery File Upload package uses jQuery UI and Bootstrap, and looks pretty good out of the box. If you want to customize it, though, there’s a lot to work through.

To display file thumbnails, jquery-file-upload-middleware uses ImageMagick, a venerable image manipulation library. This does mean your app has a dependency on ImageMagick, which could cause problems depending on your hosting situation. On Ubuntu and Debian systems, you can install ImageMagick with apt-get install imagemagick, and on OS X, you can use brew install imagemagick. For other operating systems, consult the ImageMagick documentation.

Let’s start with the server-side setup. First, install the jquery-file-upload-middleware package (npm install --save jquery-file-upload-middleware), then add the following to your app file:

var jqupload = require('jquery-file-upload-middleware');

app.use('/upload', function(req, res, next){
    var now = Date.now();
    jqupload.fileHandler({
        uploadDir: function(){
            return __dirname + '/public/uploads/' + now;
        },
        uploadUrl: function(){
            return '/uploads/' + now;
        },
    })(req, res, next);
});

If you look at the documentation, you’ll see something similar under “more sophisticated examples.” Unless you are implementing a file upload area that’s quite literally shared by all of your visitors, you’ll probably want to be able to partition off the file uploads. The example simply creates a timestamped directory to store the file uploads. A more realistic example would be to create a subdirectory that uses the user’s ID or some other unique ID. For example, if you were implementing a chat program that supports shared files, you might want to use the ID of the chat room.

Note that we are mounting the jQuery File Upload middleware on the /upload prefix. You can use whatever you want here, but make sure you don’t use that prefix for other routes or middleware, as it will interfere with the operation of your file uploads.

To hook up your views to the file uploader, you can replicate the demo uploader: you can upload the latest bundle on the project’s GitHub page. It will inevitably include a lot of things you don’t need, like PHP scripts and other implementation examples, which you are free to delete. Most of the files, you’ll put in your public directory (so they can be served statically), but the HTML files you’ll have to copy over to views.

If you just want a minimal example that you can build on, you’ll need the following scripts from the bundle: js/vendor/jquery.ui.widget.js, js/jquery.iframe-transport.js, and js/jquery.fileupload.js. You’ll also need jQuery, obviously. I generally prefer to put all of these scripts in public/vendor/jqfu for neatness. In this minimal implementation, we wrap the <input type="file"> element in a <span>, and add a <div> in which we will list the names of uploaded files:

<span class="btn btn-default btn-file">
    Upload
    <input type="file" class="form-control" required accept="image/*"
        id="fieldPhoto" data-url="/upload" multiple name="photo">
</span>
<div id="uploads"></div>

Then we attach jQuery File Upload:

{{#section 'jquery'}}
    <script src="/vendor/jqfu/js/vendor/jquery.ui.widget.js"></script>
    <script src="/vendor/jqfu/js/jquery.iframe-transport.js"></script>
    <script src="/vendor/jqfu/js/jquery.fileupload.js"></script>
    <script>
        $(document).ready(function(){

            $('#fieldPhoto').fileupload({
                dataType: 'json',
                done: function(e, data){
                    $.each(data.result.files, function(index, file){
                        $('#fileUploads').append($('<div class="upload">' +
                            '<span class="glyphicon glyphicon-ok"></span>' +
                            '&nbsp;' + file.originalName + '</div>'));
                    });
                }
            });

        });
    </script>
{{/section}}

We have to do some CSS gymnastics to style the upload button:

.btn-file {
    position: relative;
    overflow: hidden;
}
.btn-file input[type=file] {
    position: absolute;
    top: 0;
    right: 0;
    min-width: 100%;
    min-height: 100%;
    font-size: 999px;
    text-align: right;
    filter: alpha(opacity=0);
    opacity: 0;
    outline: none;
    background: white;
    cursor: inherit;
    display: block;
}

Note that the data-url attribute of the <input> tag must match the route prefix you used for the middleware. In this simple example, when a file upload successfully completes, a <div class="upload"> element is appended to <div id="uploads">. This lists only filename and size, and does not offer controls for deletion, progress, or thumbnails. But it’s a good place to start. Customizing the jQuery File Upload demo can be daunting, and if your vision is significantly different, it might be easier to start from the minimum and build your way up instead of starting with the demo and customizing. Either way, you will find the resources you need on the jQuery File Upload documentation page.

For simplicity, the Meadowlark Travel example will not continue to use jQuery File Upload, but if you wish to see this approach in action, refer to the jquery-file-upload-example branch in the repository.