JSON Data Types

JSON has the following core Data Types:

Name (or Key)/value pair

Consists of a key (a data attribute) and a value.

Object

An unordered collection of name/value pairs.

Array

A collection of ordered values.

Now that we’ve covered basic definitions, let’s dig deeper into each Data Type.

{
  "conference": "OSCON",
  "speechTitle": "JSON at Work",
  "track": "Web APIs"
}

{
  "address" : {
    "line1" : "555 Any Street",
    "city" : "Denver",
    "stateOrProvince" : "CO",
    "zipOrPostalCode" : "80202",
    "country" : "USA"
  }
}

{
  "speaker" : {
    "firstName": "Larson",
    "lastName": "Richard",
    "topics": [ "JSON", "REST", "SOA" ]
  }
}

{
  "speaker" : {
    "firstName": "Larson",
    "lastName": "Richard",
    "topics": [ "JSON", "REST", "SOA" ],
    "address" : {
      "line1" : "555 Any Street",
      "city" : "Denver",
      "stateOrProvince" : "CO",
      "zipOrPostalCode" : "80202",
      "country" : "USA"
    }
  }
}

{
  "presentations": [
    {
      "title": "JSON at Work: Overview and Ecosystem",
      "length": "90 minutes",
      "abstract": [ "JSON is more than just a simple replacement for XML when",
                    "you make an AJAX call."
                  ],
      "track": "Web APIs"
    },
    {
      "title": "RESTful Security at Work",
      "length": "90 minutes",
      "abstract": [ "You’ve been working with RESTful Web Services for a few years",
                     "now, and you’d like to know if your services are secure."
                  ],
      "track": "Web APIs"
    }
  ]
}

JSON Value Types

JSON Value Types represent the Data Types that occur on the righthand side of the colon (:) of a Name/Value Pair. JSON Value Types include the following:

• object

• array

• string

• number

• boolean

• null

We’ve already covered Objects and Arrays; now let’s focus on the remaining Value Types: string, number, boolean, and null.

[
  "fred",
  "fred\t",
  "\b",
  "",
  "\t",
  "\u004A"
]

{
  "age": 29,
  "cost": 299.99,
  "temperature": -10.5,
  "unitCost": 0.2,
  "speedOfLight": 1.23e11,
  "speedOfLight2": 1.23e+11,
  "avogadro": 6.023E23,
  "avogadro2": 6.023E+23,
  "oneHundredth": 10e-3,
  "oneTenth": 10E-2
}

{
  "isRegistered": true,
  "emailValidated": false
}

{
  "address": {
    "line1": "555 Any Street",
    "line2": null,
     "city": "Denver",
        "stateOrProvince": "CO",
        "zipOrPostalCode": "80202",
        "country": "USA"
    }
}

JSON Versions

According to Douglas Crockford, there will never be another version of the core JSON standard. This isn’t because JSON is perfect; nothing is perfect. The purpose of a sole JSON version is to avoid the pitfalls of having to support backward compatibility with previous versions. Crockford believes that a new data format should replace JSON when the need arises within the development community.

But as you’ll see in subsequent chapters, this “no versions” philosophy applies only to the core JSON data format. For example, in Chapter 5, that specification is currently at version 0.5 as of this writing. Please note that these JSON-related specifications were created by others in the JSON community.

JSON Comments

There are no comments in a JSON document. Period.

According to his postings on the Yahoo! JSON group and Google+, Crockford initially allowed comments, but removed them early on for the following reasons:

• He believed that comments weren’t useful.

• JSON parsers had difficulties supporting comments.

• People were abusing comments. For example, he noticed that comments were being used for parsing directives, which would have destroyed interoperability.

• Removing comments simplified and enabled cross-platform JSON support.

JSON File and MIME Type

According to the core JSON specification, .json is the standard JSON file type when storing JSON data on filesystems. JSON’s Internet Assigned Numbers Authority (IANA) media (or MIME) type is application/json, which can be found at the IANA Media Types site. RESTful Web Service Producers and Consumers use a technique known as content negotiation (which leverages the JSON MIME type in HTTP Headers) to indicate that they are exchanging JSON data.

JSON Style Guidelines

JSON is all about interoperability, and it’s important to provide JSON data feeds in a way that Consumers expect. Google has published a JSON Style Guide to support maintainability and best practices.

The Google JSON Style Guide is extensive, and here are the most important things for an API designer and developer:

• Property Names

• Date Property Values

• Enum Values

{
  "firstName": "John Smith"
}

{
  "dateRegistered": "2014-03-01T23:46:11-05:00"
}

{
  "empireStateBuilding": "40.748747-73.985547"
}

Our Technical Stack

We’ll start by creating a simple JSON data store for speakers and publishing it to a Stub RESTful API by taking the following steps:

1. Model JSON data with JSON Editor Online

2. Generate sample JSON data with JSON Generator

3. Create and deploy a Stub API (for future testing)

Our Architectural Style—noBackEnd

Our architectural style is based on the concept of noBackend. With noBackend, the developer doesn’t have to worry about the nuts and bolts of application servers or databases at the early stages of application development.

The first seven chapters of this book use noBackEnd architecture to maintain focus on our application from a business perspective (services and data first) so that we can support not only UI-based (e.g., mobile, tablet, and web) clients, but APIs and non-web-based client applications as well. We’ll deploy JSON data with simple tools such as json-server to emulate a RESTful API.

By using this approach, we take an interface-first approach to designing and building an API, which provides the following:

• More Agile, rapid, iterative frontend development due to the decoupling from the backend.

• Faster feedback on the API itself. Get the data and URI out there quickly for rapid review.

• A cleaner interface between the API and its Consumers.

• A separation of concerns between the Resource (e.g., speakers as JSON data) exposed by the API and its (eventual) internal implementation (e.g., application server, business logic, and data store). This makes it easier to change implementation in the future. If you create and deploy a real API with Node.js/Rails/Java (or other framework) too early, you’ve already made design decisions at a very early stage that will make it difficult to change after you start working with API Consumers.

A Stub API does the following:

• Eliminates the initial need to work with servers and databases

• Allows API Producers (those developers who write the API) to focus on API Design, how best to present the data to the Consumers, and initial testing

• Enables API Consumers (e.g., UI developers) to work with the API at an early stage and provide feedback to the API development team

By using the lightweight tools in this book, you’ll see that you can go a long way before writing code and deploying it on a server. Of course, you’ll eventually need to implement an API, and we’ll show how to do that when we cover JavaScript, Ruby on Rails, and Java in Chapters 2–4.

Model JSON Data with JSON Editor Online

Creating a valid JSON document of any real size or complexity is tedious and error-prone. JSON Editor Online is a great web-based tool that does the following:

• Enables you to model your JSON document as Objects, Arrays, and name/value pairs

• Makes it easier to rapidly generate the text for a JSON document in an iterative manner

JSONmate is another solid editor on the web, but we don’t cover it further in this book.

Speaker data in JSON Editor Online

After you’re finished modeling Speaker data, click the right-arrow button to generate a pretty-printed (indented) JSON document that represents the model. Figure 1-3 shows JSON Editor Online with our initial Speakers model.

Figure 1-3. Speaker data model in JSON Editor Online

This is just a rough model, but this initial sketch is a decent starting point. Use the initial model to visualize JSON data, get early feedback, and iterate quickly on the design. This approach enables you to refine the JSON data structure throughout the development life cycle without investing heavily in implementation and infrastructure.

Generate Sample JSON Data with JSON Generator

JSON Editor Online provides a decent start, but we want to generate lots of test data quickly. Test data can be problematic because of the sensitivity of the data, and the data volume needed to do any meaningful testing. Even with JSON Editor Online, it will take a great deal of effort to create the volume of test data we’re looking for. We need another tool to help create the data we need to create our first version of the API, and that’s where JSON Generator comes in. This excellent tool was used to create our speakers.json test data file. The template used to generate the speakers.json file is available on GitHub. Chapter 5 covers JSON Generator in more detail.

Create and Deploy a Stub API

To create the Stub API, we’ll use the Speaker data we just created and deploy it as a RESTful API. We’ll leverage the json-server Node.js module to serve up the speakers.json file as a Web API; this enables us to prototype quickly. You can find more information on the json-server GitHub page.

Before going further, please set up your development environment. Refer to Appendix A to do the following:

1. Install Node.js. json-server is a Node.js module, so you need to install Node.js first. Refer to “Install Node.js”.

2. Install json-server. See “Install npm Modules”.

3. Install JSONView and Postman. See “Install JSON Tools in the Browser”. JSONView pretty-prints JSON in Chrome and Firefox. Postman can also run as a standalone GUI application on most major operating systems.

Open a terminal session and run json-server on port 5000 from your command line:

cd chapter-1

json-server -p 5000 ./speakers.json

You should see the following:

Visit http://localhost:5000/speakers in your browser, and (with JSON pretty-printing provided by JSONView) you should see all the speakers from our Stub API as shown in Figure 1-4.

Figure 1-4. Speakers on json-server viewed from the browser with JSONView

You can also get a single speaker by adding the id to the URI as follows: http://localhost:5000/speakers/0.

This is a good start, but a web browser has limited testing functionality; it can only send HTTP GET requests. Postman provides the ability to fully test a RESTful API. It can send HTTP GET, POST, PUT, and DELETE requests and set HTTP Headers.

Let’s use Postman to delete the first speaker in the API as follows:

1. Enter the http://localhost:5000/speakers/0 URL.

2. Choose DELETE as the HTTP verb.

3. Click the Send button.

You should see that the DELETE ran properly in Postman with a 200 (OK) HTTP Status, as shown in Figure 1-5.

Figure 1-5. Postman: results from the deleting the first speaker

Now, ensure that the first speaker has truly been deleted by revisiting http://localhost:5000/speakers/0 in your browser. You should now see the empty response shown in Figure 1-6.

Figure 1-6. Verify the results of deleting the first speaker

You can stop json-server by pressing Ctrl-C at the command line.

With the Stub API in place, we can now invoke it from any HTTP client (e.g., JavaScript, Ruby, or Java) to consume the data from an external application. Although most of our examples in subsequent chapters use an HTTP GET, rest assured that json-server can handle all the core HTTP verbs (GET, POST, PUT, DELETE). Although not covered in this book, Mountebank is an alternative server that provides more robust functionality for stubbing and mocking APIs and protocols.

The main point here is that an API Producer can use JSON-based tools to prototype a testable RESTful API without having to write any code. This technique is powerful because it enables the API Consumer to test without having to wait for the API to be 100 percent complete. At the same time, the API development team can iteratively upgrade the design and prototype.

The JSON Stringifier/Parser Object

The JSON stringifier/parser Object was originally developed by Douglas Crockford, has been part of the JavaScript library as of ECMAScript 5 in 2009, and provides the following methods:

• JSON.stringify() serializes to JSON

• JSON.parse() deserializes from JSON

Additionally, the JSON Object

• Was originally developed by Crockford

• Can’t be instantiated

• Has no other functionality

JSON Serialization with Simple JavaScript Data Types

We’ll start by serializing some basic JavaScript Data Types:

• Number

• String

• Array

• Boolean

• Object (Literal)

Example 2-1 shows how to use JSON.stringify() to serialize simple Data Types.

var age = 39; // Integer
console.log('age = ' + JSON.stringify(age) + '\n');

var fullName = 'Larson Richard'; // String
console.log('fullName = ' + JSON.stringify(fullName) + '\n');

var tags = ['json', 'rest', 'api', 'oauth']; // Array
console.log('tags = ' + JSON.stringify(tags) + '\n');

var reqistered = true; // Boolean
console.log('registered = ' + JSON.stringify(reqistered) + '\n');

var speaker = {
  firstName: 'Larson',
  lastName: 'Richard',
  email: 'larsonrichard@ecratic.com',
  about: 'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
  company: 'Ecratic',
  tags: ['json', 'rest', 'api', 'oauth'],
  registered: true
};

console.log('speaker = ' + JSON.stringify(speaker));

When you run the preceding file with node from the command line, you should get the following:

JSON.stringify() doesn’t do anything too interesting with the scalar types (Number, String, Boolean). Things begin to get interesting with the speaker Object Literal because here JSON.stringify() initially generates a valid, yet unattractive, JSON String. JSON.stringify() has other parameters that enhance serialization. According to the Mozilla Developer Network (MDN) JavaScript Guide, here is the method signature:

JSON.stringify(value[, replacer [, space]])

The parameter list is as follows:

value (required)

The JavaScript value to serialize.

replacer (optional)

Either a function or an array. If a function is provided, the stringify() method invokes the replacer function for each key/value pair in an Object.

space (optional)

Indentation—either a Number or String. If a Number is used, this value specifies the number of spaces used for each indentation level.

Let’s leverage the replacer and space parameters to pretty-print the speaker Object and filter out some data elements, as shown in Example 2-2.

var speaker = {
  firstName: 'Larson',
  lastName: 'Richard',
  email: 'larsonrichard@ecratic.com',
  about: 'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
  company: 'Ecratic',
  tags: ['json', 'rest', 'api', 'oauth'],
  registered: true
};

function serializeSpeaker(key, value) {
  return (typeof value === 'string' || Array.isArray(value)) ? undefined : value;
}

// Pretty Print.
console.log('Speaker (pretty print):\n' + JSON.stringify(speaker, null, 2) + '\n');


// Pretty print and filter out Strings and Arrays.
console.log('Speaker without Strings and Arrays:\n' +
  JSON.stringify(speaker, serializeSpeaker, 2));

Running the preceding file yields the following:

The first JSON.stringify() call pretty-prints the JSON output with an indentation level of 2. The second call uses the serializeSpeaker() function as a replacer (JavaScript functions are treated as expressions and can be passed as parameters). serializeSpeaker() checks the type of each value and returns undefined for Strings and Arrays. Otherwise, this function returns the value “as is.”

JSON.stringify() does one of the following with an undefined value:

• Omits the value if it’s part of an Object

• Converts the value to null if that value belongs to an Array

JSON Serialization with an Object and toJSON()

As you’ve seen, JSON serialization makes the most sense with Objects. Let’s customize JSON.stringify()’s output by adding a toJSON() method to our speaker Object, as shown in Example 2-3.

var speaker = {
  firstName: 'Larson',
  lastName: 'Richard',
  email: 'larsonrichard@ecratic.com',
  about: 'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
  company: 'Ecratic',
  tags: ['json', 'rest', 'api', 'oauth'],
  registered: true
};

speaker.toJSON = function() {
  return "Hi there!";
}

console.log('speaker.toJSON(): ' + JSON.stringify(speaker, null, 2));

Serialization works as follows:

If an Object has a toJSON() method, JSON.stringify() outputs the value returned by the Object’s toJSON() method rather than stringifying the Object. Although the use of toJSON() is legal, it’s probably a bad idea. toJSON() defeats the whole purpose of JSON.stringify(), because the developer is now responsible for serializing the entire Object structure. This could work with simple Objects such as speaker (as currently defined), but you’ll end up writing lots of code to serialize more complex Objects that contain other Objects.

JSON Deserialization Using eval()

Originally, JavaScript developers used the eval() function to parse JSON. eval() takes a String parameter that could be a JavaScript expression, a statement, or a sequence of statements. Consider Example 2-4.

var x = '{ "sessionDate": "2014-10-06T13:30:00.000Z" }';

console.log('Parse with eval(): ' + eval('(' + x + ')').sessionDate + '\n');

console.log('Parse with JSON.parse(): ' + JSON.parse(x).sessionDate);

Running the preceding file yields the following:

In this case, both eval() and JSON.parse() work the same and parse the date properly. So what’s the problem? Let’s look at another example with a JavaScript statement embedded in the String; see Example 2-5.

var x = '{ "sessionDate": new Date() }';

console.log('Parse with eval(): ' + eval('(' + x + ')').sessionDate + '\n');

console.log('Parse with JSON.parse(): ' + JSON.parse(x).sessionDate);

When we run this, we now see the following:

We passed in text that contains a JavaScript statement, new Date(), and eval() executes that statement. Meanwhile, JSON.parse() correctly rejects the text as invalid JSON. Although we passed in only a fairly innocuous statement to create a Date, someone else could pass in malicious code and eval() would still execute it. Even though eval() can be used to parse JSON, it is considered a bad/unsafe practice because it opens the door to any valid JavaScript expression, leaving your application vulnerable to attacks. Because of this security issue, the eval() function has been deprecated (for parsing JSON) in favor of JSON.parse().

JSON Deserialization with an Object and JSON.parse()

Let’s return to our Speaker example, and use JSON.parse() to deserialize a JSON String into a speaker Object, as shown in Example 2-6.

var json = '{' +  // Multi-line JSON string.
  '"firstName": "Larson",' +
  '"lastName": "Richard",' +
  '"email": "larsonrichard@ecratic.com",' +
  '"about": "Incididunt mollit cupidatat magna excepteur do tempor ex non ...",' +
  '"company": "Ecratic",' +
  '"tags": [' +
    '"json",' +
    '"rest",' +
    '"api",' +
    '"oauth"' +
  '],' +
  '"registered": true' +
'}';

// Deserialize JSON string into speaker object.
var speaker = JSON.parse(json);

// Print 2nd speaker object.
console.log('speaker.firstName = ' + speaker.firstName);

When we run this file, we get the following:

JSON.parse() takes a JSON String as input and parses it into a fully functional JavaScript Object. We’re now able to access the speaker Object’s data members.

Node REPL

So far we’ve been using Node.js from the command line to execute JavaScript files. Let’s change things up a bit and start using Node.js’s interpreter, the Request-Eval-Print-Loop (REPL), instead. The REPL is really great because it provides instant feedback on your code, and enables you to iteratively debug and improve your application. You can find in-depth coverage of the REPL in the Node.js documentation. But nothing is perfect, and neither is the REPL. One of my pet annoyances is the following:

For each statement that doesn’t produce output, the interpreter outputs undefined. Many people find this distracting, and there’s a way to turn it off. See Appendix A (“Taming the REPL—mynode”) to configure a command alias I affectionately call mynode that I find easier to work with than the standard Node.js REPL.

Without further ado, let’s work with our speaker Object by using the mynode REPL:

In this run, you’ll notice that we can interact with the speaker Object by calling its methods and viewing the results in the interpreter.

Here are some of the commands you’ll need to use the REPL:

.clear

Clear the context of the REPL session.

.break

Go back to the REPL prompt. Use this to break out of a multiline statement.

.exit

Exit the REPL session.

.save

Save the REPL session to a file.

Where to Learn More About JavaScript Objects

We’ve glossed over many details of Object-Oriented JavaScript, and there are several other ways to interact with objects. We’ve shown just enough OO here so that we can work with JavaScript Objects and JSON in a meaningful way within an application. Complete, in-depth coverage of JavaScript Objects is far beyond the scope of this book. To gain a deeper understanding, here are a few excellent resources:

• Learn JavaScript Next by JD Isaacks (Manning).

• The Principles of Object-Oriented JavaScript by Nicholas K. Zakas (O’Reilly).

• Learning JavaScript Design Patterns by Addy Osmani (O’Reilly).

Unit Test Style—TDD and BDD

Test-Driven Development (TDD) is an approach that uses Unit Testing to drive development. Here’s a typical flow:

1. Write some tests.

2. Run the tests, which fail because there isn’t any code.

3. Write just enough code to make the tests pass.

4. Refactor the code to improve design and flexibility.

5. Rerun tests and fix code until all tests pass.

TDD-style Unit Tests tend to be procedural.

Behavior-Driven Development (BDD) is an approach that tests a User Story based on acceptance criteria and expected outcomes. BDD-style tests read like English sentences; for example: “Speakers should receive their payment from the Conference within 30 days.” For more information on BDD, please see Dan North’s excellent article, “Introducing BDD”. Some people see BDD as a refinement to TDD, and I tend to agree because a developer would follow the same workflow as TDD.

Both BDD and TDD are solid approaches, and can be combined to form a robust test suite for an application. The Unit Tests in this chapter use a BDD-style approach for assertions.

Just Enough Unit Testing with Mocha and Chai

Here are the tools for our server-side Unit Test:

Mocha

Mocha is a JavaScript Unit Test framework that runs in both Node.js and a browser. We’ll leverage Mocha from the command line within a Node.js project, and add a few features to support JSON-based API testing. You can find more details at the Mocha website.

Chai

Chai is an assertion library that complements JavaScript testing frameworks and adds a richer set of assertions, in this case to Mocha. Chai enables developers to write TDD or BDD style tests. The tests in this chapter use the expect (BDD) assertion style, but you’re free to experiment with the should (BDD) or assert (TDD) assertion styles. Use the approach that makes you comfortable. For more details on Chai, visit the Chai Asssertion Library website.

Setting Up the Unit Test

Before going further, please be sure to set up your test environment. If you haven’t installed Node.js yet, see Appendix A, and install Node.js (see “Install Node.js” and “Install npm Modules”). If you want to follow along with the Node.js project provided in the code examples, cd to chapter-2/speakers-test and do the following to install all dependencies for the project:

npm install

If you’d like to set up the Node.js project yourself, follow the instructions in the book’s GitHub repository.

Unirest

Our Unit Test will invoke an API with HTTP, so we’ll include Unirest in our testing repertoire. Unirest is an open source cross-platform REST client provided by the Mashape team. There are implementations in JS, Node.js, Ruby on Rails (RoR) and Java. Unirest is simple and works well in any client code that makes HTTP calls to REST APIs, but it’s also great for Unit Testing. Unirest enables cleaner Unit Tests because you can do a one-time setup (e.g., URI, Headers) and then make multiple HTTP calls throughout the test suite. For detailed documentation, visit the Unirest website.

Unirest is great because it’s cross-platform, and the concepts and method signatures are similar regardless of the language implementation. There are other excellent Java-based HTTP libraries (e.g., Apache Commons HTTPComponents HttpClient, but as a polyglot (multilanguage) developer, I prefer Unirest. Please note that Unirest is not just for Unit Tests. It’s widely used as an HTTP client wrapper by APIs (which invoke other APIs), and by web and mobile client applications.

Test Data

We’ll use the Speaker data from Chapter 1 as our test data and deploy it as a RESTful API. Again, we’ll leverage the json-server Node.js module to serve up the data/speakers.json file as a Web API. If you need to install json-server, please refer to “Install npm Modules” section of Appendix A.

Here’s how to run json-server on port 5000 from your local machine:

cd chapter-2/data

json-server -p 5000 ./speakers.json

Speakers Unit Test

The Unit Test in Example 2-8 shows how to use Unirest to make an API call to the Speaker Stub API provided by json-server.

'use strict';

var expect = require('chai').expect;
var unirest = require('unirest');

var SPEAKERS_ALL_URI = 'http://localhost:5000/speakers';


describe('speakers', function() {
  var req;

  beforeEach(function() {
    req = unirest.get(SPEAKERS_ALL_URI)
      .header('Accept', 'application/json');
  });

  it('should return a 200 response', function(done) {
    req.end(function(res) {
      expect(res.statusCode).to.eql(200);
      expect(res.headers['content-type']).to.eql(
        'application/json; charset=utf-8');

      done();
    });
  });

  it('should return all speakers', function(done) {
    req.end(function(res) {
      var speakers = res.body;
      var speaker3 = speakers[2];

      expect(speakers.length).to.eql(3);
      expect(speaker3.company).to.eql('Talkola');
      expect(speaker3.firstName).to.eql('Christensen');
      expect(speaker3.lastName).to.eql('Fisher');
      expect(speaker3.tags).to.eql([
        'Java', 'Spring',
        'Maven', 'REST'
      ]);

      done();
    });
  });

});

In this Unit Test, the following occurs:

• The test sets up the URI and Accept Header for unirest by using Mocha’s beforeEach() method, so that setup occurs in only one place in the code. Mocha executes beforeEach() before running each test (i.e., it) within the context of the describe.

• The should return all speakers test is the most interesting, and it works as follows:

  • req.end() executes the Unirest GET request asynchronously, and the anonymous (unnamed) function processes the HTTP response (res) from the API call.

  • We populate the speakers object with the HTTP Response Body (res.body). At this point, the JSON from the API has already been parsed by Unirest and converted to a corresponding JavaScript Object (in Object Literal form).

  • We use Chai’s BDD-style expect assertions to check for expected results:

    • We have three speakers.

    • The third speaker’s company, firstName, lastName, and tags match the values in the speakers.json file.

To run this test from the command line (in a second terminal session), do the following:

cd chapter-2/speakers-test

npm test

You should see the following results:

json-at-work => npm test

...

> mocha test

...

  speakers
    ✓ should return a 200 response
    ✓ should return all speakers


  2 passing

Yeoman

Yeoman provides an easy way to create (i.e., scaffold) a web application and simplify developer workflow, and is similar to Gradle and Maven (from the Java community), and Ruby on Rails. We’ll use Yeoman to set up, develop, and run the example application. To install Yeoman (which depends on Node.js), refer to Appendix A, and follow the instructions in “Install Yeoman”.

Yeoman provides the following functionality:

• Creates the development environment

• Runs the application

• Automatically reloads the browser when changes are saved

• Manages package dependencies

• Minifies the application’s code and packages it for deployment

Yeoman follows the philosophy of convention over configuration:

• Automates setup

• Just works

• Uses standardized directory structures

• Provides Dependency Management

• Assumes reasonable defaults

• Encourages best practices

• Enables tool-based developer workflow (e.g., test, lint, run, and package)

Please review the following Yeoman tutorials for more information:

• Let’s Scaffold a Web App with Yeoman

• Building Apps with the Yeoman Workflow

Iteration 1—Generate a Web Application with Yeoman

Let’s start with a simple application that has no real functionality, and hardcode the Speaker data into a table. We’ll add the speaker functionality in Iterations 2 and 3. With Yeoman installed, we’ll use the generator-webapp generator to create our application that comes out-of-the-box with web pages, CSS stylesheets, Bootstrap 4, jQuery, Mocha, and Chai.

If you’d like to set up the Yeoman project yourself, follow the instructions in the book’s GitHub repository. If you want to follow along with the Yeoman project provided in the code examples, cd to chapter-2/speakers-web-1. In either case, do the following to start the application from the command line:

gulp serve

This command starts a local web server and shows the main page (index.html) in your default browser. You should see the page in Figure 2-1 at http://localhost:9000.

Figure 2-1. Basic web app with Yeoman generator

Note that if you keep the application running, you can see changes take effect as you save them because this application automatically refreshes with LiveReload.

The generator-webapp Yeoman generator creates a nice starter application, and it’s time to customize it. First, let’s change the title, Header, and jumbotron (i.e., remove the Splendid! button) in index.html as shown in Example 2-9.

<!doctype html>
<html lang="">
  <head>

    ...

    <title>JSON at Work - MyConference</title>

    ...

  </head>
  <body>
    ...

    <div class="header">
      ...

      <h3 class="text-muted">JSON at Work - Speakers</h3>
    </div>

    ...

    <div class="jumbotron">
      <h1 class="display-3">Speakers</h1>
      <p class="lead">Your conference lineup.</p>
    </div>

    ...

  </body>
</html>

Let’s add a table with some hardcoded Speaker data in the index.html file, as shown in Example 2-10.

<!doctype html>
<html lang="">

  ...

  <body>

    ...

    <table class="table table-striped">
      <thead>
        <tr>
          <th>Name</th>
          <th>About</th>
          <th>Topics</th>
        </tr>
      </thead>
      <tbody id="speakers-tbody">
        <tr>
          <td>Larson Richard</td>
          <td>Incididunt mollit cupidatat magna excepteur do tempor ...
          </td>
          <td>JavaScript, AngularJS, Yeoman</td>
        </tr>
        <tr>
          <td>Ester Clements</td>
          <td>Labore tempor irure adipisicing consectetur velit. ...
          </td>
          <td>REST, Ruby on Rails, APIs</td>
        </tr>
        <tr>
          <td>Christensen Fisher</td>
          <td>Proident ex Lorem et Lorem ad. Do voluptate officia ...
          </td>
          <td>Java, Spring, Maven, REST</td>
        </tr>
      </tbody>
    </table>

    ...

  </body>
</html>

We now have a web application that displays the sample Speaker data, as shown in Figure 2-2.

Figure 2-2. Sample Speaker data in index.html

Here are the key application files and directories generated by generator-webapp:

• app/ contains the application’s code (for example, HTML, JavaScript, and CSS).

  • index.html is the application’s main page.

  • images/ holds the application’s images.

  • scripts/ is a directory that has the application’s JavaScript (and other scripting language) files.

    • main.js is the application’s main JavaScript file. We’ll work with this more in Iteration 2.

  • styles/ is the folder that holds CSS and related styling files.

• bower_components/ contains the project dependencies installed by Bower: Bootstrap, jQuery, Mocha, and Chai.

• node_modules/ contains the project dependencies required by Node.js, including Gulp.

• test/ holds test specs used by the chosen testing framework(s). In this case, we’re using Mocha and Chai.

• gulpfile.js is the Gulp build script used to build and run the application.

• package.json is used by Node.js to manage dependencies that Gulp needs to execute the project scripts.

• dist/ contains build-related artifacts created by gulp build.

To wrap up our discussion on generator-webapp, here are the other important commands you’ll need to know:

Ctrl-C

Stop the application (the web server).

gulp lint

Use lint to validate the JavaScript files in the application.

gulp +serve:test

Test the web application. In this case, it runs PhantomJS with Mocha and Chai.

gulp build

Build and package the application for deployment.

gulp clean

Clean the artifacts generated when testing and building the application.

You can get the full list of commands by typing gulp --tasks at the command line.

Please shut down the web application before moving to Iteration 2.

Iteration 2—Make an HTTP Call with jQuery

In Iteration 1, we developed a web application with Speaker data hardcoded in the main page, and now it’s time to add “live” content and functionality.

We’ll take the following steps:

1. Factor the hardcoded Speaker data out of the main page.

2. Add a separate JSON file to hold the Speaker data.

3. Use jQuery to populate the main page with Speaker data from the JSON file.

If you’d like to set up the Yeoman project for Iteration 2 by yourself, do the following:

• Follow the instructions in the book’s GitHub repository.

• Don’t forget to copy the app/index.html file from Iteration 1.

Or if you want to follow along with the Yeoman project provided in the code examples, cd to chapter-2/speakers-web-2. In either case, do the following to start the application from the command line:

gulp serve

This command starts the local web server as shown in Iteration 1. You should see the page in Figure 2-3 at http://localhost:9000.

Figure 2-3. Sample Speaker data

This has the hardcoded Speaker data table in the main page (in index.html) that you saw earlier. Please keep the web application running so you can see changes take effect as you save them.

Now, let’s remove the rows from the table body. The HTML for the speakers table now looks like Example 2-11.

<!doctype html>
<html lang="">

  ...

  <body>

    ...

    <table class="table table-striped">
      <thead>
        <tr>
          <th>Name</th>
          <th>About</th>
          <th>Topics</th>
        </tr>
      </thead>
      <tbody id="speakers-tbody">
      </tbody>
    </table>

    ...

  </body>
</html>

In this example, we now have an empty table that has only a header row. We use Bootstrap’s table-striped CSS class so that we’ll have zebra-striped rows. Notice the speakers-tbody ID on the <tbody> element that holds the table’s content. Later, jQuery will use this ID to populate the table rows.

We now need a separate JSON file to hold the Speaker data. Please see the new /speakers-web-2/app/data/speakers.json file that has the Speaker data for the application (this was copied from /chapter-2/data/speakers.json).

To complete Iteration 2, the upgraded app/scripts/main.js file now uses jQuery to populate the speakers table with the data from the app/data/speakers.json file, as shown in Example 2-12.

'use strict';

console.log('Hello JSON at Work!');

$(document).ready(function() {

  function addSpeakersjQuery(speakers) {
    $.each(speakers, function(index, speaker) {
      var tbody = $('#speakers-tbody');
      var tr = $('<tr></tr>');
      var nameCol = $('<td></td>');
      var aboutCol = $('<td></td>');
      var topicsCol = $('<td></td>');

      nameCol.text(speaker.firstName + ' ' + speaker.lastName);
      aboutCol.text(speaker.about);
      topicsCol.text(speaker.tags.join(', '));

      tr.append(nameCol);
      tr.append(aboutCol);
      tr.append(topicsCol);
      tbody.append(tr);
    });
  }

  $.getJSON('data/speakers.json',
    function(data) {
      addSpeakersjQuery(data.speakers);
    }
  );

});

In this example, we put the code inside jQuery’s $(document).ready() so that the entire page (including the DOM) is “ready” (fully loaded). $.getJSON() is a jQuery method that makes an HTTP GET request on a URL and converts the JSON response to a JavaScript object. In this case, the app/data/speakers.json file is addressable as a URL through HTTP because it is deployed as a part of the web application. The $.getJSON() callback method then delegates the job of populating the speakers table to the addSpeakersjQuery() function.

The addSpeakersjQuery() method loops through the speakers array by using the jQuery .each() method. The .each() function does the following:

• Finds the <tbody> element in the speakers table by using the speakers-tbody ID we showed in the index.html file

• Creates a row and its columns by filling in the <tr> and <td> elements with the data from the speaker object

• Appends the new row to the <tbody> element

For more information on jQuery’s getJSON() function, see the jQuery Foundation website.

If you kept the web application running, you should now see the screen in Figure 2-4.

Figure 2-4. Sample Speaker data with JSON file and jQuery

The main page looks the same, but we were expecting that. We’ve improved the application by factoring out the hardcoded Speaker data from the main page, and we’re now making an HTTP call. At this point, we have some of the elements of a real web application that populates its pages dynamically, but here are the drawbacks:

• The JSON data comes from a file within the web application, and we want it to come from a RESTful API.

• The JavaScript code knows about HTML elements on the main page. We would like to reduce the amount of HTML and DOM manipulation.

Please shut down the web application before moving to Iteration 3.

Iteration 3—Consume Speaker Data from a Stub API and Use a Template

In Iteration 2, we made an HTTP call to populate the main page with Speaker data from a JSON file, and we’re now going to get the data from the Stub API provided by json-server that was used in Chapter 1. We’re also going to factor the HTML and DOM manipulation out of the JavaScript into an external Mustache template.

We’ll take the following steps:

1. Modify the HTTP call to point to the json-server URI.

2. Use a Mustache template to remove the HTML and DOM manipulation from JavaScript.

If you’d like to set up the Yeoman project for Iteration 2 by yourself, do the following:

• Follow the instructions in the book’s GitHub Repository.

• Don’t forget to copy the following files from Iteration 2:

  • app/index.html

  • app/scripts/main.js

Or if you want to follow along with the Yeoman project provided in the code examples, cd to chapter-2/speakers-web-3.

Next, let’s modify the HTTP call in main.js to point to the Speaker Stub API (provided by json-server), as shown in Example 2-13.

...

  $.getJSON('http://localhost:5000/speakers',
    function(data) {
      addSpeakersjQuery(data);
    }
  );

...

The code now invokes the Speaker Stub API provided by json-server. Note that data is passed to addSpeakersjQuery() because json-server doesn’t emit the named speakers Array.

First, open a new terminal session and run json-server on port 5000 from your command line:

cd chapter-2/data

json-server -p 5000 ./speakers.json

Start the web application (in another terminal session) from the command line:

gulp serve

This command starts the local web server as shown in Iterations 1 and 2. You should see the same Speaker data when you visit http://localhost:9000 in your browser. But the web application is in better shape because it’s using data from an API rather than a file. Please keep the web application running so you can see changes take effect as you save them.

To complete Iteration 3, let’s factor out the HTML/DOM manipulation from our JavaScript code into a Mustache template. Mustache bills itself as providing logic-less templates, which means that there are no control statements (e.g., for or if) needed to generate HTML from JavaScript and other languages. Mustache works with multiple languages.

Example 2-14 is our Mustache template that generates HTML content based on Speaker data.

<!--
[speakers-mustache-template.html]
This is the template for items in the speakers array when the app first loads
-->
<script id="speakerTemplate" type="text/html">
  {{#.}}
    <tr>
      <td>{{firstName}} {{lastName}}</td>
      <td>{{about}}</td>
      <td>{{tags}}</td>
    </tr>
  {{/.}}
</script>

Note the following about this example:

• The template is an external file to keep the HTML out of our JavaScript code.

• The template code resides within a <script> element.

• The HTML is structured just as it would be in a regular web page.

• Mustache fills in the data by using variables enclosed in double parentheses.

• The context enables Mustache to loop through the Array of Speaker data. We have an anonymous (nameless) collection that we received from the HTTP call, so we enclose all our elements within a beginning {{#.}} and closing {{/.}} to set the context. Note that if we had a named Array (e.g., speakers), the context would begin with {{#speakers}} and end with {{/speakers}}.

• Each variable represents a field name within the specified context. For example, the {{firstName}} variable gets data from the firstName field for the current element in the Speaker data Array.

Please review Wern Ancheta’s excellent article, “Easy Templating with Mustache.js” for a deeper discussion on Mustache.

Besides Mustache, a couple of other solid templating libraries are frequently used by the JavaScript community:

Handlebars.js

Handlebars is very similar to Mustache.

Underscore.js

This is a general utility library, but it includes some templating functionality.

In addition, most MVC frameworks (AngularJS, Ember, and Backbone) have some form of templating. We’ll cover Mustache and Handlebars more thoroughly in Chapter 7.

Example 2-15 shows our refactored app/scripts/main.js file that now uses Mustache.

'use strict';

console.log('Hello JSON at Work!');

$(document).ready(function() {

  function addSpeakersMustache(speakers) {
    var tbody = $('#speakers-tbody');

    $.get('templates/speakers-mustache-template.html', function(templatePartial) {
      var template = $(templatePartial).filter('#speakerTemplate').html();
      tbody.append(Mustache.render(template, speakers));
    }).fail(function() {
      alert("Error loading Speakers mustache template");
    });

  }

  $.getJSON('http://localhost:5000/speakers',
    function(data) {
      addSpeakersMustache(data);
    }
  );

});

In this example, the addSpeakerMustache() function converts the Speaker data (that we received from json-server) into HTML by using our Mustache template. We use the jQuery’s $.get() method to pull in the external Mustache template. When the $.get() call completes, we then find the main page’s <tbody> element (just as before) and then use the append() method to append the HTML content that was created by Mustache.render() (based on the template and Speaker data).

But we’re not quite done, because we need to add Mustache to the web application:

• Use Bower to install Mustache into the web application. From the command line in the speakers-web-3 directory, type bower install mustache.

• Add Mustache to app/index.html (right after main.js) as shown in Example 2-16.

<!doctype html>
<html lang="">

  ...

  <body>

    ...

    <script src="bower_components/mustache.js/mustache.js"></script>

    ...

  </body>
</html>

If you kept the web application running, you should now see the screen in Figure 2-5.

Notice that Mustache formats the Speaker data a little differently, but we improved the web application by making an API call to the Stub API (provided by json-server) and by templating the HTML with Mustache.

Of course, you can go further by using AngularJS or React, but this is left as an exercise for you.

Please don’t forget to shut down both the web application and json-server with a Ctrl-C in each terminal session.

Figure 2-5. Speaker data using Mustache

The MultiJson Object

The MultiJson Object provides the following methods:

• MultiJson.dump() serializes Ruby to JSON.

• MultiJson.load() deserializes from JSON to Ruby.

Note that MultiJson.dump() does the following:

• Uses traditional Ruby snake case (first_name) rather than the recommended cross-platform camel case (firstName) when serializing the speaker Object with oj.

• Doesn’t generate a JSON String when serializing the speaker Object with the JSON engine. This is because the JSON gem doesn’t serialize a class unless it implements a to_json() method.

• Uses snake case (first_name) rather than camel case (firstName) for key names.

According to the RubyDoc MultiJson documentation, here is the method signature for MultiJson.dump():

#dump(object, options = {})

The options provided depend on the underlying JSON implementation (in this case oj) because MultiJson is a wrapper.

JSON Serialization/Deserialization with Simple Ruby Data Types

We’ll start by serializing some basic Ruby Data Types:

• Integer

• String

• Boolean

• Array

• Hash

• Object

Example 3-1 shows how to serialize/deserialize simple Ruby data types with MultiJson and oj.

require 'multi_json'

puts "Current JSON Engine = #{MultiJson.current_adapter()}"
puts

age = 39 # Integer
puts "age = #{MultiJson.dump(age)}"
puts

full_name = 'Larson Richard' # String
puts "full_name = #{MultiJson.dump(full_name)}"
puts

reqistered = true # Boolean
puts "reqistered = #{MultiJson.dump(reqistered)}"
puts

tags = %w(JavaScript, AngularJS, Yeoman) # Array of Strings
puts "tags = #{MultiJson.dump(tags)}"
puts

email = { email: 'larsonrichard@ecratic.com' } # Hash
puts "email = #{MultiJson.dump(email)}"
puts

class Speaker
  def initialize(first_name, last_name, email, about,
                 company, tags, registered)
    @first_name = first_name
    @last_name = last_name
    @email = email
    @about = about
    @company = company
    @tags = tags
    @registered = registered
  end
end

speaker = Speaker.new('Larson', 'Richard', 'larsonrichard@ecratic.com',
            'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
            'Ecratic', %w(JavaScript, AngularJS, Yeoman), true)

puts "speaker (using oj gem) = #{MultiJson.dump(speaker)}"
puts

When you run ruby basic_data_types_serialize.rb from the command line, you should get the following:

MultiJson.dump() doesn’t do much with the scalar types (Integer, String, and Boolean). Things begin to get interesting with the speaker Object because here MultiJson.dump() initially generates a valid, yet unattractive, JSON String. As you’ll soon see, MultiJson.dump() has other parameters that enhance serialization.

To make things more readable, we’ll leverage the :pretty ⇒ true option to pretty-print the JSON output from the speaker Object, as shown in Example 3-2. Although pretty-printing is more attractive to look at, it is inefficient, and should be used only for debugging purposes.

require 'multi_json'

...

speaker = Speaker.new('Larson', 'Richard', 'larsonrichard@ecratic.com',
            'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
            'Ecratic', %w(JavaScript, AngularJS, Yeoman), true)

puts "speaker (using oj gem) = #{MultiJson.dump(speaker, pretty: true)}"
puts

Running the preceding code yields the following pretty-printed speaker Object:

JSON Deserialization with Objects and MultiJson

MultiJson can also deserialize JSON. Let’s use the MultiJson.load() method to deserialize JSON into a Ruby Hash. But this causes an impedance mismatch because the speaker Object’s initialize() method takes Strings (which match the speaker Object’s attributes) as parameters. We’ll need to convert Hash to a set of attributes to instantiate a speaker Object. Fortunately, it’s unnecessary to write any code to convert the Hash because the well-known OpenStruct makes the Hash (from decoding JSON) look like an object.

Example 3-3 shows the use of OpenStruct.

require 'ostruct'

h = { first_name: 'Fred' }
m = OpenStruct.new(h)
puts m             # prints: #<OpenStruct first_name="Fred">
puts m.first_name  # prints: Fred

OpenStruct is a data structure that is similar to a Hash, and it allows you define key/value pairs of attributes and their values. OpenStruct is part of Ruby Core and provides the ability to access keys as attributes. For more information about OpenStruct, see the Ruby Core documentation.

When we instantiate a new speaker Object, it would be great to print out the new object in a readable manner for debugging purposes. With puts, you’d normally see something like this:

puts speaker # #<Speaker:0x007f84412e0e38>

With the awesome_print gem, the output is much more attractive. For more information, see the awesome_print GitHub repository.

Before running the code in Example 3-4, install the awesome_print gem from the command line:

gem install awesome_print

require 'multi_json'
require 'ostruct'
require 'awesome_print'

puts "Current JSON Engine = #{MultiJson.current_adapter()}"
puts

class Speaker
  def initialize(first_name, last_name, email, about,
                 company, tags, registered)
    @first_name = first_name
    @last_name = last_name
    @email = email
    @about = about
    @company = company
    @tags = tags
    @registered = registered
  end
end

speaker = Speaker.new('Larson', 'Richard', 'larsonrichard@ecratic.com',
            'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
            'Ecratic', %w(JavaScript, AngularJS, Yeoman), true)

json_speaker = MultiJson.dump(speaker, pretty: true)
puts "speaker (using oj gem) = #{MultiJson.dump(speaker)}"
puts

ostruct_spkr = OpenStruct.new(MultiJson.load(json_speaker))

speaker2 =  Speaker.new(ostruct_spkr.first_name, ostruct_spkr.last_name,
                     ostruct_spkr.email, ostruct_spkr.about, ostruct_spkr.company,
                     ostruct_spkr.tags, ostruct_spkr.registered)

puts "speaker 2 after MultiJson.load()"
ap speaker2
puts

Run this example, and we’ll see that the preceding code successfully deserialized the JSON String stored in json_speaker into an OpenStruct Object and finally into another speaker instance—speaker2. Note the use of awesome_print’s ap method rather than the built-in puts to pretty-print the Object.

Although multi_json and oj efficiently process JSON, sometimes developers need more control over the data to be serialized.

A Word on Camel Casing and JSON

If you haven’t noticed, JSON Keys/Property Names are usually in camel case form. For example, a Key that represents someone’s first name would normally be expressed as firstName. But up to this point, we’ve seen that Ruby’s JSON libraries natively express Keys in snake case (first_name). While this may be OK for small code examples and Unit Tests that no one else will use, snake case is incompatible with the rest of the world. Here’s why:

• JSON must be interoperable. Although my stance on this will probably offend many ardent Rubyists, and others may call this bike shedding, the whole point of JSON and REST is interoperability across heterogeneous applications. There are other programming languages than Ruby, and the rest of the world is expecting camel case (firstName). If your API works in a way that is unexpected, people won’t want to use it.

• The major players use camel-cased JSON:

  • Google has standardized on camel case in their Google JSON Style Guide.

  • The majority of JSON-based public APIs (e.g., Amazon AWS, Facebook, and LinkedIn) use camel-cased JSON.

• Avoid platform bleed-through. JSON should look the same regardless of the platform/programming language that generates or consumes it. The Ruby on Rails community prefers snake case, which is just fine within that platform, but this local programming language idiom shouldn’t be reflected in an API.

JSON Serialization with Objects and ActiveSupport

The ActiveSupport gem provides functionality that has been extracted from Rails, including time zones, internationalization, and JSON encoding/decoding. ActiveSupport’s JSON module provides the ability to do the following:

• Convert between camel case and snake case

• Choose which portions of an Object to serialize

You can install ActiveSupport from the command line as follows:

gem install activesupport

We’ll use ActiveSupport::JSON.encode() to serialize a speaker Object into JSON, as shown in Example 3-5.

require 'active_support/json'
require 'active_support/core_ext/string'

...

speaker = Speaker.new('Larson', 'Richard', 'larsonrichard@ecratic.com',
            'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
            'Ecratic', %w(JavaScript, AngularJS, Yeoman), true)

json = ActiveSupport::JSON.encode(speaker).camelize(first_letter = :lower)
puts "Speaker as camel-cased JSON \n#{json}"
puts

json = ActiveSupport::JSON.encode(speaker,
                       only: ['first_name', 'last_name'])
                       .camelize(first_letter = :lower)

puts "Speaker as camel-cased JSON with only firstName and lastName \n#{json}"
puts

In the code example, you’ll notice that ActiveSupport::JSON.encode() provides the following options:

• Camel case (firstName) Key names by chaining with the camelize() method. Note that the first letter of each Key is capitalized by default, so you’ll need to use the first_letter = :lower parameter to get lower camel case format.

• Limit the portions of the speaker Object to serialize by using the only: parameter.

When you run the code, you should see the following:

But if you only want to convert from snake case to camel case, the awrence gem is a simple alternative. awrence converts snake-cased Hash keys to camel case, which you can then convert to camel-cased JSON. I haven’t tried this gem yet, so this is left as an exercise for you.

JSON Deserialization with Objects and ActiveSupport

ActiveSupport also has the ability to deserialize JSON. We’ll now use the decode() method to deserialize JSON into a Ruby Hash. Just as before, we’ll leverage OpenStruct and awesome_print to help with instantiation and printing, as shown in Example 3-6.

require 'multi_json'
require 'active_support/json'
require 'active_support/core_ext/string'
require 'ostruct'
require 'awesome_print'

...

speaker = Speaker.new('Larson', 'Richard', 'larsonrichard@ecratic.com',
            'Incididunt mollit cupidatat magna excepteur do tempor ex non ...',
            'Ecratic', %w(JavaScript, AngularJS, Yeoman), true)

json_speaker = ActiveSupport::JSON.encode(speaker)
puts "speaker (using oj gem) = #{ActiveSupport::JSON.encode(speaker)}"
puts ostruct_spkr = OpenStruct.new(ActiveSupport::JSON.decode(json_speaker))

speaker2 =  Speaker.new(ostruct_spkr.first_name, ostruct_spkr.last_name,
                        ostruct_spkr.email, ostruct_spkr.about, ostruct_spkr.company,
                        ostruct_spkr.tags, ostruct_spkr.registered)

puts "speaker 2 after ActiveSupport::JSON.decode()"
ap speaker2
puts

You’ll see the following result when you run the preceding code from the command line:

The plissken gem is an alternative that converts from camel-cased Hash keys (that originated from JSON) to snake case. We’ll use plissken in our upcoming Unit Tests.

Just Enough Unit Testing with Minitest

The two most common Ruby testing frameworks are Minitest, which is part of Ruby Core and RSpec. Both Minitest and RSpec are excellent, but we can use only one of them in this chapter to keep the focus on JSON.

On one hand, Minitest

• Is part of the Ruby Standard Library, so there’s nothing else to install.

• Is lightweight and simple.

• Has most of the functionality that RSpec provides.

On the other hand, RSpec

• Requires you to install a separate rspec gem, but enjoys wide acceptance in the Ruby and Rails communities.

• Is large and complex. The RSpec code base is about eight times larger than Minitest.

• Has a richer set of matchers than Minitest.

For me, it’s really a matter of taste, and you’ll be fine with either framework. I chose Minitest because it comes standard with Ruby.

Minitest lets you choose between BDD (Minitest::Spec) and TDD (Minitest::Test) style testing. Let’s go with Minitest::Spec for the following reasons:

• I prefer BDD’s simple English-style sentences that describe each test.

• It looks similar to RSpec, so the tests will look familiar to those developers who use RSpec.

• It’s consistent with the JavaScript-base Mocha/Chai testing in the rest of this book.

This chapter covers only the basics of Minitest. To learn more, see Chris Kottom’s excellent book, The Minitest Cookbook.

Setting Up the Unit Test

Before going further, be sure to set up your test environment. If you haven’t installed Ruby on Rails yet, refer to Appendix A, and install Ruby on Rails (see “Install Ruby on Rails” and “Install Ruby Gems”). If you want to follow along with the Ruby project provided in the code examples, cd to chapter-3/speakers-test and do the following to install all dependencies for the project:

bundle install

Bundler provides dependency management for Ruby projects.

If you’d like to set up the speakers-test Ruby project yourself, follow the instructions in the book’s GitHub repository.

Test Data

We’ll use the Speaker data from earlier chapters as our test data and deploy it as a RESTful API. Again, we’ll leverage the json-server Node.js module to serve up the data/speakers.json file as a Web API. If you need to install json-server, refer to “Install npm Modules” in Appendix A.

Here’s how to run json-server on port 5000 from your local machine:

cd chapter-3/data

json-server -p 5000 ./speakers.json

You can also get a single speaker by adding the id to the URI as follows: http://localhost:5000/speakers/1. With the Stub API in place, it’s time to write some Unit Tests.

JSON and Minitest Testing with APIs

Our Unit Test will do the following:

• Make HTTP calls to the Stub Speakers API

• Check the values from the HTTP Response Body against expected values

As in previous chapters, we’ll continue to leverage the open source Unirest API wrapper, but this time we’ll use the Ruby implementation. Please note that the Unirest gem takes the JSON in the HTTP Response Body, parses it into a Ruby Hash, and returns it to the caller (inside the HTTP Response Body). This means that the Unit Test won’t be testing directly against JSON data, but rather it will test against the Hash that was populated by the JSON response from the API.

Speakers Unit Test

The Unit Test in Example 3-7 shows how to use Unirest to invoke the Speaker Stub API provided by json-server and test the response.

require 'minitest_helper'

require 'unirest'
require 'awesome_print'
require 'ostruct'
require 'plissken'
require 'jq/extend'

require_relative '../models/speaker'


describe 'Speakers API' do
  SPEAKERS_ALL_URI = 'http://localhost:5000/speakers'

  before do
    @res = Unirest.get SPEAKERS_ALL_URI,
                      headers:{ 'Accept' => "application/json" }

  end

  it 'should return a 200 response' do
    expect(@res.code).must_equal 200
    expect(@res.headers[:content_type]).must_equal 'application/json; charset=utf-8'
  end

  it 'should return all speakers' do
    speakers = @res.body
    expect(speakers).wont_be_nil
    expect(speakers).wont_be_empty
    expect(speakers.length).must_equal 3
  end

  it 'should validate the 3rd speaker as an Object' do
    speakers = @res.body
    ostruct_spkr3 = OpenStruct.new(speakers[2].to_snake_keys())

    expect(ostruct_spkr3.company).must_equal 'Talkola'
    expect(ostruct_spkr3.first_name).must_equal 'Christensen'
    expect(ostruct_spkr3.last_name).must_equal 'Fisher'
    expect(ostruct_spkr3.tags).must_equal ['Java', 'Spring', 'Maven', 'REST']

    speaker3 =  Speaker.new(ostruct_spkr3.first_name, ostruct_spkr3.last_name,
                            ostruct_spkr3.email, ostruct_spkr3.about,
                            ostruct_spkr3.company, ostruct_spkr3.tags,
                            ostruct_spkr3.registered)

    expect(speaker3.company).must_equal 'Talkola'
    expect(speaker3.first_name).must_equal 'Christensen'
    expect(speaker3.last_name).must_equal 'Fisher'
    expect(speaker3.tags).must_equal ['Java', 'Spring', 'Maven', 'REST']
  end

  it 'should validate the 3rd speaker with jq' do
    speakers = @res.body
    speaker3 = speakers[2]

    speaker3.jq('.company') {|value| expect(value).must_equal 'Talkola'}
    speaker3.jq('.tags') {|value|
        expect(value).must_equal ['Java', 'Spring', 'Maven', 'REST']}
    speaker3.jq('.email') {|value|
        expect(value).must_equal 'christensenfisher@talkola.com'}
    speaker3.jq('. | "\(.firstName) \(.lastName)"') {|value|
        expect(value).must_equal 'Christensen Fisher'}
  end

end

Note the following in this Unit Test:

• The minitest_helper consolidates configuration and setup and factors it out of this test. We’ll cover Minitest Helpers later in this chapter.

• The test executes the Unirest GET request synchronously (and gets a response) with Minitest’s before method, so that setup occurs in only one place in the code. Minitest executes before before running each test (i.e., it) within the context of the describe.

• The should return all speakers test does the following:

  • Ensures that the HTTP Response Body is not empty

  • Checks whether the Speakers API returns three speakers

• The should validate the 3rd speaker as an Object test works as follows:

  • Populate the speakers Hash from the HTTP Response Body (@res.body). At this point, the JSON from the API has already been parsed by Unirest and converted to a Ruby Hash.

  • Use OpenStruct.new() to convert the Hash for the third speaker into an OpenStruct, an Object-like structure. The to_snake_keys() method (from the plissken gem) converts the camel-cased (firstName) Hash keys to snake case (first_name) for compatibility with Ruby.

  • Use Minitest BDD-style expect assertions to check for expected results:

    • The third speaker’s company, first_name, last_name, and tags match the values in the speakers.json file.

• The should validate the 3rd speaker with jq test works as follows:

  • Use jq queries (e.g., .company) to check the same fields as in the previous test. jq simplifies Unit Testing by enabling a developer to query the JSON-based Hashes without the need to convert to an object. jq is a powerful JSON search tool, and Chapter 6 covers it in greater detail.

  • The . | "\(.firstName) \(.lastName)" query does a String interpolation to combine the firsName and lastName fields into the speaker’s full name for testing purposes.

  • The ruby-jq gem provides a solid Ruby-based jq implementation.

To run this test, use bundle exec rake from the command line, and you should see the following:

rake is a commonly used build utility for Ruby projects. In the bundle exec rake command, the following occurs:

• rake uses the gems that Bundler listed in this project’s Gemfile.

• rake has been configured to use test as the default task.

The Rakefile defines the build tasks, and looks like Example 3-8.

require 'rake/testtask'

Rake::TestTask.new(:test) do |t|
  t.libs = %w(lib test)
  t.pattern = 'test/**/*_spec.rb'
  t.warning = false
end

task :default => :test

By default, Minitest is silent and doesn’t indicate that tests are passing. In the preceding Unit Test run, notice that passing tests show in the output. The speakers-test project leverages the minitest-reporters gem to make the output more readable.

The Minitest Helper in Example 3-9 configures the minitest and minitest-reporters gems for use by the speakers_spec.

require 'minitest/spec'
require 'minitest/autorun'

require "minitest/reporters"
Minitest::Reporters.use! Minitest::Reporters::SpecReporter.new

For the sake of completeness, Example 3-10 shows the Speaker Plain Old Ruby Object (PORO) that holds the Speaker data.

class Speaker
  attr_accessor :first_name, :last_name, :email,
                :about, :company, :tags, :registered

  def initialize(first_name, last_name, email, about,
                 company, tags, registered)
    @first_name = first_name
    @last_name = last_name
    @email = email
    @about = about
    @company = company
    @tags = tags
    @registered = registered
  end
end

The preceding code is plain and simple:

• speaker.rb resides in the models directory to follow commonly accepted Ruby project conventions.

• attr_accessor defines the Speaker’s data members (e.g., first_name) and accessor methods (getters/readers and setters/writers) for the data members.

• initialize() initializes the data members when Speaker.new() is called.

Before moving on, you can stop json-server by pressing Ctrl-C at the command line.

Further Reading on Ruby and Minitest

We’ve covered only the basics of Ruby and Minitest in this chapter. To learn more, please see the following resources:

• Ruby in Practice, by Jeremy McAnally and Assaf Arkin (Manning)

• The Well-Grounded Rubyist, 2nd Ed., by David A. Black (Manning)

• Minitest Cookbook, by Chris Kottam

What Is Missing in the Unit Tests?

So far, the Unit Tests have done a decent job of testing JSON data, but something is missing. The code had to check for the existence of all the expected fields, which is clumsy and cumbersome. Imagine how arduous this would be for larger, deeper, more complex JSON documents. There’s a solution for this problem: JSON Schema (this is covered in Chapter 5).

We’ve shown how to deploy and interact with a Stub API, and now it’s time to build a small RESTful API with Ruby on Rails.

Choose a JSON Serializer

There are several options for rendering JSON in Ruby on Rails. Here’s a list of the most widely used techniques:

ActiveModel::Serializers (AMS)

AMS provides functionality to objects that need some ActiveRecord features, such as serialization and validation. AMS is part of the Rails API, and you can find documentation on GitHub.

Jbuilder

A Domain-Specific Language (DSL) builder that uses a separate template (i.e., outside the controller) that controls the output. For further details, please see Jbuilder on GitHub.

RABL

Ruby API Builder Language (RABL) generates JSON, XML, PList, MessagePack, and BSON. This gem also uses a template file. The RABL GitHub repository has details.

speakers-api-1—Create an API with Camel-Cased JSON

We’ll take the following steps to create and deploy the speakers-api-1 API with Rails 5:

1. Set up the project.

2. Write source code:

   • Model

   • Serializer

   • Controller

3. Deploy the API.

4. Test with Postman.

cd speakers-api-1

bundle exec spring binstub --all

class Speaker < ActiveModelSerializers::Model
  attr_accessor :first_name, :last_name, :email,
                :about, :company, :tags, :registered

  def initialize(first_name, last_name, email, about,
                 company, tags, registered)
    @first_name = first_name
    @last_name = last_name
    @email = email
    @about = about
    @company = company
    @tags = tags
    @registered = registered
  end
end

class SpeakerSerializer < ActiveModel::Serializer
  attributes :first_name, :last_name, :email,
             :about, :company, :tags, :registered
end

require 'speaker'

class SpeakersController < ApplicationController
  before_action :set_speakers, only: [:index, :show]

  # GET	/speakers
  def index
    render json: @speakers
  end

  # GET	/speakers/:id
  def show
    id = params[:id].to_i - 1

    if id >= 0 && id < @speakers.length
      render json: @speakers[id]
    else
      render plain: '404 Not found', status: 404
    end
  end

  private

  def set_speakers
    @speakers = []

    @speakers << Speaker.new('Larson', 'Richard', 'larsonrichard@ecratic.com',
      'Incididunt mollit cupidatat magna ...', 'Ecratic',
      ['JavaScript', 'AngularJS', 'Yeoman'], true)

    @speakers << Speaker.new('Ester', 'Clements', 'esterclements@acusage.com',
      'Labore tempor irure adipisicing consectetur ...', 'Acusage',
      ['REST', 'Ruby on Rails', 'APIs'], true)

    @speakers << Speaker.new('Christensen', 'Fisher',
      'christensenfisher@talkola.com', 'Proident ex Lorem et Lorem ad ...',
      'Talkola',
      ['Java', 'Spring', 'Maven', 'REST'], true)
  end

end

Rails.application.routes.draw do
  get 'speakers/index'

  get 'speakers/show'

  # For details on the DSL available within this file,
  # see http://guides.rubyonrails.org/routing.html
end

Rails.application.routes.draw do

  resources :speakers, :only => [:show, :index]

  # For details on the DSL available within this file,
  # see http://guides.rubyonrails.org/routing.html
end

{
  "first_name": "Larson",
  "last_name": "Richard",
  "email": "larsonrichard@ecratic.com",
  "about": "Incididunt mollit cupidatat magna ...",
  "company": "Ecratic",
  "tags": [
    "JavaScript",
    "AngularJS",
    "Yeoman"
  ],
  "registered": true
}

ActiveModelSerializers.config.key_transform = :camel_lower

Deploy the API

In the speakers-api-1 directory, run rails s to deploy the API at http://localhost:3000/speakers, and you’ll see the following:

Test the API with Postman

Now that the Speakers API is up and running, let’s test with Postman (as we did in Chapter 1) to get the first speaker. In the Postman GUI, do the following:

• Enter the http://localhost:3000/speakers/1 URL.

• Choose GET as the HTTP verb.

• Click the Send button.

You should see that the GET ran properly in Postman with the speaker JSON data in the HTTP Response Body text area and a 200 (OK) HTTP Status, as shown in Figure 3-1.

Figure 3-1. Speaker JSON data with Postman

You can stop speakers-api-1 by pressing Ctrl-C at the command line.

speakers-api-2—Create an API that Customizes the JSON Representation

AMS’s JSON customization functionality goes beyond camel-casing. The second API application will show how AMS can customize (alter) the JSON representation of each speaker. Except for the new SpeakerSerializer, speakers-api-2 has all the same code as the original speakers-api-1 project, so we’ll just focus on serialization.

Before going further, please install the gems to run the speakers-api-2 project. Do the following:

cd speakers-api-2

bundle exec spring binstub --all

class SpeakerSerializer < ActiveModel::Serializer
  attributes :name, :email, :about,
             :company, :tags, :registered

  def name
    "#{object.first_name} #{object.last_name}"
  end
end

Test the API with Postman

In the Postman GUI, invoke HTTP GET on http://localhost:3000/speakers/1 and you should see the screen in Figure 3-2.

Figure 3-2. Customized Speaker JSON data with Postman

Don’t forget to stop the speakers-api-2 application by pressing Ctrl-C at the command line.

Further Reading on Rails and Rails-based APIs

We’ve shown just enough Rails-based APIs and AMS to get a simple API to work. To go deeper, please see the following resources:

• Ruby on Rails Tutorial: Learn Web Development with Rails, by Michael Hartl

• Learn Ruby on Rails 5, by Daniel Kehoe

• APIs on Rails: Building REST APIs with Rails, by Abraham Kuri

• Get Up and Running with Rails API, by Chris Kottam

• Active Model Serializers, Rails, and JSON! OH MY!, by Kendra Uzia

Serialization/Deserialization with Simple Java Data Types

As in previous chapters, we’ll start by serializing some basic Java data types:

• integer

• string

• array

• boolean

Example 4-1 shows a simple Unit Test that uses Jackson and JUnit 4 to serialize/deserialize simple Java data types.

package org.jsonatwork.ch4;

import static org.junit.Assert.*;

import java.io.*;
import java.util.*;

import org.junit.Test;

import com.fasterxml.jackson.core.*;
import com.fasterxml.jackson.core.type.*;
import com.fasterxml.jackson.databind.*;

public class BasicJsonTypesTest {
  private static final String TEST_SPEAKER = "age = 39\n" +
    "fullName = \"Larson Richard\"\n" +
    "tags = [\"JavaScript\",\"AngularJS\",\"Yeoman\"]\n" +
    "registered = true";

  @Test
  public void serializeBasicTypes() {
    try {
      ObjectMapper mapper = new ObjectMapper();
      Writer writer = new StringWriter();
      int age = 39;
      String fullName = new String("Larson Richard");
      List<String> tags = new ArrayList<String>(
          Arrays.asList("JavaScript", "AngularJS", "Yeoman"));

      boolean registered = true;
      String speaker = null;

      writer.write("age = ");
      mapper.writeValue(writer, age);
      writer.write("\nfullName = ");
      mapper.writeValue(writer, fullName);
      writer.write("\ntags = ");
      mapper.writeValue(writer, tags);
      writer.write("\nregistered = ");
      mapper.configure(SerializationFeature.INDENT_OUTPUT, true);
      mapper.writeValue(writer, registered);
      speaker = writer.toString();
      System.out.println(speaker);
      assertTrue(TEST_SPEAKER.equals(speaker));
      assertTrue(true);
    } catch (JsonGenerationException jge) {
      jge.printStackTrace();
      fail(jge.getMessage());
    } catch (JsonMappingException jme) {
      jme.printStackTrace();
      fail(jme.getMessage());
    } catch (IOException ioe) {
      ioe.printStackTrace();
      fail(ioe.getMessage());
    }
  }

  @Test
  public void deSerializeBasicTypes() {
    try {
      String ageJson = "{ \"age\": 39 }";
      ObjectMapper mapper = new ObjectMapper();
      Map<String, Integer> ageMap = mapper.readValue(ageJson,
          new TypeReference<HashMap<String,Integer>>() {});

      Integer age = ageMap.get("age");

      System.out.println("age = " + age + "\n\n\n");
      assertEquals(39, age.intValue());
      assertTrue(true);
    } catch (JsonMappingException jme) {
      jme.printStackTrace();
      fail(jme.getMessage());
    } catch (IOException ioe) {
      ioe.printStackTrace();
      fail(ioe.getMessage());
    }
  }

}

In this example, the @Test annotation tells JUnit to run the serializeBasicTypes() and deSerializeBasicTypes() methods as part of the test. These Unit Tests don’t do many assertions on the JSON data itself. We’ll cover assertions in more detail later when we test against a Web API.

Here are the most important Jackson classes and methods that serialize/deserialize to/from JSON:

• ObjectMapper converts between Java and JSON constructs.

• ObjectMapper.writeValue() converts a Java data type to JSON (and in this case, outputs to a Writer).

• ObjectMapper.readValue() converts JSON to a Java data type.

Run a single Unit Test from the command line as follows:

cd chapter-4/speakers-test

+gradle test --tests org.jsonatwork.ch4.BasicJsonTypesTest+

You should see these results:

This example isn’t too exciting right now because it serializes/deserializes only simple data types to/from JSON. Serialization/deserialization gets more interesting when Objects are involved.

Serialization/Deserialization with Java Objects

Now that we have a decent grasp of Jackson and how to work with simple Java data types, let’s wade in deeper with Objects. Example 4-2 shows how to use Jackson to serialize/deserialize a single speaker Object, and then how to deserialize a JSON file into multiple speaker Objects.

package org.jsonatwork.ch4;

import static org.junit.Assert.*;

import java.io.*;
import java.net.*;
import java.util.*;

import org.junit.Test;

import com.fasterxml.jackson.core.*;
import com.fasterxml.jackson.databind.*;
import com.fasterxml.jackson.databind.type.*;

public class SpeakerJsonFlatFileTest {

private static final String SPEAKER_JSON_FILE_NAME = "speaker.json";
private static final String SPEAKERS_JSON_FILE_NAME = "speakers.json";
private static final String TEST_SPEAKER_JSON = "{\n" +
    "  \"id\" : 1,\n" +
    "  \"age\" : 39,\n" +
    "  \"fullName\" : \"Larson Richard\",\n" +
    "  \"tags\" : [ \"JavaScript\", \"AngularJS\", \"Yeoman\" ],\n" +
    "  \"registered\" : true\n" +
  "}";

@Test
public void serializeObject() {
    try {
      ObjectMapper mapper = new ObjectMapper();
      Writer writer = new StringWriter();
      String[] tags = {"JavaScript", "AngularJS", "Yeoman"};
      Speaker speaker = new Speaker(1, 39, "Larson Richard", tags, true);
      String speakerStr = null;

      mapper.configure(SerializationFeature.INDENT_OUTPUT, true);
      speakerStr = mapper.writeValueAsString(speaker);
      System.out.println(speakerStr);
      assertTrue(TEST_SPEAKER_JSON.equals(speakerStr));
      assertTrue(true);
    } catch (JsonGenerationException jge) {
      jge.printStackTrace();
      fail(jge.getMessage());
    } catch (JsonMappingException jme) {
      jme.printStackTrace();
      fail(jme.getMessage());
    } catch (IOException ioe) {
      ioe.printStackTrace();
      fail(ioe.getMessage());
    }
  }

  private File getSpeakerFile(String speakerFileName) throws URISyntaxException {
    ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
    URL fileUrl = classLoader.getResource(speakerFileName);
    URI fileUri = new URI(fileUrl.toString());
    File speakerFile = new File(fileUri);

    return speakerFile;
  }

  @Test
  public void deSerializeObject() {
    try {
      ObjectMapper mapper = new ObjectMapper();
      File speakerFile = getSpeakerFile(
          SpeakerJsonFlatFileTest.SPEAKER_JSON_FILE_NAME);

      Speaker speaker = mapper.readValue(speakerFile, Speaker.class);

      System.out.println("\n" + speaker + "\n");
      assertEquals("Larson Richard", speaker.getFullName());
      assertEquals(39, speaker.getAge());
      assertTrue(true);
    } catch (URISyntaxException use) {
      use.printStackTrace();
      fail(use.getMessage());
    } catch (JsonParseException jpe) {
      jpe.printStackTrace();
      fail(jpe.getMessage());
    } catch (JsonMappingException jme) {
      jme.printStackTrace();
      fail(jme.getMessage());
    } catch (IOException ioe) {
      ioe.printStackTrace();
      fail(ioe.getMessage());
    }
  }

  @Test
  public void deSerializeMultipleObjects() {
    try {
      ObjectMapper mapper = new ObjectMapper();
      File speakersFile = getSpeakerFile(
          SpeakerJsonFlatFileTest.SPEAKERS_JSON_FILE_NAME);

      JsonNode arrNode = mapper.readTree(speakersFile).get("speakers");
      List<Speaker> speakers = new ArrayList<Speaker>();
        if (arrNode.isArray()) {
          for (JsonNode objNode : arrNode) {
          System.out.println(objNode);
          speakers.add(mapper.convertValue(objNode, Speaker.class));
        }
      }

      assertEquals(3, speakers.size());
      System.out.println("\n\n\nAll Speakers\n");
      for (Speaker speaker: speakers) {
        System.out.println(speaker);
      }

      System.out.println("\n");
      Speaker speaker3 = speakers.get(2);
      assertEquals("Christensen Fisher", speaker3.getFullName());
      assertEquals(45, speaker3.getAge());
      assertTrue(true);
    } catch (URISyntaxException use) {
      use.printStackTrace();
      fail(use.getMessage());
    } catch (JsonParseException jpe) {
      jpe.printStackTrace();
      fail(jpe.getMessage());
    } catch (JsonMappingException jme) {
      jme.printStackTrace();
      fail(jme.getMessage());
    } catch (IOException ioe) {
      ioe.printStackTrace();
      fail(ioe.getMessage());
    }
  }

}

Note the following in this JUnit test:

• serializeObject() creates a Speaker Object and serializes it to Standard Output by using the ObjectMapper.writeValueAsString() method and System.out.println(). The test sets the SerializationFeature.INDENT_OUTPUT to true to indent/pretty-print the JSON output.

• deSerializeObject() calls getSpeakerFile() to read a JSON input file (which contains a single speaker JSON Object), and uses the ObjectMapper.readValue() method to deserialize it into a Speaker Java Object.

• deSerializeMultipleObjects() does the following:

  • Calls getSpeakerFile() to read a JSON input file, which contains an array of JSON speaker Objects.

  • Invokes the ObjectMapper.readTree() method to get a JsonNode Object, which is a pointer to the root node of the JSON document that was in the file.

  • Visits each node in the JSON tree and uses the ObjectMapper.convertValue() method to deserialize each speaker JSON object into a Speaker Java Object.

  • Prints out each Speaker Object in the list.

• getSpeakerFile() finds a file on the classpath and does the following:

  • Gets the ContextClassLoader from the current Thread of execution.

  • Uses the ClassLoader.getResource() method to find the filename as a resource within the current classpath.

  • Constructs a File Object based on the URI of the filename.

Each of the preceding tests uses JUnit’s assertion methods to test the results of JSON serialization/deserialization.

You’ll see the following when you run the test from the command line using gradle test --tests org.jsonatwork.ch4.SpeakerJsonFlatFileTest:

Jackson offers much more functionality than can be shown in this chapter. Refer to the following resources for some great tutorials:

• Java Jackson Tutorial, by Eugen Paraschiv

• Jackson Tutorial, Tutorials Point

• Jackson JSON Java Parser API Example Tutorial, by Pankaj (JournalDev)

• Java JSON Jackson Introduction, by Mithil Shah

Test Data

To create the Stub, we’ll use the Speaker data from earlier chapters as our test data, which is available at GitHub and deploy it as a RESTful API. We’ll leverage the json-server Node.js module to serve up the speakers.json file as a Web API. If you need to install json-server, refer to “Install npm Modules” in Appendix A. Here’s how to run json-server on port 5000 from your local machine (using a second terminal session):

cd chapter-4/speakers-test/src/test/resources

json-server -p 5000 ./speakers.json

You can also get a single speaker by adding the id to the URI as follows: http://localhost:5000/speakers/1. With the Stub API in place, it’s time to write some Unit Tests.

JSON and JUnit Testing with APIs

Our Unit Test will do the following:

• Make HTTP calls to the Stub Speakers API

• Check the JSON (from the HTTP Response) against expected values

As in earlier chapters, we’ll continue to leverage the open source Unirest API wrapper, but this time we’ll use the Java version.

In the previous JUnit tests in the chapter, we ensured that only bare minimum functionality was working (no exceptions were thrown), and it’s now time to make our tests a bit more sophisticated. The remaining Unit Tests will look at the JSON content returned from an HTTP Response, and verify that it matches the expected output. We could search through the data and do a comparison with custom code, or we could use a library to reduce the amount of work. JsonUnit has many helpful matchers to simplify JSON comparison in JUnit tests. We’ll cover the basics of JsonUnit in these Unit Tests, but it provides much deeper functionality than we can cover here, including the following:

• Regular Expressions

• More matchers

• The ability to ignore specific fields and values

The Unit Test in Example 4-3 pulls everything together by invoking the Stub API and comparing the JSON response with expected values.

package org.jsonatwork.ch4;

import static org.junit.Assert.*;

import java.io.*;
import java.net.*;
import java.util.*;

import org.apache.http.*;
import org.junit.Test;

import com.fasterxml.jackson.core.*;
import com.fasterxml.jackson.databind.*;
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.*;
import com.mashape.unirest.request.*;

import static net.javacrumbs.jsonunit.fluent.JsonFluentAssert.assertThatJson;

public class SpeakersApiJsonTest {
  private static final String SPEAKERS_ALL_URI = "http://localhost:5000/speakers";
  private static final String SPEAKER_3_URI = SPEAKERS_ALL_URI + "/3";

  @Test
  public void testApiAllSpeakersJson() {
    try {
      String json = null;
      HttpResponse <String> resp = Unirest.get(
        SpeakersApiJsonTest.SPEAKERS_ALL_URI).asString();

      assertEquals(HttpStatus.SC_OK, resp.getStatus());
      json = resp.getBody();
      System.out.println(json);
      assertThatJson(json).node("").isArray();
      assertThatJson(json).node("").isArray().ofLength(3);
      assertThatJson(json).node("[0]").isObject();
      assertThatJson(json).node("[0].fullName")
              .isStringEqualTo("Larson Richard");
      assertThatJson(json).node("[0].tags").isArray();
      assertThatJson(json).node("[0].tags").isArray().ofLength(3);
      assertThatJson(json).node("[0].tags[1]").isStringEqualTo("AngularJS");
      assertThatJson(json).node("[0].registered").isEqualTo(true);
      assertTrue(true);
    } catch (UnirestException ue) {
      ue.printStackTrace();
    }
  }

  @Test
  public void testApiSpeaker3Json() {
    try {
      String json = null;
      HttpResponse <String> resp = Unirest.get(
        SpeakersApiJsonTest.SPEAKER_3_URI).asString();

      assertEquals(HttpStatus.SC_OK, resp.getStatus());
      json = resp.getBody();
      System.out.println(json);
      assertThatJson(json).node("").isObject();
      assertThatJson(json).node("fullName")
              .isStringEqualTo("Christensen Fisher");
      assertThatJson(json).node("tags").isArray();
      assertThatJson(json).node("tags").isArray().ofLength(4);
      assertThatJson(json).node("tags[2]").isStringEqualTo("Maven");
      assertTrue(true);
    } catch (UnirestException ue) {
      ue.printStackTrace();
    }
  }

}

Note the following in this JUnit test:

• testApiAllSpeakersJson():

  • Gets a list of all speakers from the Speakers API by calling Unirest.get() with http://localhost:5000/speakers

  • Verifies that the HTTP Status Code is OK (200).

  • Gets the JSON document (which contains an array of speaker Objects) from the HTTP Response Body.

  • Makes a series of assertions on the JSON document with JSONUnit’s assertThatJson() to verify that

    • We have an array of three speaker objects.

    • Each field (for example, fullName, tags, and registered) in each speaker object matches the expected values.

  • When you run gradle test, you should see the following as part of the output:

• testApiSpeaker3Json():

  • Gets speaker 3 from the Speakers API by calling Unirest.get() with http://localhost:5000/speakers/3

  • Verifies that the HTTP Response Code is OK (200)

  • Gets the JSON document (which contains a single speaker Object) from the HTTP Response Body.

  • Makes a series of assertions on the JSON document with JSONUnit’s assertThatJson() to verify that

    • We have a single speaker Object.

    • Each field in the speaker Object has the expected values.

  • When you run gradle test, you should see the following as part of the output:

This Unit Test only touches upon the basics of the Unirest Java library, which also provides the following:

• Full HTTP verb coverage (GET, POST, PUT, DELETE, PATCH)

• The ability to do custom mappings from an HTTP Response Body to a Java Object

• Asynchronous (i.e., nonblocking) requests

• Timeouts

• File uploads

• And much more

Visit the Unirest website for further information on the Unirest Java library.

Before moving on, you can stop json-server by pressing Ctrl-C at the command line.

We’ve shown how to deploy and interact with a Stub API, and now it’s time to build a small RESTful API.

Create the Model