Objects

One of the most important things that JSON web API clients need to deal with are the JSON objects that appear in responses. Most JSON web APIs expose a unique object model via the responses. Before you can even start using the API in any meaningful way, your client app needs to understand the object model.

Figure 2-1. Twitter’s JSON object documentation

For example, as of this writing, the Twitter API Overview page lists five baseline objects:

• Users

• Tweets

• Entities

• Entities in Objects

• Places

Many of these objects contain nested dictionary and array objects, too. And there are several complete sets of JSON objects in addition to these for related Twitter APIs such as streaming services, ad services, and others.

{
  "task": [
    {
      "id": "137h96l7mpv",
      "title": "LAX",
      "completeFlag": "true",
      "assignedUser": "bob",
      "dateCreated": "2016-01-14T17:48:42.083Z",
      "dateUpdated": "2016-01-27T22:03:02.850Z"
    },
    {
      "id": "1gg1v4x46cf",
      "title": "YVR",
      "completeFlag": "false",
      "assignedUser": "carol",
      "dateCreated": "2016-01-14T18:03:18.804Z",
      "dateUpdated": "2016-01-27T17:45:46.597Z"
    },
    .... more TASK objects here
   ]
 }

{
  "id": "137h96l7mpv",
  "title": "Review API Design",
  "completeFlag": "false",
  "assignedUser": "bob",
  "dateCreated": "2016-05-14T17:48:42.083Z",
  "dateUpdated": "2016-05-27T22:03:02.850Z"
},

span.prompt {
  display:block;
  width:125px;
  font-weight:bold;
  text-align:right;
  text-transform:capitalize; 
}

Addresses

Most JSON RPC-CRUD API responses don’t include URLs—the addresses of the objects and arrays the client application is processing. Instead, the URL information is written up in human-readable documentation and it is up to the developer to work out the details. Often this involves hardcoding URLs (or URL templates) into the app, associating those addresses with objects and collections at runtime, and resolving any parameters in the URL templates before actually using them in the app.

The number of URLs and templates in an API can be very large. For example, using the Twitter API (mentioned previously) as an example, there are close to 100 URL endpoints displayed on just one page of the Twitter API documentation (see Figure 2-2). While it is likely that a single API client will not need to handle all 97 of the URLs listed on that page, any useful Twitter client app will likely need to deal with dozens of them.

For the TPS web API, there are 17 URLs and templates to deal with. They’re listed in Table 1-2. We’ll need to sort out which addresses belong to each context object (Task or User) and which of them are more than simple read actions (e.g., HTTP POST, PUT, and DELETE actions).

There are many different ways of handling URLs and templates for JSON web APIs. The approach I’ll use in our sample app is to create a JSON dictionary of all the URLs for each object. For example, Example 2-2 shows how I’ll “memorize” some of the Task operations (Note that I included a prompt element for use when displaying these URLs as links in the client UI).

actions.task = {
  tasks:   {href:"/task/", prompt:"All Tasks"},
  active:  {href:"/task/?completeFlag=false", prompt:"Active Tasks"},
  closed:  {href:"/task/?completeFlag=true", prompt:"Completed Tasks"},
}

I’ll also need to keep some information on when to display links. Should they appear on every page? Just on the pages associated with tasks? Only when a single task is displayed? Various client-side JavaScript frameworks deal with these details in different ways. Since I’m not tying my client or server to any single framework, my solution is to use an additional property of my address list, called target, which is set to values such as "all" or "list" or "single-item", etc. We’ll see that later in this chapter.

Figure 2-2. Twitter’s JSON endpoint documentation

So, Objects and Addresses. That’s pretty good, but it’s not enough. We also need to know the details on Actions that involve query parameters and actions that require construction of request bodies (e.g., POST and PUT).

Actions

The third important element that web API clients need to deal with are filters and write operations of a web API. Just like the URL construction rules, this information is typically written up in human-readable documentation and then translated into client-side code by a developer. For example, Twitter’s API for updating existing lists takes up to seven parameters and looks like Figure 2-3 in the documentation.

The documentation for our TPS web API appears in Table 1-2. Similar to the Twitter API documentation, the TPS docs show the URL, the method, and the set of parameters. This all needs to be “baked” into the web API client app, too.

Figure 2-3. Twitter’s update list API documentation

There are many different ways to encode action information into client apps—from hardcoding it into the app, keeping it as metadata in a separate local file, or even as a remote configuration file sent with the JSON response. For our TPS client, I’ll use an approach similar to the one I used when handling simple URLs (see Example 2-2).

For example, Table 2-1 shows what the UserAdd action looks like in the TPS documentation.

And the web API client app will store that information in the actions element (see the following code). Note that the pattern property information comes from another part of the TPS documentation (provided earlier in Table 1-5).

actions.user = {
  add:  {
    href:"/user/",
    prompt:"Add User",
    method:"POST",
    args:{
      nick: {
        value:"",
        prompt:"Nickname",
        required:true,
        pattern:"[a-zA-Z0-9]+"
      },
      password: {
        value:"",
        prompt:"Password",
        required:true,
        pattern:"[a-zA-Z0-9!@#$%^&*-]+"
      },
      name: {
        value:"",
        prompt:"Full Name",
        required:true
      }
    }
  }
}

And this kind of information needs to be handled for all the actions your client app needs to perform. In the TPS web API, that is 17 actions. A nontrivial app will need to handle quite a few more.

Quick Summary

So now we have a good sense of what a web API client for a JSON RPC-CRUD style API will need to deal with. Along with the usual code for requests, parsing, and rendering responses, every JSON API client will also need to know how to handle:

• Objects unique to each API

• Addresses (URLs and URL templates) for all the actions of the API

• ACTION details including HTTP methods and arguments for all non-trivial actions including HTTP POST, PUT, and DELETE requests.

With this in mind, we can now dig into the actual code for the JSON web API client.

The HTML Container

The SPA client created for the TPS web API starts with a single HTML document. This document acts as the container for the entire API client application. Once the initial HTML is loaded, all other requests and responses will be handled by the running JavaScript code parsing and rendering the JSON objects returned from the TPS web API service.

The HTML container looks like this:

<!DOCTYPE html>
<html>
  <head>
    <title>JSON</title>
    <link href="json-client.css" rel="stylesheet" /> 
  </head>
  <body>
    <h1 id="title"></h1> 
    <div id="toplinks"></div>
    <div id="content"></div>
    <div id="actions"></div>
    <div id="form"></div>
    <div id="items"></div>
    <div>
      <pre id="dump"></pre>
    </div>
  </body>
  <script src="dom-help.js">//na </script> 
  <script src="json-client.js">//na </script> 
  <script>
    window.onload = function() {
      var pg = json();
      pg.init("/home/", "TPS - Task Processing System"); 
    }
  </script>
</html>

As you can see from the preceding HTML listing, there is not much to talk about in this document. The part that all the code will be paying attention to starts at —the seven DIV elements. Each of them has a unique identifier and purpose at runtime. You can figure most of that out just by reading the names. The last DIV actually encloses a <pre> tag that will hold the full “dump” of the JSON responses at runtime. This is a handy kind of debug display and isn’t needed for production.

Along with the HTML, there is a single CSS file () and two JavaScript references; a simple DOM library () and the complete client-side code (). We’ll inspect the json-client.js library throughout this section of the chapter.

Finally, once the page is loaded, a single function is executed (see ). This initializes the client with a starting URL and (optionally) a title string. The URL shown here works fine when the client app is hosted in the same web domain as the TPS web API. If you want to run the client app from a separate domain, all you need to do is update this initial URL and the app will work just fine.

So, let’s look inside the json-client.js library and see how it works.

The Top-Level Parse Loop

The client app is designed to act in a simple, repeating loop that looks like this:

1. Execute an HTTP request.

2. Store the JSON response in memory.

3. Inspect the response for context.

4. Walk through the response and render the context-related information on screen.

We talked about context earlier in the chapter. This client is expecting multiple custom object models from the TPS web API (Task, User, and Home) and uses the returned object model as context in deciding how to parse and render the response.

// init library and start 
function init(url, title) {
  if(!url || url==='') {
    alert('*** ERROR:\n\nMUST pass starting URL to the library');
  }
  else {
    global.title = title||"JSON Client";
    global.url = url;
    req(global.url,"get"); 
  }
}

// process loop 
function parseMsg() {
  setContext();
  toplinks();
  content();
  items();
  actions();
}

When the app first loads, the init function is called (). That validates the initial URL, stores it, and eventually makes the first HTTP request to that URL (). Once the response returns (not shown here) the parseMsg function is called () and that starts the parse/render loop.

The parseMsg function does a handful of things. First, it calls setContext to inspect the response and set the app’s current context so that it knows how to interpret the response. For our app, a global context variable is set to "task", "user", or "home". Next, the page’s top links are located and rendered (toplinks), and any HTML content is displayed (content). The items function finds all the objects in the response (Task or User) and renders them on the screen, and the actions function constructs all the links and forms appropriate for the current context.

That’s quite a bit in a single function and we’ll get into some details of that in just a bit. But first, let’s look at how the JSON client keeps track of the TPS objects, addresses, and actions that were written up in the human documentation.

Objects, Addresses, and Actions

Since the TPS web API is just returning custom JSON objects and arrays, our client app needs to know what those objects are, how to address them, and what actions are possible with them.

TPS objects

The TPS objects (Task and User) are simple name–value pairs. So, all our client app needs to know are the properties of each object that need to be rendered on screen. For example, all TPS objects have dateCreated and dateUpdated properties, but our client doesn’t need to deal with them.

This app uses a simple array to contain all the object properties it needs to know about:

global.fields.task = ["id","title","completeFlag","assignedUser"];
global.fields.user = ["nick","name","password"];

Now, whenever parsing incoming objects, the client app will compare the property on the object with its own list of properties and ignore any incoming property it doesn’t already know about.

An example of how this works can be seen in the code that handles on-screen rendering of objects in the response:

// g = global storage
// d = domHelper library

// handle item collection
function items() {
  var rsp, flds, elm, coll;
  var ul, li, dl, dt, dd, p;

  rsp = g.rsp[g.context]; 
  flds = g.fields[g.context];

  elm = d.find("items");
  d.clear(elm);
  ul = d.node("ul");

  coll = rsp;
  for(var item of coll) {
    li = d.node("li");
    dl = d.node("dl");
    dt = d.node("dt");

    // emit the data elements
    dd = d.node("dd");
    for(var f of flds) { 
      p = d.data({text:f, value:item[f]});
      d.push(p,dd);
    }
    d.push(dt, dl, dd, li, lu);
  }
  d.push(ul,elm);
}

Note (in ) the first step in the routine is to use the shared context value to locate the data in the response (rsp) and select the internal properties to use when inspecting the data (flds). This information is used (in ) to make sure to only render the fields the client decides is appropriate (see Figure 2-4).

Figure 2-4. Rendering task items in the JSON client

Warning

The code examples for this book use the ES6 for..of iterator. When this book first went to press for..of was supported in some browsers, but not all. I used the Chrome browser (both the Google release and the Chromium open source release) while writing the examples and they all ran fine. Be sure to check your browser’s support for the for..of iterator.

Addresses and actions

This client app stores both the addresses (URLs) and actions (HTTP method and parameter information) in a single internal collection called actions. There is additional metadata about each action that indicates when it should be rendered (based on context information) and how it should be executed (e.g., as a simple link, form, or direct HTTP method call).

The list of actions for the TPS web API is rather long (17 separate actions), but the following code snippet gives you a good idea of how they are stored in the client app:

// task context actions
global.actions.task = {
  tasks:   {target:"app", func:httpGet, href:"/task/", prompt:"Tasks"}, 
  active:  {target:"list", func:httpGet, href:"/task/?completeFlag=false",
             prompt:"Active Tasks"
           },
  byTitle: {target:"list", func:jsonForm, href:"/task", 
             prompt:"By Title", method:"GET",
             args:{
               title: {value:"", prompt:"Title", required:true}
             }
           },
  add:     {target:"list", func:jsonForm, href:"/task/", 
             prompt:"Add Task", method:"POST",
             args:{
               title: {value:"", prompt:"Title", required:true},
               completeFlag: {value:"", prompt:"completeFlag"}
             }
           },
  item:    {target:"item", func:httpGet, href:"/task/{id}", prompt:"Item"},
  edit:    {target:"single", func:jsonForm, href:"/task/{id}", 
             prompt:"Edit", method:"PUT",
             args:{
               id: {value:"{id}", prompt:"Id", readOnly:true},
               title: {value:"{title}", prompt:"Title", required:true},
               completeFlag: {value:"{completeFlag}", prompt:"completeFlag"}
             }
           },
  del:     {target:"single", func:httpDelete, href:"/task/{id}", 
             prompt:"Delete", method:"DELETE", args:{}
           },
};

In the preceding code snippet, you can see a simple, safe, read-only action () as well as a safe action that requires user input (). There are also the classic CRUD actions (, , and ) with the expected HTTP method names, prompts, and (where appropriate) argument lists. These action definitions are selected based on runtime context information. For example, the target property indicates which actions are appropriate for app-level, list-level, item-level, and even single-item level context.

Here’s an example of the code that uses context information and scans the list of actions to render list-level links:

 // d = domHelper library

 // handle list-level actions
function actions() {
  var actions;
  var elm, coll;
  var ul, li, a;

  elm = d.find("actions");
  d.clear(elm);
  ul = d.node("ul");

  actions = g.actions[g.context]; 
  for(var act in actions) {
    link = actions[act];
    if(link.target==="list") { 
      li = d.node("li");
      a = d.anchor({
        href:link.href,
        rel:"collection",
        className:"action",
        text:link.prompt
      });
      a.onclick = link.func;
      a.setAttribute("method",(link.method||"GET"));
      a.setAttribute("args",(link.args?JSON.stringify(link.args):"{}"));
      d.push(a,li);
      d.push(li, ul);
    }
  }
  d.push(ul, elm);
}

You can see in this code that both the object context () and the internal render context () are used to select only the links appropriate for display at the moment.

The actions that contain argument details will be rendered at runtime using HTML <form> elements (see Figure 2-5). The code that handles this is shown here:

// d = domHelper library

// render inputs
coll = JSON.parse(link.getAttribute("args")); 
for(var prop in coll) {
  val = coll[prop].value;
  if(rsp[0][prop]) {
    val = val.replace("{"+prop+"}",rsp[0][prop]); 
  }
  p = d.input({ 
    prompt:coll[prop].prompt,
    name:prop,
    value:val,
    required:coll[prop].required,
    readOnly:coll[prop].readOnly,
    pattern:coll[prop].pattern
  });
  d.push(p,fs);
}

In the small snippet just shown, you can see the collection of args () from the action definition is used to create HTML form inputs. At you can see that any current object is used to populate the value of the inputs before the actual HTML input element is created (). Note the inclusion of the HTML5 properties required, readonly, and pattern in order to improve the client-side user experience, too.

Figure 2-5. JSON client rendering a form

Adding a Field and Filter

A common change that can occur in a production API is adding a new field to the data storage. For example, the team at BigCo working on the web API might decide to add a tag field to the Task storage object. This will allow users to tag tasks with common keywords and then recall all the tasks with the same keyword.

Adding the tag field means we’ll probably need a new search option, too: TaskFilterByTag. It would take a single parameter (a string) and use that to search all the task record’s tag fields, returning all Task objects where the search value is contained in the tag field.

  // task-component.js
  // valid fields for this record
  props = [
    "id",
    "title",
    "tag", 
    "completeFlag",
    "assignedUser",
    "dateCreated",
    "dateUpdated"
  ];

function addTask(elm, task, props) {
  var rtn, item;

  item = {}
  item.tags = (task.tags||""); 
  item.title = (task.title||"");
  item.assignedUser = (task.assignedUser||"");
  item.completeFlag = (task.completeFlag||"false");
  if(item.completeFlag!=="false" && item.completeFlag!=="true") {
    item.completeFlag="false";
  }
  if(item.title === "") {
    rtn = utils.exception("Missing Title");
  }
  else {
    storage(elm, 'add', utils.setProps(item, props));
  }

  return rtn;
}

curl -X POST -H "content-type:application/json" -d
  '{"title":"Run remote client tests","tags":"test"}'
  http://localhost:8181/task/

{
  "id": "1sog9t9g1ob",
  "title": "Run server-side tests",
  "tags": "test",
  "completeFlag": "false",
  "assignedUser": "",
  "dateCreated": "2016-01-28T07:16:53.044Z",
  "dateUpdated": "2016-01-28T07:16:53.044Z"
}

curl http://localhost:8181/task/?tags=test

{
  "task": [
    {
      "id": "1m80s2qgsv5",
      "title": "Run client-side tests",
      "tags": "test client",
      "completeFlag": "false",
      "assignedUser": "",
      "dateCreated": "2016-01-28T07:14:07.775Z",
      "dateUpdated": "2016-01-28T07:14:07.775Z"
    },
    {
      "id": "1sog9t9g1ob",
      "title": "Run server-side tests",
      "tags": "test",
      "completeFlag": "false",
      "assignedUser": "",
      "dateCreated": "2016-01-28T07:16:53.044Z",
      "dateUpdated": "2016-01-28T07:16:53.044Z"
    },
    {
      "id": "242hnkcko0f",
      "title": "Run remote client tests",
      "tags": "test remote",
      "completeFlag": "false",
      "assignedUser": "",
      "dateCreated": "2016-01-28T07:19:47.916Z",
      "dateUpdated": "2016-01-28T07:19:47.916Z"
    }
  ]
}

Testing the JSON API client

The easiest way to test the JSON API client’s support for the new tag field and filter option is to simply run the existing client and check the results. Figure 2-6 shows a screenshot from the JSON API client making a request to the new TPS web API server.

Figure 2-6. JSON API client without tag support

As you can see from the screenshot, even though the new task records appear in the client, the tag field is missing from the display as well as the new filter option. The good news is our JSON client didn’t crash when the new feature was added. The bad news is our client simply ignored the new functionality.

Tip

The source code for the updated (v2) TPS JSON Web Client can be found in the associated GitHub repo. A running version of the service can be found online.

The only way our JSON client will be able to take advantage of this new option is to recode and redeploy a new version of the app into production.

The New Crop of Hypermedia Formats

Starting in the early 2010s, a new crop of text-based formats emerged that offered more than just structure for describing data; they included instructions on how to manipulate the data as well. These are formats I refer to as hypermedia formats. These formats represent another trend in APIs and, as we will see later in the book, can be a valuable tool in creating API-based services that support a wide range of service changes without breaking existing client applications. In some cases, they even allow client applications to “auto-magically” acquire new features and behaviors without the need for rewriting and redeploying client-side code.

The Fallacy of The Right One

So, despite all the new hypermedia formats out there (and the continued use of XML in enterprises), with the current trend pointing toward JSON as the common output format, it would seem an easy decision, right? Any time you start implementing an API, just use JSON and you’re done. Unfortunately, that’s almost never the way it goes.

First, some industry verticals still rely on XML and SOAP-based formats. If you want to interact with them, you’ll need to support SOAP or some other custom XML-based formats. Examples might be partner APIs that you work with on a regular basis, government or other standards-led efforts that continue to focus on XML as their preferred format, and even third-party APIs that you use to solve important business goals.

Second, many companies invested heavily in XML-based APIs over the last decade and are unwilling to rewrite these APIs just to change the output format. Unless there is a clear advantage to changing the messages format (e.g., increased speed, new functionality, or some other business metric), these XML-based services are not likely to change any time soon.

Finally, some data storage systems are XML-native or default to outputting data as XML documents (e.g., dbXML, MarkLogic, etc). While some of these services may offer an option to output the data in a JSON-based format, many continue to focus on XML and the only clear way of converting this data to JSON is to move it to JSON-native data storage systems like MongoDB, CouchDB, and others.

So, deciding on a single format for your team’s service may not be feasible. And, as your point of view widens from a single team to multiple teams within your company, multiple products within an enterprise, on up to the entire WWW itself, getting everyone to agree to both produce and consume a single output format is not a reasonable goal.

As frustrating as this may be for team leaders and enterprise-level software architects, there is no single “Right One” when it comes to message formats. It may be possible to control a single team’s decision (either through consensus or fiat) but one’s ability to exert this control wanes as the scope of the community grows.

And that means the way forward is to rethink the problem, not work harder at implementing the same solution.

Reframing the Problem

One way to face a challenge that seems insurmountable is to apply the technique of reframing—to put the problem in a different light or from a new point of view. Instead of working harder to come up with a solution to the perceived problem, reframing encourages us to step outside the frame and change our perspective. Sometimes this allows us to recognize the scenario as a completely different problem—one that may have an easier or simpler solution.

In our case (the challenge of selecting a single format for your APIs), it can help to ask “Why do we need to decide on a single format?” or, to put it another way, “Why not support many formats for a single API?” Asking these questions gives us a chance to lay out some of the reasons for and against supporting multiple formats. In this way, we’ve side-stepped the challenge of picking one format. Now we’re focused on a new aspect of the same problem. Why not support multiple formats?

Separating Format from Functionality

All too often, I see service implementations that are bound too tightly to a single format. This is a common problem for SOAP implementations—usually because the developer tooling leads programmers into relying on a tight binding between the internal object model and the external output format. It is important to treat all formats (including SOAP XML output) as independent of the internal object model. This allows some changes in the internal model to happen without requiring changes to the external output format.

To manage this separation, we’ll need to employ some modularity to keep the work of converting the domain model into a message external from the work of manipulating the domain model itself. This is using modularity to split up the assignment of work. Typically modularity is used to collect related functionality in a single place (e.g., all the functionality related to users or customers or shoppingCarts). The notion of using modularity as primarily a work assignment tactic comes from David Parnas’s 1972 paper “On the Criteria to be Used in Decomposing Systems into Modules.” As Parnas states:

[M]odule is considered to be a responsibility assignment rather than a subprogram. The modularizations include the design decisions which must be made before the work on independent modules can begin. [Emphasis in the original]

Viewing the work of converting internal domain data into external output formats as a responsibility assignment leads us to isolate the conversion process into its own module. Now we can manage that module separately from the one(s) that manipulate domain data. A simple example of this clear separation might look like this:

var convert = new ConversionModule(HALConverter);
var output = convert.toMessage(domainData);

In that imaginary pseudo-code, the conversion process is accessed via an instance of the conversionModule() that accepts a message-specific converter (in this case, HALConverter) and uses the toMessage function that accepts a domainData instance to produce the desired output. This is all quite vague right now, but at least we have a clear target for implementing safe, cheap, easy support for multiple output formats.

Once the functionality of the internal domain model is cleanly separated from the external format, we need some guidance on how to consistently convert the domain model into the desired output format. But before that, we’ll need a pattern for selecting which format is appropriate.

The Selection Algorithm

An important implementation detail when supporting multiple output formats for an API is that of the output selection process. There needs to be some consistent algorithmic way to select the correct output format at runtime. The good news is that HTTP—still the most common application-level protocol for web APIs—has this algorithm already defined: content negotiation.

Section 3.4 of the HTTP specification (RFC7231) describes two patterns of content negotiation for “representing information”:

Proactive

The server selects the representation based on the client’s preferences.

Reactive

The server provides a list of possible representations to the client and the client selects the preferred format.

The most common pattern in use on the Web today is the proactive one and that’s what we’ll implement in our representor. Specifically, clients will send an Accept HTTP header that contains a list of one or more format preferences, and the server will use that list to determine which format will be used in the response (including the selected format identifier in the server’s Content-Type HTTP header).

A typical client request might be:

GET /users HTTP/1.1
Accept: application/vnd.hal+json, application/vnd.uber+json
...

And, for a service that supports HAL but does not support UBER, the response would be:

HTTP/1.1 200 OK
Content-Type: application/vnd.hal+json
...

GET /users/ HTTP/1.1
application/vnd.uber+json;q=0.3,
application/vnd.hal+json;q=1

So, that’s what it looks like on the “outside”—the actual HTTP conversation. But what pattern is used internally to make this work on the server side? Thankfully, a solution for this kind of selection process was worked out in the 1990s.

Adapting and Translating

Many of the challenges of writing solid internal code can be summed up in a common pattern. And one of the most important books on code patterns is the 1994 book Design Patterns by Gamma, Helm, Johnson, and Vlissides (Addison-Wesley Professional). Those are rather tough names to remember and, over time, this group of authors has come to be known as the Gang of Four (GoF). You’ll sometimes even hear people refer to the Gang of Four book when discussing this important text.

There are about twenty patterns in the GoF book, categorized into three types:

• Creational patterns

• Structural patterns

• Behavioral patterns

The pattern that will help us in our quest to implement support for multiple output formats for our API that is safe, cheap, and easy is the Adapter structural pattern.

The Adapter pattern

As established on OODesign.com, the intent of the Adapter pattern is to:

Convert the interface of a class into another interface clients expect. Adapter lets classes work together that couldn’t otherwise because of incompatible interfaces.

And that’s essentially what we need to do—convert an internal class (or model) into an external class (or message).

There are four participants to the Adapter pattern (Figure 3-1):

Target

Defines the domain-specific interface that client uses. This will be the message model or media type we want to use for output.

Client

Collaborates with objects conforming to the target interface. In our case, this is our API service—the app that is using the Adapter pattern.

Adaptee

Defines an existing interface that needs adapting. This is the internal model that needs to be converted into the target message model.

Adapter

Adapts the interface of adaptee to the target interface. This will be the specific media-type plug-in that we write to handle the work of converting the internal model to the message model.

So, we need to write an adapter plug-in for each target media type (HTML, HAL, Siren, Collection+JSON, etc.). That’s not too tough. But the challenge is that each internal object model (the adaptee) is going to be different. For example, writing a plug-in to handle Users then writing one to handle Tasks, and on and on. That can mean lots of code is needed to write the adapters and—even more disappointing—these adapters may not be very reusable.

To try to reduce the need for tightly coupled adapter code, I’m going to introduce another pattern—one based on the Adapter pattern: the Message Translator.

Figure 3-1. The Adapter pattern

That means we need to spend a few minutes on what the Message Translator pattern looks like and how it can be used to standardize the process of converting internal object models into external message models.

The Message Translator pattern

To cut down on lots of custom adapters, I’m going to introduce another pattern—derived from the Adapter pattern—called the Message Translator. This comes from Gregor Hohpe and his book Enterprise Integration Patterns (Addison-Wesley Professional).

Hohpe describes the Message Translator as:

Use a special filter, a Message Translator, between other filters or applications to translate one data format into another.

A message translator is a special form of the adapter class in the GoF set of patterns.

To make this all work, I’ll introduce a general message format—the Web Service Transition Language (WeSTL)—in the next section of this chapter. That will act as a standardized adaptee and make it possible to generalize the way adapter plug-ins can be coded. Now, the process of translating can be turned into an algorithm that doesn’t need to rely on any domain-specific information. As illustrated in Figure 3-2, we can write WeSTL-to-HAL or WeSTL-to-Cj or WeSTL-to-Siren translators and then the only work is to convert the internal model into a WeSTL message. This moves the complexity to a new location, but does so in a way that reduces the amount of custom code needed to support multiple formats.

So, armed with this background, we can now look at a set of concrete implementation details to make it all happen.

Figure 3-2. The Message Translator pattern

Handling the HTTP Accept Header

The first two items on that list are rather trivial to implement in any WWW-aware codebase. For example, identifying acceptable output formats for a request means reading the Accept HTTP header. Here is a snippet of NodeJS code that does that:

// rudimentary accept-header handling
var contentType = '';
var htmlType = 'text/html';
var contentAccept = req.headers['accept'];
if(!contentAccept || contentAccept==='*/*') {
  contentType = htmlType;
}
else {
  contentType = contentAccept.split(',')[0];
}

Note that the preceding code example makes two key assumptions:

1. If no Accept header is passed or the Accept header is set to “anything”, the Accept header will be set to text/html.

2. If the Accept header lists more than one acceptable format, this service will just grab the first one listed.

This implementation is very limited. It does not support the use of q values to help the server better understand client preferences, and this service defaults to the text/html type for API responses. Both of these assumptions can be altered and/or improved through additional coding, but I’ve skipped over that for this book.

Implementing the Message Translator Pattern

Now that we have the requested format value—the output context for this request—we can move on to the next step: implementing the Message Translator pattern in NodeJS. For this book, I’ve created a simple module that uses a switch … case element that matches the request context string (the accepted format) with the appropriate translator implementation.

The code looks like this:

// load representors 
var html = require('./representors/html.js');
var haljson = require('./representors/haljson.js');
var collectionJson = require('./representors/cj.js');
var siren = require('./representors/siren.js');

function processData(domainData, mimeType) {
  var doc;

  // clueless? assume HTML 
  if (!mimeType) {
    mimeType = "text/html";
  }

  // dispatch to requested representor 
  switch (mimeType.toLowerCase()) {
    case "application/vnd.hal+json":
      doc = haljson(object);
      break;
    case "application/vnd.collection+json":
      doc = collectionJson(object);
      break;
    case "application/vnd.siren+json":
      doc = siren(object);
      break;
    case "text/html":
    default: 
      doc = html(object);
      break;
  }
  return doc;
}

In the preceding code snippet, you can see that a set of representors are loaded at the top (see ). The code in these modules will be covered in “Runtime WeSTL”). Next (), if the mimeType value is not passed (or is invalid) it is automatically set to text/html. This is a bit of defensive coding. And then (at ) the switch … case block checks the incoming mimeType string with known (and supported) mime type strings in order to select the appropriate format processing module. Finally, in case an unknown/unsupported format is passed in, the default statement () makes sure that the service runs the html() module to produce valid output.

We now have the basics of the representor outlined. The next step is to actually implement each format-specific translator (HTML, HAL, etc.). To solve this challenge, we need to take a side road on our journey that establishes a general format understood by all translators—the WeSTL format.

General Representor Modules

In the Message Translator pattern, each format module (html(), haljson(), etc.) is an instance of a translator. While implementing these modules as domain-specific converters (e.g., userObjectToHTML, userObjectToHAL, etc.) would meet the needs of our implementation, that approach will not scale over time. Instead, what we need is a general-purpose approach to a translator that will not have any domain-specific knowledge. For example, the translator module used to handle user domain data will be the same translator module used to handle customer and accounting or any other domain-specific domain data.

To do that, we’ll need to create a common interface for passing domain data into format modules that is independent of any single domain model.

The WeSTL Format

For this book, I’ve worked up a common interface in the form of a standardized object model, one that service developers can quickly load with domain data and pass to format modules. I also took the opportunity to reframe the challenge of defining interfaces for web APIs. Instead of focusing on defining resources, I chose to focus on defining state transitions. For this reason, I’ve named this interface design the Web Service Transition Language, or WeSTL (pronounced wehs’-tul).

Basically, WeSTL allows API service developers to use a standardized message model for converting internal object models into external message formats. This reduces the cost (in time and effort) to craft new Translators and pushes the complexity of converting internal models into messages to a single instance—converting from the internal model to WeSTL.

Figure 3-3 provides a visual representation of the request/response life cycle of services that use the WeSTL format.

Figure 3-3. The WeSTL transformation cycle

When designing and implementing web APIs with WeSTL, the service developer collects all the possible state transitions and describes them in the WeSTL model. By state transitions, I mean all the links and forms that could appear within any service response. For example, every response might have link to the home page. Some responses will have HTML-style input forms allowing API clients to create new service data or edit existing data. There may even be service responses that list all the possible links and forms (state transitions) for the service!

A simple example of how WeSTL can be used to describe transitions is shown here:

{
  "wstl" : {
    "transitions" : [
      {
        "name" : "home", 
        "type" : "safe",
        "action" : "read",
        "prompt" : "Home Page",
      },
      {
        "name" : "user-list", 
        "type" : "safe",
        "target" : "user list"
        "prompt" : "List of Users"
      }
      {
        "name" : "change-password", 
        "type" : "unsafe",
        "action" : "append"
        "inputs" : [ 
          {
            "name" : "userName",
            "prompt" : "User",
            "readOnly" : true
          },
          {
            "name" : "old-password",
            "prompt" : "Current Password",
            "required" : true,
          },
          {
            "name" : "old-password",
            "prompt" : "New Password (5-10 chars)",
            "required" : true,
            "pattern" : "^[a-zA-Z0-9]{5,10}$"
          }
        ]
      }
    ]
  }
}

As you can see from this WeSTL document, it contains three transition descriptions named home (), user-list (), and change-password (). The first two transitions are marked safe. That means they don’t write any data, only execute reads (e.g., HTTP GET). The third one, however (change-password) is marked unsafe since it writes data to the service (à la HTTP POST). You can also see several input elements described for the change-password transition (). These details will be used when creating an API resource for the User Manager service.

There are a number of details left out in this simple example, but you can see how WeSTL works; it describes the transitions that can be used within the service. What’s important to note is that this document does not define web resources or constrain where (or even when) these transitions will appear. That work is handled by service developers elsewhere in the code.

So, this is what a WeSTL model looks like at design time, before the service is up and running. Typically, a service designer uses WeSTL in this mode. There is also another mode for WeSTL documents: runtime. That mode is typically used when implementing the service.

var transitions = require('./wstl-designtime.js');
var domainData = require('./domain.js');

function userResource(root) {
  var doc, coll, data;

  data = [];
  coll = [];

  // pull data for this resource 
  data = domain.getData('user',root.getID());

  // add transitions for this resource 
  tran = transitions("home");
  tran.href = root +"/home/";
  tran.rel = ["http:"+root+"/rels/home"];
  coll.splice(coll.length, 0, tran);

  tran = transitions("user-list");
  tran.href = root +"/user/";
  tran.rel = ["http:"+root+"/rels/collection"];
  coll.splice(coll.length, 0, tran);

  tran = transitions("change-password");
  tran.href = root +"/user/changepw/{id}";
  tran.rel = ["http:"+root+"/rels/changepw"];
  coll.splice(coll.length, 0, tran);

  // compose wstl model 
  doc = {};
  doc.wstl = {};
  doc.wstl.title = "User Management";
  doc.wstl.transitions = coll;
  doc.wstl.data =  data;

  return doc;
}

var transitions = require('./wstl-designtime.js');
var domainData = require('./domain.js');

function userResource(root) {
  var doc, coll, data;

  data = [];
  coll = [];

  // pull data for this resource 
  data = domain.getData('user',root.getID());

  // add transitions for this resource 
  tran = transitions("home");
  tran.href = root +"/home/";
  tran.rel = ["http:"+root+"/rels/home"];
  coll.splice(coll.length, 0, tran);

  tran = transitions("user-list");
  tran.href = root +"/user/";
  tran.rel = ["http:"+root+"/rels/collection"];
  coll.splice(coll.length, 0, tran);

  tran = transitions("change-password");
  tran.href = root +"/user/changepw/{id}";
  tran.rel = ["http:"+root+"/rels/changepw"];
  coll.splice(coll.length, 0, tran);

  // compose wstl model 
  doc = {};
  doc.wstl = {};
  doc.wstl.title = "User Management";
  doc.wstl.transitions = coll;
  doc.wstl.data =  data;

  return doc;
}

A Sample Representor

Now that resources are represented using a generic interface using WeSTL, we can build a general format module that converts the standardized WeSTL model into an output format. Basically, the code accepts a runtime WeSTL document and then translates domain data, element by element, into the target message format for an API response.

To see how this might look, here is a high-level look at a simplified implementation of a general HAL representor.

function haljson(wstl, root, rels) { 
  var hal;

  hal = {};
  hal._links = {};

  for(var segment in wstl) {
    hal._links = getLinks(wstl[segment], root, segment, rels);
    if(wstl[segment].data && wstl[segment].data.length===1) {
      hal = getProperties(hal, wstl[segment]);
    }
  }
  return JSON.stringify(hal, null, 2); 
}

// emit _links object 
function getLinks(wstl, root, segment, relRoot) {
  var coll, items, links, i, x;

  links = {};

  // list-level actions
  if(wstl.actions) {
    coll = wstl.transitions;
    for(i=0,x=coll.length;i<x;i++) {
      links = getLink(links, coll[i], relRoot);
    }

    // list-level objects
    if(wstl.data) {
      coll = wstl.data;
      items = [];
      for(i=0,x=coll.length;i<x;i++) {
        item = {};
        item.href = coll[i].meta.href;
        item.title = coll[i].title;
        items.push(item);
      }
      links[checkRel(segment, relRoot)] = items;
    }
  }
  return links;
}

// emit root properties 
function getProperties(hal, wstl) {
  var props;

  if(wstl.data && wstl.data[0]) {
    props = wstl.data[0];
    for(var p in props) {
      if(p!=='meta') {
        hal[p] = props[p];
      }
    }
  }
  return hal;
}

/* additional support functions appear here */

While this code example is just a high-level view, you should be able to figure out the important details. The first argument of the top level function (haljson()) accepts a WeSTL model along with some runtime request-level data (). That function “walks” the WeSTL runtime instance and (a) processes any links (transitions) in the model () and then (b) deals with any name–value pairs in the WesTL instance (). Once all the processing is done, the resulting JSON object (now a valid HAL document) is returned to the caller (). An example of what this code might produce follows:

{
  "_links" : { 
    "self" : {
      "href": "http://localhost:8282/user/mamund"
    },
    "http://localhost:8282/rels/home": {
      "href": "http://localhost:8282/",
      "title": "Home",
      "templated": false
    },
    "http://localhost:8282/rels/collection": {
      "href": "http://localhost:8282/user/",
      "title": "All Users",
      "templated": false
    },
    "http://localhost:8282/rels/changepw": {
      "href": "http://localhost:8282/user/changepw/mamund",
      "title": "Change Password"
    }
  },
  "userName": "mamund", 
  "familyName": "Amundsen",
  "givenName": "Mike",
  "password": "p@ss",
  "webUrl": "http://amundsen.com/blog/",
  "dateCreated": "2015-01-06T01:38:55.147Z",
  "dateUpdated": "2015-01-25T02:28:12.402Z"
}

Now you can see how the WeSTL document has led from design-time mode to runtime instance and finally (via the HAL translator module) to the actual HAL document. The WeSTL transitions appear in the HAL _links section () and the related data for this user appears as name–value pairs called properties in HAL documents (starting at ).

Of course, HAL is just one possible translator implementation. Throughout this book, you’ll find general message translators for a handful of formats. Hopefully, this short overview will give enough guidance to anyone who wishes to implement their own (possibly better) general representors for HAL and many other registered formats.

Image credits

• Diogo Lucas, Figures 3-1 and 3-2

Links

By far, the most important element in the HAL format is the way it standardizes Links in JSON. Here’s what a link looks like in a HAL document:

"_links": {
  "self": { "href": "/orders" },
  "next": { "href": "/orders?page=2" },
  "find": { "href": "/orders{?id}", "templated": true } 
},

Note that each link object has an identifier (self, next, find). This is required for HAL documents. It is the identifier that the client application will store in code, not the actual URL value. The URLs appear in the href property and these URLs may actually be URL templates (see ). HAL leverages the URI Template specification (RFC6570) and the templated:"true" property for these.

In HAL, all link objects appear as part of a "_links" collection. These collections can appear in several places within a single HAL document.

Objects and Properties

HAL also supports passing plain JSON objects and name–value pairs. These typically appear as a collection of properties at the root of a HAL document. For example, the Amazon API Gateway service emits HAL responses. Following are a couple of examples of the 26 resource models the AWS Gateway API produces:

{
  "domainName" : "String",
  "certificateName" : "String",
  "certificateUploadDate" : "Timestamp",
  "distributionDomainName" : "String"
}