Table of Contents for
Node.js 8 the Right Way

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Node.js 8 the Right Way by Jim Wilson Published by Pragmatic Bookshelf, 2018
  1. Title Page
  2. Node.js 8 the Right Way
  3. Node.js 8 the Right Way
  4. Node.js 8 the Right Way
  5. Node.js 8 the Right Way
  6.  Acknowledgments
  7.  Preface
  8. Why Node.js the Right Way?
  9. What’s in This Book
  10. What This Book Is Not
  11. Code Examples and Conventions
  12. Online Resources
  13. Part I. Getting Up to Speed on Node.js 8
  14. 1. Getting Started
  15. Thinking Beyond the web
  16. Node.js’s Niche
  17. How Node.js Applications Work
  18. Aspects of Node.js Development
  19. Installing Node.js
  20. 2. Wrangling the File System
  21. Programming for the Node.js Event Loop
  22. Spawning a Child Process
  23. Capturing Data from an EventEmitter
  24. Reading and Writing Files Asynchronously
  25. The Two Phases of a Node.js Program
  26. Wrapping Up
  27. 3. Networking with Sockets
  28. Listening for Socket Connections
  29. Implementing a Messaging Protocol
  30. Creating Socket Client Connections
  31. Testing Network Application Functionality
  32. Extending Core Classes in Custom Modules
  33. Developing Unit Tests with Mocha
  34. Wrapping Up
  35. 4. Connecting Robust Microservices
  36. Installing ØMQ
  37. Publishing and Subscribing to Messages
  38. Responding to Requests
  39. Routing and Dealing Messages
  40. Clustering Node.js Processes
  41. Pushing and Pulling Messages
  42. Wrapping Up
  43. Node.js 8 the Right Way
  44. Part II. Working with Data
  45. 5. Transforming Data and Testing Continuously
  46. Procuring External Data
  47. Behavior-Driven Development with Mocha and Chai
  48. Extracting Data from XML with Cheerio
  49. Processing Data Files Sequentially
  50. Debugging Tests with Chrome DevTools
  51. Wrapping Up
  52. 6. Commanding Databases
  53. Introducing Elasticsearch
  54. Creating a Command-Line Program in Node.js with Commander
  55. Using request to Fetch JSON over HTTP
  56. Shaping JSON with jq
  57. Inserting Elasticsearch Documents in Bulk
  58. Implementing an Elasticsearch Query Command
  59. Wrapping Up
  60. Node.js 8 the Right Way
  61. Part III. Creating an Application from the Ground Up
  62. 7. Developing RESTful Web Services
  63. Advantages of Express
  64. Serving APIs with Express
  65. Writing Modular Express Services
  66. Keeping Services Running with nodemon
  67. Adding Search APIs
  68. Simplifying Code Flows with Promises
  69. Manipulating Documents RESTfully
  70. Emulating Synchronous Style with async and await
  71. Providing an Async Handler Function to Express
  72. Wrapping Up
  73. 8. Creating a Beautiful User Experience
  74. Getting Started with webpack
  75. Generating Your First webpack Bundle
  76. Sprucing Up Your UI with Bootstrap
  77. Bringing in Bootstrap JavaScript and jQuery
  78. Transpiling with TypeScript
  79. Templating HTML with Handlebars
  80. Implementing hashChange Navigation
  81. Listing Objects in a View
  82. Saving Data with a Form
  83. Wrapping Up
  84. 9. Fortifying Your Application
  85. Setting Up the Initial Project
  86. Managing User Sessions in Express
  87. Adding Authentication UI Elements
  88. Setting Up Passport
  89. Authenticating with Facebook, Twitter, and Google
  90. Composing an Express Router
  91. Bringing in the Book Bundle UI
  92. Serving in Production
  93. Wrapping Up
  94. Node.js 8 the Right Way
  95. 10. BONUS: Developing Flows with Node-RED
  96. Setting Up Node-RED
  97. Securing Node-RED
  98. Developing a Node-RED Flow
  99. Creating HTTP APIs with Node-RED
  100. Handling Errors in Node-RED Flows
  101. Wrapping Up
  102. A1. Setting Up Angular
  103. A2. Setting Up React
  104. Node.js 8 the Right Way

Providing an Async Handler Function to Express

Open your lib/bundle.js and add the following inside the module.exports function, after the bundle-creation API you added previously.

 /**
  * Retrieve a given bundle.
  * curl http://<host>:<port>/api/bundle/<id>
  */
 app.​get​(​'/api/bundle/:id'​, ​async​ (req, res) => {
 const​ options = {
  url: ​`​${url}​/​${req.params.id}​`​,
  json: ​true​,
  };
 try​ {
 const​ esResBody = ​await​ rp(options);
  res.status(200).json(esResBody);
  } ​catch​ (esResErr) {
  res.status(esResErr.statusCode || 502).json(esResErr.error);
  }
 });

This code block sets up a handler for the /bundle/:id route, which will allow us to retrieve a book bundle by its ID. Note the use of async before the function parameters in the opening line to indicate that we’re using an async function. Inside the async function, like with other route handlers, the code proceeds in two parts: the setup of options and the request to Elasticsearch.

After setting up the options, we use a try/catch block to handle the success and failure modes of the Elasticsearch request. We issue the Elasticsearch request itself with the expression await rp(options). This causes the async function to suspend while waiting for the Promise to be settled.

If the Promise is resolved, then the return value of the await expression will be the Elasticsearch response body. In this case, we send it onward with a 200 OK status via the Express response object, res.

If the Promise is rejected, then the await expression throws the rejection value as an exception, which we catch and process. In this case, the rejection value is an object with rich information about the nature of the failure. We use that object’s .statusCode and .error properties to close out the Express response.

Let’s try this out using curl and jq. Open the terminal where you saved the BUNDLE_ID earlier and run the following command:

 $ ​​curl​​ ​​-s​​ ​​localhost:60702/api/bundle/$BUNDLE_ID​​ ​​|​​ ​​jq​​ ​​'.'
 {
  "_index": "b4",
  "_type": "bundle",
  "_id": "AVuFkyXcpWVRyMBC8pgr",
  "_version": 1,
  "found": true,
  "_source": {
  "name": "light reading",
  "books": []
  }
 }

The bundle object itself is in the _source property of this object. You can also try getting a bundle for a nonexistent ID to see what that returns.

 $ ​​curl​​ ​​-s​​ ​​localhost:60702/api/bundle/no-such-bundle​​ ​​|​​ ​​jq​​ ​​'.'
 {
  "_index": "b4",
  "_type": "bundle",
  "_id": "no-such-bundle",
  "found": false
 }

Back in your terminal that’s running Node.js, you should see lines like the following:

 GET /api/bundle/AVuFkyXcpWVRyMBC8pgr 200 60.512 ms - 133
 GET /api/bundle/no-such-bundle 404 40.986 ms - 69

You should know one quick thing about the try/catch block before we move on. Consider this bad implementation that omits the try/catch block:

 // BAD IMPLEMENTATION! async Express handler without a try/catch block.
 app.​get​(​'/api/bundle/:id'​, ​async​ (req, res) => {
 const​ options = {
  url: ​`​${url}​/​${req.params.id}​`​,
  json: ​true​,
  };
 
 const​ esResBody = ​await​ rp(options);
  res.status(200).json(esResBody);
 });

What would happen if the Promise returned by the rp call was rejected instead of resolved? Do you have a guess?

Let’s try it out. Comment out the try/catch lines from your async function, then save the file. Then try again to access a nonexistent bundle with curl using the verbose flag.

 $ ​​curl​​ ​​-v​​ ​​localhost:60702/api/bundle/no-such-bundle
 * Trying 127.0.0.1...
 * Connected to localhost (127.0.0.1) port 60702 (​#0)
 > GET /api/bundle/no-such-bundle HTTP/1.1
 > Host: localhost:60702
 > User-Agent: curl/7.47.0
 > Accept: */*
 >

You should notice two things. First, the curl call never seems to terminate. It just hangs there after sending the request but receiving no response.

The second thing to notice is the warning messages in the Node.js terminal.

 (node:16075) UnhandledPromiseRejectionWarning: Unhandled promise rejection
  (rejection id: 1): StatusCodeError: 404 - {"_index":"b4","_type":"bundle",
  "_id":"no-such-bundle","found":false}
 (node:16075) DeprecationWarning: Unhandled promise rejections are deprecated.
  In the future, Promise rejections that are not handled will terminate the
  Node.js process with a nonzero exit code.

It turns out that indeed the await clause triggers the rejection object to be thrown, but since it’s not caught inside the async function, it bubbles up to a Promise returned by the async function itself. That Promise is rejected, but since its rejection wasn’t handled, we get warnings.

The moral of the story is always provide a try/catch block when using an async function as an Express route handler. More generally, it’ll depend on the purpose of your async function, but as a rule of thumb you should consider the consequence of rejected Promises and take action accordingly.

Now let’s move on to adding a few more APIs.

Setting the Bundle Name with PUT

Now we’ll use an async function to implement an API endpoint that allows setting the name property of a book bundle.

Open your lib/bundle.js and add the following, after the GET bundle API.

 /**
  * Set the specified bundle's name with the specified name.
  * curl -X PUT http://<host>:<port>/api/bundle/<id>/name/<name>
  */
 app.put(​'/api/bundle/:id/name/:name'​, ​async​ (req, res) => {
 const​ bundleUrl = ​`​${url}​/​${req.params.id}​`​;
 
 try​ {
 const​ bundle = (​await​ rp({url: bundleUrl, json: ​true​}))._source;
 
  bundle.name = req.params.name;
 
 const​ esResBody =
 await​ rp.put({url: bundleUrl, body: bundle, json: ​true​});
  res.status(200).json(esResBody);
 
  } ​catch​ (esResErr) {
  res.status(esResErr.statusCode || 502).json(esResErr.error);
  }
 });

Inside the async function, first we build out the bundleUrl based on the provided ID. Next, we begin the try/catch block in which we’ll perform all of the Elasticsearch requests and response handling.

Take a look at the first line inside the try{} block. Here, we’re using await with rp to suspend as before, but it’s a parenthesized expression. Outside of the expression, we use ._source to pull out just the bundle object from the broader Elasticsearch response. This demonstrates that the results of an awaited Promise can be used in more complex expressions.

Once we have the bundle object, we overwrite its name field with the provided name parameter value. Then it’s time to PUT that object back into Elasticsearch with rp.put. The resulting Elasticsearch response body should contain information about the successful operation, which we send back through the Express response.

As usual, if anything went wrong we catch the Elasticsearch response error and report back through the Express response. One you save the file, you can try it out.

In the same terminal where you have the BUNDLE_ID still saved as an environment variable, run the following to set the bundle name to foo:

 $ ​​curl​​ ​​-s​​ ​​-X​​ ​​PUT​​ ​​localhost:60702/api/bundle/$BUNDLE_ID/name/foo​​ ​​|​​ ​​jq​​ ​​'.'
 {
  "_index": "b4",
  "_type": "bundle",
  "_id": "AVuFkyXcpWVRyMBC8pgr",
  "_version": 2,
  "result": "updated",
  "_shards": {
  "total": 2,
  "successful": 1,
  "failed": 0
  },
  "created": false
 }

You can confirm that it was indeed saved by retrieving the bundle using the GET bundle API.

 $ ​​curl​​ ​​-s​​ ​​localhost:60702/api/bundle/$BUNDLE_ID​​ ​​|​​ ​​jq​​ ​​'._source'
 {
  "name": "foo",
  "books": []
 }

Note that Express routes treat forward slashes as delimiters, so if you wanted to set the name of a bundle to foo/bar, you’d need to URI-encode the slash as %2F. The same goes for other special characters such as question marks and hash symbols.

Now let’s move on to more complex route handlers. Next you’ll learn how to manage concurrent unsettled Promises to make simultaneous asynchronous requests.

Putting a Book into a Bundle

Things are going really well for our bundle APIs. At this point, you can create a bundle, retrieve a bundle, and set the bundle’s name.

However, the important feature of the bundle is its ability to store a set of books, and we don’t yet have any way to manage them. In this section, you’ll add an API endpoint for putting a book into a bundle.

Let’s get to the code. Open your lib/bundle.js and add the following after all of the other APIs you’ve added, inside the module.exports function:

 /**
  * Put a book into a bundle by its id.
  * curl -X PUT http://<host>:<port>/api/bundle/<id>/book/<pgid>
  */
 app.put(​'/api/bundle/:id/book/:pgid'​, ​async​ (req, res) => {
 const​ bundleUrl = ​`​${url}​/​${req.params.id}​`​;
 
 const​ bookUrl =
 `http://​${es.host}​:​${es.port}​`​ +
 `/​${es.books_index}​/book/​${req.params.pgid}​`​;
 
 try​ {
 
  } ​catch​ (esResErr) {
  res.status(esResErr.statusCode || 502).json(esResErr.error);
  }
 });

This shell code sets up the API to PUT a book, identified by its Project Gutenberg ID, into a bundle identified by its bundle ID. The try{} block is currently empty; we’ll fill that in next.

This is the first API that has to make requests to both the book’s Elasticsearch index and the bundle index. So at the outset of the implementation, we compute the URL to both objects within their respective indices. We’ll need to grab the book out of the books index to add it to the bundle.

The catch{} block at the bottom predictably forwards any failed request information back through the Express request like all the other APIs have done. Now let’s fill in the try{} block, which introduces some new techniques. Start by adding the following:

 // Request the bundle and book in parallel.
 const​ [bundleRes, bookRes] = ​await​ Promise.all([
  rp({url: bundleUrl, json: ​true​}),
  rp({url: bookUrl, json: ​true​}),
 ]);

The Promise.all method takes an array of Promises (technically, an iterable object containing Promises) and returns a new Promise based on all of them.[70] The all Promise will be resolved when every one of the Promises passed in has been resolved, and its returned value will be an array of the passed-in Promises’ returned values in the same order.

When any of the passed-in Promises is rejected, then the all Promise is rejected immediately. This means if multiple passed-in Promises are rejected, the all Promise will only get the value of the first one.

The array of Promises we pass to Promise.all contains the rp invocations to retrieve the bundle and book from their respective indices. This initiates the requests in parallel.

We await the all Promise and then use destructuring assignment to pull out the respective bundle and book response objects. If either request fails, its rp Promise will be rejected, the rejection value will be thrown, and we’ll handle it in the catch{} block, passing it along to the Express response.

By contrast, consider if we’d written the code this way:

 const​ bundleRes = rp({url: bundleUrl, json: ​true​});
 const​ bookRes = rp({url: bookUrl, json: ​true​});

This would work, producing the same outcome: bundleRes and bookRes are populated. But the book request would not begin until the bundle request had already completed.

Now let’s add the rest of the code in the try{} block. Add the following after the Promise.all piece:

 // Extract bundle and book information from responses.
 const​ {_source: bundle, _version: version} = bundleRes;
 const​ {_source: book} = bookRes;
 
 const​ idx = bundle.books.findIndex(book => book.id === req.params.pgid);
 if​ (idx === -1) {
  bundle.books.push({
  id: req.params.pgid,
  title: book.title,
  });
 }
 
 // Put the updated bundle back in the index.
 const​ esResBody = ​await​ rp.put({
  url: bundleUrl,
  qs: { version },
  body: bundle,
  json: ​true​,
 });
 res.status(200).json(esResBody);

First, this code uses destructuring assignment to extract and rename some variables from the bundle and book responses. This includes the book and bundle objects themselves as well as the version of the bundle. The version is needed to detect race conditions when updating the document.

After extracting the variables, we use Array.findIndex to determine whether that book is already in the bundle. If not, we push it onto the end of the bundle.books array. Then it’s time to send the updated bundle back up to Elasticsearch.

Notice that when putting the bundle document back in the bundle index, our call to rp includes a query string (qs) field. Here we pass in the bundle version number that we previously read from the bundleRes.

When Elasticsearch receives this request, it will check that its internal version number for that document matches the query string parameter. If they don’t match, then it means that the document has changed somehow and Elasticsearch will send back a 409 Conflict HTTP status code. This would cause the await clause to throw an exception, again handled by the catch{} block.

If everything went well, we send a 200 OK back through the Express response, along with the Elasticsearch response body. If you’ve been following along, your API code to add a book should now look like this:

 /**
  * Put a book into a bundle by its id.
  * curl -X PUT http://<host>:<port>/api/bundle/<id>/book/<pgid>
  */
 app.put(​'/api/bundle/:id/book/:pgid'​, ​async​ (req, res) => {
 const​ bundleUrl = ​`​${url}​/​${req.params.id}​`​;
 
 const​ bookUrl =
 `http://​${es.host}​:​${es.port}​`​ +
 `/​${es.books_index}​/book/​${req.params.pgid}​`​;
 
 try​ {
 
 // Request the bundle and book in parallel.
 const​ [bundleRes, bookRes] = ​await​ Promise.all([
  rp({url: bundleUrl, json: ​true​}),
  rp({url: bookUrl, json: ​true​}),
  ]);
 
 // Extract bundle and book information from responses.
 const​ {_source: bundle, _version: version} = bundleRes;
 const​ {_source: book} = bookRes;
 
 const​ idx = bundle.books.findIndex(book => book.id === req.params.pgid);
 if​ (idx === -1) {
  bundle.books.push({
  id: req.params.pgid,
  title: book.title,
  });
  }
 
 // Put the updated bundle back in the index.
 const​ esResBody = ​await​ rp.put({
  url: bundleUrl,
  qs: { version },
  body: bundle,
  json: ​true​,
  });
  res.status(200).json(esResBody);
 
  } ​catch​ (esResErr) {
  res.status(esResErr.statusCode || 502).json(esResErr.error);
  }
 });

Save your lib/bundle.js if you haven’t already. nodemon should automatically restart your service, and now we can test it.

Presuming you still have your BUNDLE_ID environment variable handy from earlier, let’s add a book to it. Here we’ll add The Art of War, which has Project Gutenberg ID 132.

 $ ​​curl​​ ​​-s​​ ​​-X​​ ​​PUT​​ ​​localhost:60702/api/bundle/$BUNDLE_ID/book/pg132​​ ​​|​​ ​​jq​​ ​​'.'
 {
  "_index": "b4",
  "_type": "bundle",
  "_id": "AVuFkyXcpWVRyMBC8pgr",
  "_version": 3,
  "result": "updated",
  "_shards": {
  "total": 2,
  "successful": 1,
  "failed": 0
  },
  "created": false
 }

And now we can confirm that it was added by getting the bundle again using the retrieve API.

 $ ​​curl​​ ​​-s​​ ​​localhost:60702/api/bundle/$BUNDLE_ID​​ ​​|​​ ​​jq​​ ​​'._source'
 {
  "name": "foo",
  "books": [
  {
  "id": "pg132",
  "title": "The Art of War"
  }
  ]
 }

Great! We don’t yet have an API to remove a book from a bundle or delete a bundle entirely. But don’t worry—those will be bonus tasks described momentarily.