Table of Contents for
Node.js 8 the Right Way

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Node.js 8 the Right Way by Jim Wilson Published by Pragmatic Bookshelf, 2018
  1. Title Page
  2. Node.js 8 the Right Way
  3. Node.js 8 the Right Way
  4. Node.js 8 the Right Way
  5. Node.js 8 the Right Way
  6.  Acknowledgments
  7.  Preface
  8. Why Node.js the Right Way?
  9. What’s in This Book
  10. What This Book Is Not
  11. Code Examples and Conventions
  12. Online Resources
  13. Part I. Getting Up to Speed on Node.js 8
  14. 1. Getting Started
  15. Thinking Beyond the web
  16. Node.js’s Niche
  17. How Node.js Applications Work
  18. Aspects of Node.js Development
  19. Installing Node.js
  20. 2. Wrangling the File System
  21. Programming for the Node.js Event Loop
  22. Spawning a Child Process
  23. Capturing Data from an EventEmitter
  24. Reading and Writing Files Asynchronously
  25. The Two Phases of a Node.js Program
  26. Wrapping Up
  27. 3. Networking with Sockets
  28. Listening for Socket Connections
  29. Implementing a Messaging Protocol
  30. Creating Socket Client Connections
  31. Testing Network Application Functionality
  32. Extending Core Classes in Custom Modules
  33. Developing Unit Tests with Mocha
  34. Wrapping Up
  35. 4. Connecting Robust Microservices
  36. Installing ØMQ
  37. Publishing and Subscribing to Messages
  38. Responding to Requests
  39. Routing and Dealing Messages
  40. Clustering Node.js Processes
  41. Pushing and Pulling Messages
  42. Wrapping Up
  43. Node.js 8 the Right Way
  44. Part II. Working with Data
  45. 5. Transforming Data and Testing Continuously
  46. Procuring External Data
  47. Behavior-Driven Development with Mocha and Chai
  48. Extracting Data from XML with Cheerio
  49. Processing Data Files Sequentially
  50. Debugging Tests with Chrome DevTools
  51. Wrapping Up
  52. 6. Commanding Databases
  53. Introducing Elasticsearch
  54. Creating a Command-Line Program in Node.js with Commander
  55. Using request to Fetch JSON over HTTP
  56. Shaping JSON with jq
  57. Inserting Elasticsearch Documents in Bulk
  58. Implementing an Elasticsearch Query Command
  59. Wrapping Up
  60. Node.js 8 the Right Way
  61. Part III. Creating an Application from the Ground Up
  62. 7. Developing RESTful Web Services
  63. Advantages of Express
  64. Serving APIs with Express
  65. Writing Modular Express Services
  66. Keeping Services Running with nodemon
  67. Adding Search APIs
  68. Simplifying Code Flows with Promises
  69. Manipulating Documents RESTfully
  70. Emulating Synchronous Style with async and await
  71. Providing an Async Handler Function to Express
  72. Wrapping Up
  73. 8. Creating a Beautiful User Experience
  74. Getting Started with webpack
  75. Generating Your First webpack Bundle
  76. Sprucing Up Your UI with Bootstrap
  77. Bringing in Bootstrap JavaScript and jQuery
  78. Transpiling with TypeScript
  79. Templating HTML with Handlebars
  80. Implementing hashChange Navigation
  81. Listing Objects in a View
  82. Saving Data with a Form
  83. Wrapping Up
  84. 9. Fortifying Your Application
  85. Setting Up the Initial Project
  86. Managing User Sessions in Express
  87. Adding Authentication UI Elements
  88. Setting Up Passport
  89. Authenticating with Facebook, Twitter, and Google
  90. Composing an Express Router
  91. Bringing in the Book Bundle UI
  92. Serving in Production
  93. Wrapping Up
  94. Node.js 8 the Right Way
  95. 10. BONUS: Developing Flows with Node-RED
  96. Setting Up Node-RED
  97. Securing Node-RED
  98. Developing a Node-RED Flow
  99. Creating HTTP APIs with Node-RED
  100. Handling Errors in Node-RED Flows
  101. Wrapping Up
  102. A1. Setting Up Angular
  103. A2. Setting Up React
  104. Node.js 8 the Right Way

Creating a Command-Line Program in Node.js with Commander

In this section, you’ll produce the outline of a command-line program that provides access to some features of Elasticsearch. You’ll start by creating a package.json, then build up from there.

To begin, open a terminal and create a directory called esclu that will house our Elasticsearch Command Line Utilities project. From inside the esclu directory, run npm init to start the interactive package.json creation wizard. All of the defaults are fine except for description, for which you should provide a short sentence describing the project.

<= $ npm init
 This utility will walk you through creating a package.json file.
 It only covers the most common items, and tries to guess sensible defaults.
 
 See `npm help json` for definitive documentation on these fields
 and exactly what they do.
 
 Use `npm install <pkg> --save` afterward to install a package and
 save it as a dependency in the package.json file.
 
 Press ^C at any time to quit.
 name: (esclu)
 version: (1.0.0)
 description:
=> Elasticsearch Command Line Utilities
<= entry point: (index.js)
 test command:
 git repository:
 keywords:
 author:
 license: (ISC)
 About to write to ./code/esclu/package.json:
 
 {
  "name": "esclu",
  "version": "1.0.0",
  "description": "Elasticsearch Command Line Utilites",
  "main": "index.js",
  "scripts": {
  "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
 }
 
 
 Is this ok? (yes)

Once you’re happy with the basic structure of your package, it’s time to move on to installing the modules we’ll depend on.

Introducing the Commander and Request Modules

So far in this book, you’ve implemented a number of command-line programs. These have been relatively simple in terms of the variety of options and arguments they can accept. To take it to the next level, we’re going to use a module called Commander, which makes it easy to construct elaborate and powerful command-line tools in Node.js.

Likewise, while Node.js supplies rudimentary support for HTTP requests with its built-in http module,[51] it can take a lot of finagling to get it right. Using a higher-level module like Request simplifies the process of issuing HTTP requests and handling the asynchronous responses.

Given their power to reduce tedious boilerplate code while providing useful functionality, the Commander module and the Request module will provide the backbone of the Elasticsearch command-line program. Install them using npm as follows:

 $ ​​npm​​ ​​install​​ ​​--save​​ ​​--save-exact​​ ​​commander@2.9.0​​ ​​request@2.79.0

With these two crucial Node.js modules installed, it’s time to lay out the fundamental structure of the Elasticsearch command-line program.

Creating a Basic Command-Line Program with Commander

The Commander module handles a variety of details: enforcing required parameters, parsing command-line options, interpreting alternative short names for flags called aliases, and so on. To take advantage of these features, you should follow a basic structure to your program. We’ll set that up in this section a step at a time.

For starters, we’ll want an extensionless executable file called esclu that we’ll be able to execute directly, without explicitly running Node.js. Recall back in Creating Read and Write Streams, that using #! nomenclature in the first line of your Node.js file is an acceptable Unix convention when making a file executable. We’ll use that again here, but separate the working JavaScript into its own file. Start by making a file called esclu with these contents:

 #!/usr/bin/env node
 require(​'./index.js'​);

As you can see, all this file does is execute the code in index.js by way of the require method.

Once you save the file, use chmod to make it executable from the command line.

 $ ​​chmod​​ ​​+x​​ ​​esclu

Next, open your text editor and enter this code, which you should save as index.js:

 'use strict'​;
 
 const​ fs = require(​'fs'​);
 const​ request = require(​'request'​);
 const​ program = require(​'commander'​);
 const​ pkg = require(​'./package.json'​);
 
 program
  .version(pkg.version)
  .description(pkg.description)
  .usage(​'[options] <command> [...]'​)
  .option(​'-o, --host <hostname>'​, ​'hostname [localhost]'​, ​'localhost'​)
  .option(​'-p, --port <number>'​, ​'port number [9200]'​, ​'9200'​)
  .option(​'-j, --json'​, ​'format output as JSON'​)
  .option(​'-i, --index <name>'​, ​'which index to use'​)
  .option(​'-t, --type <type>'​, ​'default type for bulk operations'​);
 
 program.parse(process.argv);
 
 if​ (!program.args.filter(arg => ​typeof​ arg === ​'object'​).length) {
  program.help();
 }

In the intro section at the top of the file, notice how we pull package.json into a constant called pkg. Node.js’s require method can read JSON files as well as modules written in JavaScript. By pulling in the package.json we can reference configuration parameters therein.

Next, we start setting up the command-line program object provided by Commander. After setting the version, description, and usage strings, we enumerate some flags and their default values. Exactly which flags your program needs is up to you, but these are the one we’ll use in chatting with Elasticsearch.

With that out of the way, we call program.parse on the Node.js process’s command-line arguments. This causes flags to be interpreted as such.

Lastly, we check whether the program’s args array contains any objects, as opposed to just strings. Commander fills the program.args array with the user-supplied arguments as strings, except those arguments that match a named command. We don’t have any commands defined yet; those will come shortly. But this block of code ensures that if users enter arguments that we don’t recognize, they see the same thing as if they’d asked for help with -h.

Once you save the file with this content, open a terminal to your esclu project directory and try running the script:

 $ ​​./esclu
 
  Usage: esclu [options] <command> [...]
 
  Elasticsearch Command Line Utilites
 
  Options:
 
  -h, --help output usage information
  -V, --version output the version number
  -o, --host <hostname> hostname [localhost]
  -p, --port <number> port number [9200]
  -j, --json format output as JSON
  -i, --index <name> which index to use
  -t, --type <type> default type for bulk operations

As you can see, the help is already working. You can also try out the version option and confirm that it gives you the same value as specified in the package.json.

 $ ​​./esclu​​ ​​-V
 1.0.0

Now that you have the basic structure of your program ready, it’s time to start adding commands.

Adding a Command to Your Program

Throughout the rest of this chapter, we’re going to be adding commands to esclu for interacting with Elasticsearch. Since Elasticsearch is primarily a RESTful datastore, interacting with it begins with composing the correct URLs to achieve our goals.

REST is an acronym that stands for Representational State Transfer. When an API is RESTful, it is HTTP-based and its resources are identified by their URLs. Requesting or making a change to a resource comes down to issuing an HTTP request using the particular method that matches your intent. For example, the HTTP GET method retrieves a resource, and HTTP PUT sends a resource to be saved.

In Elasticsearch, the RESTful resources are JSON documents. Each document lives in an index and has a type, which defines a class of related documents. To construct a URL for an Elasticsearch document, first you append which index you’re interested in (if any) and then optionally the type of object you’re interested in, separated by slashes. To get information about your whole cluster, you could make an HTTP GET request to the root: http://localhost:9200/.

In a bit, we’ll create an index to store the book metadata we created in Chapter 5, Transforming Data and Testing Continuously. To request information about the index named books, you would GET http://localhost:9200/books.

No matter what URL we end up hitting, we’ll want to incorporate the user-provided index and type information. To do that, add this fullUrl to your esclu program after the require lines and before setting up the program:

 const​ fullUrl = (path = ​''​) => {
 let​ url = ​`http://​${program.host}​:​${program.port}​/`​;
 if​ (program.index) {
  url += program.index + ​'/'​;
 if​ (program.type) {
  url += program.type + ​'/'​;
  }
  }
 return​ url + path.replace(​/^​​\/​​*/​, ​''​);
 };

The fullUrl function takes a single parameter, path, and returns the full URL to Elasticsearch based on the program parameters. Note that here we’re taking advantage of ECMAScript’s default parameter feature to a set path to the empty string if one isn’t provided.

Constructing the URL consists of appending the index (if specified) and the type (if both it and the index were specified) and, finally, appending the path. If the path includes a leading forward slash, we strip it off using a short regular expression to avoid double slashes in the final URL.

With the fullUrl method ready, now we can add a command to log it to the console. Add the following code to your index.js, before the program.parse line.

 program
  .command(​'url [path]'​)
  .description(​'generate the URL for the options and path (default is /)'​)
  .action((path = ​'/'​) => console.log(fullUrl(path)));

Adding a command to your program consists of three things: specifying the command name and parameters, providing a description, and setting up an action callback. The string you provide to the command tells Commander the name of the command an any arguments it takes. Required arguments should be surrounded by angle brackets (like <this>) and optional arguments should use square brackets (like [this]).

The function you provide to action is the callback that will be invoked when the command is run. It will be called with the same list of arguments specified when setting up the command. In this case, we expect an optional variable called path with the default value of /.

Inside the body of the action callback, all we do here is use the console to output the result of calling fullUrl with the path.

Head back to the terminal and try it out. First, just run esclu with no arguments to see that the new url command was added.

 $ ​​./esclu
 
  Usage: esclu [options] <command> [...]
 
 
  Commands:
 
  url [path] generate the URL for the options and path (default is /)
 
  Elasticsearch Command Line Utilities
 
  Options:
 
  -h, --help output usage information
  -V, --version output the version number
  -o, --host <hostname> hostname [localhost]
  -p, --port <number> port number [9200]
  -j, --json format output as JSON
  -i, --index <name> which index to use
  -t, --type <type> default type for bulk operations

As you can see, url is now listed under Commands, including the description string. When you execute esclu url, you should see the default root URL of your local Elasticsearch cluster.

 $ ​​./esclu​​ ​​url
 http://localhost:9200/

If you provide various options, those should be constructed appropriately too.

 $ ​​./esclu​​ ​​url​​ ​​'some/path'​​ ​​-p​​ ​​8080​​ ​​-o​​ ​​my.cluster
 http://my.cluster:8080/some/path

Next we’ll add a command to perform an HTTP GET request for the URL and output the results. This will unlock a lot of utility by itself, so it’s a good place to start.