In this section, you’ll produce the outline of a command-line program that provides access to some features of Elasticsearch. You’ll start by creating a package.json, then build up from there.
To begin, open a terminal and create a directory called esclu that will house our Elasticsearch Command Line Utilities project. From inside the esclu directory, run npm init to start the interactive package.json creation wizard. All of the defaults are fine except for description, for which you should provide a short sentence describing the project.
| <= | $ npm init |
| | This utility will walk you through creating a package.json file. |
| | It only covers the most common items, and tries to guess sensible defaults. |
| | |
| | See `npm help json` for definitive documentation on these fields |
| | and exactly what they do. |
| | |
| | Use `npm install <pkg> --save` afterward to install a package and |
| | save it as a dependency in the package.json file. |
| | |
| | Press ^C at any time to quit. |
| | name: (esclu) |
| | version: (1.0.0) |
| | description: |
| => | Elasticsearch Command Line Utilities |
| <= | entry point: (index.js) |
| | test command: |
| | git repository: |
| | keywords: |
| | author: |
| | license: (ISC) |
| | About to write to ./code/esclu/package.json: |
| | |
| | { |
| | "name": "esclu", |
| | "version": "1.0.0", |
| | "description": "Elasticsearch Command Line Utilites", |
| | "main": "index.js", |
| | "scripts": { |
| | "test": "echo \"Error: no test specified\" && exit 1" |
| | }, |
| | "author": "", |
| | "license": "ISC" |
| | } |
| | |
| | |
| | Is this ok? (yes) |
Once you’re happy with the basic structure of your package, it’s time to move on to installing the modules we’ll depend on.
So far in this book, you’ve implemented a number of command-line programs. These have been relatively simple in terms of the variety of options and arguments they can accept. To take it to the next level, we’re going to use a module called Commander, which makes it easy to construct elaborate and powerful command-line tools in Node.js.
Likewise, while Node.js supplies rudimentary support for HTTP requests with its built-in http module,[51] it can take a lot of finagling to get it right. Using a higher-level module like Request simplifies the process of issuing HTTP requests and handling the asynchronous responses.
Given their power to reduce tedious boilerplate code while providing useful functionality, the Commander module and the Request module will provide the backbone of the Elasticsearch command-line program. Install them using npm as follows:
| | $ npm install --save --save-exact commander@2.9.0 request@2.79.0 |
With these two crucial Node.js modules installed, it’s time to lay out the fundamental structure of the Elasticsearch command-line program.
The Commander module handles a variety of details: enforcing required parameters, parsing command-line options, interpreting alternative short names for flags called aliases, and so on. To take advantage of these features, you should follow a basic structure to your program. We’ll set that up in this section a step at a time.
For starters, we’ll want an extensionless executable file called esclu that we’ll be able to execute directly, without explicitly running Node.js. Recall back in Creating Read and Write Streams, that using #! nomenclature in the first line of your Node.js file is an acceptable Unix convention when making a file executable. We’ll use that again here, but separate the working JavaScript into its own file. Start by making a file called esclu with these contents:
| | #!/usr/bin/env node |
| | require('./index.js'); |
As you can see, all this file does is execute the code in index.js by way of the require method.
Once you save the file, use chmod to make it executable from the command line.
| | $ chmod +x esclu |
Next, open your text editor and enter this code, which you should save as index.js:
| | 'use strict'; |
| | |
| | const fs = require('fs'); |
| | const request = require('request'); |
| | const program = require('commander'); |
| | const pkg = require('./package.json'); |
| | |
| | program |
| | .version(pkg.version) |
| | .description(pkg.description) |
| | .usage('[options] <command> [...]') |
| | .option('-o, --host <hostname>', 'hostname [localhost]', 'localhost') |
| | .option('-p, --port <number>', 'port number [9200]', '9200') |
| | .option('-j, --json', 'format output as JSON') |
| | .option('-i, --index <name>', 'which index to use') |
| | .option('-t, --type <type>', 'default type for bulk operations'); |
| | |
| | program.parse(process.argv); |
| | |
| | if (!program.args.filter(arg => typeof arg === 'object').length) { |
| | program.help(); |
| | } |
In the intro section at the top of the file, notice how we pull package.json into a constant called pkg. Node.js’s require method can read JSON files as well as modules written in JavaScript. By pulling in the package.json we can reference configuration parameters therein.
Next, we start setting up the command-line program object provided by Commander. After setting the version, description, and usage strings, we enumerate some flags and their default values. Exactly which flags your program needs is up to you, but these are the one we’ll use in chatting with Elasticsearch.
With that out of the way, we call program.parse on the Node.js process’s command-line arguments. This causes flags to be interpreted as such.
Lastly, we check whether the program’s args array contains any objects, as opposed to just strings. Commander fills the program.args array with the user-supplied arguments as strings, except those arguments that match a named command. We don’t have any commands defined yet; those will come shortly. But this block of code ensures that if users enter arguments that we don’t recognize, they see the same thing as if they’d asked for help with -h.
Once you save the file with this content, open a terminal to your esclu project directory and try running the script:
| | $ ./esclu |
| | |
| | Usage: esclu [options] <command> [...] |
| | |
| | Elasticsearch Command Line Utilites |
| | |
| | Options: |
| | |
| | -h, --help output usage information |
| | -V, --version output the version number |
| | -o, --host <hostname> hostname [localhost] |
| | -p, --port <number> port number [9200] |
| | -j, --json format output as JSON |
| | -i, --index <name> which index to use |
| | -t, --type <type> default type for bulk operations |
As you can see, the help is already working. You can also try out the version option and confirm that it gives you the same value as specified in the package.json.
| | $ ./esclu -V |
| | 1.0.0 |
Now that you have the basic structure of your program ready, it’s time to start adding commands.
Throughout the rest of this chapter, we’re going to be adding commands to esclu for interacting with Elasticsearch. Since Elasticsearch is primarily a RESTful datastore, interacting with it begins with composing the correct URLs to achieve our goals.
REST is an acronym that stands for Representational State Transfer. When an API is RESTful, it is HTTP-based and its resources are identified by their URLs. Requesting or making a change to a resource comes down to issuing an HTTP request using the particular method that matches your intent. For example, the HTTP GET method retrieves a resource, and HTTP PUT sends a resource to be saved.
In Elasticsearch, the RESTful resources are JSON documents. Each document lives in an index and has a type, which defines a class of related documents. To construct a URL for an Elasticsearch document, first you append which index you’re interested in (if any) and then optionally the type of object you’re interested in, separated by slashes. To get information about your whole cluster, you could make an HTTP GET request to the root: http://localhost:9200/.
In a bit, we’ll create an index to store the book metadata we created in Chapter 5, Transforming Data and Testing Continuously. To request information about the index named books, you would GET http://localhost:9200/books.
No matter what URL we end up hitting, we’ll want to incorporate the user-provided index and type information. To do that, add this fullUrl to your esclu program after the require lines and before setting up the program:
| | const fullUrl = (path = '') => { |
| | let url = `http://${program.host}:${program.port}/`; |
| | if (program.index) { |
| | url += program.index + '/'; |
| | if (program.type) { |
| | url += program.type + '/'; |
| | } |
| | } |
| | return url + path.replace(/^\/*/, ''); |
| | }; |
The fullUrl function takes a single parameter, path, and returns the full URL to Elasticsearch based on the program parameters. Note that here we’re taking advantage of ECMAScript’s default parameter feature to a set path to the empty string if one isn’t provided.
Constructing the URL consists of appending the index (if specified) and the type (if both it and the index were specified) and, finally, appending the path. If the path includes a leading forward slash, we strip it off using a short regular expression to avoid double slashes in the final URL.
With the fullUrl method ready, now we can add a command to log it to the console. Add the following code to your index.js, before the program.parse line.
| | program |
| | .command('url [path]') |
| | .description('generate the URL for the options and path (default is /)') |
| | .action((path = '/') => console.log(fullUrl(path))); |
Adding a command to your program consists of three things: specifying the command name and parameters, providing a description, and setting up an action callback. The string you provide to the command tells Commander the name of the command an any arguments it takes. Required arguments should be surrounded by angle brackets (like <this>) and optional arguments should use square brackets (like [this]).
The function you provide to action is the callback that will be invoked when the command is run. It will be called with the same list of arguments specified when setting up the command. In this case, we expect an optional variable called path with the default value of /.
Inside the body of the action callback, all we do here is use the console to output the result of calling fullUrl with the path.
Head back to the terminal and try it out. First, just run esclu with no arguments to see that the new url command was added.
| | $ ./esclu |
| | |
| | Usage: esclu [options] <command> [...] |
| | |
| | |
| | Commands: |
| | |
| | url [path] generate the URL for the options and path (default is /) |
| | |
| | Elasticsearch Command Line Utilities |
| | |
| | Options: |
| | |
| | -h, --help output usage information |
| | -V, --version output the version number |
| | -o, --host <hostname> hostname [localhost] |
| | -p, --port <number> port number [9200] |
| | -j, --json format output as JSON |
| | -i, --index <name> which index to use |
| | -t, --type <type> default type for bulk operations |
As you can see, url is now listed under Commands, including the description string. When you execute esclu url, you should see the default root URL of your local Elasticsearch cluster.
| | $ ./esclu url |
| | http://localhost:9200/ |
If you provide various options, those should be constructed appropriately too.
| | $ ./esclu url 'some/path' -p 8080 -o my.cluster |
| | http://my.cluster:8080/some/path |
Next we’ll add a command to perform an HTTP GET request for the URL and output the results. This will unlock a lot of utility by itself, so it’s a good place to start.