Chapter 6
Commanding Databases

In this chapter you’ll be crafting a rich command-line utility program for interacting with Elasticsearch—a schema-free, RESTful, NoSQL database that stores and indexes JSON documents over HTTP. Your program will be configurable through many options, and will support advanced query capabilities. Importantly, it will be able to import documents in bulk, like the corpus of Project Gutenberg documents produced in the preceding chapter. The following image shows where we’re headed.

With the experience you gain here, you’ll be able to develop your own richly featured Node.js command-line programs, as well as communicate with other RESTful, JSON-based services. You’ll also be in a good position to judge whether Elasticsearch is the right database for your future Node.js projects.

Doing this work will introduce you to these aspects of Node.js development:

Node.js Core

In this chapter, we’ll revisit piping streams of data, but this time in the context of communicating with remote services over HTTP. We’ll also use npm to interactively create a package.json file, the cornerstone of a Node.js project.

Patterns

You’ll use the Commander module’s method-chaining pattern for building out command-line programs. And you’ll use the Request module for performing HTTP requests.

JavaScriptisms

In JavaScript, a caller can invoke your function without providing all the arguments that the function expects. To compensate for missing arguments, when defining a function, you can supply a default parameter value that is evaluated at runtime. You’ll use this technique, as well as some useful Array methods such as filter and join.

Supporting Code

Outside of Node.js proper, this chapter will introduce you to Elasticsearch, a JSON-based document datastore. In addition, to manipulate JSON you’ll use an exciting command-line tool called jq.

Over the course of this book, you’ve already been briefly exposed to a couple of domain-specific languages (DSLs) like RDF (XML), as well as CSS selectors and how to make good use of them with Node.js. In this chapter we’ll touch on a few more. You’ll learn Elasticsearch’s query syntax, which is needed to find matching documents. In addition, you’ll learn how to use jq’s filter expressions to manipulate JSON on the command line.

These DSLs could warrant whole books in their own right, but I can only afford partial coverage in this book. It is my sincere goal to provide you with the best footing on your way to becoming a great Node.js developer. For that reason, I feel it’s necessary to introduce you to these topics, even if you’ll have to take the rest of the journey on your own.

At the outset of this chapter, you’ll install Elasticsearch and confirm that it’s running. After that, for the rest of the chapter, you’ll be developing a program called esclu from the ground up. This process will provide a rough blueprint of what it’s like to develop a featureful command-line program in Node.js.

Rather than dive deep on Elasticsearch features at the outset, we’ll explore them as we go, meeting each one as we expand the capabilities of the command-line program. Let’s get to it!