In this chapter, you learned how to use Node.js to interact with the popular document-oriented datastore Elasticsearch. Elasticsearch stores JSON documents over HTTP and provides a rich search API, making it a great example to learn from as you start to think about creating your own RESTful APIs.
By taking advantage of the Commander module, you developed a command-line utility program with a number of useful commands for getting information into and out of Elasticsearch. In turn, this allowed us to discuss default parameters for JavaScript functions, and the useful Array methods filter and join.
As for DSLs, you learned the basics of Elasticsearch’s query language, which we’ll utilize more in upcoming chapters. And, importantly, you used jq to dig into and reshape JSON messages. JSON over HTTP is ubiquitous in Node.js development, and knowing how to quickly dive into an unfamiliar dataset or API is a valuable skill.
Using the Request module made it relatively painless to issue GET, PUT, and POST requests to HTTP endpoints. It even made it easy to stream content from a file to the server, and from the server response to standard output.
Although the Request module follows the Node.js core convention of taking a single callback handler, it can be used with Promises for more fluid handling of asynchronous responses. We’ll cover how to do this in upcoming chapters.
The next chapter will build on your knowledge of RESTful/JSON APIs as we use Node.js and Express to develop HTTP endpoints that operate on your Elasticsearch indices. But before we get to that, the following bonus tasks invite you to add more functionality to the incomplete esclu program. See you again soon!
Any database you work with will offer at least the following four operations: Create, Read, Update, and Delete (CRUD). RESTful datastores like Elasticsearch use a different HTTP method (or verb) for each operation. You use POST to create, GET to read, PUT to update, and DELETE to (you guessed it) delete records.
In this chapter, you’ve already used three of these four verbs. You used PUT to create an index, GET to query for documents, and POST to upload documents in bulk.
For this task, implement a new command called delete-index, which checks for an index specified with the --index flag and issues an HTTP DELETE request to remove it. (Hint: request.del issues a DELETE request.)
When creating the esclu program, we made the bulk command, which provided a way for the user to perform bulk operations on Elasticsearch. However, we did not provide a command for performing a single action, such as inserting a new document.
For this task, you’ll add a new command called put, which inserts a new document for indexing (or overwrites the existing document if there’s a collision).
With the get command, you can already retrieve a book by its _id. For example, here’s how to look up The Art of War by its ID:
| | $ ./esclu get pg132 --index books --type book | jq '.' |
| | { |
| | "_index": "books", |
| | "_type": "book", |
| | "_id": "pg132", |
| | "_version": 1, |
| | "found": true, |
| | "_source": { |
| | "id": 132, |
| | "title": "The Art of War", |
| | "authors": [ |
| | "Sunzi, active 6th century B.C.", |
| | "Giles, Lionel" |
| | ], |
| | "subjects": [ |
| | "Military art and science -- Early works to 1800", |
| | "War -- Early works to 1800" |
| | ] |
| | } |
| | } |
Putting a document into Elasticsearch is roughly the opposite maneuver, but the API should be quite similar. For example, say we save the document part of the above response to a file, like so:
| | $ ./esclu get pg132 -i books -t book | jq '._source' > ../data/art_of_war.json |
Ideally, we should be able to reinsert the document from the file using the following command:
| | $ ./esclu put ../data/art_of_war.json -i books -t book --id pg132 |
To make this work, you’ll need to do the following:
Add a new, optional, --id flag.
Update the fullUrl function to append the ID in the returned URL.
Add a new command called put that takes a single required parameter called file (same as the bulk command).
Inside the action callback of your new command, assert that an ID was specified, or fail loudly.
Stream the contents of the file to Elasticsearch through the request object and stream the results to standard output.
For a reference on the expectation of the Elasticsearch API, see the Index documentation.[63]
If you get all of this working, great! Next, think about how you might relax the requirement that the document came from an actual file. For example, could you make the file part of the put command optional? If the file wasn’t specified, how would you read the JSON document content from standard input instead?
http://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html
https://www.elastic.co/downloads/past-releases/elasticsearch-5-2-2
https://www.elastic.co/guide/en/elasticsearch/reference/5.2/system-config.html
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/fetch
https://stedolan.github.io/jq/manual/v1.5/#Builtinoperatorsandfunctions
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html
https://www.elastic.co/guide/en/elasticsearch/reference/5.2/docs-index_.html