Node.js 8 the Right Way

Responding to Requests

The REQ/REP pattern is quite common in networked programming, particularly in Node. We’ll use this pattern often in the next couple of chapters, and ØMQ has great support for it. As you’ll see in a minute, this is where the “Q” of ØMQ becomes apparent.

In ØMQ, a REQ/REP pair communicates in lockstep. A request comes in, then a reply goes out. Additional incoming requests are queued and later dispatched by ØMQ. Your application, however, is aware of only one request at a time.

Let’s see how this works, again using the filesystem as a source of information for building a microservice. In this scenario, a responder waits for a request for file data, then serves up the content when asked. We’ll start with the responder—the REP (reply) part of the REQ/REP pair.

Implementing a Responder

Open an editor and enter the following:

microservices/zmq-filer-rep.js

	'use strict';
	const fs = require('fs');
	const zmq = require('zeromq');

	// Socket to reply to client requests.
	const responder = zmq.socket('rep');

	// Handle incoming requests.
	responder.on('message', data => {

	// Parse the incoming message.
	const request = JSON.parse(data);
	console.log(`Received request to get: ${request.path}`);

	// Read the file and reply with content.
	fs.readFile(request.path, (err, content) => {
	console.log('Sending response content.');
	responder.send(JSON.stringify({
	content: content.toString(),
	timestamp: Date.now(),
	pid: process.pid
	}));
	});

	});

	// Listen on TCP port 60401.
	responder.bind('tcp://127.0.0.1:60401', err => {
	console.log('Listening for zmq requesters...');
	});

	// Close the responder when the Node process ends.
	process.on('SIGINT', () => {
	console.log('Shutting down...');
	responder.close();
	});

Save the file as zmq-filer-rep.js. The program creates a ØMQ REP socket and uses it to respond to incoming requests.

When a message event happens, we parse out the request from the raw data. Next we call fs.readFile to asynchronously retrieve the requested file’s content. When it arrives, we use the responder’s send method to reply with a JSON serialized response, including the file content and a timestamp. We also include the process ID (pid) of the Node.js process in the response.

The responder binds to TCP port 60401 of the loopback interface (IP 127.0.0.1) to wait for connections. This makes the responder the stable endpoint of the REP/REQ pair.

Since this service reads content from your filesystem and serves it to any requester, be sure to set the IP to 127.0.0.1 (localhost)! This is for demonstration purposes only.

Finally, we listen for SIGINT events on the Node.js process. This Unix signal indicates that the process has received an interrupt signal from the user—typically invoked by pressing Ctrl-C in the terminal. The clean thing to do in this case is ask the responder to gracefully close any outstanding connections.

Start the program in a terminal as usual:

	$ node zmq-filer-rep.js
	Listening for zmq requesters...

Looks like our responder is ready! But to connect to it, we’ll need to develop a client, so let’s put one together.

Issuing Requests

Creating a requester to work with our responder is pretty short. Open an editor and enter this:

microservices/zmq-filer-req.js

	'use strict';
	const zmq = require('zeromq');
	const filename = process.argv[2];

	// Create request endpoint.
	const requester = zmq.socket('req');

	// Handle replies from the responder.
	requester.on('message', data => {
	const response = JSON.parse(data);
	console.log('Received response:', response);
	});

	requester.connect('tcp://localhost:60401');

	// Send a request for content.
	console.log(`Sending a request for ${filename}`);
	requester.send(JSON.stringify({ path: filename }));

Save the file as zmq-filer-req.js.

This program starts off by creating a ØMQ REQ socket. Then we listen for incoming message events and interpret the data as a JSON serialized response (which we log to the console). The end of the program kicks off the request by connecting to the REP socket over TCP and finally calling requester.send. The JSON request message contains the requested file’s path as specified on the command line.

Let’s see how these REQ and REP sockets work together. With the zmq-filer-rep program still running in one terminal, run this command in another:

	$ node zmq-filer-req.js target.txt
	Sending a request for target.txt
	Received response: { content: '', timestamp: 1458898367933, pid: 24815 }

Success! The REP endpoint received the request, processed it, and sent back a response.

Trading Synchronicity for Scale

There is a catch to using ØMQ REP/REQ socket pairs with Node. Each endpoint of the application operates on only one request or one response at a time. There is no parallelism.

We can see this in action by making a small change to the requester program. Open the zmq-filer-req.js file from last section. Find the code that sends the request and wrap it in a for loop like this:

	for (let i = 1; i <= 5; i++) {
	console.log(`Sending request ${i} for ${filename}`);
	requester.send(JSON.stringify({ path: filename }));
	}

Save this file as zmq-filer-req-loop.js. With the responder still running, invoke the new script:

	$ node zmq-filer-req-loop.js target.txt
	Sending request 1 for target.txt
	Sending request 2 for target.txt
	Received response: { content: '', timestamp: 1458902785998, pid: 24674 }
	Sending request 3 for target.txt
	Sending request 4 for target.txt
	Sending request 5 for target.txt
	Received response: { content: '', timestamp: 1458902786010, pid: 24674 }
	Received response: { content: '', timestamp: 1458902786011, pid: 24674 }
	Received response: { content: '', timestamp: 1458902786011, pid: 24674 }
	Received response: { content: '', timestamp: 1458902786012, pid: 24674 }

We see that the requests were queued in the loop, and then we received the responses. The sending and receiving lines may be interleaved, depending on how quickly the responses become available.

This output shouldn’t be too surprising, but let’s take a look at the responder window:

	$ node zmq-filer-rep.js
	Listening for zmq requesters...
	Received request to get: target.txt
	Sending response content.
	Received request to get: target.txt
	Sending response content.
	Received request to get: target.txt
	Sending response content.
	Received request to get: target.txt
	Sending response content.
	Received request to get: target.txt
	Sending response content.

Notice that the responder program sent a response to each request before even becoming aware of the next queued request. This means Node.js’s event loop was left spinning while the fs.readFile for each request was being processed.

For this reason, a simple REQ/REP pair is probably not going to suit your high-performance Node.js needs. Next we’ll construct a cluster of Node.js processes using more advanced ØMQ socket types to scale up our throughput.

Previous Chapter

Publishing and Subscribing to Messages

Next Chapter

Routing and Dealing Messages

Table of Contents for Node.js 8 the Right Way

Responding to Requests

Implementing a Responder

Issuing Requests

Trading Synchronicity for Scale

Table of Contents for
Node.js 8 the Right Way