HTTP Sniffer – listening to the HTTP conversation

The events emitted by the HTTPServer object can be used for additional purposes beyond the immediate task of delivering a web application. The following code demonstrates a useful module that listens to all of the HTTPServer events. It could be a useful debugging tool, which also demonstrates how HTTPServer objects operate.

Node.js's HTTPServer object is an EventEmitter object, and HTTP Sniffer simply listens to every server event, printing out information pertinent to each event.

Create a file named httpsniffer.mjs, containing the following code:

import * as util from 'util';
import * as url from 'url';

const timestamp = () => { return new Date().toISOString(); }

export function sniffOn(server) {
    server.on('request', (req, res) => {
        console.log(`${timestamp()} request`);
        console.log(`${timestamp()} ${reqToString(req)}`);
    });

    server.on('close', errno => { console.log(`${timestamp()} 
        close errno=${errno}`); });

    server.on('checkContinue', (req, res) => {
        console.log(`${timestamp()} checkContinue`);
        console.log(`${timestamp()} ${reqToString(req)}`);
        res.writeContinue();
    });

    server.on('upgrade', (req, socket, head) => {
        console.log(`${timestamp()} upgrade`);
        console.log(`${timestamp()} ${reqToString(req)}`);
    });

    server.on('clientError', () => { console.log('clientError'); });

    // server.on('connection', e_connection);
}

export function reqToString(req) {
    var ret = `request ${req.method} ${req.httpVersion} ${req.url}` 
       +'\n';
    ret += JSON.stringify(url.parse(req.url, true)) +'\n';
    var keys = Object.keys(req.headers);
    for (var i = 0, l = keys.length; i < l; i++) {
        var key = keys[i];
        ret += `${i} ${key}: ${req.headers[key]}` +'\n';
    }
    if (req.trailers)
        ret += util.inspect(req.trailers) +'\n';
    return ret;
}

The key here is the sniffOn function. When given an HTTPServer object, it attaches listener functions to each HTTPServer event to print relevant data. This gives us a fairly detailed trace of the HTTP traffic on an application.

In order to use it, make two simple modifications to server.mjs. To the top, add the following import statement:

import { sniffOn } from '../events/httpsniffer.mjs';

Then, change the server setup, as follows:

server.listen(new URL(listenOn).port);
sniffOn(server);
console.log(`listening to ${listenOn}`);

Here, we're importing the sniffOn function and then using it to attach listener methods to the server object.

With this in place, run the server as we did earlier. You can visit http://localhost:8124/ in your browser and see the following console output:

$ node server.mjs 
listening to http://localhost:8124
2020-01-05T02:33:09.864Z request
2020-01-05T02:33:09.868Z request GET 1.1 /osinfo
{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/osinfo","path":"/osinfo","href":"/osinfo"}
0 host: localhost:8124
1 connection: keep-alive
2 cache-control: max-age=0
3 dnt: 1
4 upgrade-insecure-requests: 1
5 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36
6 sec-fetch-user: ?1
7 accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/.jpg,*/*;q=0.8,application/signed-exchange;v=b3
8 sec-fetch-site: same-origin
9 sec-fetch-mode: navigate
10 referer: http://localhost:8124/
11 accept-encoding: gzip, deflate, br
12 accept-language: en-US,en;q=0.9
{}

You now have a tool for snooping on HTTPServer events. This simple technique prints a detailed log of event data. This pattern can be used for any EventEmitter objects. You can use this technique as a way to inspect the actual behavior of EventEmitter objects in your program.

Before we move on to using Express, we need to discuss why we use application frameworks at all.