Net internals

Chrome net-internals² can be accessed by typing chrome://net-internals in Chrome’s address bar. These tools provide a look at the network data, including capturing/exporting/importing low-level network data, checking network and DNS logs, and visualizing network activity.

The net-internal tools can be used to capture HTTP/2 traffic, which is useful to illustrate some HTTP/2 concepts like:

• Stream IDs

• Priorities

• Dependencies

• Server-Push promises

• Session PINGs

Here are the steps to do a capture of HTTP/2 traffic:

1. Type chrome://net-internals in Chrome’s address bar.

2. Select HTTP/2 from the left sidebar.

3. Open a new tab and type the your favorite URL in the address bar.

4. Go back to the net-internals tab, where you will see a list of all the active sessions to hostnames that use the HTTP/2 protocol.

5. Click the ID link located on the right side of the hostname of the entered URL (see Figure 8-1).

Figure 8-1. Net-internals HTTP/2

6. The context will change to Events. Click the checkbox located on the left side of the hostname (Figure 8-2).

Figure 8-2. Net-internals Events

7. On the right side of this page you should see a capture of all the traffic between the web server and the client.

Let’s review some of the fields used in the capture. Each “event” starts with something like this:

t=timestamp [st=  milliseconds]    EVENT_TYPE

For example:

t=123808 [st=    1]    HTTP2_SESSION_SEND_HEADERS

• The first field “t” indicates the timestamp, in milliseconds, since the browser session started. For example, a value of 123808 indicates the HTTP/2 session has started 123.8 seconds after the browser’s session started.

• The second field “st” indicates the relative number of milliseconds since the HTTP/2 session started. For example, a value of 1 indicates the event happened 1 millisecond after the HTTP/2 was started.

• The third field shows the type of event recorded.

Let’s look at a sample capture to give you a feel for how to read them and to see what HTTP/2 information we can pull out:

t=791301 [st=   1]    HTTP2_SESSION_SEND_HEADERS 
                      --> exclusive = true
                      --> fin = true             
                      --> has_priority = true    
                      --> :method: GET           
                          :authority: akah2san.h2book.com
                          :scheme: https
                          :path: /
                          cache-control: max-age=0
                          upgrade-insecure-requests: 1
                          user-agent: Mozilla/5.0 (Macintosh; Intel Mac...
                          accept: text/html,application/xhtml+xml,...
                          accept-encoding: gzip, deflate, sdch, br
                          accept-language: en-US,en;q=0.8
                          cookie: [30 bytes were stripped]
                          if-none-match: "11168351bd3324ad3e43ed68195063c5:1464989325"
                      --> parent_stream_id = 0   
                      --> stream_id = 1          
                      --> weight = 256           
...

Here is the event information line as described previously

fin = true indicates that there is no additional head frame coming

This request has priorities set

The HTTP headers on the frame start here

The associated parent stream in stream 0

This stream has ID 1 (the first client request)

The relative weight for dependencies is 256

By looking at the net-internal events you can get a clear picture of precisely what is happening when and even look inside the protocol.

Server Push visualization

The Network tab of Chrome’s Developer Tools panel is useful for visually following the communication between the client and the server, while displaying a bunch of useful information in a table format like:

• Object’s name

• Object’s size

• Status code

• Priority

• Total load time

• Breakdown of load times within a timeline

Let’s look at an example. Load the web page https://akah2san.h2book.com/ (a simple page that loads over HTTP/2 and uses Server Push).

In the Network tab, you should see something like Figure 8-3.

Figure 8-3. Server Push timeline

The Network tab tells us that the HTML loads three stylesheets and four PNG images. From those seven objects, two of them (/resources/push.css and /resources/http2-banner-0614.png) are “pushed” to the client, while the other five load normally.

If we mouse over one of the objects in the waterfall on the righthand side of the pane, we will see a breakdown on the different stages while the object was fully loaded. Here is the explanation of the information displayed (see Figure 8-4):

• Connection Setup

  • Queueing: Time the request was postponed by the rendering engine or network layer.

  • Stalled: Time the request spent waiting before it could be sent.

• Request/Response

  • Request Sent: Time spent issuing the network request.

  • Waiting (TTFB): Time spent waiting for the initial response, also known as the Time To First Byte. This number captures the latency of a round-trip to the server in addition to the time spent waiting for the server to deliver the response.

  • Content Download: Time spent receiving the response data.

• Total time (labeled Explanation)

Figure 8-4. Server Push timeline h2.css

If we mouse over one of the objects that are pushed by the server, we will see the information shown in Figure 8-5:

• Server Push

  • Receiving Push: Time it takes to receive all the bytes for the pushed object

• Connection Setup

  • Queueing: Time the request was postponed by the rendering engine or network layer

• Request/Response

  • Reading Push: Time it takes for the browser to read the bytes pushed from the temporary cache

• Total time (labeled Explanation)

Figure 8-5. Server Push timeline push.css

Chrome session key logging

Both Chrome and Firefox provide the ability to log the TLS session key used for encrypting particular encrypted connections in the browser. It is incredibly useful for using external tools such as Wireshark (see “Wireshark”) to examine the traffic and look at the HTTP/2 frames. This functionality is triggered by setting the appropriate environment variable to the location you want the keys logged before you launch the browser. For example, on OS X you could do:

$ SSLKEYLOGFILE=~/keylog.txt
$ open /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome

Mozilla has a good description of this process written up at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format.

Logging HTTP sessions

Logging HTTP sessions can be useful for debugging purposes, and Firefox’s HTTP Session logging can provide you a lot of insight into what is going on over the wire. There is no in-browser mechanism for getting those captures from Firefox at this time. A bit of command-line hacking is required.

Let’s use Firefox’s HTTP Session logging to do a capture of HTTP/2 traffic, which will illustrate some HTTP/2 concepts like:

• Stream IDs

• Priorities

• Dependencies

• Server-Push promises

• Session PINGs

To enable logging, use the following commands in the Windows command prompt:

cd c:\
set NSPR_LOG_MODULES=timestamp,nsHttp:5,nsSocketTransport:5,nsStreamPump:5, ^
   nsHostResolver:5
set NSPR_LOG_FILE=%TEMP%\firefox_http_log.txt
cd "Program Files (x86)\Mozilla Firefox"
.\firefox.exe

or MAC OS terminal:

export NSPR_LOG_MODULES=timestamp,nsHttp:5,nsSocketTransport:5, \
   nsStreamPump:5,nsHostResolver:5
export NSPR_LOG_FILE=~/Desktop/firefox_http_log.txt
cd /Applications/Firefox.app/Contents/MacOS
./firefox-bin

You can control the verbosity of the log file by updating the values of the shell variable NSPR_LOG_FILE in a couple of ways:

• Reducing the number on the right of the module. For example, nsHttp:3 indicates less verbosity than nsHttp:5. See Table 8-1 for a list of debug values.

• Removing a module from the list. For example, NSPR_LOG_MODULES=timestamp,nsHttp:5 is less verbose than NSPR_LOG_MODULES=timestamp,nsHttp:5,nsSocketTransport:5,nsStreamPump:5. You can get the most verbose logging by using NSPR_LOG_MODULES=all:5.

Firefox session key logging

Firefox, like Chrome, can log the TLS session keys. See “Chrome session key logging” for a description.

iOS Simulators

To get Charles Proxy set up to debug h2 in an iOS Simulator running on the same machine, follow these steps:

1. Quit your iOS Simulator.

2. Launch Charles Proxy.

3. Open the Help menu and choose SSL Proxying → Install Charles Root Certificate in iOS Simulators (this will install it into all of your local iOS Simulators).

Now when you start the iOS Simulator, you should be able to access TLS websites with Charles Proxy using TLS Proxying.

iOS devices

In order to debug HTTPS traffic on iOS devices you need to run a proxy on a computer and change the iOS device network settings to use the proxy running on that computer.

On the host computer:

1. Start Charles Proxy.

2. On the Proxy menu select Proxy Settings and ensure the port is set to 8888 (you can use Dynamic Port if you have problems connecting to the 8888 port).

3. On the Proxy menu select SSL Proxy Settings, click Add, and type the hostname you want to monitor on the Host field (or use * to monitor any site).

4. On the Proxy menu select Access Control Settings and add the IP of the iOS device (or just 0.0.0.0/0 to allow all devices to connect).

5. On the Help menu select Local IP Address, write down your local IP address, and close the window (you will need to use that IP as a proxy server on the iOS device).

6. On the File menu select New Session to start recording traffic.

For more information on this process, see https://www.charlesproxy.com/documentation/proxying/ssl-proxying/.

Next, on the iOS device:

1. Open the Settings app → “Wifi settings,” and then click the Information icon that looks like (i) located on the right of the WiFi network you are connected to.

2. In the list that appears, scroll down to the HTTP PROXY section and select Manual.

3. In the Server field that appears, put the IP of the host computer, and in the port field put 8888 (or enter a different number if you are using a dynamic port).

4. Click the home button, start Safari, and point the browser to chls.pro/ssl (or http://www.charlesproxy.com/getssl/) to install Charles’s proxy root certificate (needed to decrypt TSL traffic).

Now you should be able to access TLS websites with Charles using SSL Proxying (except for apps that use pinned certificates–like some Apple native apps).

Measuring page load times

You can use curl’s w parameter to print useful performance metrics (see the curl man page⁸ for reference).

By adding the following parameter to a curl request (note that the parameter includes some text formatting):

-w "Connection time: %{time_connect}\t1st byte transfer:
  %{time_starttransfer}\tDownload time: %{time_total}
    (sec)\tDownload Speed: %{speed_download} (bps)\n"

you will see metrics like:

• Connection time

• First byte time

• Download time

• Total time

• Download speed (indicating how many bytes you can send per second)

For example:

$ curl -v --http2 https://akah2san.h2book.com/hello-world.html -w \
       "Connection time: %{time_connect}\t                        \
       1st byte transfer: %{time_starttransfer}\t                 \
       Download time: %{time_total} (sec)\t                       \
       Download Speed: %{speed_download} (bps)\n"

...omitting a bunch of lines...
* Connection #0 to host akah2san.h2book.com left intact
Connection time: 0.054  1st byte transfer: 0.166  Download time: 0.166 (sec)
Download Speed: 1016.000 (bps)