Last time we looked at a HTTP/1.1 request and response using Wireshark. Over the next few posts I’m building towards a better understanding of gRPC. gRPC uses HTTP/2, which departs from HTTP/1.1 in several ways. Before getting into those changes, let’s look at how HTTP can be used for secure communication.

Transport Layer Security, or TLS for short, is an approach to security which sits just above TCP (hence the name). At a high level, the protocol describes a way to open an encrypted channel on top of an underlying TCP stream. An initial handshake settles cryptographic conventions and identities, after which encrypted application data is sent over ordinary TCP segments.

There are a lot of cryptographic details that can obscure the high level structure of the protocol. To get a feel for how TLS works, we can instead just try it out with an HTTP request (so-called HTTPS) and look at some of the captured packets.

Last time, we used cURL to generate an HTTP request. Today, we’ll use Python. In the following code we use the ssl package to wrap an ordinary TCP socket with TLS:

 1import socket
 2import ssl
 3
 4host = 'google.com'
 5port = 443
 6
 7context = ssl.create_default_context()
 8# Log TLS key information for use with Wireshark
 9context.keylog_filename =  "sslkeylog.log"
10
11with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
12    sock.connect((host, port))
13    with context.wrap_socket(sock, server_hostname=host) as ssock:    
14        request = (
15            "GET / HTTP/1.1\r\n"
16            f"Host: {host}\r\n"
17            "Connection: close\r\n"
18            "\r\n"
19        )
20    
21        ssock.sendall(request.encode('ascii'))
22    
23        response_parts = []
24        while True:
25            data = ssock.recv(4096)
26            if not data:
27                break
28            response_parts.append(data)
29            
30        ssock.close()
31    
32        response = b''.join(response_parts).decode('utf-8', errors='replace')
33        print(response)

Note that in Python, we need an SSLContext in order to wrap a socket. In the above code we explicitly indicate that the context should save the session’s TLS keys. We’ll need these to decrypt the packet contents in Wireshark.

After running the above and catching the packets, we see the following:

Although we can’t follow every detail in this, we can at least understand the shape of the interaction:

1. There’s an ordinary TCP connection establishment.
2. Then, some back and forth between the client and server to set up the TLS session. Wireshark labels many of these packets with TLSv1.3 as the protocol, and in some cases gives suggestive names like “Server Hello, Change Cipher Spec”.
3. There are a number of TLSv1.3 segments that just get labeled “Application Data”. These are encrypted but presumably include the HTTP GET and response.
4. Finally, the TCP connection closes.

To further inspect these packets we need the keys we recorded during our request. These are ephemeral to the session so it’s safe to share the contents of sslkeylog.log :

# TLS secrets log file, generated by OpenSSL / Python
SERVER_HANDSHAKE_TRAFFIC_SECRET bb058323a16b445da4c69a211dd9fb3718a5574fef4a13e59a00c94eb46b5731 f431df302f018369fe4540bf3d148ea62faedeff5b518db06e6077267ca2807241c7934238702e288641759d473caaab
EXPORTER_SECRET bb058323a16b445da4c69a211dd9fb3718a5574fef4a13e59a00c94eb46b5731 b24fc91dddbe4c1fd5cd308d1cdd0db3c0dacfc3756ac9661d7c5a20d04a10c0759c55fcc3b8a38fa2793e191d7b8992
SERVER_TRAFFIC_SECRET_0 bb058323a16b445da4c69a211dd9fb3718a5574fef4a13e59a00c94eb46b5731 d8a62b42acffc95f5730618010de5cbdeec02c8d0e26a85d53da989e806cd6f0b297772b9c082bae2d2c86b6276848a6
CLIENT_HANDSHAKE_TRAFFIC_SECRET bb058323a16b445da4c69a211dd9fb3718a5574fef4a13e59a00c94eb46b5731 f58ff4ceb0cd5ff5dd668c7f6217896e17eb5a1b3a00a2af0a8a6118d0ab3bc57f9de4cf67a432285db4dcee7b43a1b9
CLIENT_TRAFFIC_SECRET_0 bb058323a16b445da4c69a211dd9fb3718a5574fef4a13e59a00c94eb46b5731 5312948edcc68e012acd1622c0f2191bb36b2ed1b26a4d76a7b6dd8df2a807239fcdae09da5e5581a5198bed25e4588e

In Wireshark you can provide this file (Preference -> Protocols -> TLS -> (Pre)-Master-Secret log filename) and then see something like this:

Here we see a few more details that weren’t visible before:

1. The HTTP request and response, as expected.
2. A TLSv1.3 message to provide a “New Session Ticket”. This allows a client to resume this session in a future connection (e.g. reducing the cost of a handshake).

This was just a quick “anatomical exercise”. I won’t dwell too much on TLS, but it’s ubiquitous and so worth getting familiar with. For a more detailed dive into how TLS drives the underlying TCP connection, see The Illustrated TLS 1.3 Connection.