CHAPTER 6

A Usable, Secure Communications Protocol: Client-Side TLS

Armed with symmetric encryption to protect sensitive data from eavesdroppers, public-key encryption to exchange keys securely over an insecure medium, message authentication to ensure message integrity, and certificates and their digital signatures to establish trust, it's possible to create a secure protocol that operates over an insecure line without any prior interaction between parties. This is actually pretty amazing when you think about it. You can assume that anybody who's interested in snooping on your traffic has full and complete access to it. Nevertheless, it's possible to securely send data such that only the intended recipient can read it, and be assured, within reason, that you're communicating with the intended recipient and not an impostor.

Even with all the pieces in place, though, it's possible to get this subtly wrong. This is why the TLS protocol was developed — even if you use the strongest cryptography, key exchange, MAC and signature algorithms available, you can still leave yourself vulnerable by improper use of random numbers, improper seeding of random number generation, improper verification of parameters, and a lot of other, subtle, easy-to-overlook flaws. TLS was designed as a standard for secure communications. You must, of course, use strong, secure cryptographic algorithms; the best way to ensure this is to use standard algorithms that were designed and have been thoroughly reviewed by security professionals for years. To ensure that you're using them correctly, your best bet is to also follow a standard protocol that was also designed and has been thoroughly reviewed by security professionals for years.

Implementing the TLS 1.0 Handshake (Client Perspective)

As much as possible, TLS aims to be completely transparent to the upper-layer protocol. Effectively, this means that it tries to be completely transparent to the application programmer; the application programmer implements the protocol in question as if TLS was not being used. As long as nothing goes wrong, TLS succeeds admirably in this goal; although, as you'll see, if something does go wrong, everything fails miserably and the developer is left scratching his head, trying to figure out what he missed.

Of course, TLS can't be completely transparent. The application must indicate in some way that it wants to negotiate a secure channel. Perhaps surprisingly, TLS doesn't specify how the application should do this nor does it even provide any guidance. Remembering that SSL was initially developed as an add-on to HTTP, this makes some sense. The protocol designers weren't thinking about applicability to other protocols at the time. In fact, they didn't even specify how to use HTTP with SSL, assuming that there was only way to do so. It actually wasn't until 2000 that Eric Rescorla finally drafted RFC 2818 that describes how it should be done.

TLS requires that the handshake — a secure key exchange — takes place before it can protect anything. Effectively the question is when the handshake should take place; anything that's transmitted before the handshake is complete is transmitted in plaintext and is theoretically interceptable. HTTPS takes an extreme position on this. The very first thing that must take place on the channel is the TLS handshake; no HTTP data can be transmitted until the handshake is complete.

You can probably spot a problem with this approach. HTTP expects the very first byte(s) on the connection to be an HTTP command such as GET, PUT, POST, and so on. The client has to have some way of warning the server that it's going to start with a TLS negotiation rather than a plaintext HTTP command. The solution adopted by HTTPS is to require secure connections to be established on a separate port. If the client connects on port 80, the next expected communication is a valid HTTP command. If the client connects on port 443, the next expected communication is a TLS handshake after which, if the handshake is successful, an encrypted, authenticated valid HTTP command is expected.

Adding TLS Support to the HTTP Client

To add TLS support to the HTTP client developed in Chapter 1, you define four new top-level functions as shown in Listing 6-1.

Listing 6-1: "tls.h" top-level function prototypes

/**
 * Negotiate an TLS channel on an already-established connection
 * (or die trying).
 * @return 1 if successful, 0 if not.
 */
int tls_connect( int connection,
                TLSParameters *parameters );

/**
 * Send data over an established TLS channel.  tls_connect must already
 * have been called with this socket as a parameter.
 */
int tls_send( int connection,
             const char *application_data,
             int length,
             int options,
             TLSParameters *parameters );
/**
 * Received data from an established TLS channel.
 */
int tls_recv( int connection,
             char *target_buffer,
             int buffer_size,
             int options,
             TLSParameters *parameters );

/**
 * Orderly shutdown of the TLS channel (note that the socket itself will

* still be open after this is called).
 */
int tls_shutdown( int connection, TLSParameters *parameters );

The primary "goal" of tls_connect is to fill in the TLSParameters structure that is passed in. It contains, among other things, the negotiated encryption and authentication algorithms, along with the negotiated keys. Because this structure is large and complex, it is built up incrementally throughout the course of this chapter; the bulk of this chapter is dedicated to filling out the tls_connect function and the TLSParamaters structure.

To apply these to an HTTP connection, open it as usual but immediately call tls_connect, which performs a TLS handshake. Afterward, assuming it succeeds, replace all calls to send and recv with tls_send and tls_recv. Finally, just before closing the socket, call tls_shutdown. Note that SSLv2 didn't have a dedicated shutdown function — this opened the connection to subtle attacks.

In order to support HTTPS, the first thing you'll need to do is to modify the main routine in http.c to start with a TLS handshake as shown in Listing 6-2.

Listing 6-2: "https.c" main routine

#define HTTPS_PORT      443
...
int main( int argc, char *argv[ ] )
{
 int client_connection;
 char *host, *path;
 struct hostent *host_name;
 struct sockaddr_in host_address;
 int port = HTTPS_PORT;

 TLSParameters tls_context;
...
 printf( "Connection complete; negotiating TLS parameters\n" );

 if ( tls_connect( client_connection, &tls_context ) )
 {
   fprintf( stderr, "Error: unable to negotiate TLS connection.\n" );
   return 3;
 }

 printf( "Retrieving document: '%s'\n", path );

 http_get( client_connection, path, host, &tls_context );
 display_result( client_connection, &tls_context );

 tls_shutdown( client_connection, &tls_context );
...

Here, http_get and display_result change only slightly, as shown in Listing 6-3; they take an extra parameter indicating the new tls_context, and they call tls_send and tls_recv to send and receive data; otherwise, they're identical to the functions presented in Chapter 1:

Listing 6-3: "https.c" http_get and display_result

int http_get( int connection, const char *path, const char *host,
             TLSParameters *tls_context )
{
 static char get_command[ MAX_GET_COMMAND ];
 sprintf( get_command, "GET /%s HTTP/1.1\r\n", path );
 if ( tls_send( connection, get_command,
      strlen( get_command ), 0, tls_context ) == −1 )
 {
   return −1;
 }

 sprintf( get_command, "Host: %s\r\n", host );
 if ( tls_send( connection, get_command,
      strlen( get_command ), 0, tls_context ) == −1 )
 {
   return −1;
 }

 strcpy( get_command, "Connection: Close\r\n\r\n" );
 if ( tls_send( connection, get_command,
      strlen( get_command ), 0, tls_context ) == −1 )
 {
   return −1;
 }

 return 0;
}
void display_result( int connection, TLSParameters *tls_context )
{
...
 while ( ( received = tls_recv( connection, recv_buf,
           BUFFER_SIZE, 0, tls_context ) ) >= 0 )
 {

   recv_buf[ received ] = '\0';
   printf( "data: %s", recv_buf );
 }
...

Notice that the proxy negotiation part of http_get is missing from Listing 6-3. Negotiating proxies is a major complication for SSL; by now you can probably see why. The proxy performs the HTTP connection on behalf of the client and then returns the results back to it. Unfortunately this is by definition a man-in-the-middle attack. HTTPS can, of course, be extended to work correctly behind a proxy. This topic is revisited in Chapter 10.

Otherwise, this is it. After the tls_connect, tls_send, tls_recv, and tls_shutdown routines are complete, this client is HTTPS-compliant. If you are inclined to extend display_result to parse the HTML response and build a renderable web page, you can do so without giving a single thought to whether or not the connection is secure. If you add support for POST, HEAD, PUT, DELETE, and so on into the client-side implementation, you do so just as if the connection was plaintext; just be sure to call tls_send instead of send. Of course, you should probably extend this to actually pay attention to the protocol and perform a TLS connection only if the user requested "https" instead of "http." I'll leave that as an exercise for you if you're interested.

Understanding the TLS Handshake Procedure

Most of the complexity is in the handshake; after the handshake has been completed, sending and receiving is just a matter of encrypting/decrypting, MAC'ing/verifying data before/after it's received. At a high-level, the handshake procedure is as shown in Figure 6-1.

Figure 6-1: TLS handshake

The client is responsible for sending the client hello that gets the ball rolling and informs the server, at a minimum, what version of the protocol it understands and what cipher suites (cryptography, key exchange, and authentication triples) it is capable of working with. It also transmits a unique random number, which is important to guard against replay attacks and is examined in depth later.

The server selects a cipher suite, generates its own random number, and assigns a session ID to the TLS connection; each connection gets a unique session ID. The server also sends enough information to complete a key exchange. Most often, this means sending a certificate including an RSA public key.

The client is then responsible for completing the key exchange using the information the server provided. At this point, the connection is secured, both sides have agreed on an encryption algorithm, a MAC algorithm, and respective keys. Of course, the whole process is quite a bit more complex than this, but you may want to keep this high-level overview in mind as you read the remainder of this chapter.

TLS Client Hello

Every step in the TLS handshake is responsible for updating some aspect of the TLSParameters structure. As you can probably guess, the most important values are the MAC secret, the symmetric encryption key, and, if applicable, the initialization vector. These are defined in the ProtectionParameters structure shown in Listing 6-4.

Listing 6-4: "tls.h" ProtectionParameters

typedef struct
{
 unsigned char *MAC_secret;
 unsigned char *key;
 unsigned char *IV;
 ...
}
ProtectionParameters;

Tracking the Handshake State in the TLSParameters Structure

TLS actually allows a different MAC secret, key, and IV to be established for the sender and the receiver. Therefore, the TLSParameters structure keeps track of two sets of ProtectionParameters as shown in Listing 6-5.

Listing 6-5: "tls.h" TLSParameters

#define TLS_VERSION_MAJOR 3
#define TLS_VERSION_MINOR 1

#define MASTER_SECRET_LENGTH  48
typedef unsigned char master_secret_type[ MASTER_SECRET_LENGTH ];

#define RANDOM_LENGTH 32
typedef unsigned char random_type[ RANDOM_LENGTH ];

typedef struct
{
 master_secret_type    master_secret;
 random_type           client_random;
 random_type           server_random;

 ProtectionParameters  pending_send_parameters;
 ProtectionParameters  pending_recv_parameters;
 ProtectionParameters  active_send_parameters;
 ProtectionParameters  active_recv_parameters;

 // RSA public key, if supplied
 public_key_info       server_public_key;

 // DH public key, if supplied (either in a certificate or ephemerally)
 // Note that a server can legitimately have an RSA key for signing and
 // a DH key for key exchange (e.g. DHE_RSA)
 dh_key                server_dh_key;
...
}
TLSParameters;

TLS 1.0 is SSL version 3.1, as described previously. The pending_send_parameter and pending_recv_parameters are the keys currently being exchanged; the TLS handshake fills these out along the way, based on the computed master secret. The master secret, server random, and client random values are likewise provided by various hello and key exchange messages; the public keys' purpose ought to be clear to you by now.

What about this active_send_parameters and active_recv_parameters? After the TLS handshake is complete, the pending parameters become the active parameters, and when the active parameters are non-null, the parameters are used to protect the channel. Separating them this way simplifies the code; you could get away with a single set of send and recv parameters in the TLSParameters structure, but you'd have to keep track of a lot more state in the handshake code.

Both the TLSParameters and ProtectionParameters structures are shown partially filled out in Listings 6-4 and 6-5; you add to them along the way as you develop the client-side handshake routine.

As always, you need a couple of initialization routines, shown in Listing 6-6.

Listing 6-6: "tls.c" init_parameters

static void init_protection_parameters( ProtectionParameters *parameters )
{
 parameters->MAC_secret = NULL;
 parameters->key = NULL;
 parameters->IV = NULL;
...
}
static void init_parameters( TLSParameters *parameters )
{
 init_protection_parameters( &parameters->pending_send_parameters );
 init_protection_parameters( &parameters->pending_recv_parameters );
 init_protection_parameters( &parameters->active_send_parameters );
 init_protection_parameters( &parameters->active_recv_parameters );

 memset( parameters->master_secret, '\0', MASTER_SECRET_LENGTH );
 memset( parameters->client_random, '\0', RANDOM_LENGTH );
 memset( parameters->server_random, '\0', RANDOM_LENGTH );
...
}

So, tls_connect, shown partially in Listing 6-7, starts off by calling init_parameters.

Listing 6-7: "tls.c" tls_connect

/**
 * Negotiate TLS parameters on an already-established socket.
 */
int tls_connect( int connection,
                TLSParameters *parameters )
{
 init_parameters( parameters );
 // Step 1. Send the TLS handshake "client hello" message
 if ( send_client_hello( connection, parameters ) < 0 )
 {
   perror( "Unable to send client hello" );
   return 1;
 }
...

Recall from the overview that the first thing the client should do is send a client hello message. The structure of this message is defined in Listing 6-8.

Listing 6-8: "tls.h" client hello structure

typedef struct
{
 unsigned char major, minor;
}
ProtocolVersion;

typedef struct
{
 unsigned int  gmt_unix_time;
 unsigned char  random_bytes[ 28 ];
}
Random;

/**
 * Section 7.4.1.2
 */
typedef struct
{
 ProtocolVersion client_version;
 Random random;
 unsigned char session_id_length;
 unsigned char *session_id;
 unsigned short cipher_suites_length;
 unsigned short *cipher_suites;
 unsigned char compression_methods_length;
 unsigned char *compression_methods;
}
ClientHello;

Listing 6-9 shows the first part of the send_client_hello function, which is responsible for filling out a ClientHello structure and sending it on to the server.

Listing 6-9: "tls.c" send_client_hello

/**
 * Build and submit a TLS client hello handshake on the active
 * connection.  It is up to the caller of this function to wait
 * for the server reply.
 */
static int send_client_hello( int connection, TLSParameters *parameters )
{
 ClientHello       package;
 unsigned short    supported_suites[ 1 ];
 unsigned char     supported_compression_methods[ 1 ];
 int               send_buffer_size;
 char              *send_buffer;
 void              *write_buffer;
 time_t            local_time;
 int               status = 1;

 package.client_version.major = TLS_VERSION_MAJOR;
 package.client_version.minor = TLS_VERSION_MINOR;
 time( &local_time );
 package.random.gmt_unix_time = htonl( local_time );
 // TODO - actually make this random.
 // This is 28 bytes, but client random is 32 - the first four bytes of

// "client random" are the GMT unix time computed above.
 memcpy( parameters->client_random, &package.random.gmt_unix_time, 4 );
 memcpy( package.random.random_bytes, parameters->client_random + 4, 28 );
 package.session_id_length = 0;
 package.session_id = NULL;
 // note that this is bytes, not count.
 package.cipher_suites_length = htons( 2 );
 supported_suites[ 0 ] = htons( TLS_RSA_WITH_3DES_EDE_CBC_SHA );
 package.cipher_suites = supported_suites;
 package.compression_methods_length = 1;
 supported_compression_methods[ 0 ] = 0;
 package.compression_methods = supported_compression_methods;

The session ID is left empty, indicating that a new session is being requested (session reuse is examined in Chapter 8). To complete the ClientHello structure, the supported cipher suites and compression methods are indicated. Only one of each is given here: For the cipher suite, it's RSA key exchange; 3DES (EDE) with CBC for encryption; and SHA-1 for MAC. The compression method selected is "no compression." TLS allows the client and sender to agree to compress the stream before encrypting.

You may be wondering, legitimately, what compression has to do with security. Nothing, actually — however, it was added to TLS and, at the very least, both sides have to agree not to compress. If the stream is going to be compressed, however, it is important that compression be applied before encryption. One property of secure ciphers is that they specifically not be compressible, so if you try to compress after encrypting, it will be too late.

Describing Cipher Suites

So, what about this TLS_RSA_WITH_3DES_EDE_CBC_SHA value? Strictly speaking, it's not always safe to "mix and match" encryption functions with key exchange and MAC functions, so TLS defines them in triples rather than allowing the two sides to select them à la carte. As a result, each allowed triple has a unique identifier: TLS_RSA_WITH_3DES_EDE_CBC_SHA is 10 or 0x0A hex. Go ahead and define a CipherSuiteIdentifier enumeration as shown in Listing 6-10.

Listing 6-10: "tls.h" CipherSuiteIdentifier list

typedef enum
{
 TLS_NULL_WITH_NULL_NULL               = 0x0000,

TLS_RSA_WITH_NULL_MD5                 = 0x0001,
 TLS_RSA_WITH_NULL_SHA                 = 0x0002,
 TLS_RSA_EXPORT_WITH_RC4_40_MD5        = 0x0003,
 TLS_RSA_WITH_RC4_128_MD5              = 0x0004,
 TLS_RSA_WITH_RC4_128_SHA              = 0x0005,
 TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5    = 0x0006,
 TLS_RSA_WITH_IDEA_CBC_SHA             = 0x0007,
 TLS_RSA_EXPORT_WITH_DES40_CBC_SHA     = 0x0008,
 TLS_RSA_WITH_DES_CBC_SHA              = 0x0009,
 TLS_RSA_WITH_3DES_EDE_CBC_SHA         = 0x000A,
 ...
} CipherSuiteIdentifier;

Notice the NULL cipher suites 0, 1 and 2. TLS_NULL_WITH_NULL_NULL indicates that there's no encryption, no MAC and no key exchange. This is the default state for a TLS handshake — the state it starts out in. Cipher suites 1 and 2 allow a non-encrypted, but MAC'ed, cipher suite to be negotiated. This can actually be pretty handy when you're trying to debug something and you don't want to have to decrypt what you're trying to debug. Unfortunately for the would-be debugger, for obvious security reasons, most servers won't allow you to negotiate this cipher suite by default.

There's no particular rhyme or reason to the identifiers assigned to the various cipher suites. They're just a sequential list of every combination that the writers of the specification could think of. They're not even grouped together meaningfully; the RSA key exchange cipher suites aren't all in the same place because after the specification was drafted, new cipher suites that used the RSA key exchange method were identified. It would certainly have been nicer, from an implementer's perspective, if they had allocated, say, three bits to identify the key exchange, five bits to identify the symmetric cipher, two for the MAC, and so on.

Additional cipher suites are examined later on. For now, you're just writing a client that understands only 3DES, RSA, and SHA-1.

Flattening and Sending the Client Hello Structure

Now that the ClientHello message has been built, it needs to be sent on. If you look at RFC 2246, which describes TLS, you see that the formal description of the client hello message looks an awful lot like the C structure defined here. You may be tempted to try to just do something like this:

send( connection, ( void * ) &package, sizeof( package ), 0 );

This is tempting, but your compiler thwarts you at every turn, expanding some elements, memory-aligning others, and generally performing unexpected optimizations that cause your code to run faster and work better (the nerve!). Although it is possible to include enough compiler directives to force this structure to appear in memory just as it needs to appear on the wire, you'd be, at the very least, locking yourself into a specific platform. As a result, you're better off manually flattening the structure to match the expected wire-level interface as shown in Listing 6-11:

Listing 6-11: "tls.c" send_client_hello (continued in Listing 6-13)

// Compute the size of the ClientHello message after flattening.
 send_buffer_size = sizeof( ProtocolVersion ) +
    sizeof( Random ) +
    sizeof( unsigned char ) +
    ( sizeof( unsigned char ) * package.session_id_length ) +
    sizeof( unsigned short ) +
    ( sizeof( unsigned short ) * 1 ) +
    sizeof( unsigned char ) +
    sizeof( unsigned char );

 write_buffer = send_buffer = ( char * ) malloc( send_buffer_size );

 write_buffer = append_buffer( write_buffer, ( void * )
    &package.client_version.major, 1 );
 write_buffer = append_buffer( write_buffer, ( void * )
    &package.client_version.minor, 1 );
 write_buffer = append_buffer( write_buffer, ( void * )
    &package.random.gmt_unix_time, 4 );
 write_buffer = append_buffer( write_buffer, ( void * )
    &package.random.random_bytes, 28 );
 write_buffer = append_buffer( write_buffer, ( void * )
    &package.session_id_length, 1 );
 if ( package.session_id_length > 0 )
 {
   write_buffer = append_buffer( write_buffer,
     ( void * )package.session_id,
     package.session_id_length );
 }
 write_buffer = append_buffer( write_buffer,
    ( void * ) &package.cipher_suites_length, 2 );
 write_buffer = append_buffer( write_buffer,
   ( void * ) package.cipher_suites, 2 );
 write_buffer = append_buffer( write_buffer,
   ( void * ) &package.compression_methods_length, 1 );
 if ( package.compression_methods_length > 0 )
 {
   write_buffer = append_buffer( write_buffer,
      ( void * ) package.compression_methods, 1 );
 }

The append_buffer function, in Listing 6-12, is a convenience routine designed to be called incrementally as in Listing 6-11.

Listing 6-12: "tls.c" append buffer

/**
 * This is just like memcpy, except it returns a pointer to dest + n instead
 * of dest, to simplify the process of repeated appends to a buffer.
 */
static char *append_buffer( char *dest, char *src, size_t n )
{
 memcpy( dest, src, n );
 return dest + n;
}

This flattened structure is illustrated in Figure 6-2.

Figure 6-2: Client hello structure

Finally, the client hello is sent off in Listing 6-13:

Listing 6-13: "tls.c" send_client_hello (continued from Listing 6-11)

assert( ( ( char * ) write_buffer - send_buffer ) == send_buffer_size );

 status = send_handshake_message( connection, client_hello, send_buffer,
    send_buffer_size );

 free( send_buffer );

 return status;
}

Notice that send still isn't called. Instead, you invoke send_handshake_message. Like TCP and IP, and network programming in general, TLS is an onion-like nesting of headers. Each handshake message must be prepended with a header indicating its type and length. The definition of the handshake header is shown in Listing 6-14.

Listing 6-14: "tls.h" handshake structure

/**
 * Handshake message types (section 7.4)
 */
typedef enum
{
 hello_request = 0,
 client_hello = 1,
 server_hello = 2,
 certificate = 11,
 server_key_exchange = 12,
 certificate_request = 13,
 server_hello_done = 14,
 certificate_verify = 15,
 client_key_exchange = 16,
 finished = 20
}
HandshakeType;

/**
 * Handshake record definition (section 7.4)
 */
typedef struct
{
 unsigned char         msg_type;
 unsigned int        length;       // 24 bits(!)
}
Handshake;

This structure is illustrated in Figure 6-3.

Figure 6-3: TLS handshake header

The send_handshake_message function that prepends this header to a handshake message is shown in Listing 6-15.

Listing 6-15: "tls.c" send_handshake_message

static int send_handshake_message( int connection,
                                  int msg_type,
                                  const unsigned char *message,
                                  int message_len )
{
 Handshake     record;
 short         send_buffer_size;
 unsigned char *send_buffer;

int           response;

 record.msg_type = msg_type;
 record.length = htons( message_len ) << 8; // To deal with 24-bits...
 send_buffer_size = message_len + 4; // space for the handshake header

 send_buffer = ( unsigned char * ) malloc( send_buffer_size );
 send_buffer[ 0 ] = record.msg_type;
 memcpy( send_buffer + 1, &record.length, 3 );
 memcpy( send_buffer + 4, message, message_len );

 response = send_message( connection, content_handshake,
                      send_buffer, send_buffer_size );

 free( send_buffer );

 return response;
}

This would be a bit simpler except that, for some strange reason, the TLS designers mandated that the length of the handshake message must be given in a 24-bit field, which no compiler that I'm aware of can generate. Of course, on a big-endian machine, this wouldn't be a problem; just truncate the high-order byte of a 32-bit integer and you'd have a 24-bit integer. Unfortunately, most general purpose computers these days are little-endian, so it's necessary to convert it and then truncate it.

But send_handshake_message still doesn't call send! TLS mandates not only that every handshake message be prepended with a header indicating its type and length, but that every message, including the already-prepended handshake messages, be prepended with yet another header indicating its type and length!

So, finally, define yet another header structure and some supporting enumerations in Listing 6-16.

Listing 6-16: "tls.h" TLSPlaintext header

/** This lists the type of higher-level TLS protocols that are defined */
typedef enum {
 content_change_cipher_spec = 20,
 content_alert = 21,
 content_handshake = 22,
 content_application_data = 23
}
ContentType;

typedef enum { warning = 1, fatal = 2 } AlertLevel;

/**
 * Enumerate all of the error conditions specified by TLS.

*/
typedef enum
{
 close_notify = 0,
 unexpected_message = 10,
 bad_record_mac = 20,
 decryption_failed = 21,
 record_overflow = 22,
 decompression_failure = 30,
 handshake_failure = 40,
 bad_certificate = 42,
 unsupported_certificate = 43,
 certificate_revoked = 44,
 certificate_expired = 45,
 certificate_unknown = 46,
 illegal_parameter = 47,
 unknown_ca = 48,
 access_denied = 49,
 decode_error = 50,
 decrypt_error = 51,
 export_restriction = 60,
 protocol_version = 70,
 insufficient_security = 71,
 internal_error = 80,
 user_canceled = 90,
 no_renegotiation = 100
}
AlertDescription;

typedef struct
{
 unsigned char level;
 unsigned char description;
}
Alert;

/**
 * Each packet to be encrypted is first inserted into one of these structures.
 */
typedef struct
{
 unsigned char   type;
 ProtocolVersion version;
 unsigned short  length;
}
TLSPlaintext;

There are four types of TLS messages defined: handshake messages, alerts, data, and "change cipher spec," which is technically a handshake message, but is broken out for specific implementation types that are examined later. Also, the protocol version is included on every packet.

The TLS Message header is illustrated in Figure 6-4.

Figure 6-4: TLS Message header

Notice that this header is added to every packet that is sent over a TLS connection, not just the handshake messages. If, after handshake negotiation, either side receives a packet whose first byte is not greater than or equal to 20 and less than or equal to 23 then something has gone wrong, and the whole connection should be terminated.

Finally, you need one last send function that prepends this header on top of the handshake message as shown in Listing 6-17.

Listing 6-17: "tls.c" send_message

static int send_message( int connection,
                        int content_type,
                        const unsigned char *content,
                        short content_len )
{
 TLSPlaintext header;
 unsigned char *send_buffer;
 int send_buffer_size;

 send_buffer_size = content_len;
 send_buffer_size +=5;

 send_buffer = ( unsigned char * ) malloc( send_buffer_size );

 header.type = content_type;
 header.version.major = TLS_VERSION_MAJOR;
 header.version.minor = TLS_VERSION_MINOR;
 header.length = htons( content_len );
 send_buffer[ 0 ] = header.type;
 send_buffer[ 1 ] = header.version.major;
 send_buffer[ 2 ] = header.version.minor;
 memcpy( send_buffer + 3, &header.length, sizeof( short ) );
 memcpy( send_buffer + 5, content, content_len );

if ( send( connection, ( void * ) send_buffer,
      send_buffer_size, 0 ) < send_buffer_size )
 {
   return −1;
 }

 free( send_buffer );

 return 0;
}

At this point, the actual socket-level send function is called. Now the client hello message, with its handshake message header, with its TLS header, are sent to the server for processing. After all of this prepending, the final wire-level structure is as shown in Figure 6-5.

Figure 6-5: TLS Client Hello with all headers

TLS Server Hello

The server should now select one of the supported cipher suites and respond with a server hello response. The client is required to block, waiting for an answer; nothing else can happen on this socket until the server responds. Expand tls_connect:

// Step 2. Receive the server hello response
 if ( receive_tls_msg( connection, parameters ) < 0 )
 {
   perror( "Unable to receive server hello" );
   return 2;
 }

The function receive_tls_msg, as you can probably imagine, is responsible for reading a packet off the socket, stripping off the TLS header, stripping off the handshake header if the message is a handshake message, and processing the message itself. This is shown in Listing 6-18.

Listing 6-18: "tls.c" receive_tls_msg

/**
 * Read a TLS packet off of the connection (assuming there's one waiting)
 * and try to update the security parameters based on the type of message
 * received.  If the read times out, or if an alert is received, return an error
 * code; return 0 on success.
 * TODO - assert that the message received is of the type expected (for example,
 * if a server hello is expected but not received, this is a fatal error per
 * section 7.3).
 * returns −1 if an error occurred (this routine will have sent an
 * appropriate alert). Otherwise, return the number of bytes read if the packet
 * includes application data; 0 if the packet was a handshake.  −1 also
 * indicates that an alert was received.
 */
static int receive_tls_msg( int connection,
                           TLSParameters *parameters )
{
 TLSPlaintext  message;
 unsigned char *read_pos, *msg_buf;
 unsigned char header[ 5 ];  // size of TLSPlaintext
 int bytes_read, accum_bytes;

 // STEP 1 - read off the TLS Record layer
 if ( recv( connection, header, 5, 0 ) <= 0 )
 {
   // No data available; it's up to the caller whether this is an error or not.
   return −1;
 }

 message.type = header[ 0 ];
 message.version.major = header[ 1 ];
 message.version.minor = header[ 2 ];
 memcpy( &message.length, header + 3, 2 );
 message.length = htons( message.length );

Adding a Receive Loop

First, the TLSPlaintext header is read from the connection and validated. The error handling here leaves a bit to be desired, but ignore that for the time being. If everything goes correctly, message.length holds the number of bytes remaining in the current message. Because TCP doesn't guarantee that all bytes are available right away, it's necessary to enter a receive loop in Listing 6-19:

Listing 6-19: "tls.c" receive_tls_msg (continued in Listing 6-21)

msg_buf = ( char * ) malloc( message.length );

   // keep looping & appending until all bytes are accounted for
   accum_bytes = 0;
   while ( accum_bytes < message.length )
   {
     if ( ( bytes_read = recv( connection, ( void * ) msg_buf,
            message.length - accum_bytes, 0 ) ) <= 0 )
     {
       int status;
       perror( "While reading a TLS packet" );

       if ( ( status = send_alert_message( connection,
              illegal_parameter ) ) )
       {
         free( msg_buf );
         return status;
       }
       return −1;
     }
     accum_bytes += bytes_read;
     msg_buf += bytes_read;
   }

This loop, as presented here, is vulnerable to a denial of service attack. If the server announces that 100 bytes are available but never sends them, the client hangs forever waiting for these bytes. There's not much you can do about this, though. You can (and should) set a socket-level timeout, but if it expires, there's not much point in continuing the connection.

Sending Alerts

Notice that if recv returns an error, a function send_alert_message is invoked. Remember the four types of TLS messages? Alert was one of them. This is how clients and servers notify each other of unexpected conditions. In theory, an alert can be recoverable — expired certificate is defined as an alert, for example — but this poses a problem for the writer of a general-purpose TLS implementation. If the client tells the server that its certificate has expired then in theory the server could present a new certificate that hadn't expired. But why did it send an expired certificate in the first place, if it had one that was current? In general, all alerts are treated as fatal errors.

Alerts are also frustratingly terse. As you can see, if the client wasn't able to receive the entire message, it just returns an illegal parameter with no further context. Although it logs a more detailed reason, the server developer probably doesn't have access to those logs and has no clue what he did wrong. It would certainly be nice if the TLS alert protocol allowed space for a descriptive error message.

send_alert_message is shown in Listing 6-20.

Listing 6-20: "tls.c" send_alert_message

static int send_alert_message( int connection,
                              int alert_code )
{
 char buffer[ 2 ];

 // TODO support warnings
 buffer[ 0 ] = fatal;
 buffer[ 1 ] = alert_code;

 return send_message( connection, content_alert, buffer, 2 );
}

By reusing the send_message routine from above, sending an alert message is extremely simple.

Parsing the Server Hello Structure

Assuming nothing went wrong, the message has now been completely read from the connection and is contained in msg_buf. For the moment, the only type of message you're interested in is content_handshake, whose parsing is shown in Listing 6-21:

Listing 6-21: "tls.c" receive_tls_msg (continued from Listing 6-19)

read_pos = msg_buf;

 if ( message.type == content_handshake )
 {
   Handshake handshake;

   // Now, read the handshake type and length of the next packet
   // TODO - this fails if the read, above, only got part of the message
   read_pos = read_buffer( ( void * ) &handshake.msg_type,
              ( void * ) read_pos, 1 );
   handshake.length = read_pos[ 0 ] << 16 | read_pos[ 1 ] << 8 | read_pos[ 2 ];

   read_pos += 3;

   // TODO check for negative or unreasonably long length
   // Now, depending on the type, read in and process the packet itself.
   switch ( handshake.msg_type )
   {
     // Client-side messages
     case server_hello:

read_pos = parse_server_hello( read_pos, handshake.length,
           parameters );
       if ( read_pos == NULL )  /* error occurred */
       {
         free( msg_buf );
         send_alert_message( connection, illegal_parameter );
         return −1;
       }
       break;
     default:
       printf( "Ignoring unrecognized handshake message %d\n",
          handshake.msg_type );
       // Silently ignore any unrecognized types per section 6
       // TODO However, out-of-order messages should result in a fatal alert
       // per section 7.4
       read_pos += handshake.length;
       break;
   }
 }
 else
 {
   // Ignore content types not understood, per section 6 of the RFC.
   printf( "Ignoring non-recognized content type %d\n", message.type );
 }

 free( msg_buf );

 return message.length;
}

As I'm sure you can imagine, you fill this out quite a bit more throughout this chapter. For now, though, just focus on the parse_server_hello function. The Server Hello message is illustrated in Figure 6-6.

Figure 6-6: Server Hello structure

As with the client hello, go ahead and define a structure to hold its value in Listing 6-22.

Listing 6-22: "tls.h" ServerHello structure

typedef struct
{
 ProtocolVersion   server_version;
 Random            random;
 unsigned char     session_id_length;
 unsigned char     session_id[ 32 ]; // technically, this len should be dynamic.
 unsigned short    cipher_suite;
 unsigned char     compression_method;
}
ServerHello;

Because the TLSParameters were passed into the receive_tls_message function, the parse_server_hello can go ahead and update the ongoing state as it's parsed, as in Listing 6-23.

Listing 6-23: "tls.c" parse_server_hello

static char *parse_server_hello( char *read_pos,
                                int pdu_length,
                                TLSParameters *parameters )
{
 ServerHello hello;

 read_pos = read_buffer( ( void * ) &hello.server_version.major,
   ( void * ) read_pos, 1 );
 read_pos = read_buffer( ( void * ) &hello.server_version.minor,
   ( void * ) read_pos, 1 );
 read_pos = read_buffer( ( void * ) &hello.random.gmt_unix_time,
   ( void * ) read_pos, 4 );
 // *DON'T* put this in host order, since it's not used as a time!  Just
 // accept it as is
 read_pos = read_buffer( ( void * ) hello.random.random_bytes,
   ( void * ) read_pos, 28 );
 read_pos = read_buffer( ( void * ) &hello.session_id_length,
    ( void * ) read_pos, 1 );
 read_pos = read_buffer( ( void * ) hello.session_id,
    ( void * ) read_pos, hello.session_id_length );
 read_pos = read_buffer( ( void * ) &hello.cipher_suite,
    ( void * ) read_pos, 2 );
 hello.cipher_suite = ntohs( hello.cipher_suite );

 // TODO check that these values were actually in the client hello
 // list.
 parameters->pending_recv_parameters.suite = hello.cipher_suite;
 parameters->pending_send_parameters.suite = hello.cipher_suite;

 read_pos = read_buffer( ( void * ) &hello.compression_method,
    ( void * ) read_pos, 1 );
 if ( hello.compression_method != 0 )

{
   fprintf( stderr, "Error, server wants compression.\n" );
   return NULL;
 }

 // TODO - abort if there's more data here than in the spec (per section
 // 7.4.1.2, forward compatibility note)
 // TODO - abort if version < 3.1 with "protocol_version" alert error

 // 28 random bytes, but the preceding four bytes are the reported GMT unix
 // time
 memcpy( ( void * ) parameters->server_random, &hello.random.gmt_unix_time, 4 );
 memcpy( ( void * ) ( parameters->server_random + 4 ),
    ( void * ) hello.random.random_bytes, 28 );
 return read_pos;
}

Note that if the server asked for compression, this function returns null because this implementation doesn't support compression. This is recognized by the calling routine and is used to generate an alert. Here the terseness of the TLS alert protocol shows. If the server asked for compression, it just gets back a nondescript illegal parameter but receives no indication of which parameter was illegal. It certainly would be more robust if you were allowed to tell it which parameter you were complaining about. This is generally not a problem for users of TLS software — if you get an illegal parameter while using, say, a browser, that means that the programmer of the browser did something wrong — but is a hassle when developing/testing TLS software like the library developed in this book. When developing, therefore, it's best to test against a client or server with its debug levels set to maximum so that if you do get back an illegal parameter (or any other nondescript alert message), you can go look at the server logs to see what you actually did wrong.

This routine stores the server random, of course, because it is needed later on in the master secret computation. Primarily, though, it sets the values pending_send_parameters and pending_recv_parameters with the selected suite. Expand the definition of ProtectionParameters to keep track of this in Listing 6-24.

Listing 6-24: "tls.h" ProtectionParameters with cipher suite

typedef struct
{
 unsigned char *MAC_secret;
 unsigned char *key;
 unsigned char *IV;
 CipherSuiteIdentifier suite;
}
ProtectionParameters;

Recall that CipherSuiteIdentifier was defined as part of the client hello.

parse_server_hello is something of the opposite of send_client_hello and it even makes use of a function complementary to append_buffer, shown in Listing 6-25.

Listing 6-25: "tls.c" read_buffer

static char *read_buffer( char *dest, char *src, size_t n )
{
 memcpy( dest, src, n );
 return src + n;
}

Reporting Server Alerts

What if the server doesn't happen to support TLS_RSA_WITH_3DES_EDE_CBC_SHA (or any of the cipher suites on the list the client sends)? It doesn't return a server_hello at all; instead it responds with an alert message. You need to be prepared to deal with alerts at any time, so extend receive_tls_message to handle alerts as shown in Listing 6-26.

Listing 6-26: "receive_tls_message" with alert support

static int receive_tls_msg( int connection,
                           TLSParameters *parameters )
{
...
 if ( message.type == content_handshake )
 {
    ...
 }
 else if ( message.type == content_alert )
 {
   while ( ( read_pos - decrypted_message ) < decrypted_length )
   {
     Alert alert;

     read_pos = read_buffer( ( void * ) &alert.level,
        ( void * ) read_pos, 1 );
     read_pos = read_buffer( ( void * ) &alert.description,
        ( void * ) read_pos, 1 );

     report_alert( &alert );

     if ( alert.level == fatal )
     {
       return −1;
     }
   }
 }

Notice that alert level is checked. If the server specifically marks an alert as a fatal, the handshake is aborted; otherwise, the handshake process continues. Effectively this means that this implementation is ignoring warnings, which is technically a Bad Thing. However, as noted previously, there's really not much that can be done about the few alerts defined as warnings anyway. In any case, the alert itself is written to stdout via the helper function report_alert in Listing 6-27.

Listing 6-27: "tls.c" report_alert

static void report_alert( Alert *alert )
{
 printf( "Alert - " );

 switch ( alert->level )
 {
   case warning:
     printf( "Warning: " );
     break;
   case fatal:
     printf( "Fatal: " );
     break;
   default:
     printf( "UNKNOWN ALERT TYPE %d (!!!): ", alert->level );
     break;
 }

 switch ( alert->description )
 {
   case close_notify:
     printf( "Close notify\n" );
     break;
   case unexpected_message:
     printf( "Unexpected message\n" );
     break;
   case bad_record_mac:
     printf( "Bad Record Mac\n" );
     break;
...
   default:
     printf( "UNKNOWN ALERT DESCRIPTION %d (!!!)\n", alert->description );
     break;
 }

TLS Certificate

According to the handshake protocol, the next message after the server hello ought to be the certificate that both identifies the server and provides a public key for key exchange. The client, then, should accept the server hello and immediately start waiting for the certificate message that follows.

The designers of TLS recognized that it is somewhat wasteful to rigidly separate messages this way, so the TLS format actually allows either side to concatenate multiple handshake messages within a single TLS message. This capability was one of the big benefits of TLS over SSLv2. Remember the TLS header that included a message length which was then followed by the seemingly superfluous handshake header that included essentially the same length? This is why it was done this way; a single TLS header can identify multiple handshake messages, each with its own independent length. Most TLS implementations do take advantage of this optimization, so you must be prepared to handle it.

This slightly complicates the design of receive_tls_msg, though. The client must now be prepared to process multiple handshake messages within a single TLS message. Modify the content_handshake handler to keep processing the TLS message until there are no more handshake messages remaining as in Listing 6-28.

Listing 6-28: "tls.c" receive_tls_msg with multiple handshake support

if ( message.type == content_handshake )
 {
   while ( ( read_pos - decrypted_message ) < decrypted_length )
   {
     Handshake handshake;
     read_pos = read_buffer( ( void * ) &handshake.msg_type,
       ( void * ) read_pos, 1 );
...
     switch ( handshake.msg_type )
     {
...
       case certificate:
         read_pos = parse_x509_chain( read_pos, handshake.length,
           &parameters->server_public_key );
         if ( read_pos == NULL )
         {
           printf( "Rejected, bad certificate\n" );
           send_alert_message( connection, bad_certificate );
           return −1;
         }
         break;

Notice that the call is made, not directly to the parse_x509_certificate function developed in Chapter 5, but to a new function parse_x509_chain. TLS actually allows the server to pass in not just its own certificate, but the signing certificate of its certificate, and the signing certificate of that certificate, and so on, until a top-level, self-signed certificate is reached. It's up to the client to determine whether or not it trusts the top-level certificate. Of course, each certificate after the first should be checked to ensure that it includes the "is a certificate authority" extension described in Chapter 5 as well.

Therefore, the TLS certificate handshake message starts off with the length of the certificate chain so that the receiver knows how many bytes of certificate follow. If you are so inclined, you can infer this from the length of the handshake message and the ASN.1 structure declaration that begins each certificate, but explicitness can never hurt.

Certificate chain parsing, then, consists of reading the length of the certificate chain from the message, and then reading each certificate in turn, using each to verify the last. Of course, the first must also be verified for freshness and domain name validity. At a bare minimum, though, in order to complete a TLS handshake, you need to read and store the public key contained within the certificate because it's required to perform the key exchange. Listing 6-29 shows a bare-bones certificate chain parsing routine that doesn't actually verify the certificate signatures or check validity parameters.

Listing 6-29: "x509.c" parse_x509_chain

/**
 * This is called by "receive_server_hello" when the "certificate" PDU
 * is encountered.  The input to this function should be a certificate chain.
 * The most important certificate is the first one, since this contains the
 * public key of the subject as well as the DNS name information (which
 * has to be verified against).
 * Each subsequent certificate acts as a signer for the previous certificate.
 * Each signature is verified by this function.
 * The public key of the first certificate in the chain will be returned in
 * "server_public_key" (subsequent certificates are just needed for signature
 * verification).
 * TODO verify signatures.
 */
char *parse_x509_chain( unsigned char *buffer,
                       int pdu_length,
                       public_key_info *server_public_key )
{
 int pos;
 signed_x509_certificate certificate;
 unsigned int chain_length, certificate_length;
 unsigned char *ptr;
 ptr = buffer;

 pos = 0;

 // TODO this won't work on a big-endian machine
 chain_length = ( *ptr << 16 ) | ( *( ptr + 1 ) << 8 ) | ( *( ptr + 2 ) );
 ptr += 3;

 // The chain length is actually redundant since the length of the PDU has
 // already been input.
 assert ( chain_length == ( pdu_length - 3 ) );

while ( ( ptr - buffer ) < pdu_length )
 {
   // TODO this won't work on a big-endian machine
   certificate_length = ( *ptr << 16 ) | ( *( ptr + 1 ) << 8 ) |
     ( *( ptr + 2 ) );
   ptr += 3;

    init_x509_certificate( &certificate );

   parse_x509_certificate( ( void * ) ptr, certificate_length, &certificate );
   if ( !pos++ )
   {
     server_public_key->algorithm =
       certificate.tbsCertificate.subjectPublicKeyInfo.algorithm;
     switch ( server_public_key->algorithm )
     {
       case rsa:
         server_public_key->rsa_public_key.modulus = ( huge * ) malloc( sizeof(
huge ) );
         server_public_key->rsa_public_key.exponent = ( huge * ) malloc( sizeof(
huge ) );
         set_huge( server_public_key->rsa_public_key.modulus, 0 );
         set_huge( server_public_key->rsa_public_key.exponent, 0 );
         copy_huge( server_public_key-> rsa_public_key.modulus,
           certificate.tbsCertificate.subjectPublicKeyInfo.
           rsa_public_key.modulus );
         copy_huge( server_public_key-> rsa_public_key.exponent,
           certificate.tbsCertificate.subjectPublicKeyInfo.
           rsa_public_key.exponent );
         break;
       default:
         break;
     }
   }

   ptr += certificate_length;

   // TODO compute the hash of the certificate so that it can be validated by
   // the next one

   free_x509_certificate( &certificate );
 }

 return ptr;
}

This blindly accepts whatever certificate is presented by the server. It doesn't check the domain name parameter of the subject name, doesn't check to see that it's signed by a trusted certificate authority, and doesn't even verify that the validity period of the certificate contains the current date. Clearly, an industrial-strength TLS implementation needs to do at least all of these things.

TLS Server Hello Done

After the certificate the server should send a hello done message — at least that's what happens in the most common type of TLS handshake being examined here. This indicates that the server will not send any more unencrypted handshake messages on this connection.

This may seem surprising, but if you think about it, there's nothing more for the server to do. It has chosen a cipher suite acceptable to the client and provided enough information — the public key — to complete a key exchange in that cipher suite. It's incumbent on the client to now come up with a key and exchange it. Parsing the server hello done message is trivial, as in Listing 6-30.

Listing 6-30: "tls.c" receive_tls_message with server hello done support

switch ( handshake.msg_type )
     {
...
       case server_hello_done:
         parameters->server_hello_done = 1;
         break;

As you can see, there's really nothing there; the server hello done message is just a marker and contains no data. Note that this will almost definitely be piggy-backed onto a longer message that contains the server hello and the server certificate.

This routine just sets a flag indicating that the server hello done message has been received. Add this flag to TLSParameters as shown in Listing 6-31; it's used internally to track the state of the handshake.

Listing 6-31: "tls.h" TLSParameters with state tracking included

typedef struct
{
...
 int                   server_hello_done;
}
TLSParameters;

Finally, recall that this whole process was being controlled by tls_connect. It sent a client hello message and then received the server hello. Due to piggy-backing of handshake messages, though, that call to receive probably picked up all three expected messages and processed them, culminating in the setting of server_hello_done. This isn't guaranteed, though; the server could legitimately split these up into three separate messages (the server you develop in the next chapter does this, in fact). To handle either case, modify tls_connect to keep receiving TLS messages until server_hello_done is set, as in Listing 6-32.

Listing 6-32: "tls.c" tls_connect multiple handshake messages

// Step 2. Receive the server hello response (will also have gotten
 // the server certificate along the way)
 parameters->server_hello_done = 0;
 while ( !parameters->server_hello_done )
 {
   if ( receive_tls_msg( connection, parameters ) < 0 )
   {
     perror( "Unable to receive server hello" );
     return 2;
   }
 }

TLS Client Key Exchange

Now it's time for the client to do a key exchange, which is the most critical part of the whole TLS handshake. You might reasonably expect that if RSA is used as a key exchange method then the client selects a set of keys, encrypts them, and sends them on. If DH was used as a key exchange method, both sides would agree on Z and that would be used as the key. As it turns out, however, TLS mandates a bit more complexity here; the key exchange is used to exchange a premaster secret, which is expanded using a pseudo-random function into a master secret which is used for keying material. This procedure guards against weaknesses in the client's key generation routines.

Sharing Secrets Using TLS PRF (Pseudo-Random Function)

In several places during the TLS negotiation, the algorithm calls for a lot of data to be generated deterministically so that both sides agree on the same result, based on a seed. This process is referred to as pseudo-random, just like the software pseudo-random generator that's built into every C implementation. TLS has a fairly complex pseudo-random function called the PRF that generates data from a seed in such a way that two compliant implementations, given the same seed data, generate the same data, but that the output is randomly distributed and follows no observable pattern.

It should come as no surprise at this point that this pseudo-random function is based on secure hash algorithms, which deterministically generate output from input in a non-predictable way. TLS's PRF is actually based on the HMAC algorithm. It takes as input three values: the seed, a label, and a secret. The seed and the label are both used as input to the HMAC algorithm.

The PRF for TLS v1.0 involves both MD5 and SHA-1 (and the use of these specific hash algorithms is hard-coded into the specification). MD5 and SHA-1 are each used, along with the HMAC algorithm specified in Chapter 4, to generate an arbitrary quantity of output independently. Then the results of both the MD5 HMAC and the SHA HMAC are XORed together to produce the final result. The secret is split up so that the MD5 routine gets the first half and the SHA routine gets the second half:

Consider using the triple ("abcd", "efgh", "ijkl" ) to generate 40 bytes of output through the PRF as shown in Figure 6-7.

Figure 6-7: TLS's pseudo-random function

So what are these P_MD5 and P_SHA1 blocks that are XORed together to produce the final result? Well, if you recall from Chapter 4, MD5 produces 16 bytes of output, regardless of input length, and SHA-1 produces 20. If you want to produce an arbitrary amount of data based on the secret, the label, and the seed using these hashing algorithms, you have to call them more than once. Of course, you have to call them with different data each time, otherwise you get the same 16 bytes back each time. P_[MD5|SHA1] actually use the HMAC algorithm, again, to produce the input to the final HMAC algorithm. So what goes into the HMAC algorithms that go into the HMAC algorithms? More HMAC output, of course! The seed is HMAC'ed once to produce the HMAC input for the first n bytes (where n is 16 or 20 depending on the algorithm), and then that is HMAC'ed again to produce the input for the next n bytes.

All of this sounds almost self-referential, but it actually does work. Figure 6-8 shows the P_MD5 algorithm, illustrated out to three iterations (to produce 48 = 16 * 3 bytes of output).

Figure 6-8: P_MD5

So, given a secret of "ab" and a seed of "efghijkl", A(1) is HMAC_MD5("ab", "efghijkl"), or 0xefe3a7027ddbdb424cabd0935bfb3898. A(2), then, is HMAC_MD5( "ab", 0xefe3a7027ddbdb424cabd0935bfb3898), or 0xda55f448c81b 93ce1231cb7668bee2a2. Because you need 40 bytes of output, and MD5 only produces 16 per iteration, you need to iterate three times to produce 48 bytes and throw away the last 8. This means that you need A(3) as well, which is HMAC_MD5( "ab", A(2) = 0xda55f448c81b93ce1231cb7668bee2a2), or 0xbfa8ec7eda156ec26478851358c7a1fa.

With all the As computed, you now have enough information to feed into the "real" HMAC operations that generate the requisite 48 bytes of output. The final 48 bytes of output (remembering that you discard the last 8) are

HMAC( "ab", A(1) . "efghijkl" ) .
HMAC( "ab", A(2) . "efghijkl" ) .
HMAC( "ab", A(3), "efghijkl" )

Notice that the secret and the seed are constant throughout each HMAC operation; the only difference in each are the A values. These operations produce the 48 output bytes:

0x1b6ca10d18faddfbeb92b2d95f55ce2607d6c81ebe4b96d
1bec81813b9a0275725564781eda73ac521548d7d1f982c17

P_SHA1 is identical. It just replaces the SHA-1 hash algorithm with the MD5 hash algorithm. Because SHA-1 produces 20 bytes of output per iteration, though, it's only necessary to iterate twice and none of the output is discarded. P_SHA is fed the exact same seed, but only the last half of the secret, as diagrammed in Figure 6-9.

Figure 6-9: P_SHA1

This produces the 40 bytes:

0xcbb3de5db9295cdb68eb1ab18f88939cb3146849fe167cf8f9ec5f131790005d7f27b
2515db6c590

Finally, these two results are XORed together to produce the 40-byte pseudo-random combination:

0xd0df7f50a1d381208379a868d0dd5dbab4c2a057405dea2947244700ae30270
a5a71f5d0b011ff55

Notice that there's no predictable repetition here, and no obvious correlation with the input data. This procedure can be performed to produce any arbitrarily large amount of random data that two compliant implementations, both of which share the secret, can reproduce consistently.

In code, this is shown in Listing 6-33.

Listing 6-33: "prf.c" PRF function

/**
 * P_MD5 or P_SHA, depending on the value of the "new_digest" function
 * pointer.
 * HMAC_hash( secret, A(1) + seed ) + HMAC_hash( secret, A(2) + seed ) + ...
 * where + indicates concatenation and A(0) = seed, A(i) =
 * HMAC_hash( secret, A(i - 1) )
 */
static void P_hash( const unsigned char *secret,
                   int secret_len,
                   const unsigned char *seed,
                   int seed_len,
                   unsigned char *output,
                   int out_len,
                   void (*new_digest)( digest_ctx *context ) )
{
 unsigned char *A;
 int hash_len; // length of the hash code in bytes
 digest_ctx A_ctx, h;
 int adv;
 int i;
 new_digest( &A_ctx );
 hmac( secret, secret_len, seed, seed_len, &A_ctx );

 hash_len = A_ctx.hash_len * sizeof( int );
 A = malloc( hash_len + seed_len );
 memcpy( A, A_ctx.hash, hash_len );
 memcpy( A + hash_len, seed, seed_len );

 i = 2;

 while ( out_len > 0 )
 {
   new_digest( &h );
   // HMAC_Hash( secret, A(i) + seed )
   hmac( secret, secret_len, A, hash_len + seed_len, &h );
   adv = ( h.hash_len * sizeof( int ) ) < out_len ?
      h.hash_len * sizeof( int ) : out_len;
   memcpy( output, h.hash, adv );
   out_len -= adv;
   output += adv;
   // Set A for next iteration
   // A(i) = HMAC_hash( secret, A(i-1) )
   new_digest( &A_ctx );
   hmac( secret, secret_len, A, hash_len, &A_ctx );
   memcpy( A, A_ctx.hash, hash_len );

}

 free( A );
}

/**
 * P_MD5( S1, label + seed ) XOR P_SHA1(S2, label + seed );
 * where S1 & S2 are the first & last half of secret
 * and label is an ASCII string.  Ignore the null terminator.
 *
 * output must already be allocated.
 */
void PRF( const unsigned char *secret,
         int secret_len,
         const unsigned char *label,
         int label_len,
         const unsigned char *seed,
         int seed_len,
         unsigned char *output,
         int out_len )
{
 int i;
 int half_secret_len;
 unsigned char *sha1_out = ( unsigned char * ) malloc( out_len );
 unsigned char *concat = ( unsigned char * ) malloc( label_len + seed_len );
 memcpy( concat, label, label_len );
 memcpy( concat + label_len, seed, seed_len );

 half_secret_len = ( secret_len / 2 ) + ( secret_len % 2 );
 P_hash( secret, half_secret_len, concat, ( label_len + seed_len ),
   output, out_len, new_md5_digest );
 P_hash( secret + ( secret_len / 2 ), half_secret_len, concat,
   ( label_len + seed_len ), sha1_out, out_len, new_sha1_digest );

 for ( i = 0; i < out_len; i++ )
 {
   output[ i ] ^= sha1_out[ i ];
 }

 free( sha1_out );
 free( concat );
}

To see the PRF in action, put together a short test main routine in Listing 6-34.

Listing 6-34: "prf.c" main routine

#ifdef TEST_PRF
int main( int argc, char *argv[ ] )

{
 unsigned char *output;
 int out_len, i;
 int secret_len;
 int label_len;
 int seed_len;
 unsigned char *secret;
 unsigned char *label;
 unsigned char *seed;

 if ( argc < 5 )
 {
   fprintf( stderr,
     "usage: %s [0x]<secret> [0x]<label> [0x]<seed> <output len>\n",
     argv[ 0 ] );
   exit( 0 );
 }

 secret_len = hex_decode( argv[ 1 ], &secret );
 label_len = hex_decode( argv[ 2 ], &label );
 seed_len = hex_decode( argv[ 3 ], &seed );
 out_len = atoi( argv[ 4 ] );
 output = ( unsigned char * ) malloc( out_len );

 PRF( secret, secret_len,
      label, label_len,
      seed, seed_len,
      output, out_len );

 for ( i = 0; i < out_len; i++ )
 {
   printf( "%.02x", output[ i ] );
 }
 printf( "\n" );

 free( secret );
 free( label );
 free( seed );
 free( output );

 return 0;
}
#endif

You can try out the PRF, although it's not earth-shatteringly interesting:

[jdavies@localhost ssl]$ ./prf secret label seed 20
b5baf4722b91851a8816d22ebd8c1d8cc2e94d55

Creating Reproducible, Unpredictable Symmetric Keys with Master Secret Computation

The client selects the premaster secret and sends it to the server (or agrees on it, in the case of DH key exchange). The premaster secret is, as the name implies, secret — in fact, it's really the only important bit of handshake material that's hidden from eavesdroppers. However, the premaster secret itself isn't used as a session key; this would open the door to replay attacks. The premaster secret is combined with the server random and client random values exchanged earlier in the handshake and then run through the PRF to generate the master secret, which is used, indirectly, as the keying material for the symmetric encryption algorithms and MACs that actually protect the data in transit.

Given, for example, a premaster secret

030102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e
1f202122232425262728292a2b2c2d2e2f

a client random

4af0a38100000000000000000000000000000000000000000000000000000000

and a server random

4af0a3818ff72033b852b9b9c09e7d8045ab270eabc74e11d565ece018c9a5ec

you would compute the final master secret from which the actual keys are derived using — you guessed it — the PRF.

Remember that the PRF takes three parameters: a secret, a label, and a seed. The premaster secret is the secret, the label is just the unimaginative text string "master secret", and the seed is the client random and the server random concatenated one after the other, client random first.

The PRF is the XOR of the SHA-1 and the MD5 HMACs of the secret and the label concatenated with the seed, expanded out iteratively. With the PRF function defined above, master secret expansion is actually simple to code, as in Listing 6-35.

Listing 6-35: "tls.c" master secret computation

/**
 * Turn the premaster secret into an actual master secret (the
 * server side will do this concurrently) as specified in section 8.1:
 * master_secret = PRF( pre_master_secret, "master secret",
 * ClientHello.random + ServerHello.random );
 * ( premaster_secret, parameters );
 * Note that, with DH, the master secret len is determined by the generator (p)
 * value.
 */
static void compute_master_secret( const unsigned char *premaster_secret,
                                  int premaster_secret_len,
                                  TLSParameters *parameters )

{
 const char *label = "master secret";
 PRF( premaster_secret, premaster_secret_len,
      label, strlen( label ),
      // Note - cheating, since client_random & server_random are defined
      // sequentially in the structure
      parameters->client_random, RANDOM_LENGTH * 2,
      parameters->master_secret, MASTER_SECRET_LENGTH );
}

RSA Key Exchange

After the server hello done has been received, the server believes that the client has enough information to complete the key exchange specified in the selected cipher suite. If the key exchange is RSA, this means that the client now has the server's public key. It's the client's problem whether to trust that key or not, based on the certificate chain.

The client should thus send a key exchange as shown in Listing 6-36, in tls_connect.

Listing 6-36: "tls.c" tls_connect with key exchange

// Step 3. Send client key exchange, change cipher spec (7.1) and encrypted
 // handshake message
 if ( !( send_client_key_exchange( connection, parameters ) ) )
 {
   perror( "Unable to send client key exchange" );
   return 3;
 }

send_client_key_exchange is slightly complex because RSA and DH key exchanges are so different. For now, just focus on RSA in Listing 6-37.

Listing 6-37: "tls.c" send_client_key_exchange

/**
 * Send the client key exchange message, as detailed in section 7.4.7
 * Use the server's public key (if it has one) to encrypt a key. (or DH?)
 * Return true if this succeeded, false otherwise.
 */
static int send_client_key_exchange( int connection, TLSParameters *parameters )
{
 unsigned char *key_exchange_message;
 int key_exchange_message_len;
 unsigned char *premaster_secret;
 int premaster_secret_len;

 switch ( parameters->pending_send_parameters.suite ) {
   case TLS_NULL_WITH_NULL_NULL:
     // XXX this is an error, exit here
     break;

case TLS_RSA_WITH_NULL_MD5:
   case TLS_RSA_WITH_NULL_SHA:
...
   case TLS_RSA_WITH_DES_CBC_SHA:
   case TLS_RSA_WITH_3DES_EDE_CBC_SHA:
   case TLS_RSA_WITH_AES_128_CBC_SHA:
   case TLS_RSA_WITH_AES_256_CBC_SHA:
     premaster_secret_len = MASTER_SECRET_LENGTH;
     premaster_secret = malloc( premaster_secret_len );
     key_exchange_message_len = rsa_key_exchange(
       &parameters->server_public_key.rsa_public_key,
       premaster_secret, &key_exchange_message );
     break;
   default:
     return 0;
 }

 if ( send_handshake_message( connection, client_key_exchange,
      key_exchange_message, key_exchange_message_len ) )
 {
   free( key_exchange_message );
   return 0;
 }

 free( key_exchange_message );

 // Now, turn the premaster secret into an actual master secret (the
 // server side will do this concurrently).
 compute_master_secret( premaster_secret, premaster_secret_len, parameters );

 // XXX - for security, should also "purge" the premaster secret from
 // memory.
 calculate_keys( parameters );

 free( premaster_secret );

 return 1;
}

The goal of the key exchange is to exchange a premaster secret, turn it into a master secret, and use that to calculate the keys that are used for the remainder of the connection. As you can see, send_client_key_exchange starts by checking if the key exchange method is RSA. If the key exchange method is RSA, send_client_key_exchange calls rsa_key_exchange to build the appropriate handshake message. compute_master_secret has already been examined in Listing 6-35, and calculate_keys is examined later in Listing 6-41.

This routine goes ahead and lets the rsa_key_exchange function select the premaster secret. There's no reason why send_client_key_exchange couldn't do this, and pass the premaster secret into rsa_key_exchange. However, this configuration makes DH key exchange easier to support because the upcoming dh_key_exchange necessarily has to select the premaster secret, due to the nature of the Diffie-Hellman algorithm.

The rsa_key_exchange message is built in Listing 6-38.

Listing 6-38: "tls.c" rsa_key_exchange

int rsa_key_exchange( rsa_key *public_key,
                     unsigned char *premaster_secret,
                     unsigned char **key_exchange_message )
{
 int i;
 unsigned char *encrypted_premaster_secret = NULL;
 int encrypted_length;

 // first two bytes are protocol version
 premaster_secret[ 0 ] = TLS_VERSION_MAJOR;
 premaster_secret[ 1 ] = TLS_VERSION_MINOR;
 for ( i = 2; i < MASTER_SECRET_LENGTH; i++ )
 {
   // XXX SHOULD BE RANDOM!
   premaster_secret[ i ] = i;
 }

 encrypted_length = rsa_encrypt( premaster_secret, MASTER_SECRET_LENGTH,
   &encrypted_premaster_secret, public_key );

 *key_exchange_message = ( unsigned char * ) malloc( encrypted_length + 2 );
 (*key_exchange_message)[ 0 ] = 0;
 (*key_exchange_message)[ 1 ] = encrypted_length;
 memcpy( (*key_exchange_message) + 2, encrypted_premaster_secret,
   encrypted_length );

 free( encrypted_premaster_secret );

 return encrypted_length + 2;
}

This function takes as input the RSA public key, generates a "random" premaster secret, encrypts it, and returns both the premaster secret and the key exchange message. The format of the key exchange message is straightforward; it's just a two-byte length followed by the PKCS #1 padded, RSA encrypted premaster secret. Notice that the specification mandates that the first two bytes of the premaster secret must be the TLS version — in this case, 3.1. In theory, the server is supposed to verify this. In practice, few servers do the verification because there are a few buggy TLS implementations floating around that they want to remain compatible with.

Now the only thing left to do is to turn the master secret into a set of keys. The amount, and even the type, of keying material needed depends on the cipher suite. If the cipher suite uses SHA-1 HMAC, the MAC requires a 20-byte key; if MD5, it requires a 16-byte key. If the cipher suite uses DES, it requires an 8-byte key; if AES-256, it requires a 32-byte key. If the encryption algorithm uses CBC, initialization vectors are needed; if the algorithm is a stream algorithm, no initialization vector is involved.

Rather than build an enormous switch/case statement for each possibility, define a CipherSuite structure as in Listing 6-39.

Listing 6-39: "tls.h" CipherSuite structure

typedef struct
{
 CipherSuiteIdentifier id;

 int                   block_size;
 int                   IV_size;
 int                   key_size;
 int                   hash_size;

 void (*bulk_encrypt)( const unsigned char *plaintext,
                       const int plaintext_len,
                       unsigned char ciphertext[],
                       void *iv,
                       const unsigned char *key );
 void (*bulk_decrypt)( const unsigned char *ciphertext,
                       const int ciphertext_len,
                       unsigned char plaintext[],
                       void *iv,
                       const unsigned char *key );
 void (*new_digest)( digest_ctx *context );
}
CipherSuite;

This includes everything you need to know about a cipher suite; by now, the utility of declaring the encrypt, decrypt, and hash functions with identical signatures in the previous chapters should be clear. Now, for each supported cipher suite, you need to generate a CipherSuite instance and index it as shown in Listing 6-40.

Listing 6-40: "tls.c" cipher suites list

static CipherSuite suites[] =
{
 { TLS_NULL_WITH_NULL_NULL, 0, 0, 0, 0, NULL, NULL, NULL },
 { TLS_RSA_WITH_NULL_MD5, 0, 0, 0, MD5_BYTE_SIZE, NULL, NULL, new_md5_digest },
 { TLS_RSA_WITH_NULL_SHA, 0, 0, 0, SHA1_BYTE_SIZE, NULL, NULL, new_sha1_digest },
 { TLS_RSA_EXPORT_WITH_RC4_40_MD5, 0, 0, 5, MD5_BYTE_SIZE, rc4_40_encrypt,

rc4_40_decrypt, new_md5_digest },
 { TLS_RSA_WITH_RC4_128_MD5, 0, 0, 16, MD5_BYTE_SIZE, rc4_128_encrypt,
    rc4_128_decrypt, new_md5_digest },
 { TLS_RSA_WITH_RC4_128_SHA, 0, 0, 16, SHA1_BYTE_SIZE, rc4_128_encrypt,
    rc4_128_decrypt, new_sha1_digest },
 { TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5, 0, 0, 0, MD5_BYTE_SIZE, NULL, NULL,
    new_md5_digest },
 { TLS_RSA_WITH_IDEA_CBC_SHA, 0, 0, 0, SHA1_BYTE_SIZE, NULL, NULL,
    new_sha1_digest },
 { TLS_RSA_EXPORT_WITH_DES40_CBC_SHA, 0, 0, 0, SHA1_BYTE_SIZE, NULL, NULL,
    new_sha1_digest },
 { TLS_RSA_WITH_DES_CBC_SHA, 8, 8, 8, SHA1_BYTE_SIZE, des_encrypt, des_decrypt,
    new_sha1_digest },
 { TLS_RSA_WITH_3DES_EDE_CBC_SHA, 8, 8, 24, SHA1_BYTE_SIZE, des3_encrypt,
    des3_decrypt, new_sha1_digest },
...

Because these instances are referred to by position, you have to list each one, even if it's not supported. Notice, for example, that TLS_RSA_WITH_IDEA_CBC_SHA is declared, but left empty. It is never used by this implementation, but by allocating space for it, the rest of the code is allowed to refer to elements in the CipherSuite structure by just referencing the suites array.

If you wanted to create a key for a 3DES cipher suite, for example, you could invoke

suites[ TLS_RSA_WITH_3DES_EDE_CBC_SHA ].key_size

In fact, because the CipherSuiteIdentifier was added to ProtectionParameters, the key computation code can just invoke

suites[ parameters->suite ].key_size

when it needs to know how much keying material to retrieve from the master secret.

Now, recall that MASTER_SECRET_LENGTH is 48 bytes, regardless of cipher suite. If the selected cipher suite is AES 256, CBC, with SHA-1, you need 136 bytes of keying material — 32 bytes each for the client and server keys, 16 bytes each for the initialization vectors, and 20 bytes each for the MAC secrets. Therefore, the master secret itself must be expanded. As you can probably guess, this is done via the PRF; the only difference between the use of the PRF in key calculation and the use of the PRF in master secret expansion is that the label passed in is "key expansion" rather than "master secret".

The key calculation routine is shown in Listing 6-41.

Listing 6-41: "tls.c" calculate_keys

/**
6.3: Compute a key block, including MAC secrets, keys, and IVs for client & server.
Notice that the seed is server random followed by client random (whereas for master
secret computation, it's client random followed by server random).  Sheesh!

*/
static void calculate_keys( TLSParameters *parameters )
{
 // XXX assuming send suite & recv suite will always be the same
 CipherSuite *suite = &( suites[ parameters->pending_send_parameters.suite ] );
 const char *label = "key expansion";
 int key_block_length =
   suite->hash_size * 2 +
   suite->key_size * 2 +
   suite->IV_size * 2;
 char seed[ RANDOM_LENGTH * 2 ];
 unsigned char *key_block = ( unsigned char * ) malloc( key_block_length );
 unsigned char *key_block_ptr;
 ProtectionParameters *send_parameters = &parameters->pending_send_parameters;
 ProtectionParameters *recv_parameters = &parameters->pending_recv_parameters;

 memcpy( seed, parameters->server_random, RANDOM_LENGTH );
 memcpy( seed + RANDOM_LENGTH, parameters->client_random, RANDOM_LENGTH );

 PRF( parameters->master_secret, MASTER_SECRET_LENGTH,
   label, strlen( label ),
   seed, RANDOM_LENGTH * 2,
   key_block, key_block_length );

 send_parameters->MAC_secret = ( unsigned char * ) malloc( suite->hash_size );
 recv_parameters->MAC_secret = ( unsigned char * ) malloc( suite->hash_size );
 send_parameters->key = ( unsigned char * ) malloc( suite->key_size );
 recv_parameters->key = ( unsigned char * ) malloc( suite->key_size );
 send_parameters->IV = ( unsigned char * ) malloc( suite->IV_size );
 recv_parameters->IV = ( unsigned char * ) malloc( suite->IV_size );

 key_block_ptr = read_buffer( send_parameters->MAC_secret, key_block,
   suite->hash_size );
 key_block_ptr = read_buffer( recv_parameters->MAC_secret, key_block_ptr,
   suite->hash_size );
 key_block_ptr = read_buffer( send_parameters->key, key_block_ptr,
   suite->key_size );
 key_block_ptr = read_buffer( recv_parameters->key, key_block_ptr,
   suite->key_size );
 key_block_ptr = read_buffer( send_parameters->IV, key_block_ptr,
   suite->IV_size );
 key_block_ptr = read_buffer( recv_parameters->IV, key_block_ptr,
   suite->IV_size );

 free( key_block );
}

Diffie-Hellman Key Exchange

TLS 1.0 supports Diffie-Hellman key exchange in addition to RSA key exchange. Remember that, in Diffie-Hellman key exchange, neither side gets to pick the negotiated secret, but both sides end up computing the same value. This works out in the context of TLS key exchange; both sides can agree on the premaster secret, which is expanded to the master secret, which is expanded to the keying material.

Add support for DH key exchange in send_client_key_exchange as shown in Listing 6-42.

Listing 6-42: "tls.c" send_client_key_exchange with Diffie-Hellman key exchange

switch ( parameters->pending_send_parameters.suite ) {
   case TLS_NULL_WITH_NULL_NULL:
     // XXX this is an error, exit here
     break;
...
   case TLS_DH_DSS_EXPORT_WITH_DES40_CBC_SHA:
   case TLS_DH_DSS_WITH_DES_CBC_SHA:
   case TLS_DH_DSS_WITH_3DES_EDE_CBC_SHA:
...
     premaster_secret_len = parameters->server_dh_key.p.size;
     premaster_secret = malloc( premaster_secret_len );
     key_exchange_message_len = dh_key_exchange( &parameters->server_dh_key,
       premaster_secret, &key_exchange_message );
     break;

The Diffie-Hellman key exchange procedure continues as described in Chapter 3. Recall that you didn't code a specific Diffie-Hellman routine because it was essentially just a couple of calls to mod_pow. These calls can be integrated into a premaster secret exchange as shown in Listing 6-43.

Listing 6-43: "tls.c" dh_key_exchange

/**
 * Just compute Yc = g^a % p and return it in "key_exchange_message".  The
 * premaster secret is Ys ^ a % p.
 */
int dh_key_exchange( dh_key *server_dh_key,
                    unsigned char *premaster_secret,
                    unsigned char **key_exchange_message )
{
 huge Yc;
 huge Z;

huge a;
 int message_size;
 short transmit_len;

 // TODO obviously, make this random, and much longer
 set_huge( &a, 6 );
 mod_pow( &server_dh_key->g, &a, &server_dh_key->p, &Yc );
 mod_pow( &server_dh_key->Y, &a, &server_dh_key->p, &Z );

 // Now copy Z into premaster secret and Yc into key_exchange_message
 memcpy( premaster_secret, Z.rep, Z.size );
 message_size = Yc.size + 2;
 transmit_len = htons( Yc.size );
 *key_exchange_message = malloc( message_size );
 memcpy( *key_exchange_message, &transmit_len, 2 );
 memcpy( *key_exchange_message + 2, Yc.rep, Yc.size );

 free_huge( &Yc );
 free_huge( &Z );
 free_huge( &a );

 return message_size;
}

If you've been following closely, you may be wondering where the server's dh_key value — the p, g and Y values that this key exchange relies on — come from? Although it's possible to get one from a certificate (it's officially defined, anyway), practically speaking this never happens. Instead, there's a specific server key exchange handshake type where the server can provide these values as well as authenticate them. This is examined in Chapter 8.

TLS Change Cipher Spec

After the key exchange has been successfully completed, the client should send a change cipher spec message. Although change cipher spec can never be legally sent outside of the context of a handshake, it's not declared as a handshake message. Why? According to the specification,

This isn't made particularly clear, but it appears that they're concerned with the possibility of an implementation that automatically piggy-backs handshake messages into one large TLS message doing so with change cipher spec messages and having the other side lose this.

The change cipher spec message is a marker message, just like server hello done was, that doesn't include any data. It is a major milestone in the handshake process, though, because the reception of a change cipher spec message tells the other side that the key exchange has completed, and every subsequent message sent by this peer is encrypted and authenticated using it. Note that neither side should assume that the negotiated parameters are in place before receiving a change cipher spec message — it's entirely possible that, although a change cipher spec message is expected, an alert could appear in plaintext instead.

So, at this point in the key exchange, the client should send a change cipher spec and make its pending send parameters active, but not touch the pending receive parameters. This is shown in Listing 6-44.

Listing 6-44: "tls.c" send_change_cipher_spec

static int send_change_cipher_spec( int connection, TLSParameters *parameters )
{
 char          send_buffer[ 1 ];
 send_buffer[ 0 ] = 1;
 send_message( connection, content_change_cipher_spec, send_buffer, 1 );

 memcpy( &parameters->active_send_parameters,
         &parameters->pending_send_parameters,
         sizeof( ProtectionParameters ) );

 init_protection_parameters( &parameters->pending_send_parameters );

 return 1;
}

As promised, this is pretty simple. The server then sends a change cipher spec of its own, and this should be accounted for as in Listing 6-45.

Listing 6-45: "tls.c" receive_tls_msg with support for change cipher spec

...
 if ( message.type == content_handshake )
 {
...
 }
 else if ( message.type == content_change_cipher_spec )
 {
   while ( ( read_pos - decrypted_message ) < decrypted_length )
   {
     unsigned char change_cipher_spec_type;

     read_pos = read_buffer( ( void * ) &change_cipher_spec_type,
       ( void * ) read_pos, 1 );

     if ( change_cipher_spec_type != 1 )
     {
       printf( "Error - received message ChangeCipherSpec, but type != 1\n" );
       exit( 0 );
     }

else
     {
       memcpy( &parameters->active_recv_parameters,
               &parameters->pending_recv_parameters,
               sizeof( ProtectionParameters ) );
       init_protection_parameters( &parameters->pending_recv_parameters );
     }
   }
 }

Now, when the server sends its change_cipher_spec message, the pending receive parameters are updated and made the active receive parameters. Technically, the change cipher spec messages can "cross" on the wire. The server may legally send its change cipher spec message as soon as it receives a proper key exchange from the client; it doesn't strictly have to wait until the client sends a change cipher spec.

TLS Finished

So, the key exchange has been completed, both sides have agreed that the pending parameters are now the active parameters. It's time to start using the connection, right?

Well, that's what the designers of SSLv2 thought, too. As it turns out, this was the fatal flaw in SSLv2. After the key exchange had completed, the connection was used immediately to transfer data. The problem with this is that it doesn't take into account man-in-the-middle attacks that occur before the key exchange. Although the key exchange protocol designed around X.509 certificates does an admirable job of protecting against man-in-the-middle attacks against the public key, the malicious man in the middle can intercept and modify all the exchanges prior to this.

How can an attacker use this to his advantage? Well, he could, for instance, change the client hello message to list only one possibility for a cipher suite — DES with MD5 MAC. If he has a DES-cracking machine, the key exchange can proceed, and he can decode the communications at his leisure.

In general, both sides need a way to strongly authenticate that what they sent was what was received. The way TLS accomplishes this is to require both sides to send a finished message before the handshake can be considered complete. Both of these finished messages must be sent before the negotiated parameters can be used for application data, and the finished messages themselves are sent using the negotiated encryption and authentication parameters. The contents of this finished message are a 12-byte verify array whose contents are based on the hash of the contents of all of the handshake messages to this point.

Computing the Verify Message

To compute this verify message, then, it's necessary to keep a running hash of every byte that's sent or received with a message type of handshake. This is, incidentally, why I spent so much time in Chapter 4 on creating an "updateable" HMAC function; without the updateable HMAC function, it would have been necessary here to buffer all this data and pass it as a gigantic memory array into the HMAC function.

Instead, following these steps:

1. Add a pair of digest_ctx objects to the TLSParameters as shown in Listing 6-46; the verify data is actually based on a combination of both MD5 and SHA (similar to the PRF).

   Listing 6-46: "tls.h" TLSParameters with digest contexts

   typedef struct
   {
   ...
    int                   server_hello_done;
    digest_ctx            md5_handshake_digest;
    digest_ctx            sha1_handshake_digest;
   }
   TLSParameters;

2. At the top of tls_connect, initialize them both, in Listing 6-47.

   Listing 6-47: "tls.c" tls_connect with handshake digests

   int tls_connect( int connection,
                   TLSParameters *parameters )
   {
    init_parameters( parameters );
   
    new_md5_digest( &parameters->md5_handshake_digest );
    new_sha1_digest( &parameters->sha1_handshake_digest );

3. Modify send_handshake_message, as shown in Listing 6-48, to update the running digest every time a handshake message is sent.

   Listing 6-48: "tls.c" send_handshake_message with handshake digest update

   static int send_handshake_message( int connection,
                                     int msg_type,
                                     const unsigned char *message,
                                     int message_len,
                                     TLSParameters *parameters )
   {
   ...
    update_digest( &parameters->md5_handshake_digest, send_buffer,
      send_buffer_size );

   update_digest( &parameters->sha1_handshake_digest, send_buffer,
      send_buffer_size );
   
    response = send_message( connection, content_handshake, send_buffer,
      send_buffer_size );

4. Because send_handshake_message now takes a new parameter — the TLSParameters — update the invocations to it to include this, as shown in Listing 6-49.

   Listing 6-49: "tls.c" send_handshake_message updates

   static int send_client_hello( int connection, TLSParameters *parameters )
   
   {
   
   ...
    status = send_handshake_message( connection, client_hello, send_buffer,
      send_buffer_size, parameters );
   
   ...
   static int send_client_key_exchange( int connection, TLSParameters *parameters )
   
   {
   ...
    if ( send_handshake_message( connection, client_key_exchange,
   
         key_exchange_message, key_exchange_message_len, parameters ) )
   
    {

5. Update the running digest within receive_tls_message, if the type of the message is content_handshake, as in Listing 6-50.

   Listing 6-50: "tls.c" receive_tls_message with handshake digest update

   static int receive_tls_msg( int connection,
                              TLSParameters *parameters )
   {
   ...
    if ( message.type == content_handshake )
    {
      while ( ( read_pos - decrypted_message ) < decrypted_length )
      {
        Handshake handshake;
        // Keep track of beginning of message for handshake digest update below
        const unsigned char *handshake_msg_start = read_pos;
   ...
        update_digest( &parameters->md5_handshake_digest, handshake_msg_start,
          handshake.length + 4 );
        update_digest( &parameters->sha1_handshake_digest, handshake_msg_start,
          handshake.length + 4 );
      }

6. With the handshake digest defined and updated, the client can send its finished message, in Listing 6-51.

   Listing 6-51: "tls.c" tls_connect with client finished message

   if ( !( send_change_cipher_spec( connection, parameters ) ) )
    {
      perror( "Unable to send client change cipher spec" );
      return 4;
    }
   
    // This message will be encrypted using the newly negotiated keys
    if ( !( send_finished( connection, parameters ) ) )
    {
      perror( "Unable to send client finished" );
      return 5;
    }

7. send_finished itself is straightforward, as shown in Listing 6-52.

   Listing 6-52: "tls.c" send_finished

   static int send_finished( int connection,
                            TLSParameters *parameters )
   {
    unsigned char verify_data[ VERIFY_DATA_LEN ];
   
    compute_verify_data( "client finished", parameters, verify_data );
    send_handshake_message( connection, finished, verify_data, VERIFY_DATA_LEN,
      parameters );
   
    return 1;
   }

8. Of course, as you can likely guess, the challenge is in the computation of verify data. This is shown in Listing 6-53.

   Listing 6-53: "tls.c" compute_verify_data

   /**
    * 7.4.9:
    * verify_data = PRF( master_secret, "client finished", MD5(handshake_messages)
    * + SHA-1(handshake_messages)) [0..11]
    *
    * master_secret = PRF( pre_master_secret, "master secret", ClientHello.random +
    *  ServerHello.random );
    * always 48 bytes in length.
    */
   #define VERIFY_DATA_LEN 12
   
   static void compute_verify_data( const char *finished_label,
                                   TLSParameters *parameters,

   char *verify_data )
   {
    unsigned char handshake_hash[ ( MD5_RESULT_SIZE * sizeof( int ) ) +
                               ( SHA1_RESULT_SIZE * sizeof( int ) ) ];
   
    finalize_digest( &parameters->md5_handshake_digest );
    finalize_digest( &parameters->sha1_handshake_digest );
   
    memcpy( handshake_hash, parameters->md5_handshake_digest.hash, MD5_BYTE_SIZE );
    memcpy( handshake_hash + MD5_BYTE_SIZE, parameters->sha1_handshake_digest.hash,
      SHA1_BYTE_SIZE );
   
    PRF( parameters->master_secret, MASTER_SECRET_LENGTH,
         finished_label, strlen( finished_label ),
         handshake_hash,
         MD5_RESULT_SIZE * sizeof( int ) + SHA1_RESULT_SIZE * sizeof( int ),
         verify_data, VERIFY_DATA_LEN );
   }

   The verify data is a PRF expansion of "client finished" with both hashes concatenated next to one another. The result is 12 bytes, and both sides end up computing the same value.

9. Of course, the client must wait for the server to send its finished message as well. Update tls_connect to wait for the server_finished as shown in Listing 6-54.

   Listing 6-54: "tls.c" tls_connect with server finished support

   parameters->server_finished = 0;
    while ( !parameters->server_finished )
    {
      if ( receive_tls_msg( connection, parameters ) < 0 )
      {
        perror( "Unable to receive server finished" );
        return 6;
      }
    }

   NOTE This call will also be the first time the client receives the change cipher spec message.

10. This requires that you also add a server_finished flag, similar to the server_hello_done flag, in TLSParameters, in Listing 6-55:

    Listing 6-55: "tls.c" TLSParameters

    typedef struct
    {

    ...
     int                   server_hello_done;
     int                   server_finished;
    ...
    }
    TLSParameters;

11. Update tls_receive_message to process the server_finished message in Listing 6-56.

    Listing 6-56: "tls.c" tls_receive_message with server finished support

    switch ( handshake.msg_type )
         {
    ...
           case finished:
             {
               read_pos = parse_finished( read_pos, handshake.length, parameters );
               if ( read_pos == NULL )
               {
                 send_alert_message( connection, illegal_parameter );
                 return −1;
               }
             }
             break;

12. Now you can parse the finished message, in Listing 6-57.

    Listing 6-57: "tls.c" parse_finished

    static unsigned char *parse_finished( unsigned char *read_pos,
                                         int pdu_length,
                                         TLSParameters *parameters )
    {
     unsigned char verify_data[ VERIFY_DATA_LEN ];
    
     parameters->server_finished = 1;
    
     compute_verify_data( "server finished", parameters, verify_data );
    
     if ( memcmp( read_pos, verify_data, VERIFY_DATA_LEN ) )
     {
       return NULL;
     }
    
     return read_pos + pdu_length;
    }

Here, compute_verify_data is called again to recompute the verification data, and the received data is compared with the computed data.

Correctly Receiving the Finished Message

Unfortunately, parse_finished, in Listing 6-57, doesn't work. The handshake digests were finalized when verify_data was called the first time around, but they need to be finalized again when the server sends its finished message. And because the server's finished message is based on a hash including the client's finished message you can't just reuse the original hashes; the server sends verification of a different hash code.

Therefore, it's necessary to modify compute_verify_data so that it doesn't operate on the running hash. The easiest way to do this is to make a temporary copy and operate on that temporary copy, as in Listing 6-58.

Listing 6-58: "tls.c" compute_verify_data with temporary copy

void compute_handshake_hash( TLSParameters *parameters, unsigned char *handshake_hash )
{
 digest_ctx tmp_md5_handshake_digest;
 digest_ctx tmp_sha1_handshake_digest;

 // "cheating".  Copy the handshake digests into local memory (and change
 // the hash pointer) so that we can finalize twice (again in "recv")
 memcpy( &tmp_md5_handshake_digest, &parameters->md5_handshake_digest,
   sizeof( digest_ctx ) );
 memcpy( &tmp_sha1_handshake_digest, &parameters->sha1_handshake_digest,
   sizeof( digest_ctx ) );

 tmp_md5_handshake_digest.hash = ( unsigned int * ) malloc( MD5_BYTE_SIZE );
 tmp_sha1_handshake_digest.hash = ( unsigned int * ) malloc( SHA1_BYTE_SIZE );
 memcpy( tmp_md5_handshake_digest.hash, parameters->md5_handshake_digest.hash,
   MD5_BYTE_SIZE );
 memcpy( tmp_sha1_handshake_digest.hash, parameters->sha1_handshake_digest.hash,
   SHA1_BYTE_SIZE );

 finalize_digest( &tmp_md5_handshake_digest );
 finalize_digest( &tmp_sha1_handshake_digest );

 memcpy( handshake_hash, tmp_md5_handshake_digest.hash, MD5_BYTE_SIZE );
 memcpy( handshake_hash + MD5_BYTE_SIZE, tmp_sha1_handshake_digest.hash,
   SHA1_BYTE_SIZE );

 free( tmp_md5_handshake_digest.hash );
 free( tmp_sha1_handshake_digest.hash );
}

static void compute_verify_data( const char *finished_label,
                                TLSParameters *parameters,
                                char *verify_data )
{
 // Per 6.2.3.1 - encrypted data should always be followed by a MAC

unsigned char handshake_hash[ ( MD5_RESULT_SIZE * sizeof( int ) ) +
                            ( SHA1_RESULT_SIZE * sizeof( int ) ) ];

 compute_handshake_hash( parameters, handshake_hash );

 // First, compute the verify data
 PRF( parameters->master_secret, MASTER_SECRET_LENGTH,
      finished_label, strlen( finished_label ),
      handshake_hash,
      MD5_RESULT_SIZE * sizeof( int ) + SHA1_RESULT_SIZE * sizeof( int ),
      verify_data, VERIFY_DATA_LEN );
}

Now, the same compute_verify_data function can be used both when sending and receiving finished messages.

That's it, right? The key exchange is complete, and the finished messages have been exchanged and verified. Everything is in place except for the small matter of actually encrypting and MAC'ing the data.

Secure Data Transfer with TLS

Conceptually, applying TLS is simple after the keys have been agreed upon. First, the whole block of data to be sent, including the TLS message header, is run through the MAC algorithm and the result is appended to the message. There's a chicken-and-the-egg problem here, though. The MAC includes the TLS header, which includes the length of the following buffer, which includes the MAC in its length. So when MAC'ing, what length is used? The transmitted length is the length of the content, plus the MAC; what's MAC'ed is just the length of the content.

If the bulk encryption algorithm requires padding, the length also indicates padding. Again, the MAC buffer does not reflect the padding length. And, of course, the whole thing — header, padding, MAC and all — are encrypted using the bulk encryption algorithm in force before being sent.

Assigning Sequence Numbers

As a protection against replay attacks, each packet is also assigned a sequence number. The sequence numbers start at 0 whenever a change_cipher_spec is received and is incremented each time a new TLSMessage is sent or received. Each side maintains a separate counter, and this counter is prepended to each message before MAC'ing it.

Declare the sequence number as shown in Listing 6-59 and initialize it as shown in Listing 6-60.

Listing 6-59: "tls.h" ProtectionParameters with seq_num

typedef struct
{
...
 unsigned long         seq_num;
}
ProtectionParameters;

Listing 6-60: "tls.c" init_protection_parameters with seq_num

void init_protection_parameters( ProtectionParameters *parameters )
{
 parameters->MAC_secret = NULL;
 parameters->key = NULL;
 parameters->IV = NULL;

 parameters->seq_num = 0;
 parameters->suite = TLS_NULL_WITH_NULL_NULL;
}

static int send_change_cipher_spec( int connection, TLSParameters *parameters )
{
 send_message( connection, content_change_cipher_spec, send_buffer, 1,
               &parameters->active_send_parameters );
...
 // Per 6.1: The sequence number must be set to zero whenever a connection
 // state is made the active state... the first record which is transmitted
 // under a particular connection state should use sequence number 0.
 parameters->pending_send_parameters.seq_num = 0;

 memcpy( &parameters->active_send_parameters,
         &parameters->pending_send_parameters,
         sizeof( ProtectionParameters ) );
...
static int receive_tls_msg( int connection,
                           TLSParameters *parameters )
{
...
 else if ( message.type == content_change_cipher_spec )
 {
...
     if ( change_cipher_spec_type != 1 )
     {
       printf( "Error - received message of type ChangeCipherSpec, but type !=
1\n" );
       exit( 0 );
     }
     else
     {
       parameters->pending_recv_parameters.seq_num = 0;

memcpy( &parameters->active_recv_parameters,
               &parameters->pending_recv_parameters,
               sizeof( ProtectionParameters ) );
...

Note that the sequence number is never transmitted, encrypted or otherwise; it's just prepended to the MAC buffer before the packet is MAC'ed. Therefore, given a content buffer of "content", the buffer that is MAC'ed looks like this:

A digest is produced over this data.

What's actually transmitted, on the other hand, is

But this is, of course, encrypted before sending. Notice that the buffer is MAC'ed first and then encrypted. The order is clearly important so that the other side can correctly receive it.

Supporting Outgoing Encryption

To support encryption — outgoing — the only function that needs to be updated is the send_message function. In order to apply the active cipher suite, it needs to be sent the active ProtectionParameters so that it can check to see what cipher suite is active, and apply that cipher suite. Remember that TLS_NULL_WITH_NULL_NULL is a valid cipher suite, and it's the one that's active when the handshake first starts. It just tells send_message to do no MAC nor encrypt. In this way, there's always a cipher suite active, even if it's a "do nothing" cipher suite.

The first thing send_message must do is to create the MAC buffer and compute the MAC. To do this, follow these steps:

1. Check for an active digest and apply it to the contents as shown in Listing 6-61.

   Listing 6-61: "tls.c" send_message with MAC support

   static int send_message( int connection,
                           int content_type,
                           const unsigned char *content,
                           short content_len,

   ProtectionParameters *parameters )
   {
    TLSPlaintext header;
    unsigned char *send_buffer;
    int send_buffer_size;
    unsigned char *mac = NULL;
    digest_ctx digest;
    CipherSuite *active_suite;
   
    active_suite = &suites[ parameters->suite ];
   
    if ( active_suite->new_digest )
    {
      // Allocate enough space for the 8-byte sequence number, the 5-byte pseudo
      // header, and the content.
      unsigned char *mac_buffer = malloc( 13 + content_len );
      int sequence_num;
   
      mac = ( unsigned char * ) malloc( active_suite->hash_size );
      active_suite->new_digest( &digest );
   
      memset( mac_buffer, 0x0, 8 );
      sequence_num = htonl( parameters->seq_num );
      memcpy( mac_buffer + 4, &sequence_num, sizeof( int ) );
   
      // These will be overwritten below
      header.type = content_type;
      header.version.major = 3;
      header.version.minor = 1;
      header.length = htons( content_len );
      mac_buffer[ 8 ] = header.type;
      mac_buffer[ 9 ] = header.version.major;
      mac_buffer[ 10 ] = header.version.minor;
      memcpy( mac_buffer + 11, &header.length, sizeof( short ) );
   
      memcpy( mac_buffer + 13, content, content_len );
      hmac( parameters->MAC_secret,
            active_suite->hash_size,
            mac_buffer, 13 + content_len,
            &digest );
   
      memcpy( mac, digest.hash, active_suite->hash_size );
   
      free( mac_buffer );
   }
   
    send_buffer_size = content_len + active_suite->hash_size;
   ...
   
    parameters->seq_num++;

   free( send_buffer );
   
    return 0;
   }

   If the current active protection parameters include a digest, a MAC buffer is built as described above, MAC'ed, and thrown away; after the MAC itself has been computed, the MAC buffer is immaterial to the remainder of the function.

2. Check to see if the active cipher suite has a block size (that is, is not a stream cipher). If so, add any required padding, as shown in Listing 6-62.

   Listing 6-62: "tls.c" send_message with padding support

   unsigned char padding_length = 0;
   ...
    send_buffer_size = content_len + active_suite->hash_size;
   
    if ( active_suite->block_size )
    {
      padding_length = active_suite->block_size -
        ( send_buffer_size % active_suite->block_size );
      send_buffer_size += padding_length;
    }
   
    // Add space for the header, but only after computing padding
    send_buffer_size +=5;

3. Build the actual send buffer. Recall Listing 6-15 where send_message was initially defined; the send buffer was simply the TLS header followed by the contents, verbatim. Now, it's the TLS header, followed by the contents, followed by any required padding, followed by the MAC. The updated send buffer is shown in Listing 6-63.

   Listing 6-63: "tls.c" send buffer

   send_buffer = ( unsigned char * ) malloc( send_buffer_size );
    if ( mac )
    {
      memcpy( send_buffer + content_len + 5, mac, active_suite->hash_size );
      free( mac );
    }
   
    if ( padding_length > 0 )
    {
      unsigned char *padding;
      for ( padding = send_buffer + send_buffer_size - 1;
            padding > ( send_buffer + ( send_buffer_size - padding_length - 1 ) );
            padding-- )

   {
        *padding = ( padding_length - 1 );
      }
    }
   
    header.type = content_type;
    header.version.major = 3;
    header.version.minor = 1;
    header.length = htons( content_len + active_suite->hash_size + padding_length );

4. The send buffer itself needs to be encrypted if the cipher suite calls for it (remember that TLS_NULL_WITH_MD5 is a legitimate cipher suite that calls for authentication but no encryption). The encryption is shown in Listing 6-64.

   Listing 6-64: "tls.c" send_message with encryption

   memcpy( send_buffer + 5, content, content_len );
   
    if ( active_suite->bulk_encrypt )
    {
      unsigned char *encrypted_buffer = malloc( send_buffer_size );
      // The first 5 bytes (the header) aren't encrypted
      memcpy( encrypted_buffer, send_buffer, 5 );
      active_suite->bulk_encrypt( send_buffer + 5, send_buffer_size - 5,
        encrypted_buffer + 5, parameters->IV, parameters->key );
      free( send_buffer );
      send_buffer = encrypted_buffer;
    }
   
    if ( send( connection, ( void * ) send_buffer, send_buffer_size, 0 ) <
         send_buffer_size )

The original send buffer is encrypted and then thrown away; the encrypted buffer is the send buffer, which is what's transmitted over the wire. Notice that the header itself is not encrypted; this is necessary so that the receiver knows how many bytes to process in the current packet. Everything following the header is encrypted, though.

Adding Support for Stream Ciphers

The encryption routine in Listing 6-64 almost works, but not quite. Recall from the calculate_keys routine in Listing 6-41 that if the cipher suite called for an initialization vector, one was computed, but otherwise, parameters->IV was left as a null pointer. This is fine for block ciphers, but causes a failure if you try to use this routine for the RC4 routine, which expects a state vector in this position.

You might try to get around this by changing the IV size in the cipher suite declaration to 256 so that you'd get a state vector here. Unfortunately, this won't work: You'd get 256 pseudo-random bytes rather than an array of 0's as rc4_encrypt expects on its first call. The ideal way to handle this would be to define a cipher_init routine that should be called on first invocation. However, the simple hack in Listing 6-65 works well enough:

Listing 6-65: "tls.c" calculate_keys with a special RC4 exception

static void calculate_keys( TLSParameters *parameters )

{
...
 switch ( suite->id )
 {
   case TLS_RSA_EXPORT_WITH_RC4_40_MD5:
   case TLS_RSA_WITH_RC4_128_MD5:
   case TLS_RSA_WITH_RC4_128_SHA:
   case TLS_DH_anon_EXPORT_WITH_RC4_40_MD5:
   case TLS_DH_anon_WITH_RC4_128_MD5:
     {
     rc4_state *read_state = malloc( sizeof( rc4_state ) );
     rc4_state *write_state = malloc( sizeof( rc4_state ) );
     read_state->i = read_state->j = write_state->i = write_state->j = 0;
     send_parameters->IV = ( unsigned char * ) read_state;
     recv_parameters->IV = ( unsigned char * ) write_state;
     memset( read_state->S, '\0', RC4_STATE_ARRAY_LEN );
     memset( write_state->S, '\0', RC4_STATE_ARRAY_LEN );
     }
     break;
   default:
     break;
 }

 free( key_block );

At this point, send_parameters->IV is no longer necessarily an IV, but a void pointer to the state of the cipher suite. Although the code would be clearer if it were renamed, the specification refers specifically to IV in several places, so leave it this way.

Updating Each Invocation of send_message

Of course, because you're now applying the active encryption function to every sent message, you must also go through and update each invocation of send_ message to include the active ProtectionParameters, as in Listing 6-66.

Listing 6-66: "tls.c" with protection parameters sent to send_message

static int send_handshake_message( int connection,
                                  int msg_type,
                                  const unsigned char *message,
                                  int message_len,
                                  TLSParameters *parameters )
{
...
 return send_message( connection, content_handshake, send_buffer,
   send_buffer_size, &parameters->active_send_parameters );
...
static int send_alert_message( int connection,
                              int alert_code,
                              ProtectionParameters *parameters )
{
...
 return send_message( connection, content_alert, buffer, 2, parameters );
...
static int send_change_cipher_spec( int connection, TLSParameters *parameters )
{
...
 send_message( connection, content_change_cipher_spec, send_buffer, 1,
               &parameters->active_send_parameters );xs
...
static int receive_tls_msg( int connection,
                           TLSParameters *parameters )
{
...
       if ( ( status = send_alert_message( connection, illegal_parameter,
              &parameters->active_send_parameters ) ) )
...
         read_pos = parse_server_hello( read_pos, handshake.length, parameters );
         if ( read_pos == NULL )  /* error occurred */
         {
           send_alert_message( connection, illegal_parameter,
             &parameters->active_send_parameters );
...
           read_pos = parse_finished( read_pos, handshake.length, parameters );
           if ( read_pos == NULL )
           {
             send_alert_message( connection, illegal_parameter,
               &parameters->active_send_parameters );

Notice that the alert messages need to be updated, too. If an alert is sent after a handshake has completed, the alert itself must be encrypted. Although this won't come up during an initial handshake, send_alert_message must include a ProtectionParameters value because send_message does.

Decrypting and Authenticating

Just as encryption was handled entirely within send_message, decrypting can be handled entirely within receive_tls_msg. This time, you don't even need to update the parameter list because it already accepts a TLSParameters; it just needs to look at the active receive parameters and apply them as necessary. Decryption and authentication is shown in Listing 6-67.

Listing 6-67: "tls.c" receive_tls_msg with decrypt support

static int receive_tls_msg( int connection,
                           TLSParameters *parameters )
{
 TLSPlaintext  message;
 unsigned char *read_pos, *msg_buf, *decrypted_message, *encrypted_message;
 unsigned char header[ 5 ];  // size of TLSPlaintext
 int bytes_read, accum_bytes;
 int decrypted_length;
// Read header as usual – header is not encrypted
...
   encrypted_message = ( char * ) malloc( message.length );

   // keep looping & appending until all bytes are accounted for
   accum_bytes = 0;

   msg_buf = encrypted_message;
   while ( accum_bytes < message.length )
   {
// Read the buffer as before, but update encrypted_message now
...
   }
   // If a cipherspec is active, all of "encrypted_message" will be encrypted.
   // Must decrypt it before continuing.  This will change the message length
   // in all cases, since decrypting also involves verifying a MAC (unless the
   // active cipher spec is NULL_WITH_NULL_NULL).
   decrypted_message = NULL;
   decrypted_length = tls_decrypt( header, encrypted_message, message.length,
     &decrypted_message, &parameters->active_recv_parameters );

   free( encrypted_message );

   if ( decrypted_length < 0 )
   {
     send_alert_message( connection, bad_record_mac,
       &parameters->active_send_parameters );
     return −1;
   }
   parameters->active_recv_parameters.seq_num++;
 }

read_pos = decrypted_message;
 if ( message.type == content_handshake )
 {
...
 }
 free( decrypted_message );

 return decrypted_length;
}

And everything else stays the same; at this point, the message has been decrypted and can be processed just as if it had been received in plaintext. In fact, this code is necessary to process a server finished handshake message because, per the specification, server finished is always sent using the active cipher spec.

The heavy lifting of decryption is handled by tls_decrypt, which is responsible for not only decrypting the buffer, but verifying its MAC as well. If the MAC doesn't verify, tls_decrypt returns –1; otherwise it returns the length of the decrypted buffer. As mentioned earlier, this is always different than the original buffer length, even for stream ciphers, as the tls_decrypt function strips off the MAC after decrypting the block.

Without further ado, tls_decrypt itself is presented in Listing 6-68.

Listing 6-68: "tls.c" tls_decrypt

/**
 * Decrypt a message and verify its MAC according to the active cipher spec
 * (as given by "parameters").  Free the space allocated by encrypted message
 * and allocate new space for the decrypted message (if decrypting is "identity",
 * then decrypted will point to encrypted).  The caller must always issue a
 * "free decrypted_message".
 * Return the length of the message, or −1 if the MAC doesn't verify.  The return
 * value will almost always be different than "encrypted_length", since it strips
 * off the MAC if present as well as bulk cipher padding (if a block cipher
 * algorithm is being used).
 */
static int tls_decrypt( const unsigned char *header, // needed for MAC verification
                       unsigned char *encrypted_message,
                       short encrypted_length,
                       unsigned char **decrypted_message,
                       ProtectionParameters *parameters )
{
 short decrypted_length;
 digest_ctx digest;
 unsigned char *mac_buffer;
 int sequence_number;
 short length;
 CipherSuite *active_suite = &( suites[ parameters->suite ] );

*decrypted_message = ( unsigned char * ) malloc( encrypted_length );

 if ( active_suite->bulk_decrypt )
 {
   active_suite->bulk_decrypt( encrypted_message, encrypted_length,
     *decrypted_message, parameters->IV, parameters->key );
   decrypted_length = encrypted_length;
   // Strip off padding
   if ( active_suite->block_size )
   {
     decrypted_length -= ( (*decrypted_message)[ encrypted_length - 1 ] + 1 );
   }
 }
 else
 {
   // Do nothing, no bulk cipher algorithm chosen.
   // Still have to memcpy so that "free" in caller is consistent
   decrypted_length = encrypted_length;
   memcpy( *decrypted_message, encrypted_message, encrypted_length );
 }

 // Now, verify the MAC (if the active cipher suite includes one)
 if ( active_suite->new_digest )
 {
   active_suite->new_digest( &digest );

   decrypted_length -= ( digest.hash_len * sizeof( int ) );

   // Allocate enough space for the 8-byte sequence number, the TLSPlainText
   // header, and the fragment (e.g. the decrypted message).
   mac_buffer = malloc( 13 + decrypted_length );
   memset( mac_buffer, 0x0, 13 + decrypted_length );
   sequence_number = htonl( parameters->seq_num );
   memcpy( mac_buffer + 4, &sequence_number, sizeof( int ) );

   // Copy first three bytes of header; last two bytes reflected the
   // message length, with MAC attached.  Since the MAC was computed
   // by the other side before it was attached (obviously), that MAC
   // was computed using the original length.
   memcpy( mac_buffer + 8, header, 3 );
   length = htons( decrypted_length );
   memcpy( mac_buffer + 11, &length, 2 );
   memcpy( mac_buffer + 13, *decrypted_message, decrypted_length );

   hmac( parameters->MAC_secret, digest.hash_len * sizeof( int ),
     mac_buffer, decrypted_length + 13, &digest );

   if ( memcmp( digest.hash,
                (*decrypted_message) + decrypted_length,
                digest.hash_len * sizeof( int ) ) )

{
     return −1;
   }

   free( mac_buffer );
 }

 return decrypted_length;
}

This is essentially the inverse of the encryption performed in send_message. There aren't any real surprises here; the buffer is decrypted and a MAC buffer is built and verified. The MAC buffer is built the same as it is in send_message; if you're so inclined, you could probably refactor this into a common routine and share it between the two. After the local MAC has been computed, verifying it is just a matter of comparing it, byte-for-byte, with the MAC that the server sent.

TLS Send

That's it. The TLS handshake is complete, and tls_connect returns control back up to the calling function, which, in this case, is the main routine of the HTTPS utility. From this point on, nothing can be sent on this connection unless it's encrypted, authenticated, and prepended with a correct TLS header. So, as you recall from Listing 6-3, each call to the socket-level send function is replaced with a call to tls_send. With the infrastructure developed in the previous sections, this is simple to implement, in Listing 6-69.

Listing 6-69: "tls.c" tls_send

int tls_send( int connection,
             const char *application_data,
             int length,
             int options,
             TLSParameters *parameters )
{
 send_message( connection, content_application_data, application_data, length,
   &parameters->active_send_parameters );

return length;
}

This is nothing but a wrapper around send_message, which examines the active protection parameters and applies them to the data buffer before sending it on.

TLS Receive

tls_recv, shown in Listing 6-70, can take advantage of the decrypt support added to receive_tls_message. However, receive_tls_message expected to handle the received packet after decrypting it. tls_recv wants the data itself, so receive_tls_message also needs to be updated as shown in Listing 6-71 to optionally pass back a chunk of decrypted data.

Listing 6-70: "tls_recv"

int tls_recv( int connection, char *target_buffer, int buffer_size, int options,
             TLSParameters *parameters )
{
 int bytes_decrypted = 0;

 bytes_decrypted = receive_tls_msg( connection, target_buffer, buffer_size,
   parameters );

 return bytes_decrypted;
}

Listing 6-71: "receive_tls_msg" with optional response buffer

static int receive_tls_msg( int connection,
                           char *buffer,
                           int bufsz,
                           TLSParameters *parameters )
{
 if ( message.type == content_handshake )
 {
...
 else if ( message.type == content_application_data )
 {
   memcpy( buffer, decrypted_message, decrypted_length );
 }
...

This means that the invocations of receive_tls_msg in tls_connect must also be updated to pass a null pointer, shown in Listing 6-72.

Listing 6-72: "tls.c" tls_connect with receive_tls_msg calls updated

int tls_connect( int connection,
                TLSParameters *parameters )
{

...
 while ( !parameters->server_hello_done )
 {
   if ( receive_tls_msg( connection, NULL, 0, parameters ) < 0 )
...
 while ( !parameters->server_finished )
 {
   if ( receive_tls_msg( connection, NULL, 0, parameters ) < 0 )
...

This change is just necessary to satisfy the compiler; at no point does receive_tls_msg actually check this pointer for NULL. If a handshake message is expected and application data arrives, you get an unhelpful segmentation violation.

There's still a problem here. If you glance back at the implementation of display_result in Listing 6-3, you notice that it calls for a buffer of data at a time — in this case, 255 bytes. However, there's no guarantee that this much data is available in the next TLS packet, nor is there a guarantee that there isn't more. Therefore, whenever the application calls tls_recv with a buffer set to receive a chunk of application data, the routine must check to see if there's any left over from a previous call, as well as store any that was decrypted but that the user didn't ask for.

To accommodate variable-length input, follow these steps:

1. Add a buffer to the end of the TLSParameters structure as shown in Listing 6-73.

   Listing 6-73: "tls.h" TLSParameters with buffering support

   typedef struct
   {
   ...
    char                  *unread_buffer;
    int                   unread_length;
   }
   TLSParameters;

2. receive_tls_msg must also be updated to check this buffer for any unread bytes whenever the application asks for more data, as well as to buffer any data that was decrypted but that the user didn't ask for. This is shown in Listing 6-74.

Listing 6-74: "tls.c" receive_tls_msg with buffering support

void init_parameters( TLSParameters *parameters,
                     int renegotiate )
{
...
 parameters->server_hello_done = 0;
 parameters->server_finished = 0;

parameters->unread_buffer = NULL;
 parameters->unread_length = 0;
}

static int receive_tls_msg( int connection,
                           char *buffer,
                           int bufsz,
                           TLSParameters *parameters )
{
...
 int bytes_read, accum_bytes;
 int decrypted_length;

 // First, check to see if there's any data left over from a previous read.
 // If there is, pass that back up.
 // This means that if the caller isn't quick about reading available data,
 // TLS alerts can be missed.
 if ( parameters->unread_buffer != NULL )
 {
   decrypted_message = parameters->unread_buffer;
   decrypted_length = parameters->unread_length;
   parameters->unread_buffer = NULL;
   parameters->unread_length = 0;

   message.type = content_application_data;
 }
 else
 {
   if ( recv( connection, header, 5, 0 ) <= 0 )
...
 }

 read_pos = decrypted_message;
...
 else if ( message.type == content_application_data )
 {
   if ( decrypted_length <= bufsz )
   {
     memcpy( buffer, decrypted_message, decrypted_length );
   }
   else
   {
     // Need to hang on to a buffer of data here and pass it back for the
     // next call
     memcpy( buffer, decrypted_message, bufsz );
     parameters->unread_length = decrypted_length - bufsz;
     parameters->unread_buffer = malloc( parameters->unread_length );
     memcpy( parameters->unread_buffer, decrypted_message + bufsz,
       parameters->unread_length );

decrypted_length = bufsz;
   }
 }

Now, if there's any data left over from a previous call, as much as the caller asked for is returned. If there's less than the caller asked for, only what's available is returned; the caller must invoke another call if it wants the next chunk of data. This could be made more robust — and more complicated — if it went ahead and read the next available TLSMessage, concatenated that on top of whatever it had buffered, and tried to fill up the buffer the client requested. Of course, in any case, if the caller requested less data than is available in the buffer, the remaining data must be held on to. receive_tls_message accomplishes this by masquerading any buffered data as decrypted_message; whether the client consumes all of it or not, it ends up in the decrypted_length <= bufsz else-case and is rebuffered.

If you closely compared the display_result listing of 6-3 to the display_result Listing 1-7 in Chapter 1, you may have noticed one seemingly trivial difference: The plaintext HTTP routine reads until recv returns 0 bytes, indicating EOF. The secured implementation reads until tls_recv returns less than 0. Why?

To frustrate attackers, it's acceptable for compliant TLS implementations to return empty packets consisting of nothing but padding. When this function receives such a packet, it removes the padding and the MAC and reports the returned data length as 0.

"So," you must certainly be wondering, "if tls_recv can't return 0 to indicate an EOF condition, how does TLS handle an end-of-stream?" Read on.

Implementing TLS Shutdown

SSLv2 didn't have a specific shutdown mechanism; when either side was done using the connection, it just issued a regular TCP FIN packet. The problem with this was that it's easy for a man in the middle to generate a FIN packet; it's not encrypted or authenticated in any way. This can be used to perform truncation attacks as detailed in Chapter 4.

As a result, TLS has a special way to indicate shutdown. The side wishing to shut the connection down sends an alert with the close_notify code of 0. Because this is a TLS message, it's subject to the standard encryption and authentication values currently in force and is protected. tls_shutdown is shown in Listing 6-75.

Listing 6-75: "tls.c" tls_shutdown

int tls_shutdown( int connection, TLSParameters *parameters )
{
 send_alert_message( connection, close_notify,

&parameters->active_send_parameters );
 if ( parameters->unread_buffer )
 {
   free( parameters->unread_buffer );
 }
 free_protection_parameters( &parameters->pending_send_parameters );
 free_protection_parameters( &parameters->pending_recv_parameters );
 free_protection_parameters( &parameters->active_send_parameters );
 free_protection_parameters( &parameters->active_recv_parameters );

 return 1;
}

This routine goes ahead and frees any memory that was allocated by the connection; it mostly relies on free_protection_parameters to free the MAC secrets, keys, and IVs. This is shown in Listing 6-76.

Listing 6-76: "tls.c" free_protection_parameters

static void free_protection_parameters( ProtectionParameters *parameters )
{
 if ( parameters->MAC_secret )
 {
   free( parameters->MAC_secret );
 }
 if ( parameters->key )
 {
   free( parameters->key );
 }
 if ( parameters->IV )
 {
   free( parameters->IV );
 }
}

Another benefit of the dedicated TLS shutdown protocol is that either side can switch back to plaintext if desired without severing the connection. This could potentially be useful if for regulatory reasons some data had to be sent in the clear. I'm not aware of any applications that take advantage of this, but it's nice to know that the flexibility is there if you need it.

Examining HTTPS End-to-End Examples (TLS 1.0)

You can, and should, compile the code presented in this chapter and try to connect to a few different public secure websites. You may have to scrounge around to find one that doesn't require a context to be previously established; your bank's login landing page might be a good choice (if it's not SSL enabled, consider a different bank).

For illustration purposes, though, you can also run an instance of the Apache web server locally, configure it to accept HTTPS connections, and connect to it. The tcpdump utility can be used to monitor exactly what's passed back and forth over a socket connection.

Dissecting the Client Hello Request

Now you can run your https command-line client to connect to an SSL-enabled website and monitor the packets exchanged. After invoking tcpdump to run in the background, start up an instance of https:

[jdavies@localhost ssl]$ ./https https://localhost/index.html

The tcpdump output starts with the standard expected TCP 3-way handshake:

[root@localhost ssl]# /usr/sbin/tcpdump -i lo -s 0 -X tcp port 443
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
12:37:03.937423 IP localhost.localdomain.56047 > localhost.localdomain.https: S
506618802:506618802(0) win 32792
<mss 16396,sackOK,timestamp 12673266 0,nop,wscale 7>
       0x0000:  4500 003c 0340 4000 4006 397a 7f00 0001  E..<.@@.@.9z....
       0x0010:  7f00 0001 daef 01bb 1e32 63b2 0000 0000  .........2c.....
       0x0020:  a002 8018 cf4a 0000 0204 400c 0402 080a  .....J....@.....
       0x0030:  00c1 60f2 0000 0000 0103 0307            ..`.........
12:37:03.937430 IP localhost.localdomain.https > localhost.localdomain.56047: S
505995792:505995792(0) ack 506618803 win 32768
<mss 16396,sackOK,timestamp 12673267 12673266,nop,wscale 7>
       0x0000:  4500 003c 0000 4000 4006 3cba 7f00 0001  E..<..@.@.<.....
       0x0010:  7f00 0001 01bb daef 1e28 e210 1e32 63b3  .........(...2c.
       0x0020:  a012 8000 6d64 0000 0204 400c 0402 080a  ....md....@.....
       0x0030:  00c1 60f3 00c1 60f2 0103 0307            ..`...`.....
12:37:03.937459 IP localhost.localdomain.56047 > localhost.localdomain.https: .
       ack 1 win 257 <nop,nop,timestamp 12673267 12673267>
       0x0000:  4500 0034 0341 4000 4006 3981 7f00 0001  E..4.A@.@.9.....
       0x0010:  7f00 0001 daef 01bb 1e32 63b3 1e28 e211  .........2c..(..
       0x0020:  8010 0101 5587 0000 0101 080a 00c1 60f3  ....U.........`.
       0x0030:  00c1 60f3                                ..`.

After the three-way handshake is complete, the TLS protocol takes over:

• The first actual packet exchanged is the client hello. The client hello starts at byte 0x0035. The first five bytes of the data packet is the TLS header 160301002d. 0x16 is the type of the message (content_handshake); 0x0301 is the version of SSL (3.1); and 0x002d is the length of the contained packet — 45 bytes. Every packet sent over this connection must now start with a TLS header.
• Next is the handshake header 0x01000029. 0x01 is the handshake message type (client hello), and 0x000029 is the strange three-byte length that indicates that the actual payload is 41 bytes.
• Following this, finally, is the client hello message itself. The first two bytes are the protocol version 0x0301 again, followed by 32 bytes of random data (although as you can see the data isn't particularly random in this case). Remember that the random structure is required to begin with a four-byte "seconds since January 1, 1970" time, followed by 28 actually random bytes. As you can see, this is passed in the clear. Anything you can see with tcpdump, a malicious intruder can see with a packet sniffer as well.

After the random bytes, a 0 byte is supplied indicating that no session ID follows. The session ID structure is followed by the cipher suites list, which must be supplied. First the two-byte length of the cipher suites indicates that there are two bytes of cipher suite (that is, one cipher suite). This length declaration is followed by the cipher suite list itself — here 0x002F, which is the single cipher suite TLS_RSA_WITH_AES_128_CBC_SHA. Notice that the length of 2 indicates that there follows two bytes of cipher suites, which is one cipher suite because each suite is two bytes long. The client hello message ends with the list of supported compression methods, which is simply "no compression."

After the client hello, the server acknowledges the packet according to the standard rules of TCP.

12:37:03.938244 IP localhost.localdomain.https > localhost.localdomain.56047: .
 ack 51 win 256 <nop,nop,timestamp 12673267 12673267>
       0x0000:  4500 0034 82e2 4000 4006 b9df 7f00 0001  E..4..@.@.......
       0x0010:  7f00 0001 01bb daef 1e28 e211 1e32 63e5  .........(...2c.
       0x0020:  8010 0100 5556 0000 0101 080a 00c1 60f3  ....UV........`.
       0x0030:  00c1 60f3                                ..`.

Because this is a book about SSL/TLS and not about TCP, packet acknowledgments are omitted from the remainder of this section.

Dissecting the Server Response Messages

The server then sends back the server hello, certificate and server hello done messages. Notice that in this case, they're not included in a single TLS packet, although the specification allows for this. This single TCP packet includes three individual top-level TLS messages.

The first is of length 75 bytes (0x004a) and contains a handshake message of type 2 (server hello). The server hello packet, of course, starts with the version number 0x0301, followed by the 32-byte server random structure. Servers normally assign a session ID to every connection, so the session ID in this case is non-empty; it is 32 bytes long and is equal to 07e76fed29bfc73b710b0c2757fcd 1a7b325561b232906ceb3d8a0347f3bd2f5. The server hello finishes out by selecting a cipher suite and a compression method. Obviously it chose 0x002F and 0x00, respectively, because those were the only two choices it was given. If it didn't recognize, or didn't support, any of these choices then this server hello message would have instead been an alert.

A server hello is generally followed by a server certificate message. This is not necessarily always the case, as you see in Chapter 8, but generally it is. Here the TLS header indicates that the following message is 344 (0x02d8) bytes long and is a server handshake certificate message (handshake message 0x0b). This message in turns starts out with a three-byte length declaration — 0x0002d1 indicating that the following certificate chain is 337 bytes long. After this is yet another length declaration 0x0002ce, indicating that the first certificate in this certificate chain — in this case, the only certificate in this chain — is 334 bytes long. Finally, the ASN.1 DER representation of the server's certificate follows.

16 0301 02d8 0b00 02d4 0002 d100  ./..............
       0x0090:  02ce 3082 02ca 3082 0274 a003 0201 0202  ..0...0..t......
       0x00a0:  0900 a72f c757 5f51 e56f 300d 0609 2a86  .../.W_Q.o0...*.
       0x00b0:  4886 f70d 0101 0505 0030 7931 0b30 0906  H........0y1.0..
....

The server's side of the exchange is rounded out by the server hello done message. As shown in the next line of code, this contains no actual data; it's just a TLS header indicating that the packet is four bytes whose contents are an empty handshake message of type 0x0e.

0x0360:  1603 0100 040e 0000 00                   .........

Dissecting the Key Exchange Message

If the client was able to parse the certificate and has decided to trust it, and the certificate contains enough information to satisfy the key exchange method in the selected cipher suite, it must now send a key exchange message. Because the key exchange method in this case is RSA, this involves making up a random premaster secret, encrypting it using the public key that the server sent in the certificate message, and sending that on.

After the standard TLS header and handshake message header, an RSA key exchange message starts with a two-byte length, followed by the data. Because the key used here is 512 bits, the RSA-encrypted data is also 512 bits, 64 bytes.

12:37:04.007143 IP localhost.localdomain.56047 > localhost.localdomain.https: P
 51:126(75) ack 822 win 270 <nop,nop,timestamp 12673336 12673268>
       0x0000:  4500 007f 0344 4000 4006 3933 7f00 0001  E....D@.@.93....
       0x0010:  7f00 0001 daef 01bb 1e32 63e5 1e28 e546  .........2c..(.F
       0x0020:  8018 010e fe73 0000 0101 080a 00c1 6138  .....s........a8
       0x0030:  00c1 60f4 1603 0100 4610 0000 4200 407a  ..`.....F...B.@z
       0x0040:  8d74 369f 97e3 86e4 494f 5e71 1e0f 2059  .t6.....IO^q...Y
       0x0050:  6583 04d2 d432 ce33 1067 251c 5a4b edef  e....2.3.g%.ZK..
       0x0060:  d149 935b 9256 1a20 959a b9e4 0427 175e  .I.[.V.......'.^
       0x0070:  6d70 cd0d af00 e3c2 c977 ab11 5af5 f7    mp.......w..Z..

Because I have access to the private key that corresponds with the public key used in this exchange, I can decrypt this message using the rsa code developed in Chapter 3. If I didn't have this private key then I'd be out of luck trying to interpret this or any subsequent message in this connection. Of course, the private key appears nowhere in this exchange; the security of TLS hinges around this fact. The private key exponent is

EAFF403432CBD12A7F7174C209F5364398E62F4A1B8F9B7C32B6CE190E716696D3E866E09
AF5367743EA5CC7903515D05D667E5480C562BCC0821F4A670B27F9

and the modulus is

EEB4761CAAE2E34F56CBC3AFE479E88589A9AB398250687ADE502D53EEFAD78C6E3CF8946
301095BD0BD7A60089737E2F1BB40A152E12DDCDBC95BD86661DA4F

so the RSA-encrypted message can be decoded as

[jdavies@localhost ssl]$ ./rsa -d \
0xEEB4761CAAE2E34F56CBC3AFE479E88589A9AB398250687ADE502D53EEFAD78\
C6E3CF8946301095BD0BD7A60089737E2F1BB40A152E12DDCDBC95BD86661DA4F \

0xEAFF403432CBD12A7F7174C209F5364398E62F4A1B8F9B7C32B6CE190E71669\
6D3E866E09AF5367743EA5CC7903515D05D667E5480C562BCC0821F4A670B27F9 \
0x7a8d74369f97e386e4494f5e711e0f2059658304d2d432ce331067251c5a4be\
defd149935b92561a20959ab9e40427175e6d70cd0daf00e3c2c977ab115af5f7
02 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 00 03 01 02 03 04 05 06 07 08 09 0a 0b
0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26
27 28 29 2a 2b 2c 2d 2e 2f 00
030102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f2021222324252627
28292a2b2c2d2e2f

After stripping off the padding, you're left with a premaster secret whose first two bytes are the SSL version 0301 and then the premaster secret which, in this case, isn't particularly secure.

According to the protocol, the client must now send a change cipher spec message. This is a pretty boring message, consisting of a single byte:

12:37:04.046440 IP localhost.localdomain.56047 > localhost.localdomain.https: P
 126:185(59) ack 822 win 270 <nop,nop,timestamp 12673376 12673376>
       0x0000:  4500 006f 0345 4000 4006 3942 7f00 0001  E..o.E@.@.9B....
       0x0010:  7f00 0001 daef 01bb 1e32 6430 1e28 e546  .........2d0.(.F
       0x0020:  8018 010e fe63 0000 0101 080a 00c1 6160  .....c........a`
       0x0030:  00c1 6160 1403 0100 0101

Notice that the message type is 14, not 16; change cipher spec is a separate top-level type. Other than this, it's just the single, plain byte 0x01.

Decrypting the Encrypted Exchange

Change cipher spec is followed by a finished message. This message is encrypted using the newly negotiated parameters, so must be decrypted. My network stack decided to aggregate these two messages into a single TCP push, but there's no particular reason why it would have to be this way.

The finished message starts with the standard, expected (unencrypted) TLS message header, and tells the recipient that the message that follows is 48 bytes long. Everything following the header is encrypted using the negotiated session keys.

1603 0100 301c  ..a`..........0.
       0x0040:  e1a2 aaf3 1267 749d b1e7 701d 8f95 98d8  .....gt...p.....
       0x0050:  cd65 526b 90a4 d8d0 7536 1dd4 6a26 4787  .eRk....u6..j&G.
       0x0060:  4f49 5ac7 8c89 7dcc 1ee3 ad37 b25e 8d    OIZ...}....7.^.

Because you know the private key that was used for the key exchange, you know the premaster secret. Therefore, you can work forward and discover the master secret and the key material. Recall that the master secret was generated by applying the PRF to the premaster secret. Specifically, the PRF takes three input parameters: the secret, the label, and the seed. Recall that, for master secret computation, the label is the fixed string "master secret", and the seed is the client random followed by the server random. The secret is, of course, the premaster secret.

You can reproduce the master secret with this information:

[jdavies@localhost ssl]$ ./prf \
0x030102030405060708090a0b0c0d0e0f101112131415161718\
191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f "master secret" \
0x4c4f193f00000000000000000000000000000000000000000000000000000000\
4c4f193fe58513ecbaf3966b6ab96fbfbee41052ae1b643c93174f38a6548e53 48
2dee06e1ba5e41722dd1c24286ae5a0cfbd89b38bd4688fa
fb97c3dc05a2647be55490ba733406807df8023ae75d0a0a

The master secret is always exactly 48 bytes long.

The master secret itself isn't used for key material; instead, it's used as a seed into the PRF again. The PRF also needs to know how many bytes to generate in order to determine how many times to iterate. How many bytes are needed for the selected cipher suite? AES-128 uses 16-byte keys, and a 16-byte block size, so you need 32 bytes of keying material and 32 bytes of IV — one of each for each side of the conversation. SHA-1 uses a 20-byte MAC key, so you need 40 bytes of MAC secret; this works out to 104 bytes of keying material.

You can reproduce the keys by running the PRF algorithm with this input:

[jdavies@localhost ssl]$ ./prf \
0x2dee06e1ba5e41722dd1c24286ae5a0cfbd89b38bd4688\
fafb97c3dc05a2647be55490ba733406807df8023ae75d0a0a "key expansion" \
0x4c4f193fe58513ecbaf3966b6ab96fbfbee41052ae1b643c93174f38a6548e\
534c4f193f00000000000000000000000000000000000000000000000000000000 104
3a1ee25b3fa7efb9a2c8f112de47c3276917a2bbb0f81a9a389dbc82c3fc2a073e97aa31087f312
96dbb1276d318c6551ef8245888420cf4c2a545f7a8515c42c367599cdd52cf6ef6bb0cc22615db
9c0d93ad3c21d2f58ed18324dbfb7645103f191455421cceca

Notice that the seed in this case is the server random, followed by the client random, whereas for the master secret expansion, it was the other way around. If you overlook this fact, you will end up tearing your hair out for days trying to figure out why your code isn't working. (Don't ask me how I know).

The key material block starts with the MAC secrets, then the keys, then the initialization vectors, so this works out to what is shown in Table 6-1.

Table 6-1: The Key Material Block

Armed with the keys and initialization vectors, you can go ahead and decrypt the 48-byte finished message:

[jdavies@localhost ssl]$ ./aes -d 0x6dbb1276d318c6551ef8245888420cf4 \
0xf6bb0cc22615db9c0d93ad3c21d2f58e \
0x1ce1a2aaf31267749db1e7701d8f9598d8cd65526b90a4d8\
d075361dd46a2647874f495ac78c897dcc1ee3ad37b25e8d
1400000ce3945aa7b226794d96cfcaf7b79febcb41c2bd7b48a77d7b26f
958296dc1467c0b0b0b0b0b0b0b0b0b0b0b0b

This is, as expected, a handshake message of type 0x14 (finished), followed by a three-byte length indicating that the message is 12 bytes long. If you recall, this is correct; the finished message is 12 bytes, generated by the PRF, seeded with the hash values of all of the handshake messages up to this point.

However, the message body is significantly longer. It includes 20 bytes of MAC and an extra 12 bytes of padding, because 4+12+20 (header + contents + MAC) is equal to 36, which is not an even multiple of the 16-byte block size. Notice that there are 12 bytes of padding, but the value of the padding byte is 0x0b (11); there's always one byte on the end that indicates how much padding there is, which isn't actually considered padding.

You can verify the MAC as well. Remember that the MAC is the HMAC function of the sequence number, the TLS header, and the contents. Because this is the first securely exchanged packet, the sequence number is 0. However, be careful. If you try to do this

[jdavies@localhost ssl]$ ./hmac -sha1 \
0x3a1ee25b3fa7efb9a2c8f112de47c3276917a2bb \
0x000000000000000016010100301400000ce3945aa7b226794d96cfcaf7
dfc0e0ef8c50a3bc3f9db3d87168a99ce3c4f99d

you'll get the wrong answer. The MAC is actually b79febcb41c2bd7b48a77d 7b26f958296dc1467c; what went wrong?

Remember that the MAC was generated from the sequence number and what the TLS header would have looked like, had there been no encryption or MAC — that is, the length before padding and MAC. So, to get the correct answer, you have to subtract the 32 bytes of MAC and padding from the length in the TLS header and compute:

[jdavies@localhost ssl]$ ./hmac -sha1 \
0x3a1ee25b3fa7efb9a2c8f112de47c3276917a2bb \
0x000000000000000016030100101400000ce3945aa7b226794d96cfcaf7
b79febcb41c2bd7b48a77d7b26f958296dc1467c

The server receives the finished message, decrypts it, verifies the MAC, and compares the verify data of e3945aa7b226794d96cfcaf7 to what it has computed so far. It then responds with its own change cipher spec and finished message:

12:37:04.047107 IP localhost.localdomain.https > localhost.localdomain.56047: P
  822:881(59) ack 185 win 256 <nop,nop,timestamp 12673376 12673376>
       0x0000:  4500 006f 82e6 4000 4006 b9a0 7f00 0001  E..o..@.@.......

0x0010:  7f00 0001 01bb daef 1e28 e546 1e32 646b  .........(.F.2dk
       0x0020:  8018 0100 fe63 0000 0101 080a 00c1 6160  .....c........a`
       0x0030:  00c1 6160 1403 0100 0101 1603 0100 30c8  ..a`..........0.
       0x0040:  5afc e4c0 1560 ec3b 4db9 6185 f4f4 f1b1  Z....`.;M.a.....
       0x0050:  bcb1 3528 c8a6 5862 f512 30e6 02d5 62a8  ..5(..Xb..0...b.
       0x0060:  6e4d f925 8048 d19b 0a2d 6296 4b6c e9    nM.%.H...-b.Kl.

The finished message is encrypted just as the client's was, but because you know the keys you can decrypt it:

[jdavies@localhost ssl]$ ./aes -d 0xc2a545f7a8515c42c367599cdd52cf6e \
0xd18324dbfb7645103f191455421cceca \
0xc85afce4c01560ec3b4db96185f4f4f1b1bcb13528c8a65\
862f51230e602d562a86e4df9258048d19b0a2d62964b6ce9
1400000c45c4904ac71a5948a7198e18b8618774e12b8f58f49216bcf
59a914f236b6fef0b0b0b0b0b0b0b0b0b0b0b0b

Of course, you must use the second set of keys to decrypt this properly. You can verify the MAC as well.

Also, notice that, in this case, the verify data is 45c4904ac71a5948a7198e18 — it does not match the verify data that the client sent. Why not? Because the client's finished message is included in the computation of the verify data that the server sends.

Exchanging Application Data

The TLS handshake is complete; it's time for the top-level protocol, in this case HTTP, to take over. The TLS header is present, but this time, it identifies application data:

12:37:04.049299 IP localhost.localdomain.56047 > localhost.localdomain.https: P
185:238(53) ack 881 win 270 <nop,nop,timestamp 12673378 12673376>
       0x0000:  4500 0069 0347 4000 4006 3946 7f00 0001  E..i.G@.@.9F....
       0x0010:  7f00 0001 daef 01bb 1e32 646b 1e28 e581  .........2dk.(..
       0x0020:  8018 010e fe5d 0000 0101 080a 00c1 6162  .....]........ab
       0x0030:  00c1 6160 1703 0100 301d 6070 ca35 be42  ..a`....0.`p.5.B
       0x0040:  29da cf8a 9654 391c 08a5 981a 8d15 e87a  )....T9........z
       0x0050:  c058 437c 834d 957a d446 b9eb dd78 f392  .XC|.M.z.F...x..
       0x0060:  0375 de85 e852 b6e6 c0                   .u...R...

This is decrypted just like the finished message was. However, remember that the initialization vector used to decrypt this packet is the last 16 bytes of the previously sent packet:

[jdavies@localhost ssl]$ ./aes -d 0x6dbb1276d318c6551ef8245888420cf4 \
0x874f495ac78c897dcc1ee3ad37b25e8d \
0x1d6070ca35be4229dacf8a9654391c08a5981a8d15e87ac\
058437c834d957ad446b9ebdd78f3920375de85e852b6e6c0
474554202f696e6465782e68746d6c20485454502f312e310d0a200e496b9e2dadcf20bb5c9
2c4047baf348b1f7b0101

After the MAC is verified and removed and the padding is stripped off, the payload of 474554202f696e6465782e68746d6c20485454502f312e310d0a is handed off to the HTTP protocol. This is just the ASCII encoding of "GET /index.html HTTP/1.0" and the CRLF delimiter. HTTP doesn't indicate its length; the TLS header gave the TLS layer enough information to decrypt and strip off the MAC, but it's up to HTTP to figure out what to do with this message.

The server's response is omitted here. After the server has responded, though, it sends

12:37:04.089204 IP localhost.localdomain.https > localhost.localdomain.56047: P
  1291:1328(37) ack 344 win 256 <nop,nop,timestamp 12673418 12673418>
       0x0000:  4500 0059 82ea 4000 4006 b9b2 7f00 0001  E..Y..@.@.......
       0x0010:  7f00 0001 01bb daef 1e28 e71b 1e32 650a  .........(...2e.
       0x0020:  8018 0100 fe4d 0000 0101 080a 00c1 618a  .....M........a.
       0x0030:  00c1 618a 1503 0100 20ae dd34 8655 8551  ..a........4.U.Q
       0x0040:  3836 6592 0d73 dcda 4770 9798 dc2a c22c  86e..s..Gp...*.,
       0x0050:  79da e8c2 0945 6c4f 61                   y....ElOa

As you can see from the header, this is an alert. Of course, it's encrypted, but you know by now that this is a close_notify alert. This is followed by the normal TCP shutdown.

Differences Between SSL 3.0 and TLS 1.0

As mentioned previously, TLS is a minor revision to SSL 3.0, which was a major overhaul of SSLv2. There are few differences between SSL 3.0 and TLS 1.0; TLS defined a handful of new alert types and removed support for the Fortezza key exchange algorithm.

The most significant difference, and what necessitated a new version, was the introduction of the PRF. SSL 3.0 had a premaster secret, just like TLS 1.0, but it became the master secret by taking the MD5 hash of the premaster secret plus the SHA hash of the letter A plus the premaster secret, the client random, and the server random, followed by the same MD5 hash with A replaced by BB, and the same MD5 hash again with CCC instead of BB. The finished messages, the other place where the PRF shows up in TLS, were based directly on MD5 and SHA-1 hashes as well.

There's not much reason to go into any more detail on SSLv3 here. Because SSLv3 and TLS 1.0 are almost identical, it's a good bet that any server that supports one supports the other.

Differences Between TLS 1.0 and TLS 1.1

The TLS 1.0 protocol stood untouched for seven years after it was standardized in 1999. In 2006, Tim Dierks and Eric Rescorla drafted RFC 4346, specifying TLS 1.1 and making TLS 1.0 obsolete.

By and large, TLS 1.1 is not a significant change from TLS 1.0. It added some new cipher suites and some clarifications and implementation notes, but the most important change is that initialization vectors are no longer computed from the master secret; instead, they're prepended to each packet.

Why, you ask? Well, each record's IV is the most recently transmitted block. Although an attacker can't decrypt the packet, even a passive eavesdropper can see what the next IV is going to be. The attack is complex, but a dedicated attacker who can inject known plaintext into the stream can guess what the last plaintext block is and verify his guess by injecting a specially crafted plaintext block into the stream. It's not clear under what circumstances an attacker might be able to inject known plaintext, but not have access to the keys; perhaps he's sniffing traffic coming out of a call center, calls that call center, makes up an account ID, and hopes that that account ID is the next packet on a live session.

Theoretical or not, this is an easy enough attack to defend against, so TLS 1.1 does so. TLS 1.1 is not particularly common on the public Internet, even today, four years after it was drafted.

Two years after TLS 1.1 was drafted, it was made obsolete by RFC 5246, which specifies TLS 1.2. TLS 1.2 was a major modification of the TLS protocol and Chapter 9 is devoted to detailing the changes it introduced.