Table of Contents for

Network Security with OpenSSL

We need to examine three relevant object types: SSL_METHOD, SSL_CTX, and SSL. An SSL_METHOD represents an implementation of SSL functionality. In other words, it specifies a protocol version. OpenSSL provides populated SSL_METHOD objects and some accessor methods for them. They are listed in Table 5-1. The extent of our interaction with this type of object will be to select the protocol version we wish to support by making a function call from the table.

OpenSSL provides implementations for SSL Version 2, SSL Version 3, and TLS Version 1. Also, some SSLv23 functions don’t indicate a specific protocol version but rather a compatibility mode. In such a mode, a connection will report that it can handle any of the three SSL/TLS protocol versions. To reiterate, applications should not use SSLv2, since this protocol is known to have security flaws. Using an SSL_METHOD object retrieved by one of the functions in Table 5-1, we create an SSL_CTX object.

An SSL_CTX object will be a factory for producing SSL connection objects. This context allows us to set connection configuration parameters before the connection is made, such as protocol version, certificate information, and verification requirements. It is easiest to think of SSL_CTX objects as the containers for default values for the SSL connections to be made by a program. Objects of this type are created with the function SSL_CTX_new. This function takes only one argument, generally supplied from the return value of one of the functions in Table 5-1.

In general, an application will create just one SSL_CTX object for all of the connections it makes. From this SSL_CTX object, an SSL type object can be created with the SSL_new function. This function causes the newly created SSL object to inherit all of the parameters set forth in the context. Even though most of the settings are copied to the SSL object on invocation of SSL_new, the order in which calls are made to OpenSSL functions can cause unexpected behavior if we’re not careful.

The SSL protocol usually requires the server to present a certificate. The certificate contains credentials that the client may look at to determine if the server is authentic and can be trusted. As we know, a peer validates a certificate through verification of its chain of signers. Thus, to implement an SSL server correctly, we must provide certificate and chain information to the peer. The SSL protocol also allows the client to optionally present certificate information so that the server may authenticate it.

In general, server applications should always provide certificates to peers, and clients can do so optionally. The purpose and the desired security of the application should dictate whether client certificates are used. For instance, a server may request a client certificate, and if our client does not have one to present, we may not be able to establish a secure connection. Thus, it’s a good idea to implement client certificates if it makes sense to do so. On the other hand, server certificates are usually required, and, unless our goal is to create a completely nonauthenticated connection, we should implement them.

OpenSSL presents the client certificate to the server during handshakes, as long as we assign a certificate to the client and the server asks for it. This is actually a small violation of the TLS protocol. The protocol calls for the server to present a list of valid CAs, and the client should send a certificate only if it matches. In practice, this infraction of the standard should not affect anything, but the behavior may be fixed in future versions of OpenSSL.

The SSL API has several ways to incorporate certificate information into an SSL_CTX object. The function to use is SSL_CTX_use_certificate_chain_file . It loads the chain of certificates from the filename specified by the second argument. The file should contain the certificate chain in order, starting with the certificate for the application and ending with the root CA certificate. Each of these entries must be in PEM format.

In addition to loading the certificate chain, the SSL_CTX object must have the application’s private key. This key must correspond to the public key embedded within the certificate. The easiest way that we can supply this key to the context is through the SSL_CTX_use_PrivateKey_file function. The second argument specifies the filename, and the third specifies the type of encoding. The type is specified by using a defined name—either SSL_FILETYPE_PEM or SSL_FILETYPE_ASN1. It bears mentioning that this private key must be kept secret for the application to remain secure. Therefore, using an encrypted PEM format for on-disk storage is recommended; using triple DES in CBC mode is a good choice. The SSL_CTX will fail to incorporate an encrypted private key correctly unless the correct passphrase is supplied.

OpenSSL collects passphrases through a callback function. The default callback prompts the user on the terminal. For some applications, the default will not be acceptable. The function SSL_CTX_set_default_passwd_cb allows us to set the callback to something more appropriate for the application. The assigned function is invoked during the call to SSL_CTX_use_PrivateKey_file if the indicated file contains an encrypted key. Therefore, the callback should be set before making that call. In fact, the certificates in our chain could be encrypted even though there is nothing secret about them, and our callback function would be invoked to gather the passphrase. More accurately, we could state that the passphrase function is called any time a piece of encrypted information is loaded as a parameter to the SSL_CTX.

The callback function’s obligation is to copy the passphrase into the buffer that is supplied to it when it is called. The callback function is called with four arguments.

int passwd_cb(char * buf, int size, int flag, void *userdata);

There are two approaches to implementing the passphrase callback. The first method is simply to have the callback prompt the user, copy the collected passphrase to the buffer, and return. This method is viable for applications that need to decrypt a key only once, commonly on application startup. The second way to implement the callback is for the application to prompt the user for a passphrase on startup and store the collected information in a buffer. The passphrase can be added to the SSL_CTX as user data via the function SSL_CTX_set_default_passwd_cb_userdata . With this method, the callback itself only needs to copy the data from the fourth parameter to the first. This method is viable for applications that need to decrypt keys during normal operation in which constant user prompting is a nuisance.

At this stage, we will limit our discussion to the process of providing certificate information to the peer, rather than discussing the processes of validation. The validation problem will be discussed in Step 2.

Using the same example applications that we’ve already provided, let’s modify them with what we’ve learned about making SSL connections. Keep in mind that this example is not yet secure. It does not validate anything about the peer to which it connects; it merely provides the authentication information. The new version of our client in client1.c is shown in Example 5-5. The bold lines are those that we have added or changed.

  1  #include "common.h"
  2   
  3  #define CERTFILE "client.pem"
  4  SSL_CTX *setup_client_ctx(void)
  5  {
  6      SSL_CTX *ctx;
  7  
  8      ctx = SSL_CTX_new(SSLv23_method(  ));
  9      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 10          int_error("Error loading certificate from file");
 11      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 12          int_error("Error loading private key from file");  
 13      return ctx;
 14  }
 15  
 16  int do_client_loop(SSL *ssl)
 17  {
 18      int  err, nwritten;
 19      char buf[80];
 20   
 21      for (;;)
 22      {
 23          if (!fgets(buf, sizeof(buf), stdin))
 24              break;
 25          for (nwritten = 0;  nwritten < sizeof(buf);  nwritten += err)
 26          {
 27              err = SSL_write(ssl, buf + nwritten, strlen(buf) - nwritten);
 28              if (err <= 0)
 29                  return 0;
 30          }
 31      }
 32      return 1;
 33  }
 34   
 35  int main(int argc, char *argv[])
 36  {
 37      BIO     *conn;
 38      SSL     *ssl;
 39      SSL_CTX *ctx;
 40  
 41      init_OpenSSL(  );
 42      seed_prng(  );
 43  
 44      ctx = setup_client_ctx(  );
 45  
 46      conn = BIO_new_connect(SERVER ":" PORT);
 47      if (!conn)
 48          int_error("Error creating connection BIO");
 49   
 50      if (BIO_do_connect(conn) <= 0)
 51          int_error("Error connecting to remote machine");
 52   
 53      if (!(ssl = SSL_new(ctx)))
 54          int_error("Error creating an SSL context");
 55      SSL_set_bio(ssl, conn, conn);
 56      if (SSL_connect(ssl) <= 0)
 57          int_error("Error connecting SSL object");
 58  
 59      fprintf(stderr, "SSL Connection opened\n");
 60      if (do_client_loop(ssl))
 61          SSL_shutdown(ssl);
 62      else
 63          SSL_clear(ssl);
 64      fprintf(stderr, "SSL Connection closed\n");
 65  
 66      SSL_free(ssl);
 67      SSL_CTX_free(ctx);
 68      return 0;
 69  }

In this example, we make a call to the function seed_prng . As its name suggests, this function seeds the OpenSSL PRNG. Its implementation is left out; see Chapter 4 for details on an appropriate implementation. It is very important to maintaining the security of SSL for the PRNG to be properly seeded, so this function should never be left out in real-world applications.

The function setup_client_ctx performs the actions as discussed earlier to provide certificate data to the server properly. In this setup process, we leave out any calls to SSL_CTX_set_default_passwd_cb since the default OpenSSL passphrase callback is acceptable for our purposes. The only other point of interest is the error checking performed. This example prints errors and exits if anything goes wrong; a more robust error handling technique was left out for clarity. The sidebar contains more information about the contents of the file client.pem.

$ openssl req -newkey rsa:1024 -sha1 -keyout rootkey.pem -out rootreq.pem
$ openssl x509 -req -in rootreq.pem -sha1 -extfile myopenssl.cnf \
>  -extensions v3_ca -signkey rootkey.pem -out rootcert.pem 
$ cat rootcert.pem rootkey.pem > root.pem
$ openssl x509 -subject -issuer -noout -in root.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA

$ openssl req -newkey rsa:1024 -sha1 -keyout serverCAkey.pem -out \
>  serverCAreq.pem
$ openssl x509 -req -in serverCAreq.pem -sha1 -extfile \
>  myopenssl.cnf -extensions v3_ca -CA root.pem -CAkey root.pem \
>  -CAcreateserial -out serverCAcert.pem
$ cat serverCAcert.pem serverCAkey.pem rootcert.pem > serverCA.pem
$ openssl x509 -subject -issuer -noout -in serverCA.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/OU=Server Division/CN=Server CA
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA

$ openssl req -newkey rsa:1024 -sha1 -keyout serverkey.pem -out \
>  serverreq.pem
$ openssl x509 -req -in serverreq.pem -sha1 -extfile myopenssl.cnf \
>  -extensions usr_cert -CA serverCA.pem -CAkey serverCA.pem \
>  -CAcreateserial -out servercert.pem
$ cat servercert.pem serverkey.pem serverCAcert.pem rootcert.pem > \
>  server.pem
$ openssl x509 -subject -issuer -noout -in server.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=splat.zork.org
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/OU=Server Division/CN=Server CA

$ openssl req -newkey rsa:1024 -sha1 -keyout clientkey.pem -out \
>  clientreq.pem
$ openssl x509 -req -in clientreq.pem -sha1 -extfile myopenssl.cnf \
>  -extensions usr_cert -CA root.pem -CAkey root.pem \
>  -CAcreateserial -out clientcert.pem
$ cat clientcert.pem clientkey.pem rootcert.pem > client.pem
$ openssl x509 -subject -issuer -noout -in client.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=shell.zork.org
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA

$ openssl dhparam -check -text -5 512 -out dh512.pem
$ openssl dhparam -check -text -5 1024 -out dh1024.pem

Lines 54-58 create the SSL object and connect it. There are some functions used here that we have not yet discussed. The call to SSL_new creates our SSL object and copies the settings we’ve already placed in the SSL_CTX to the newly created object. At this point, the SSL object is still in a generic state. In other words, it could play the role of the server or the client in an SSL handshake.

Another factor left unspecified is the path of communications for the SSL object. Since SSL objects are flexible in the sense that they can perform SSL functions on top of many different types of I/O methods, we must specify a BIO for our object to use. Line 56 does this through a call to SSL_set_bio . This function is passed our connection BIO twice since SSL objects are robust enough to operate on two one-way I/O types instead of requiring a single full-duplex I/O method. Basically, we must specify the BIO to use for writing separately from the BIO used for reading. In this case, they are the same object, since sockets allow two-way communication.

The last unfamiliar function used here is SSL_connect . This function causes the SSL object to initiate the protocol using the underlying I/O. In other words, it begins the SSL handshake with the application on the other end of the underlying BIO. This function will return an error for problems such as incompatible protocol versions.

The do_client_loop function is almost identical to that of our non-SSL client. We’ve simply changed the parameter to an SSL object instead of a BIO, and the BIO_write becomes an SSL_write. In addition, we’ve added a return value to this function. If no errors occur, we can call SSL_shutdown to stop the SSL connection; otherwise, we call SSL_clear. This is done to force OpenSSL to remove any session with errors from the session cache. We will look at session caching in more detail later in this chapter, but for now, it is worth noting that session caching is effectively disabled in the examples we’ve provided so far.

The last point to make about this example is that we removed the call to BIO_free. This is done because SSL_free automatically frees the SSL object’s underlying BIOs for us.

Example 5-6 has the contents of server1.c, the file containing the implementation of our SSL-enabled server. Again, it isn’t yet secure since it validates nothing about the peer; it simply provides its certificate information to the client.

  1  #include "common.h"
  2   
  3  #define CERTFILE "server.pem"
  4  SSL_CTX *setup_server_ctx(void)
  5  {
  6      SSL_CTX *ctx;
  7  
  8      ctx = SSL_CTX_new(SSLv23_method(  ));
  9      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 10          int_error("Error loading certificate from file");
 11      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 12          int_error("Error loading private key from file");
 13      return ctx;
 14  }
 15  
 16  int do_server_loop(SSL *ssl)
 17  {
 18      int  err, nread;
 19      char buf[80];
 20   
 21      do
 22      {
 23          for (nread = 0;  nread < sizeof(buf);  nread += err)
 24          {
 25              err = SSL_read(ssl, buf + nread, sizeof(buf) - nread);
 26              if (err <= 0)
 27                  break;
 28          }
 29          fwrite(buf, 1, nread, stdout);
 30      }
 31      while (err > 0);
 32      return (SSL_get_shutdown(ssl) & SSL_RECEIVED_SHUTDOWN) ? 1 : 0;
 33  }
 34   
 35  void THREAD_CC server_thread(void *arg)
 36  {
 37      SSL *ssl = (SSL *)arg;
 38  
 39  #ifndef WIN32
 40      pthread_detach(pthread_self(  ));
 41  #endif
 42      if (SSL_accept(ssl) <= 0)
 43          int_error("Error accepting SSL connection");
 44      fprintf(stderr, "SSL Connection opened\n");
 45      if (do_server_loop(ssl))
 46          SSL_shutdown(ssl);
 47      else
 48          SSL_clear(ssl);
 49      fprintf(stderr, "SSL Connection closed\n");
 50      SSL_free(ssl);
 51  
 52  ERR_remove_state(0);
 53  
 54  #ifdef WIN32
 55      _endthread(  );
 56  #endif
 57  }
 58   
 59  int main(int argc, char *argv[])
 60  {
 61      BIO         *acc, *client;
 62      SSL         *ssl;
 63      SSL_CTX     *ctx;
 64      THREAD_TYPE tid;
 65   
 66      init_OpenSSL(  );
 67      seed_prng(  );
 68  
 69      ctx = setup_server_ctx(  );
 70  
 71      acc = BIO_new_accept(PORT);
 72      if (!acc)
 73          int_error("Error creating server socket");
 74   
 75      if (BIO_do_accept(acc) <= 0)
 76          int_error("Error binding server socket");
 77   
 78      for (;;)
 79      {
 80          if (BIO_do_accept(acc) <= 0)
 81              int_error("Error accepting connection");
 82   
 83          client = BIO_pop(acc);
 84          if (!(ssl = SSL_new(ctx)))
 85              int_error("Error creating SSL context");
 86  
 87          SSL_set_bio(ssl, client, client);
 88          THREAD_create(tid, server_thread, ssl);
 89      }
 90      
 91      SSL_CTX_free(ctx);
 92      BIO_free(acc);
 93      return 0;
 94  }

After looking at the new client program, the modifications to the server should be clear. Since this is the server side of the SSL negotiation, a different function call is made: SSL_accept . The SSL_accept function initiates communication on the underlying I/O layer to perform the SSL handshake.

In the do_server_loop function, we use a call to SSL_get_shutdown to check into the error status of the SSL object. This essentially allows us to differentiate normal client terminations from actual errors. If the SSL_RECEIVED_SHUTDOWN flag is set, we know the session hasn’t had an error and it’s safe to cache. In other words, we can call SSL_shutdown rather than simply clear the connection. The remainder of the modifications to the server program parallel those made to the client.

In summary, we’ve taken our applications and built them to the point of creating the objects necessary for SSL connections. Each application does provide its certificate data to the peers to which it connects, but they do not verify the certificates they receive. We will build further in the next step and learn how to validate peer certificates.

Certificate verification can often be confusing, so we’ll discuss the theory before delving into the details. As we already know, a certificate is a set of credentials that has been cryptographically signed by a CA. Each certificate, including a CA certificate, contains a public key with a private key counterpart held in secret by the certificate owner. Moving forward, the process of signing a certificate involves using the private key of a CA to sign the public key in the new certificate. Thus, it should be clear that the process of verification will involve using the public key in a CA certificate to verify the signature on a certificate.

Aside from using a CA to create an entity certificate, a CA may also sign a certificate and give it permissions, via X.509v3 extensions, to act as a CA itself. Generally, this procedure allows for a CA to permit another certificate to act as a CA for a specialized purpose. Through this mechanism, we become aware of certificate hierarchies, i.e., a certificate tree. Understanding this, we can see that a single entity certificate may have a list of signing certificates leading up to the original, self-signed root certificate. This list of certificates, each signed by the next, is called a certificate chain .

Jumping back to a simple example of a root CA signing a single entity certificate, any party may verify the entity certificate by checking the signature on it, presuming it trusts the root CA. The process of validating an entity certificate is that simple. Extending this to a certificate chain, we must validate each subsequent signature in the list until we reach a trusted CA certificate or until we reach the end of the list. If we hit a CA we trust and the signatures are all valid, our entity certificate is verified; if we find an invalid signature or reach the end of the chain without reaching a trusted certificate, the entity certificate is not verified.

As we previously discussed, verifying a certificate’s authenticity requires that the verifying agent have a list of CAs that it trusts. Therefore, we must provide our application with such a list in order for it to verify the peer. We will start our discussion of peer verification by first concentrating on accomplishing this task.

Loading trusted CA certificates into our application is manifested as additional setup to the SSL_CTX object. The function SSL_CTX_load_verify_locations performs this task. This function will load certificates from files, directories, or both.

int SSL_CTX_load_verify_locations(SSL_CTX *ctx, const char *CAfile,
                                    const char *CApath);

We can call SSL_CTX_load_verify_locations with either the second or the third arguments specified as NULL, but not both. The behavior of this function is to perform the loading for the non-NULL arguments. An important difference between file and directory storage is the time when the certificates get loaded. With a flat file, the file is parsed and certificates loaded during the call to SSL_CTX_load_verify_locations. However, with a directory, certificates are read only when needed, i.e., during the verification phase that occurs during the SSL handshake.

OpenSSL also has default CA certificate locations. When building the library, these paths are hardcoded into the library based on the parameters that are used to build it. In general, there is an OpenSSL directory (commonly /usr/local/openssl on Unix systems). The default certificate file is named cert.pem , and it lives in this OpenSSL directory. Likewise, the default certificate directory is named certs , and it too lives in the OpenSSL directory. These default locations provide a convenient place to store system-wide CA certificates that all of the OpenSSL-based applications running on the machine require. Using the default files, we will not need to keep separate copies of common certificates for each application. The function SSL_CTX_set_default_verify_paths loads these default locations into our SSL_CTX object. For calls to this function, the same rules for determining when the CA certificates actually get loaded apply as with a call to SSL_CTX_load_verify_locations.

Using these two functions, we can load trusted CA certificates into our SSL_CTX. Even though they are loaded, the certificates are still not used to verify the peer. We will explore the details of enabling this in the next section.

Certificate verification entails checking the cryptographic signatures on a certificate to be sure an entity we trust has signed that certificate. It also involves checking the certificate’s notBefore and notAfter dates, trust settings, purpose, and revocation status. Verification of a certificate takes place during the SSL handshake (during the call to SSL_connect or SSL_accept, depending on whether the SSL object is a client or a server).

Once we’ve properly loaded trusted certificates into the SSL_CTX object, OpenSSL has a built-in function to verify the peer’s certificate chain automatically. The routine used to verify the certificate chain could be changed from the default via a call to SSL_CTX_set_cert_verify_callback , but under almost all circumstances, this is undesirable since the default routine for signature verification is amply complete and robust. Instead, the developer can specify a different callback that filters the return status of the default verification and returns the new verification status. The function to perform this task is SSL_CTX_set_verify .

Aside from assigning a filter verify callback, this function’s primary purpose is to assign the type of verification our SSL_CTX object’s connections will perform. More accurately, we can use it to control how certificates and requests are handled during a handshake. The second argument to this function is a set of flags that determine this. Four flags are defined with names that can be combined with a logical OR operation. Depending on whether the context is being used in client mode or server mode, these flags can have different meanings.

The third argument to SSL_CTX_set_verify is a pointer to the verification filter callback. Since the internal verification routine is called for each level of the peer certificate chain, our filter routine will be called just after each step. This function’s first argument is nonzero if the verification succeeded, and zero otherwise. The second argument is an X509_STORE_CTX object. This type of object contains the information necessary to verify a certificate. The object holds the current certificate being verified and the verification result. The return value from the function should be either zero or nonzero to indicate whether the certificate should be considered valid or not.

As in most of the callbacks used with OpenSSL, there is a default supplied that simply returns the value of the first argument. When implementing our own version of this function, we need to maintain this behavior. If we return nonzero when the first parameter is actually zero, an unverified client certificate will be accepted as a verified one. Likewise, the reverse would cause a valid certificate to fail verification. At this point, it may seem as if there is no real purpose to implementing our own version, but this isn’t accurate. The reason we should supply our own is so that more detailed information about the results of verification can be obtained, especially when verification fails. For instance, if a peer presents an expired certificate and we do not implement a verify callback to check status, then we find out only that the call to SSL_connect or SSL_accept failed because of “handshake failure.” Example 5-7 shows an implementation for this callback that we will use in our example applications. To use it in our examples, it should be implemented in common.c and prototyped in common.h.

int verify_callback(int ok, X509_STORE_CTX *store)
{
    char data[256];
 
    if (!ok)
    {
        X509 *cert = X509_STORE_CTX_get_current_cert(store);
        int  depth = X509_STORE_CTX_get_error_depth(store);
        int  err = X509_STORE_CTX_get_error(store);
 
        fprintf(stderr, "-Error with certificate at depth: %i\n", depth);
        X509_NAME_oneline(X509_get_issuer_name(cert), data, 256);
        fprintf(stderr, "  issuer   = %s\n", data);
        X509_NAME_oneline(X509_get_subject_name(cert), data, 256);
        fprintf(stderr, "  subject  = %s\n", data);
        fprintf(stderr, "  err %i:%s\n", err, X509_verify_cert_error_string(err));
    }
 
    return ok;
}

This callback employs several functions from the X509 family of functions to report the detailed error information.

The call to SSL_CTX_set_verify is done before any SSL objects are created from the context. We should also make a call to SSL_CTX_set_verify_depth . This function sets the maximum allowable depth for peer certificates. In other words, it limits the number of certificates that we are willing to verify in order to ensure the chain is trusted. For example, if the depth was set to four and six certificates are present in the chain to reach the trusted certificate, the verification would fail because the required depth would be too great. For nearly all applications, the default depth of nine is more than high enough to ensure that the peer certificate will not fail due to too large of a certificate chain. On the other hand, if we know that our application will be used only with peers presenting certificates of some smaller chain length, it is a good idea to set the value to exclude certificates composed of longer chains from being verified successfully. Setting the depth to zero allows chains of unlimited length to be used.

A large problem with SSL security is the availability and usage of certificate revocation lists. Since certificates can be revoked by the issuing CA, we must somehow account for this in our SSL implementation. To do this, an application must load CRL files in order for the internal verification process to ensure each certificate it verifies is not revoked. Unfortunately, OpenSSL’s CRL functionality is incomplete in Version 0.9.6. The features necessary to utilize CRL information will be complete in new versions starting with 0.9.7.

Because this functionality is not available at the time of writing, CRL usage will not be incorporated in this example. However, we can tell you what you will need to do once newer releases are made. Remember, it is paramount to include CRL checking when verifying certificates. If any new version of OpenSSL is used when building applications, this step is required for security.

The SSL interface itself does not support CRL incorporation directly; instead, we must use the underlying X509 interface.

The function SSL_CTX_get_cert_store retrieves the internal X509_STORE object from the SSL_CTX object. Operations on this store object allow us to perform a variety of tweaks on the verification process. In fact, the functions SSL_CTX_load_verify_locations and SSL_CTX_set_default_paths call functions against this same X509_STORE object to perform their respective operations.

X509_STORE *SSL_CTX_get_cert_store(SSL_CTX *ctx);

All of the details for interacting with certificate stores to set further verification parameters or incorporate CRL data are discussed in Chapter 10 with the verification of certificate chains. We strongly recommend that developers implementing applications consult the verification process in Chapter 10 that uses X509_STORE objects to learn the proper method of SSL certificate verification against CRLs. The process involves adding the CRL files to the X509_STORE via a file lookup method and then setting the store’s flags to check certificates against the CRLs.

Essentially, SSL_CTX_set_verify and SSL_CTX_set_verify_depth are all we need to use in order for OpenSSL to verify the peer certificate chain upon connection. There is more, however. After connecting the SSL object, we need to assert that some assumed properties about the connection are indeed true. OpenSSL provides several functions that allow us to create a post-connection verification routine to make sure that we haven’t been fooled by a malicious peer. This post-connection verification routine is very important because it allows for much finer grained control over the certificate that is presented by the peer, beyond the certificate verification that is required by the SSL protocol proper.

The function SSL_get_peer_certificate will return a pointer to an X509 object that contains the peer’s certificate. While the handshake is complete and, presumably, the verification completed correctly, we must still use this function. Consider the case in which the peer presents no certificate when one is requested but not required. The certificate verification routines—both the built-in and the filter—will not return errors since there was nothing wrong with the NULL certificate. Thus, to prevent this condition, we must call this function and check that the return value is not NULL. If this function returns a non-NULL value, the reference count of the return object is increased. In order to prevent memory leaks, we must call X509_free to decrement the count after we’re done using the object.

Our application will be vulnerable if we do not check the peer certificate beyond verification of the chain. For example, let’s say that we’re making a web browsing application. To keep it simple, we’ll allow just one trusted CA. When we do this, any SSL peer with a certificate signed by the same CA will be verified correctly. This isn’t secure. Nothing prevents an attacker from getting his own certificate signed by the CA and then hijacking all your sessions. We thwart this kind of masquerade by tying the certificate to some piece of information unique to the machine. For purposes of SSL, this piece of information is the entity’s fully qualified domain name (FQDN), also called the DNS name.

The common practice with X.509v1 certificates was to put the FQDN in the certificate’s commonName field of the subjectName field. This practice is no longer recommended for new applications since X.509v3 allows certificate extensions to hold the FQDN as well as other identifying information, such as IP address. The proper place for the FQDN is in the dNSName field of the subjectAltName extension.

We use the function post_connection_check to perform these checks for us. We recommend always checking for the dNSName field first, and if it isn’t present, we can check the commonName field. Checking the commonName field is strictly for backward compatibility, so if this isn’t a concern, it can safely be omitted. Our example function will check for the extension first and then fall back to the commonName. One feature our example does omit is the optional wildcard expansion. RFC 2818 specifies a paradigm for allowing FQDNs in certificates to contain wildcards. Implementing this functionality is simply a text-processing issue and is thus omitted for clarity.

SSL_get_verify_result is another API function that we will employ in our post-connection check. This function returns the error code last generated by the verification routines. If no error has occurred, X509_V_OK is returned. We should call this function and make sure the returned value equals X509_V_OK. When browsing the example application, it is obvious that robust error handling has been left out for clarity. For example, the programs simply exit when an error occurs. In most cases, we will want to do something better to handle errors in some application-specific way. Checking the verify result is always a good idea. It makes an assertion that no matter what error handling occurred up to this point, if the result isn’t OK now, we should disconnect.

Example 5-8 shows a function that performs the checks that we’ve just described. In the example, we’ll check to be sure that the certificate contains the FQDN of the peer to which we expect to be connecting. For the client, we’ll want to make sure that the server presents a certificate that contains the FQDN of the server’s address. Likewise, for the server, we’ll want to make sure that the client presents a certificate that contains the FQDN of the client’s address. In this case, our checking of the client certificate will be very strict because we’ll be expecting the client to be using a specific FQDN, and we’ll allow only that one. For the purposes of our example client and server, this function should appear in common.c and be prototyped in common.h.

long post_connection_check(SSL *ssl, char *host)
{
    X509      *cert;
    X509_NAME *subj;
    char      data[256];
    int       extcount;
    int       ok = 0;
 
    /* Checking the return from SSL_get_peer_certificate here is not strictly
     * necessary.  With our example programs, it is not possible for it to return
     * NULL.  However, it is good form to check the return since it can return NULL
     * if the examples are modified to enable anonymous ciphers or for the server
     * to not require a client certificate.
     */
    if (!(cert = SSL_get_peer_certificate(ssl)) || !host)
        goto err_occured;
    if ((extcount = X509_get_ext_count(cert)) > 0)
    {
        int i;
 
        for (i = 0;  i < extcount;  i++)
        {
            char              *extstr;
            X509_EXTENSION    *ext;
 
            ext = X509_get_ext(cert, i);
            extstr = OBJ_nid2sn(OBJ_obj2nid(X509_EXTENSION_get_object(ext)));
 
            if (!strcmp(extstr, "subjectAltName"))
            {
                int                  j;
                unsigned char        *data;
                STACK_OF(CONF_VALUE) *val;
                CONF_VALUE           *nval;
                X509V3_EXT_METHOD    *meth;
 
                if (!(meth = X509V3_EXT_get(ext)))
                    break;
                data = ext->value->data;
 
                val = meth->i2v(meth, 
                                meth->d2i(NULL, &data, ext->value->length),
                                NULL);
                for (j = 0;  j < sk_CONF_VALUE_num(val);  j++)
                {
                    nval = sk_CONF_VALUE_value(val, j);
                    if (!strcmp(nval->name, "DNS") && !strcmp(nval->value, host))
                    {
                        ok = 1;
                        break;
                    }
                }
            }
            if (ok)
                break;
        }
    }
 
    if (!ok && (subj = X509_get_subject_name(cert)) &&
        X509_NAME_get_text_by_NID(subj, NID_commonName, data, 256) > 0)
    {
        data[255] = 0;
        if (strcasecmp(data, host) != 0)
            goto err_occured;
    }
 
    X509_free(cert);
    return SSL_get_verify_result(ssl);
 
err_occured:
    if (cert)
        X509_free(cert);
    return X509_V_ERR_APPLICATION_VERIFICATION;
}

At a high level, the function post_connection_check is implemented as a wrapper around SSL_get_verify_result, which performs our extra peer certificate checks. It uses the reserved error code X509_V_ERR_APPLICATION_VERIFICATION to indicate errors where there is no peer certificate present or the certificate presented does not match the expected FQDN. This function will return an error in the following circumstances:

As long as none of the above errors occurs, the value X509_V_OK will be returned. Our example programs are further extended below to employ this function.

Unfortunately, the code to check the dNSName is not very clear. We use the X509 functions to access the extensions. We then iterate through the extensions and use the extension-specific parsing routines to find all extensions that are subjectAltName fields. Since a subjectAltName field may itself contain several fields, we must then iterate through those fields to find any dNSName fields (they are tagged with the short name DNS). Since it is rather confusing, we will step through this function to explain its behavior. Having a stronger understanding of some of the more advanced programming techniques presented in Chapter 10 will help demystify the implementation of this function.

At the onset, we simply get the peer certificate from the SSL object. If the function X509_get_ext_count returns a positive number, we know there are some X.509v3 extensions present in the peer’s certificate. Given this, we iterate over the extensions to look for a subjectAltName. The function X509_get_ext will retrieve an extension for us based on the counter. We also use the variable extstr to hold the extracted short name of the extension. Unfortunately, we must use three functions to perform this task: the innermost extracts the ASN1_OBJECT, the next fetches the NID, and the outermost function gets the short name as a const char * from the NID.

Next, we check if the extension is what we’re looking for by comparing the short name against the string constant subjectAltName. Once we’re sure it’s the right extension, we need to extract the X509V3_EXT_METHOD object from the extension. This object is a container of extension-specific functions for manipulating the data within the extension. We access the data we wish to manipulate directly through the value member of the X509_EXTENSION structure. The d2i and i2v functions serve the purpose of converting the raw data in the subjectAltName to a stack of CONF_VALUE objects. This is necessary to make it simple to iterate over the several kinds of fields in the subjectAltName so that we may find the dNSName field(s). We check each member of this CONF_VALUE stack to see if we have a match for the host string in a dNSName field. Keep in mind that the dNSName field is named DNS in the extension itself, since it’s referenced by its short name. As soon as we find a match, we stop the iterations over all the extensions.

We only pursue checking the commonName of the certificate if no match is found in a dNSName field. If we fail to find an FQDN that matches the host argument in either the dNSName field or the commonName, we return the code for an application-specific error. In most real-world cases, matching one specific FQDN is not desirable. Most often, a server would have a list (known as a whitelist) that contains all of the acceptable FQDNs for connecting clients. If the client’s certificate contains an FQDN that appears on this list, the certificate is accepted; otherwise, it is rejected.

Employing what we know about verifying the authenticity of the peer, we can extend our example applications to make them one step closer to being secure. For these examples, we’ve added the functions verify_callback and post_connection_check to common.c and their prototypes to common.h.

The code for our revised client application, client2.c, is provided in Example 5-9. The lines that differ from client1.c are marked. Line 3 defines the file we use to store trusted certificates. We define CADIR to be NULL on line 4 since we will use a flat file instead. Nothing prevents us from specifying both a file and a directory; but in this case, we do not need a directory.

  1  #include "common.h"
  2   
  3  #define CAFILE "root.pem"
  4  #define CADIR NULL
  5  #define CERTFILE "client.pem"
  6  SSL_CTX *setup_client_ctx(void)
  7  {
  8      SSL_CTX *ctx;
  9   
 10      ctx = SSL_CTX_new(SSLv23_method(  ));
 11      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 12          int_error("Error loading CA file and/or directory");
 13      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 14          int_error("Error loading default CA file and/or directory");
 15      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 16          int_error("Error loading certificate from file");
 17      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 18          int_error("Error loading private key from file");
 19      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
 20      SSL_CTX_set_verify_depth(ctx, 4);
 21      return ctx;
 22  }
 23   
 24  int do_client_loop(SSL *ssl)
 25  {
 26      int  err, nwritten;
 27      char buf[80];
 28   
 29      for (;;)
 30      {
 31          if (!fgets(buf, sizeof(buf), stdin))
 32              break;
 33          for (nwritten = 0;  nwritten < sizeof(buf);  nwritten += err)
 34          {
 35              err = SSL_write(ssl, buf + nwritten, strlen(buf) - nwritten);
 36              if (err <= 0)
 37                  return 0;
 38          }
 39      }
 40      return 1;
 41  }
 42   
 43  int main(int argc, char *argv[])
 44  {
 45      BIO     *conn;
 46      SSL     *ssl;
 47      SSL_CTX *ctx;
 48      long    err;
 49  
 50      init_OpenSSL(  );
 51      seed_prng(  );
 52   
 53      ctx = setup_client_ctx(  );
 54   
 55      conn = BIO_new_connect(SERVER ":" PORT);
 56      if (!conn)
 57          int_error("Error creating connection BIO");
 58   
 59      if (BIO_do_connect(conn) <= 0)
 60          int_error("Error connecting to remote machine");
 61   
 62      ssl = SSL_new(ctx);
 63      SSL_set_bio(ssl, conn, conn);
 64      if (SSL_connect(ssl) <= 0)
 65          int_error("Error connecting SSL object");
 66      if ((err = post_connection_check(ssl, SERVER)) != X509_V_OK)
 67      {
 68          fprintf(stderr, "-Error: peer certificate: %s\n",
 69                  X509_verify_cert_error_string(err));
 70          int_error("Error checking SSL object after connection");
 71      }
 72      fprintf(stderr, "SSL Connection opened\n");
 73      if (do_client_loop(ssl))
 74          SSL_shutdown(ssl);
 75      else
 76          SSL_clear(ssl);
 77      fprintf(stderr, "SSL Connection closed\n");
 78   
 79      SSL_free(ssl);
 80      SSL_CTX_free(ctx);
 81      return 0;
 82  }

To load the trusted certificates from root.pem, we call SSL_CTX_load_verify_locations on line 11 and check for errors. Since we trust the users of the system on which this client will run, we also call SSL_CTX_set_default_verify_paths to load the built-in certificate stores. Our example does not explicitly require the loading of the default locations; it is included for illustration. It is good design practice to load these defaults only when the application will run on a trusted system and when the application itself needs to incorporate these extra certificates.

After loading the trusted certificates, we set the verification mode to SSL_VERIFY_PEER and assign the callback (line 19). When implementing SSL clients, the verification mode should always include SSL_VERIFY_PEER. Without this, we could never tell if the server we connect with is properly authenticated. As we discussed in the section above, the verify_callback function simply reports errors in more detail and does not change the behavior of the internal verification process. The following line, line 20, sets the maximum depth for verification to four. For this client example, four levels of verification ought to be plenty because our certificate hierarchy is not too complex. Given the details from the previous sidebar describing our example PKI, the minimum depth we can assign to our client is two, since the server’s certificate is signed by the server CA, which, in turn, is signed by the root CA that we trust.

The last major change to this version of the client is in lines 66-71. These lines use the post_connection_check function that we developed in Example 5-8. This call asserts that the server we are connected with did present a certificate and the certificate it provided has “splat.zork.org” as the FQDN. If any errors occur, we can call X509_verify_cert_error_string to convert our error code into a string to print to the console.

Example 5-10 shows the contents of server2.c, our example server program. The changes made to it are congruent with the changes made to the client application.

  1  #include "common.h"
  2   
  3  #define CAFILE "root.pem"
  4  #define CADIR NULL
  5  #define CERTFILE "server.pem"
  6  SSL_CTX *setup_server_ctx(void)
  7  {
  8      SSL_CTX *ctx;
  9   
 10      ctx = SSL_CTX_new(SSLv23_method(  ));
 11      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 12          int_error("Error loading CA file and/or directory");
 13      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 14          int_error("Error loading default CA file and/or directory");
 15      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 16          int_error("Error loading certificate from file");
 17      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 18          int_error("Error loading private key from file");
 19      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT,
 20                         verify_callback);
 21      SSL_CTX_set_verify_depth(ctx, 4);
 22      return ctx;
 23  }
 24   
 25  int do_server_loop(SSL *ssl)
 26  {
 27      int  err, nread;
 28      char buf[80];
 29   
 30      for (;;)
 31      {
 32          for (nread = 0;  nread < sizeof(buf);  nread += err)
 33          {
 34              err = SSL_read(ssl, buf + nread, sizeof(buf) - nread);
 35              if (err <= 0)
 36                  break;
 37          }
 38          fwrite(buf, 1, nread, stdout);
 39      }
 40      return (SSL_get_shutdown(ssl) & SSL_RECEIVED_SHUTDOWN) ? 1 : 0;
 41  }
 42   
 43  void THREAD_CC server_thread(void *arg)
 44  {
 45      SSL *ssl = (SSL *)arg;
 46  long err;
 47   
 48  #ifndef WIN32
 49      pthread_detach(pthread_self(  ));
 50  #endif
 51      if (SSL_accept(ssl) <= 0)
 52          int_error("Error accepting SSL connection");
 53      if ((err = post_connection_check(ssl, CLIENT)) != X509_V_OK)
 54      {
 55          fprintf(stderr, "-Error: peer certificate: %s\n",
 56                  X509_verify_cert_error_string(err));
 57          int_error("Error checking SSL object after connection");
 58      }
 59      fprintf(stderr, "SSL Connection opened\n");
 60      if (do_server_loop(ssl))
 61          SSL_shutdown(ssl);
 62      else
 63          SSL_clear(ssl);
 64      fprintf(stderr, "SSL Connection closed\n");
 65      SSL_free(ssl);
 66  ERR_remove_state(0)
 67  #ifdef WIN32
 68      _endthread(  );
 69  #endif
 70  }
 71   
 72  int main(int argc, char *argv[])
 73  {
 74      BIO     *acc, *client;
 75      SSL     *ssl;
 76      SSL_CTX *ctx;
 77   THREAD_TYPE tid;
 78  
 79      init_OpenSSL(  );
 80      seed_prng(  );
 81   
 82      ctx = setup_server_ctx(  );
 83   
 84      acc = BIO_new_accept(PORT);
 85      if (!acc)
 86          int_error("Error creating server socket");
 87   
 88      if (BIO_do_accept(acc) <= 0)
 89          int_error("Error binding server socket");
 90   
 91      for (;;)
 92      {
 93          if (BIO_do_accept(acc) <= 0)
 94              int_error("Error accepting connection");
 95   
 96          client = BIO_pop(acc);
 97          if (!(ssl = SSL_new(ctx)))
 98          int_error("Error creating SSL context");
 99          SSL_set_accept_state(ssl);
100          SSL_set_bio(ssl, client, client);
101          THREAD_create(tid, server_thread, ssl);
102      }
103   
104      SSL_CTX_free(ctx);
105      BIO_free(acc);
106      return 0;
107  }

One way the changes to the server stray from the changes to the client is in the verification mode. In server applications, the behavior of SSL_VERIFY_PEER is slightly different; it causes the server to solicit a certificate from the client. The other verification flag used is SSL_VERIFY_FAIL_IF_NO_PEER_CERT. This flag instructs the server to fail on the handshake if the client presents no certificate.

The client’s usage of the SERVER-defined value and the server’s usage of CLIENT is an oversimplification. In many cases, especially with that of a server, the string containing the peer’s FQDN will not be so readily available. Instead, we may need to look at the IP address to which we are connected and use it to discover the FQDN. For purposes of clarity, this process was omitted, and the names were hardcoded. Additionally, when verifying peer certificates, the owner will often not be a machine with a FQDN but rather a person, organization, etc. For these cases, it’s important to change the post_connection_check routine accordingly to allow for successful connections.

Using the tricks we learned in this step, we have now developed a viable framework for verifying a peer and assuring authentication is done properly. The majority of the battle to SSL-enable our application is over; however, we still have one more step.

The SSL_CTX_set_options function provides the developer with finer-grained control over the SSL connections spawned from the context. Using this function, we can enable the bug workarounds built into the OpenSSL library. For instance, a particular version of a Netscape product (Netscape-Commerce 1.12) will truncate the material used for key generation. In order for our SSL programs to establish a connection to a peer with such a bug, we need to enable the workaround. These fixes are useful only to programs that will communicate with a peer known to have bugs, but enabling the workarounds does not hurt anything as a rule. These bug fixes can be enabled individually, but instead we should set the SSL_OP_ALL flag, which will enable all of the workaround code.

Like the function SSL_CTX_set_verify, the second parameter to this function is a set of flags. Again, the flags can be combined with the logical OR operation. An important fact about this call is that once an option is set, it can’t be cleared: this function only adds the options presented by the second argument to the options set contained in the SSL_CTX object. The new set of options is returned by this function.

In addition to the workarounds for buggy SSL peers, this function allows us to tighten the security of our SSL connections. By setting the option SSL_OP_NO_SSLv2, we prevent the SSLv2 protocol from being used. As we noted in Step 1, this is a very useful feature. Using this option, we can create an SSL_CTX object based on the compatibility method, SSLv23_method, and the context will not allow SSLv2 peers. This is useful since electing to base our context upon either SSLv3_method or TLSv1_method would prevent the other from connecting correctly.

Two server-side-only options that bear consideration are SSL_OP_EPHEMERAL_RSA and SSL_OP_SINGLE_DH_USE. The former causes our context object to attempt to use a temporary RSA key for the key exchange. The details of this process are discussed below, but generally, this option should never be used, since it violates the SSL/TLS protocol specification. We discuss the SSL_OP_SINGLE_DH_USE flag in the next section.

In our examples thus far, both the server and the client certificates have been based on RSA key pairs. Because the RSA algorithm can be used for most signing and encrypting, SSL uses it to perform the key agreement necessary to create the shared key under which the data in the stream is encrypted. This technique, such as when the key exchange is conducted through a persistent key, is called static keying . Building on this definition, ephemeral keying is defined as key exchange through a temporary key. At first, it may seem that temporary keys may not allow for proper authentication—not true. Generally, with ephemeral keying, the authentication is accomplished through signature verification with persistent keys, and the temporary keys are used only for key agreement. There are two main advantages of ephemeral keying over static keying from a security perspective.

We’ve said that our example uses RSA keys in the certificates, but consider a case in which the certificates are based upon DSA keys. The DSA algorithm provides a mechanism for signing but not for encrypting. Thus, having only DSA keys on either side of an SSL connection leaves the protocol unable to perform key exchange. It follows that static keying is not even an option for DSA-based certificates; we must supplement them with ephemeral keys.

The second advantage of using temporary keys is that they provide forward secrecy . At a high level, forward secrecy means that if the private key is obtained by a third party, that third party will not be able to decode previous sessions conducted with that key or even future sessions conducted by someone else with the compromised key. This is due to the ephemeral keys used for the secrecy of the sessions. When using static keys, there is no forward secrecy since the secrecy of the key exchange, and hence the following transmission stream, is based on the private key. With an ephemeral key, the data on which the key exchange was performed no longer exists after the key exchange, and thus the session remains private regardless of private key compromise. Thus, forward secrecy means that private key compromise will only allow an attacker to masquerade as the key owner, but not access the key owner’s private data.

These two points make the benefits of using ephemeral keying clear. In the case of DSA certificates, it’s necessary for the protocol to succeed, and in the case of RSA certificates, it affords us forward secrecy. In terms of SSL, using ephemeral keys essentially mandates that the keys embedded in the certificates shall be used only for signatures and not for encryption. Understanding this, it seems as though we’ve left a hole in the protocol since we do not have a method for key exchange. OpenSSL provides two options: temporary RSA keys or Diffie-Hellman (DH) key agreement. Of these two choices, DH is better because temporary RSA keys violate the SSL/TLS protocols. The RSA keying was originally implemented to make sure export restrictions on cryptography were not violated.^[1] Today, this issue is not a primary concern; thus, ephemeral RSA keys tend not to be used. Additionally, generation of these temporary RSA keys is much slower than using DH, presuming the DH parameters are pre-generated.

In order to allow OpenSSL to use ephemeral Diffie-Hellman (EDH), we must set up the server-side SSL_CTX object properly. Providing DH parameters directly or, alternatively, a callback that returns the DH parameters accomplishes this goal. The function SSL_CTX_set_tmp_dh sets the DH parameters for a context, while the function SSL_CTX_set_tmp_dh_callback sets the callback. Since the callback mechanism subsumes the functionality of the former method, applications should provide only the callback. The callback function has the following signature:

DH *tmp_dh_callback(SSL *ssl, int is_export, int keylength);

The first argument to the callback is the SSL object representing the connection on which the parameters will be used. The second argument indicates whether an export-restricted cipher is being used; the argument value is nonzero, and zero otherwise. The main advantage of the callback is its ability to provide different functionality based on the third parameter, the key size. The DH parameters returned should have a key size equal to the last argument’s value. Below, our server application is extended with a fully functional callback. The server application in its final form shows an implementation of this callback.

We’ve deferred discussion of the SSL option SSL_OP_SINGLE_DH_USE to this point. Some of the details from Chapter 8 will be helpful in understanding the impact of this option. Essentially, DH parameters are public information. A private key is generated and used for the key exchange from these parameters. Setting this option causes the server to generate a new private key for each new connection. In the end, setting this option provides better security at the cost of more computational power to make new connections. Unless there are special processor usage considerations, we should enable this option.

A cipher suite is a set of algorithms that SSL uses to secure a connection. In order to make a suite, we need to provide algorithms for four functions: signing/authentication, key exchange, cryptographic hashing, and encrypting/decrypting. Keep in mind that some algorithms can serve multiple purposes. For example, RSA can be used for signing and for key exchange.

OpenSSL implements a variety of algorithms and cipher suites when it comes to SSL connections. When designing secure applications, it is essential that algorithms having known security vulnerabilities not be allowed.

The SSL_CTX_set_cipher_list function allows us to set the list of cipher suites that we authorize our SSL objects to use. The list of ciphers is specified by a specially formatted string. This string is a colon-delimited list of algorithms. Given the number of possible combinations, specifying all the acceptable ones explicitly would be quite cumbersome. OpenSSL allows for several keywords in the list, which are shortcuts for sets of ciphers. For instance, “ALL” is a shortcut for every available combination. Additionally, we can precede a keyword with the “!” operator to remove all ciphers associated with the keyword from the list. Using this, we will create a string to define our custom cipher list. There are other operators such as “+” or “-”, but they are not essential for specifying a secure list. For applications that need a custom definition, the ciphers manpage is a good reference on string formation.

SSL allows the use of anonymous ciphers . Anonymous ciphers allow the SSL connection to succeed without proper authentication of the peer by using the DH algorithm. In almost all circumstances, we want to block these ciphers; they are identified by the "ADH” keyword. In addition to suites that do not allow us to authenticate properly, we want to block low-security algorithms. The "LOW” keyword refers to ciphers that use a key of 64 bits or 56 bits without export crippling. Accordingly, the "EXP” keyword marks the ciphers that are export-crippled to 56 or 40 bits. Finally, we should block algorithms that have known weaknesses, e.g., "MD5”.

We can also use the special keyword " @STRENGTH”. Using this indicates that the list of cipher suites should be sorted by their strength (their key size) in order of highest to lowest. Employing this keyword causes our SSL connections to attempt to select the most secure suite possible, and if necessary, back off to the next most secure, and so on down the list. This keyword should be specified last on the list.

Using our knowledge of SSL options, ephemeral keying, and cipher suite selection, we will implement the last step necessary to make our examples fully SSL-enabled, secure applications. After looking at the code for the client and server, we will discuss some of the simplifications that our applications employ.

Example 5-11 contains the code for client3.c, the final client application. Because the only changes we’re making are to the setup_client_ctx function, we truncated the example to only the contents of the source file up to and including that function.

  1  include "common.h"
  2   
  3  #define CIPHER_LIST "ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH"
  4  #define CAFILE "root.pem"
  5  #define CADIR NULL
  6  #define CERTFILE "client.pem"
  7  SSL_CTX *setup_client_ctx(void)
  8  {
  9      SSL_CTX *ctx;
 10   
 11      ctx = SSL_CTX_new(SSLv23_method(  ));
 12      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 13          int_error("Error loading CA file and/or directory");
 14      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 15          int_error("Error loading default CA file and/or directory");
 16      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 17          int_error("Error loading certificate from file");
 18      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 19          int_error("Error loading private key from file");
 20      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
 21      SSL_CTX_set_verify_depth(ctx, 4);
 22      SSL_CTX_set_options(ctx, SSL_OP_ALL|SSL_OP_NO_SSLv2);
 23      if (SSL_CTX_set_cipher_list(ctx, CIPHER_LIST) != 1)
 24          int_error("Error setting cipher list (no valid ciphers)");
 25      return ctx;
 26  }

Line 3 contains a definition of the cipher list we discussed in the previous section. Translated to plain terms, the list is composed of all cipher suites in order of strength except those containing anonymous DH ciphers, low bit-size ciphers, export-crippled ciphers, or the MD5 hash algorithm.

As discussed at the beginning of this step, we enable all bug workarounds and disable SSL Version 2 with the call on line 22. Finally, lines 23-24 actually load our cipher list into the SSL_CTX object.

The server application appears in Example 5-12. We’ve made significantly more changes than in the client, but most of our changes are the addition of new functions that are used only by the SSL context setup function. We’ve similarly truncated the source listing for the server example.

  1  #include "common.h"
  2   
  3  DH *dh512 = NULL;
  4  DH *dh1024 = NULL;
  5  
  6  void init_dhparams(void)
  7  {
  8      BIO *bio;
  9  
 10      bio = BIO_new_file("dh512.pem", "r");
 11      if (!bio)
 12          int_error("Error opening file dh512.pem");
 13      dh512 = PEM_read_bio_DHparams(bio, NULL, NULL, NULL);
 14      if (!dh512)
 15          int_error("Error reading DH parameters from dh512.pem");
 16      BIO_free(bio);
 17  
 18      bio = BIO_new_file("dh1024.pem", "r");
 19      if (!bio)
 20          int_error("Error opening file dh1024.pem");
 21      dh1024 = PEM_read_bio_DHparams(bio, NULL, NULL, NULL);
 22      if (!dh1024)
 23          int_error("Error reading DH parameters from dh1024.pem");
 24      BIO_free(bio);
 25  }
 26  
 27  DH *tmp_dh_callback(SSL *ssl, int is_export, int keylength)
 28  {
 29      DH *ret;
 30  
 31      if (!dh512 || !dh1024)
 32          init_dhparams(  );
 33  
 34      switch (keylength)
 35      {
 36          case 512:
 37              ret = dh512;
 38              break;
 39          case 1024:
 40          default: /* generating DH params is too costly to do on the fly */
 41              ret = dh1024;
 42              break;
 43      }
 44      return ret;
 45  }
 46  
 47  #define CIPHER_LIST "ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH"
 48  #define CAFILE "root.pem"
 49  #define CADIR NULL
 50  #define CERTFILE "server.pem"
 51  SSL_CTX *setup_server_ctx(void)
 52  {
 53      SSL_CTX *ctx;
 54   
 55      ctx = SSL_CTX_new(SSLv23_method(  ));
 56      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 57          int_error("Error loading CA file and/or directory");
 58      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 59          int_error("Error loading default CA file and/or directory");
 60      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 61          int_error("Error loading certificate from file");
 62      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 63          int_error("Error loading private key from file");
 64      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT,
 65                         verify_callback);
 66      SSL_CTX_set_verify_depth(ctx, 4);
 67      SSL_CTX_set_options(ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2 |
 68                               SSL_OP_SINGLE_DH_USE);
 69      SSL_CTX_set_tmp_dh_callback(ctx, tmp_dh_callback);
 70      if (SSL_CTX_set_cipher_list(ctx, CIPHER_LIST) != 1)
 71          int_error("Error setting cipher list (no valid ciphers)");
 72      return ctx;
 73  }

The most obvious change to this file is the addition of the functions init_dhparams and tmp_dh_callback . The initializing function reads the DH parameters from the files dh512.pem and dh1024.pem and loads them into the global parameters. The callback function simply switches on the required key size and returns either a 512-bit DH parameter set or a 1,024-bit set. This function intentionally does not try to perform any on-the-fly generation of parameters because it is simply too computationally expensive to be worthwhile.

The only other changes to the server not done to the client, aside from the call on line 60 to set the callback, is the inclusion of the SSL option SSL_OP_SINGLE_DH_USE . As discussed earlier, this causes the private part of the DH key exchange to be recomputed for each client connecting.

For purposes of clarity in the example code, we have avoided several serious considerations for real applications. The most obvious is error handling. Our examples simply exit if errors of any kind occur. For most applications, extension of the example to more robustly handle errors is application-specific. While specific functions may return different error codes, OpenSSL functions and macros will generally return 1 on success; thus, implementing better error handling should be straightforward.

In addition, the examples we’ve built do two-way authentication. When making a new client and server application, this should always be done. However, when making applications that are meant to connect with other SSL peers, such as a web server, we should take into account that the stringent security requirements of our example are not always desirable. For instance, to entirely remove client authentication from a server, we simply need to remove the calls that load the verify locations, the call to set the verify mode, and the call to the post-connection verification routine. This isn’t the best approach, though. When making such compatibility-first applications, we need to try to incorporate as much security as possible. For instance, instead of removing all of the verification calls, we could still load the verification locations and request a peer certificate with the SSL option SSL_VERIFY_PEER . We can, however, omit the SSL_VERIFY_FAIL_IF_NO_PEER_CERT option and modify the post-connection verification so that if the client does present a certificate, we go on with high security. If the client does not present the server with a certificate, the condition and associated information can be logged so that we can keep track of unauthenticated connections.

Another point we avoided was the password callback for the encrypted private key for the certificate. For most server applications, common practice is to leave the private key in a file that isn’t encrypted so that the application can start up and stop without user input. This convention, born of simplicity of implementation, can easily be subverted. When doing something like this, it is essential that we make sure users on the machine do not have read permission on the private key file; ideally, the file will also be owned by the root or administrator user so that the likelihood of compromise is further reduced. For most client applications, tying the encryption passphrase to the user should not be a problem.

DSA parameters can be converted to DH parameters. This method is often utilized since the computational power necessary to generate the DSA parameters is smaller. Often, parameters generated in this fashion are used for ephemeral DH parameters. Without having to get into the mathematics behind the algorithms, the SSL option SSL_OP_SINGLE_DH_USE should always be used in these cases. Without it, applications are susceptible to a subtle attack.

A large flaw of our example programs is their handling of I/O on SSL connections. The examples rely on blocking I/O operations. For most real applications, this is unacceptable. In the following section, we broach the important topic of non-blocking I/O on SSL connections. We have also neglected to consider renegotiations (requesting that a handshake be performed during an already established connection) on SSL connections and their impact on I/O. While this should occur automatically when the peer requests it, the I/O routines must be robust enough to handle its subtle impacts. In the next section, we begin by addressing server efficiency with respect to session caching and then move into a more in-depth look at I/O paradigms.

When a connection is established, the function SSL_get_session returns the SSL_SESSION object representing the parameters used on the SSL connection. This function returns NULL if there is no session established on the SSL connection; otherwise, it returns the object. Actually, this function is called as either SSL_get0_session or SSL_get1_session . These two variants are used to ensure that the reference counting on the SSL_SESSION object is updated correctly. The former does not change the reference count, and the latter increments it. In general, we want to use the latter function since SSL_SESSION objects can timeout asynchronously. If this occurs, our object disappears and we are holding an invalid reference to it. Using SSL_get1_session requires that we call SSL_SESSION_free on the object when we’re done using it, in order to prevent memory leaks.

After saving a reference to the SSL_SESSION object, we can close down the SSL connection and its underlying transport (normally a socket). For most clients, there will not be a large number of SSL sessions established at one time, so caching them in memory is adequate. If this isn’t the case, we can always write the sessions out to disk using PEM_write_bio_SSL_SESSION or PEM_write_SSL_SESSION , and reread them later using PEM_read_bio_SSL_SESSION or PEM_read_SSL_SESSION . The syntax of these functions is the same as the functions used for reading and writing public-key objects. They are discussed in Chapter 8 in Section 8.6.2.

To reuse a saved session, we need to call SSL_set_session before calling SSL_connect. The reference count of the SSL_SESSION object will be incremented automatically so we should follow the call with SSL_SESSION_free. After reusing a session, it is a good idea to call SSL_get1_session before disconnecting to replace the SSL_SESSION object that we’ve cached. The reason is that renegotiations may occur during the connection. Renegotiations cause the creation of a new SSL_SESSION, so we should keep only the most recent (renegotiation is discussed in more detail later).

Now that we understand the basics of enabling session caching, we’ll take a brief look at incorporating it into a client application. Example 5-13 shows pseudocode for our client-side session caching. Now that we’ve established an implementation for a client, we’ll elaborate upon some of the details of session caching as we explore the implementation necessary for server-side caching.

ssl = SSL_new(ctx)
... setup underlying communications layer for ssl ...
... connect to host:port ...
if (saved session for host:port in cache)
  SSL_set_session(ssl, saved session)
  SSL_SESSION_free(saved session)
SSL_connect(ssl)
call post_connection_check(ssl, host) and check return value
... normal application code here ...
saved session = SSL_get1_session(ssl)
if (saved session != NULL)
  enter saved session into cache under host:port
SSL_shutdown(ssl)
SSL_free(ssl)

All sessions must have a session ID context. For the server, session caching is disabled by default unless a call to SSL_CTX_set_session_id_context is made. The purpose of the session ID context is to make sure the session is being reused for the same purpose for which it was created. For instance, a session created for an SSL web server should not be automatically allowed for an SSL FTP server. Along the same lines, we can use session ID contexts to exercise a finer-grained control over sessions within our application. For example, authenticated clients could have a different session ID context than unauthenticated ones. The context itself can be any data we choose. We set the context through a call to the above function, passing in our context data as the second argument and its length as the third.

After setting the session ID context, session caching is enabled on the server side; however, it isn’t configured completely yet. Sessions have a limited lifetime. The default value for session timeout with OpenSSL is 300 seconds. If we want to change this lifetime, a call to SSL_CTX_set_timeout is necessary. Although the default server automatically flushes expired sessions, we may still want to call SSL_CTX_flush_sessions manually, e.g., when we disable automatic session flushing.

One important function that allows us to tweak the behavior of a server with respect to caching is SSL_CTX_set_session_cache_mode . Like several other mode-setting calls in OpenSSL, the mode is set using flags that are joined by a logical OR operation. One such flag is SSL_SESS_CACHE_NO_AUTO_CLEAR. This disables the automatic flushing of expired sessions. It is useful for servers with tighter processor usage constraints. The automatic behavior can cause unexpected delays; thus, disabling it and manually calling the flush routine when free cycles are available can improve performance. Another flag that can be set is SSL_SESS_CACHE_NO_INTERNAL_LOOKUP. Up to this point, we’ve relied solely on the internal lookup, but in the next section, we’ll outline an on-disk caching method that can replace the internal lookup methods.

Session caching adds subtleties when we also use connection renegotiations. Before implementing a caching server, we should be aware of the potential pitfalls. The section below on renegotiation explores some of these problems after explaining a little more about what renegotiation entails.

OpenSSL’s session caching features include API calls to set three callbacks for synchronization of sessions with external caches. Like other OpenSSL callbacks, three functions are used to set a pointer to the callback function. For each one, the first argument is an SSL_CTX object, and the second argument is a pointer to the callback function.

The callback set by SSL_CTX_sess_set_new_cb is invoked whenever a new SSL_SESSION is created by the SSL_CTX object. This callback enables us to add the new session to our external container. If the callback returns zero, the session object will not be cached. A nonzero return allows the session to be cached.

int new_session_cb(SSL *ctx, SSL_SESSION *session);

The callback set by SSL_CTX_sess_set_remove_cb is invoked whenever an SSL_SESSION is destroyed. It is called just before the session object is destroyed because it is invalid or has expired.

void remove_session_cb(SSL_CTX *ctx, SSL_SESSION *session);

The SSL_CTX_sess_set_get_cb function is used to set a cache retrieval callback. The assigned function is called whenever the internal cache cannot find a hit for the requested session resumption. In other words, this callback should query our external cache in hopes of finding a match.

SSL_SESSION *get_session_cb(SSL *ctx, unsigned char *id, int len, int *ref);

Some of the features required by the caching mechanism in Example 5-14 are easily implemented. Since we are using files to store the sessions, we can use the filesystem’s built-in locking mechanisms. In order to write the keys to disk, we can use the macro PEM_write_bio_SSL_SESSION, but it doesn’t allow for encryption. Remember, SSL_SESSION objects hold the shared secrets that were negotiated; thus, their contents shoud be protected when serialized. Instead, we can call the underlying function PEM_ASN1_write_bio directly. Alternatively, it may be sufficient for some applications to simply use a secure directory and write the sessions unencrypted. In general, it is far safer to use encryption with an in-memory key.

new_session_cb(  )
{
  acquire a lock for the file named according to the session id
  open file named according to session id
  encrypt and write the SSL_SESSION object to the file
  release the lock
}

remove_session_cb(  )
{
  acquire a lock for the file named according to the session id
  remove the file named according to the session id
  release the lock
}

get_session_cb(  )
{
  acquire a lock for the file named according to the session id in the 2nd arg
  read and decrypt contents of file and create a new SSL_SESSION object
  release the lock
  set the integer referenced by the fourth parameter to 0
  return the new session object
}

This framework, when implemented, provides a powerful mechanism for session caching. By using the filesystem, the cache isn’t constrained by memory restrictions. Additionally, it is easier to use than an in-memory caching scheme since the on-disk scheme allows multiple processes to access the session cache.

In the examples, we used the I/O functions SSL_read and SSL_write for SSL connections, but we didn’t discuss them in any detail. The arguments for these calls are similar to the system calls read and write. The way these calls differ from their system call counterparts is in their return values. Table 5-2 provides the details of the possible return values.

Knowing these return values helps, but we still can’t tell if an error actually occured without some more information. OpenSSL provides the function SSL_get_error , which accepts the return value of an SSL I/O function and returns different values based on what actually happened.

This function inspects the I/O routine’s return value, the SSL object, and the current thread’s error queue to determine what the effect of the call was. Because the internal errors for OpenSSL are stored on a per-thread basis, the call to check the errors must be made from the same thread as the I/O call. Robust applications should always check the detailed error status of all I/O operations with SSL_get_error. Remember that functions like SSL_connect and SSL_accept are also I/O operations.

The return value of SSL_get_error can be many different values. Our application needs to be able to handle a subset of these conditions and provide a reasonable default behavior of shutting down the SSL connection. Table 5-3 provides the descriptions of the values we should always be able to handle.

The handling of the first error code in the table, SSL_ERROR_NONE, is straightforward. SSL_ERROR_ZERO_RETURN should be handled in an application-specific way, knowing that the SSL connection has been closed. In order to handle SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE, we need to retry the I/O operation; this is discussed in more detail below. Any of the other possible return values of SSL_get_error should be considered errors.

In order to implement a call to an I/O function correctly, we should check for these different return values. A sample piece of code is given in Example 5-15. The handlers for SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE have been omitted; the details on handling them properly are discussed below, since the correct actions vary based on whether the application is using blocking or non-blocking I/O. This example’s purpose is to provide a template of what a robust I/O call should look like. Using the switch statement, we handle all the conditions for which we have interest and error on any others. As stated above, there are other return values possible, and we may wish to handle them with specific error messages, for instance.

code = SSL_read(ssl, buf + offset, size - offset);
switch (SSL_get_error(ssl, code))
{
    case SSL_ERROR_NONE:
        /* update the offset value */
        offset += code;
        break;
    case SSL_ERROR_ZERO_RETURN:
        /* react to the SSL connection being closed */
        do_cleanup(ssl);
        break;
    case SSL_ERROR_WANT_READ:
        /* handle this in an application specific way to retry the SSL_read */
        break;
    case SSL_ERROR_WANT_WRITE:
        /* handle this in an application specific way to retry the SSL_read */
        break;
    default:
        /* an error occurred. shutdown the connection */
        shutdown_connection(ssl);
}

Typically, using blocking I/O alleviates the complications of retrying failed calls. With SSL, this is not the case. There are times when a call to SSL_read or SSL_write on a blocking connection requires a retry.

Blocking I/O means that an operation will wait until it can be completed or until an error occurs. Thus, with blocking I/O in general, we shouldn’t have to retry an I/O call, since a single call’s failure indicates an error with the communication channel, such that the channel is no longer usable; with blocking I/O using OpenSSL, this is not true. The SSL connection is based upon an underlying I/O layer that handles the actual transfer of data from one side to the other. The blocking property of an SSL connection is based solely on the underlying communication layer. For instance, if we create an SSL connection from a socket, the SSL connection will be blocking if and only if the socket is blocking. It is also possible to change this property in a connection we’ve already established; we change the property on the underlying layer and then handle all SSL I/O functions appropriate for the new paradigm.

Conceptually, SSL_read and SSL_write read and write data from the peer. Actually, a call to SSL_read may write data to the underlying I/O layer, and a call to SSL_write may read. This usually occurs during a renegotiation. Since a renegotiation may occur at any time, this behavior can cause unexpected results when using a blocking I/O layer; namely, an I/O call may require a retry. Thus, our implementation must handle this.

In order to handle a blocking call correctly, we need to retry the SSL I/O operation if we receive SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE . Don’t let the names of these errors confuse you. Even though they tell us the SSL connection needs to wait until we can read or write, respectively, we just need to retry whatever operation caused the condition. For instance, if an SSL_read caused the SSL_ERROR_WANT_WRITE error, we must retry the SSL_read rather than making the call to SSL_write. It is worth taking a moment to understand the potential errors for a single I/O call. Though nonintuitive, a call to SSL_read may indeed return the error SSL_ERROR_WANT_WRITE due to the possibility of renegotiations at any point.

In many ways, implementing a blocking call is similar to implementing a non-blocking one. It’s similar in the sense that we must loop for retries, but it differs in that we don’t need to check for I/O availability on the underlying layer, as with poll or select. In the end, we will not show an example of looping to accomplish the blocking call; there is an easier way.

Using the SSL_CTX_set_mode function or its counterpart for SSL objects, SSL_set_mode, we can set some I/O behaviors of SSL connections. The second parameter is a set of defined properties joined with the logical OR operator. SSL_MODE_AUTO_RETRY is one such mode. Setting this on a blocking SSL object (or on the context that will create a object) will cause all I/O operations to automatically retry all reads and complete all negotiations before returning.

Using this option allows us to implement I/O as simply as normal blocking I/O with the system calls read and write. In general, we should set this option on the SSL_CTX object before creating the SSL object. We can set the option later on the SSL object itself, but it’s best to do so before any calls to I/O routines on that object.

If we elect not to use this option and instead implement our blocking I/O with our own loops, we might fall into a few traps. This is due to some special requirements for function call retries, which are detailed with our discussion of non-blocking I/O.

This paradigm causes all of our I/O calls never to block. If the underlying layer is unable to handle a request, it reports its requirement immediately, without waiting. As we’ve hinted, this adds complexity to our I/O routines.

A non-blocking SSL I/O call returns the reason of failure, but only the application can check to see if that status has been cleared. This is the source of the complexity in implementation. For instance, a call to SSL_read may return SSL_ERROR_WANT_READ, which tells the application that once the underlying I/O layer is ready to fulfill a read request, the SSL call may be retried. Generally, the application’s I/O loop will need to serve both read and write requests, however. The problem we need to solve in the I/O loop is that once we’ve made a call to an SSL I/O function, and it requires a retry, we should not call other I/O functions until the original call has succeeded.

Since the logic for correctly implementing I/O routines for the application can have several subtleties, especially with multiple input and output sources, we’ll look at a detailed example. Example 5-16 provides the code for the function data_transfer . This function takes two SSL objects as parameters, which are expected to have connections to two different peers. The data_transfer function will read data from one connection (A) and write it to the other (B) and at the same time read data from B and write it to A.

  1  #include <openssl/ssl.h>
  2  #include <openssl/err.h>
  3  #include <string.h>
  4   
  5  #define BUF_SIZE 80
  6   
  7  void data_transfer(SSL *A, SSL *B)
  8  {
  9      /* the buffers and the size variables */
 10      unsigned char A2B[BUF_SIZE];
 11      unsigned char B2A[BUF_SIZE];
 12      unsigned int A2B_len = 0;
 13      unsigned int B2A_len = 0;
 14      /* flags to mark that we have some data to write */
 15      unsigned int have_data_A2B = 0;
 16      unsigned int have_data_B2A = 0;
 17      /* flags set by check_availability(  ) that poll for I/O status */
 18      unsigned int can_read_A = 0;
 19      unsigned int can_read_B = 0;
 20      unsigned int can_write_A = 0;
 21      unsigned int can_write_B = 0;
 22      /* flags to mark all the combinations of why we're blocking */
 23      unsigned int read_waiton_write_A = 0;
 24      unsigned int read_waiton_write_B = 0;
 25      unsigned int read_waiton_read_A = 0;
 26      unsigned int read_waiton_read_B = 0;
 27      unsigned int write_waiton_write_A = 0;
 28      unsigned int write_waiton_write_B = 0;
 29      unsigned int write_waiton_read_A = 0;
 30      unsigned int write_waiton_read_B = 0;
 31      /* variable to hold return value of an I/O operation */
 32      int code;
 33   
 34      /* make the underlying I/O layer behind each SSL object non-blocking */
 35      set_non-blocking(A);
 36      set_non-blocking(B);
 37      SSL_set_mode(A, SSL_MODE_ENABLE_PARTIAL_WRITE|
 38                      SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); 
 39      SSL_set_mode(B, SSL_MODE_ENABLE_PARTIAL_WRITE| 
 40                      SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER);
 41   
 42      for (;;)
 43      {
 44          /* check I/O availability and set flags */
 45          check_availability(A, &can_read_A, &can_write_A,
 46                             B, &can_read_B, &can_write_B);
 47   
 48          /* this "if" statement reads data from A. it will only be entered if
 49           * the following conditions are all true:
 50           * 1. we're not in the middle of a write on A
 51           * 2. there's space left in the A to B buffer
 52           * 3. either we need to write to complete a previously blocked read
 53           *    and now A is available to write, or we can read from A
 54           *    regardless of whether we're blocking for availability to read.
 55           */
 56          if (!(write_waiton_read_A || write_waiton_write_A) &&
 57              (A2B_len != BUF_SIZE) &&
 58              (can_read_A || (can_write_A && read_waiton_write_A)))
 59          {
 60              /* clear the flags since we'll set them based on the I/O call's
 61               * return
 62               */
 63              read_waiton_read_A = 0;
 64              read_waiton_write_A = 0;
 65   
 66              /* read into the buffer after the current position */
 67              code = SSL_read(A, A2B + A2B_len, BUF_SIZE - A2B_len);
 68              switch (SSL_get_error(A, code))
 69              {
 70                  case SSL_ERROR_NONE:
 71                      /* no errors occured.  update the new length and make
 72                       * sure the "have data" flag is set.
 73                       */
 74                      A2B_len += code;
 75                      have_data_A2B = 1;
 76                      break;
 77                  case SSL_ERROR_ZERO_RETURN:
 78                      /* connection closed */
 79                      goto end;
 80                  case SSL_ERROR_WANT_READ:
 81                      /* we need to retry the read after A is available for
 82                       * reading
 83                       */
 84                      read_waiton_read_A = 1;
 85                      break;
 86                  case SSL_ERROR_WANT_WRITE:
 87                      /* we need to retry the read after A is available for
 88                       * writing
 89                       */
 90                      read_waiton_write_A = 1;
 91                      break;
 92                  default:
 93                      /* ERROR */
 94                      goto err;
 95              }
 96          }
 97   
 98          /* this "if" statement is roughly the same as the previous "if"
 99           * statement with A and B switched
100           */
101          if (!(write_waiton_read_B || write_waiton_write_B) &&
102              (B2A_len != BUF_SIZE) &&
103              (can_read_B || (can_write_B && read_waiton_write_B)))
104          {
105              read_waiton_read_B = 0;
106              read_waiton_write_B = 0;
107   
108              code = SSL_read(B, B2A + B2A_len, BUF_SIZE - B2A_len);
109              switch (SSL_get_error(B, code))
110              {
111                  case SSL_ERROR_NONE:
112                      B2A_len += code;
113                      have_data_B2A = 1;
114                      break;
115                  case SSL_ERROR_ZERO_RETURN:
116                      goto end;
117                  case SSL_ERROR_WANT_READ:
118                      read_waiton_read_B = 1;
119                      break;
120                  case SSL_ERROR_WANT_WRITE:
121                      read_waiton_write_B = 1;
122                      break;
123                  default:
124                      goto err;
125              }
126          }
127   
128          /* this "if" statement writes data to A. it will only be entered if
129           * the following conditions are all true:
130           * 1. we're not in the middle of a read on A
131           * 2. there's data in the A to B buffer
132           * 3. either we need to read to complete a previously blocked write
133           *    and now A is available to read, or we can write to A
134           *    regardless of whether we're blocking for availability to write
135           */
136          if (!(read_waiton_write_A || read_waiton_read_A) &&
137              have_data_B2A &&
138              (can_write_A || (can_read_A && write_waiton_read_A)))
139          {
140              /* clear the flags */
141              write_waiton_read_A = 0;
142              write_waiton_write_A = 0;
143   
144              /* perform the write from the start of the buffer */
145              code = SSL_write(A, B2A, B2A_len);
146              switch (SSL_get_error(A, code))
147              {
148                  case SSL_ERROR_NONE:
149                      /* no error occured. adjust the length of the B to A
150                       * buffer to be smaller by the number bytes written.  If
151                       * the buffer is empty, set the "have data" flags to 0,
152                       * or else, move the data from the middle of the buffer
153                       * to the front.
154                       */
155                      B2A_len -= code;
156                      if (!B2A_len)
157                          have_data_B2A = 0;
158                      else
159                          memmove(B2A, B2A + code, B2A_len);
160                      break;
161                  case SSL_ERROR_ZERO_RETURN:
162                      /* connection closed */
163                      goto end;
164                  case SSL_ERROR_WANT_READ:
165                      /* we need to retry the write after A is available for
166                       * reading
167                       */
168                      write_waiton_read_A = 1;
169                      break;
170                  case SSL_ERROR_WANT_WRITE:
171                      /* we need to retry the write after A is available for
172                       * writing
173                       */
174                      write_waiton_write_A = 1;
175                      break;
176                  default:
177                      /* ERROR */
178                      goto err;
179              }
180          }
181   
182          /* this "if" statement is roughly the same as the previous "if"
183           * statement with A and B switched
184           */
185          if (!(read_waiton_write_B || read_waiton_read_B) &&
186              have_data_A2B &&
187              (can_write_B || (can_read_B && write_waiton_read_B)))
188          {
189              write_waiton_read_B = 0;
190              write_waiton_write_B = 0;
191   
192              code = SSL_write(B, A2B, A2B_len);
193              switch (SSL_get_error(B, code))
194              {
195                  case SSL_ERROR_NONE:
196                      A2B_len -= code;
197                      if (!A2B_len)
198                          have_data_A2B = 0;
199                      else
200                          memmove(A2B, A2B + code, A2B_len);
201                      break;
202                  case SSL_ERROR_ZERO_RETURN:
203                      /* connection closed */
204                      goto end;
205                  case SSL_ERROR_WANT_READ:
206                      write_waiton_read_B = 1;
207                      break;
208                  case SSL_ERROR_WANT_WRITE:
209                      write_waiton_write_B = 1;
210                     break;
211                  default:
212                      /* ERROR */
213                      goto err;
214              }
215          }
216      }
217   
218  err:
219      /* if we errored, print then before exiting */
220      fprintf(stderr, "Error(s) occured\n");
221      ERR_print_errors_fp(stderr);
222  end:
223      /* close down the connections. set them back to blocking to simplify. */
224      set_blocking(A);
225      set_blocking(B);
226      SSL_shutdown(A);
227      SSL_shutdown(B);
228  }

Since the code sample is rather large, we’ll dissect it and explain the reasons behind the implementation decisions. We use two buffers, A2B and B2A, to hold the data read from A to be written to B, and the data read from B to be written to A, respectively. The length variables corresponding to each buffer (A2B_len and B2A_len) are initialized to zero; throughout our function, they will hold the number of bytes of data in their counterpart buffer. An important observation to make at this point is that we do not use an offset variable for pointing into our buffer; the data we want to write will always be at the front of our buffers.

We also declare three sets of flags. The first set (have_data_A2B and have_data_B2A) indicates whether there is any data in our buffers. We could have left these two variables out, since throughout the function they will be zero only if the corresponding buffer length’s variable is also zero. We opted to use them for code readability. The next set of variables is availability flags, which are of the form can_read_A, can_write_B, etc. These flags are set to indicate that the named operation on the named object can be performed, i.e., the connection is available to perform the operation without needing to block. The last set of flags is the blocking flags. When one of these is set, it tells us that a particular kind of I/O operation requires the availability of the connection to perform another kind of I/O operation. For instance, if write_waiton_read_B is set, it means that the last write operation performed on B must be retried after B is available to read.

We use three functions in this example that are not explicitly defined. The first is set_nonblocking . This function takes an SSL object as its only argument and must be implemented in a platform-specific way to set the underlying I/O layer of the SSL object to be non-blocking. Likewise, set_blocking needs to set the connection to a blocking state. The last platform-specific function is check_availability . This function’s obligation is to check the I/O status of the underlying layers of both A and B and set the variables appropriately. Additionally, this function should wait until at least one variable is set before returning. We cannot perform any I/O operations if nothing is available for either connection. These omitted functions can be implemented easily. For instance, on a Unix system with SSL objects based on sockets, set_nonblocking and set_blocking can be implemented using the fcntl system call, and the check_availability function can use fd_set data structures along with the select system call.

The calls to SSL_set_mode set two mode variables we’ve not discussed. Normally, a call to SSL_write will not return success until all of the data in the buffer has been written, even with non-blocking I/O. Thus, a call that requests a large amount of data to be written can return many retry requests before succeeding. This behavior is undesirable in our function because we wish to interlace the reading routines with those that write, and to do this effectively, we cannot retry a single call for too long because it will prevent us from reading more data from that connection. To alleviate this problem, the SSL_MODE_ENABLE_PARTIAL_WRITE mode flag instructs the library to allow partially complete writes to count as successes. This allows our function to perform several write operations successfully, and between those calls, we can read more data. Because the SSL protocol is built around sending complete messages on the channel, sometimes we’ll be required to retry a call because only a part of the requested data was actually sent. As a result, all retried operations must be called with the exact same arguments as the original call that caused the error. This behavior can be cumbersome since it does not allow us to add to the write buffer after making a call to SSL_write that needed a retry. To change this property partially, we use the SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER flag. This flags allows us to retry the write operation with different parameters, i.e., a different buffer and size, so long as the original data is still contained in the new buffer. In our example, we use the same buffer, but enable this mode so that we can keep reading into the end of the buffer without causing errors on attempts to retry writes.

The function has one main loop, beginning on line 42 and ending on line 216. In this loop, we have four separated subsections, lines 56-96, lines 101-126, lines 136-180 and lines 185-215. Each of these conditional blocks corresponds respectively to reading data from A, reading data from B, writing data to A, and writing data to B. Looking carefully, we can see that the first and second section mirror each other just as the third and fourth do.

We must be sure it is safe to read. The entry conditions on the if statements do this. They assert we are not in the middle of a read operation by checking the write blocking flags for the connection (!(write_waiton_read_... || write_waiton_write_...)). Additionally, it needs to be sure there is space in the buffers (..._len != BUF_SIZE). In addition to these conditions, we must be sure this is the right time to call the read function, i.e., either we’re waiting to retry a failed read because it was waiting for the connection to be available for writing (can_write_... && read_waiton_write_...), or we simply have data to read (can_read_...). If all of these conditions are met, we can attempt a read. Before we do, though, we reset the blocking flags for the read operation, and then perform the I/O call and check the error code. The I/O call itself instructs the SSL library to write the data into the buffer offset by the number of bytes already stored within it. If the call succeeds, we’ll update the length counter and make sure that the have_data_... flag is set. If we are instructed to retry through either SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, we will set the appropriate blocking flag and carry on. If an error occurs, or the end of the SSL connection is detected, we will break out of the I/O loop.

As with read operations, write operations are protected by an if statement to assure entry conditions. These statements ensure that we are not in the middle of a read operation (!(read_waiton_write_... || read_waiton_read_...)). Additionally, there must be data in the buffer (have_data_...). Lastly, the write operation must make sure that writing is possible, i.e., either we’ve been waiting for read availability to retry a write operation (can_read_... && write_waiton_read_...), or there is write availability (can_write_...). Before attempting the write, we must zero the blocking flags. The write operation always tries to write from the beginning of the buffer. If we are successful, we move the data in the buffer forward so that data already written is pushed out. If the buffer is empty after writing, we zero the have_data_... flag. As with the read operation, if we are instructed to retry, we set the corresponding blocking flag and continue. If errors occur, or if the SSL connection has been closed, we break out of the I/O loop.

These paradigms for performing reading and writing enable us to do effective non-blocking communication when combined in the loop. For instance, if B cannot be written to, we will continue to buffer data from A until the buffers fill and we are forced to wait for writing on B. We can extend the general form presented in this function to allow for many different non-blocking I/O needs that applications may have.

As we’ve said before, to do a passive renegotiation (i.e., one in which the handshake occurs during application I/O) we need to call only SSL_renegotiate. In general, applications can get away with having the handshake occur during application I/O, but making sure that it did indeed happen is important. Unfortunately, this isn’t easy to do with OpenSSL Version 0.9.6. In fact, many aspects of renegotiations don’t work cleanly in this version. However, the forthcoming Version 0.9.7 promises to improve this considerably. These changes are described in the next section.

With Version 0.9.6, we should always use explicit renegotiation since there is no way to determine if a renegotiation request was ignored by the peer. This is the same problem we find in renegotiating client credentials; we will focus on determining if a request was ignored, since it is far more common. Renegotiating for session key refreshment is a subset of this problem. Example 5-17 shows an incomplete code fragment to force a renegotiation from a server.

/* assume ssl is connected and error free up to here */
set_blocking(ssl); /* this is unnecessary if it is already blocking */
SSL_renegotiate(ssl);
SSL_do_handshake(ssl);
if (ssl->state != SSL_ST_OK)
    int_error("Failed to send renegotiation request");
ssl->state |= SSL_ST_ACCEPT;
SSL_do_handshake(ssl);
if (ssl->state != SSL_ST_OK)
    int_error("Failed to complete renegotiation");
/* our renegotiation is complete */

This example uses some functions we’ve seen before. To avoid extra complication, we ensure the SSL object is blocking so we don’t need to retry failed I/O calls. The call to SSL_renegotiate sends out the request to the peer. The function SSL_do_handshake is a generic routine that calls the accept function for server objects or the connect function for client objects. This first call to SSL_do_handshake sends out our request and returns. After doing this, we need to check that the SSL connection hasn’t received any errors. We do this by making sure its state is SSL_ST_OK. At this point, if we call the handshake function again, it will just return if the peer chose not to renegotiate. This occurs because the SSL/TLS protocols allow requests to be ignored. Since we have a reason for renegotiating, and we need it to complete before continuing, we must manually set the SSL_ST_ACCEPT state of the server object. This will cause the subsequent call to SSL_do_handshake, which will force a handshake to occur before continuing.

Obviously, this method of renegotiation isn’t very clean because it requires us to set internal variables of the SSL object manually. Unfortunately, it is the only way to accomplish a forced renegotiation. This code fragment is not complete, though. Consider session caching, which allows a client to skip the handshake by resuming a previously created session. This can be extremely bad when our purpose is to collect stronger client credentials since the client has already obtained a valid session with weak credentials. When we attempt to renegotiate a connection from a server that does session caching, we must take extra precautions that the client doesn’t simply present the previously negotiated session and bypass the handshake. To make this discrepancy between the sessions, we need to change the session ID context. Recall that the session ID context’s function is to discern sessions established with clients for different purposes. To change its value, we use the function SSL_set_session_id_context . It behaves exactly as the SSL_CTX version discussed above, except that it operates on SSL objects. The change to the session ID context must be made before the renegotiation is started.

We haven’t discussed the minor detail of setting stronger requirements for verification of the client during renegotiation. To do this, we use the function SSL_set_verify and pass it our new verify flags. Example 5-18 shows the code fragment that must be built around the fragment shown in Example 5-17 in order for the renegotiation to be effective. This fragment is for a caching server that wishes to upgrade client authentication; if our server isn’t caching, we can omit the calls to set the session ID context.

/* assume ctx is an SSL_CTX object that is setup to not have any verify
   options. */
int normal_user = 1;
int admin_user = 2;
SSL_CTX_set_session_id_context(ctx, &normal_user, sizeof(int));
/* perform rest of ctx setup, create an ssl object, and connect it */
/* normal SSL I/O operations and application code go here */
/* if we want to upgrade client privilege, we enter the following code block */
SSL_set_verify(ssl, SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT,
               verify_callback);
SSL_set_session_id_context(ssl, &admin_user, sizeof(int));
/* code fragment from Example 5-18 goes here.  the new session is made */
post_connection_check(ssl, host);
/* if everything is error-free, we have properly authenticated the client */

The code in Example 5-18 realizes the solution to the problem we laid out earlier for upgrading client authentication. By changing the session context ID to admin_user, we allow clients previously verified as admin users to resume connections, but no others. This is effective at keeping resumed sessions from being mistaken as privileged sessions. In addition, we set the verify options for the SSL object explicitly, forcing the renegotiation to demand a client certificate or fail. After renegotiation is complete, we call the post-connection check function. In some cases, we may want to tailor the post-connection function to meet application-specific needs.

Overall, renegotiation in Version 0.9.6 is inferior to the functions and simplicity that Version 0.9.7 promises. A new function has been added, SSL_regenotiate_pending . This function will return nonzero if a request is sent, but the handshake hasn’t been finished. It will return zero once the handshake is complete. Using this function, we can eliminate most of the ugliness associated with renegotiations in 0.9.6. Before looking at forced renegotiations, we’ll briefly return to passive renegotiations.

In most applications, renegotiations for changing the session key rather than upgrading client authentication are started by byte transfer thresholds. In other words, once our connection has transferred a certain number of bytes, we will renegotiate. Because of this new function, we can simply call SSL_renegotiate when the byte limit is reached. Then we periodically check the value of SSL_renegotiate_pending to determine if the renegotiation completed. Doing this, we can programmatically fail if the handshake isn’t completed in a certain amount of time after the request.

Furthermore, a new SSL option to aid us has been added in Version 0.9.7. By setting SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION with a call to SSL_CTX_set_options, we can automatically prevent clients from being able to resume sessions when we ask for renegotiations, regardless of the session ID context. When our goal is to refresh session keys, this option is invaluable.

Using these two new additions will also allow us to make a much cleaner forced renegotiation for client authentication. We can call SSL_renegotiate to set the flag and make a single call to SSL_do_handshake to send the request out. Instead of setting internal state of the SSL object, we can now just call SSL_do_handshake until we either programmatically timeout or SSL_renegotiate_pending returns zero. If the latter condition is met, our renegotiation completed successfully. Ideally, we want to leave the session ID context changing and not set the new SSL option when performing renegotiation for client authentication. This is better because it allows authenticated users to resume authenticated sessions rather than always perform the full handshake.

We’ve limited our discussion of renegotiations to server-side implementation. In general, applications will be made this way since servers dictate session caching, and a server almost always presents credentials. However, it is possible for a client to request or force renegotiation, though it is less common. Doing this is a logical extension of the methods used by a server. We now have a better understanding of renegotiations, why they occur, and why they’re needed for certain operations. Forcing a renegotiation to force the client to provide a certificate to continue the connection is also a popular paradigm for implementing SSL applications. As we discussed early in this section, the alternatives to renegotiation for accomplishing this task are often too burdensome to be a general solution.

One major point that’s been missing up to now is how to make an application react to renegotiation requests. The good news here is that it’s all handled by the OpenSSL library for us. Recall the complications with I/O. The reason we had to handle all the different varieties of retries is that a renegotiation request could be received at any time. When an SSL connection is requested to renegotiate, the implementation automatically does so and completes the new handshake to generate a new session.