The primary goal of cryptography is to secure important data as it passes through a medium that may not be secure itself. Usually, that medium is a computer network.

There are many different cryptographic algorithms, each of which can provide one or more of the following services to applications:

These simple services can be used to stop a wide variety of network attacks, including:

Many people assume that some (or all) of the above attacks aren’t actually feasible in practice. However, that’s far from the truth. Especially due to tool sets such as dsniff (http://www.monkey.org/~dugsong/dsniff/), it doesn’t even take much experience to launch all of the above attacks if access to any node on a network between the two endpoints is available. Attacks are equally easy if you’re on the same local network as one of the endpoints. Talented high school students who can use other people’s software to break into machines and manipulate them can easily manage to use these tools to attack real systems.

Traditionally, network protocols such as HTTP, SMTP, FTP, NNTP, and Telnet don’t provide adequate defenses to the above attacks. Before electronic commerce started taking off in mid-1990, security wasn’t really a large concern, especially considering the Internet’s origins as a platform for sharing academic research and resources. While many protocols provided some sort of authentication in the way of password-based logins, most of them did not address confidentiality or integrity at all. As a result, all of the above attacks were possible. Moreover, authentication information could usually be among the information “snooped” off a network.

SSL is a great boon to the traditional network protocols, because it makes it easy to add transparent confidentiality and integrity services to an otherwise insecure TCP-based protocol. It can also provide authentication services, the most important being that clients can determine if they are talking to the intended server, not some attacker that is spoofing the server.

The SSL protocol covers many cryptographic needs. Sometimes, though, it isn’t good enough. For example, you may wish to encrypt HTTP cookies that will be placed on an end user’s browser. SSL won’t help protect the cookies while they’re being stored on that disk. For situations like this, OpenSSL exports the underlying cryptographic algorithms used in its implementation of the SSL protocol.

Generally, you should avoid using cryptographic algorithms directly if possible. You’re not likely to get a totally secure system simply by picking an algorithm and applying it. Usually, cryptographic algorithms are incorporated into cryptographic protocols. Plenty of nonobvious things can be wrong with a protocol based on cryptographic algorithms. That is why it’s better to try to find a well-known cryptographic protocol to do what you want to do, instead of inventing something yourself. In fact, even the protocols invented by cryptographers often have subtle holes.

If not for public review, most protocols in use would be insecure. Consider the original WEP protocol for IEEE 802.11 wireless networking. WEP (Wired Equivalent Privacy) is the protocol that is supposed to provide the same level of security for data that physical lines provide. It is a challenge, because data is transmitted through the air, instead of across a wire. WEP was designed by veteran programmers, yet without soliciting the opinions of any professional cryptographers or security protocol developers. Although to a seasoned developer with moderate security knowledge the protocol looked fine, in reality, it was totally lacking in security.

Nonetheless, sometimes you might find a protocol that does what you need, but can’t find an implementation that suits your needs. Alternatively, you might find that you do need to come up with your own protocol. For those cases, we do document the SSL cryptographic API.

Five types of cryptographic algorithms are discussed in this book: symmetric key encryption, public key encryption, cryptographic hash functions, message authentication codes, and digital signatures.

Symmetric key encryption

Symmetric key algorithms encrypt and decrypt data using a single key. As shown in Figure 1-1, the key and the plaintext message are passed to the encryption algorithm, producing ciphertext. The result can be sent across an insecure medium, allowing only a recipient who has the original key to decrypt the message, which is done by passing the ciphertext and the key to a decryption algorithm. Obviously, the key must remain secret for this scheme to be effective.

Figure 1-1. Symmetric key cryptography

The primary disadvantage of symmetric key algorithms is that the key must remain secret at all times. In particular, exchanging secret keys can be difficult, since you’ll usually want to exchange keys on the same medium that you’re trying to use encryption to protect. Sending the key in the clear before you use it leaves open the possibility of an attacker recording the key before you even begin to send data.

One solution to the key distribution problem is to use a cryptographic key exchange protocol. OpenSSL provides the Diffie-Hellman protocol for this purpose, which allows for key agreement without actually divulging the key on the network. However, Diffie-Hellman does not guarantee the identity of the party with whom you are exchanging keys. Some sort of authentication mechanism is necessary to ensure that you don’t accidentally exchange keys with an attacker.

Right now, Triple DES (usually written 3DES, or sometimes DES3) is the most conservative symmetric cipher available. It is in wide use, but AES, the new Advanced Encryption Standard, will eventually replace it as the most widely used cipher. AES is certainly faster than 3DES, but 3DES has been around a lot longer, and thus is a more conservative choice for the ultra-paranoid. It is worth mentioning that RC4 is widely supported by existing clients and servers. It is faster than 3DES, but is difficult to set up properly (don’t worry, SSL uses RC4 properly). For purposes of compatibility with existing software in which neither AES nor 3DES are supported, RC4 is of particular interest. We don’t recommend supporting other algorithms without a good reason. For the interested, we discuss cipher selection in Chapter 6.

Security is related to the length of the key. Longer key lengths are, of course, better. To ensure security, you should only use key lengths of 80 bits or higher. While 64-bit keys may be secure, they likely will not be for long, whereas 80-bit keys should be secure for at least a few years to come. AES supports only 128-bit keys and higher, while 3DES has a fixed 112 bits of effective security.^[1] Both of these should be secure for all cryptographic needs for the foreseeable future. Larger keys are probably unnecessary. Key lengths of 56 bits (regular DES) or less (40-bit keys are common) are too weak; they have proven to be breakable with a modest amount of time and effort.

Public key encryption

Public key cryptography suggests a solution to the key distribution problem that plagues symmetric cryptography. In the most popular form of public key cryptography, each party has two keys, one that must remain secret (the private key ) and one that can be freely distributed (the public key ). The two keys have a special mathematical relationship. For Alice to send a message to Bob using public key encryption (see Figure 1-2), Alice must first have Bob’s public key. She then encrypts her message using Bob’s public key, and delivers it. Once encrypted, only someone who has Bob’s private key can successfully decrypt the message (hopefully, that’s only Bob).

Figure 1-2. Public key cryptography

Public key encryption solves the problem of key distribution, assuming there is some way to find Bob’s public key and ensure that the key really does belong to Bob. In practice, public keys are passed around with a bunch of supporting information called a certificate , and those certificates are validated by trusted third parties. Often, a trusted third party is an organization that does research (such as credit checks) on people who wish to have their certificates validated. SSL uses trusted third parties to help address the key distribution problem.

Public key cryptography has a significant drawback, though: it is intolerably slow for large messages. Symmetric key cryptography can usually be done quickly enough to encrypt and decrypt all the network traffic a machine can manage. Public key cryptography is generally limited by the speed of the cryptography, not the bandwidth going into the computer, particularly on server machines that need to handle multiple connections simultaneously.

As a result, most systems that use public key cryptography, SSL included, use it as little as possible. Generally, public key encryption is used to agree on an encryption key for a symmetric algorithm, and then all further encryption is done using the symmetric algorithm. Therefore, public key encryption algorithms are primarily used in key exchange protocols and when non-repudiation is required.

RSA is the most popular public key encryption algorithm. The Diffie-Hellman key exchange protocol is based on public key technology and can be used to achieve the same ends by exchanging a symmetric key, which is used to perform actual data encryption and decryption. For public key schemes to be effective, there usually needs to be an authentication mechanism involving a trusted third party that is separate from the encryption itself. Most often, digital signature schemes, which we discuss below, provide the necessary authentication.

Keys in public key algorithms are essentially large numbers with particular properties. Therefore, bit length of keys in public key ciphers aren’t directly comparable to symmetric algorithms. With public key encryption algorithms, you should use keys of 1,024 bits or more to ensure reasonable security. 512-bit keys are probably too weak. Anything larger than 2,048 bits may be too slow, and chances are it will not buy security that is much more practical. Recently, there’s been some concern that 1,024-bit keys are too weak, but as of this writing, there hasn’t been conclusive proof. Certainly, 1,024 bits is a bare minimum for practical security from short-term attacks. If your keys potentially need to stay protected for years, then you might want to go ahead and use 2,048-bit keys.

When selecting key lengths for public key algorithms, you’ll usually need to select symmetric key lengths as well. Recommendations vary, but we recommend using 1,024-bit keys when you are willing to work with symmetric keys that are less than 100 bits in length. If you’re using 3DES or 128-bit keys, we recommend 2,048-bit public keys. If you are paranoid enough to be using 192-bit keys or higher, we recommend using 4,096-bit public keys.

Requirements for key lengths change if you’re using elliptic curve cryptography (ECC), which is a modification of public key cryptography that can provide the same amount of security using faster operations and smaller keys. OpenSSL currently doesn’t support ECC, and there may be some lingering patent issues for those who wish to use it. For developers interested in this topic, we recommend the book Implementing Elliptic Curve Cryptography, by Michael Rosing (Manning).

SSL is a lot slower than a traditional unsecured TCP/IP connection. This problem is a direct result of providing adequate security. When a new SSL session is being established, the server and the client exchange a sizable amount of information that is required for them to authenticate each other and agree on a key to be used for the session. This initial handshake involves heavy use of public key cryptography, which, as we’ve already mentioned, is very slow. It’s also the biggest slowdown when using SSL. On current high-end PC hardware, OpenSSL struggles to make 100 connections per second under real workloads.

Once the initial handshake is complete and the session is established, the overhead is significantly reduced, but some of it still remains in comparison with an unsecured TCP/IP connection. Specifically, more data is transferred than normal. Data is transmitted in packets, which contain information required by the SSL protocol as well as any padding required by the symmetric cipher that is in use. Of course, there is the overhead of encrypting and decrypting the data as well, but the good news is that a symmetric cipher is in use, so it usually isn’t a bottleneck. The efficiency of symmetric cryptography can vary greatly based on the algorithms used and the strength of the keys. However, even the slowest algorithms are efficient enough that they are rarely a bottleneck at all.

Because of the inefficiency of public key cryptography, many people decide not to use SSL when they realize it can’t handle a large enough load. Some people go without security at all, which is obviously not a good idea. Other people try to design their own protocols to compensate. This is a bad idea, because there are many nonobvious pitfalls that can besiege you. Protocols that aren’t designed by a skilled cryptographer inevitably have problems. SSL’s design does consider efficiency; it simply isn’t willing to sacrifice security for a speed improvement. You should be skeptical of using protocols that are more efficient.

There are ways to ameliorate this problem without abandoning the protocol. SSL does support a connection resumption mechanism so that clients that reconnect shortly after disconnecting can do so without incurring the full overhead of establishing a connection. While that is useful for HTTP,^[3] it often isn’t effective for other protocols.

In a typical SSL installation, the server maintains credentials so that clients can authenticate the server. In addition to a certificate that is presented at connection time, the server also maintains a private key, which is necessary for establishing that the server presenting a certificate is actually presenting its own certificate.

That private key needs to live somewhere on the server. The most secure solution is to use cryptographic acceleration hardware. Most of these devices can generate and store key material, and additionally prevent the private key from being accessed by an attacker who has broken into the machine. To do this, the private key is used only on the card, and is not allowed off except under special circumstances.

In cases in which hardware solutions aren’t feasible, there is no absolute way to protect the private key from an attacker who has obtained root access, because, at the very least, the key must be unencrypted in memory when handling a new connection.^[4] If an attacker has root, she can generally attach a debugger to the server process, and pull out the unencrypted key.

There are two options in these situations. First, you can simply keep the key unencrypted on disk. This is the easiest solution, but it also makes the job of an attacker simple if he has physical access, since he can power off the machine and pull out the disk, or simply reboot to single-user mode. Alternatively, you can keep the key encrypted on disk using a passphrase, which an administrator must type when the SSL server starts. In such a situation, the key will only be unencrypted in the address space of the server process, and thus won’t be available to someone who can shut the machine off and directly access the disk.

Furthermore, many attackers are looking for low-hanging fruit, and will not likely go after the key even if they have the skills to do so. The downside to this solution is that unattended reboots are not possible, because whenever the machine restarts (or the SSL server process crashes), someone must type in the passphrase, which is often not very practical, especially in a lights-out environment. Storing the key in the clear obviously does not exhibit this problem.

In either case, your best defense is to secure the host and your network with the best available lockdown techniques (including physical lockdown techniques). Such solutions are outside the scope of this book.

What exactly does it mean if the server’s private key is compromised? The most obvious result is that the attacker can masquerade as the server, which we discuss in the next section. Another result (which may not be as obvious) is that all past communications that used the key can likely be decrypted. If an attacker is able to compromise a private key, it is also likely that the attacker could have recorded previous communications. The solution to this problem is to use ephemeral keying . This means a temporary key pair is generated when a new SSL session is created. This is then used for key exchange and is subsequently destroyed. By using ephemeral keying, it is possible to achieve forward secrecy , meaning that if a key is compromised, messages encrypted with previous keys will not be subject to attack.^[5] We discuss ephemeral keying and forward secrecy in more detail in Chapter 5.

A server’s private key can be stolen. In such a case, an attacker can usually masquerade as the server with impunity. Additionally, Certification Authorities sometimes sign certificates for people who are fraudulently representing themselves, despite the efforts made by the CA to validate all of the important information about the party that requests the certificate signing.^[6] For example, in early 2001, VeriSign signed certificates that purported to belong to Microsoft, when in reality they did not. However, since they had been signed by a well-known Certification Authority, they would look authentic to anyone validating the signature on those certificates.

SSL has a mechanism for thwarting these problems: Certificate Revocation Lists. Once the Certification Authority learns that a certificate has been stolen or signed inappropriately, the Authority adds the certificate’s serial number to a CRL. The client can access CRLs and validate them using the CA’s certificate, since the server signs CRLs with its private key.

One problem with CRLs is that windows of vulnerability can be large. It can take time for an organization to realize that a private key may have been stolen and to notify the CA. Even when that happens, the CA must update its CRLs, which generally does not happen in real time (the time it takes depends on the CA). Then, once the CRLs are updated, the client must download them in order to detect that a presented certificate has been revoked. In most situations, clients never download or update CRLs. In such cases, compromised certificates tend to remain compromised until they expire.

There are several reasons for this phenomenon. First, CRLs tend to be large enough that they can take significant time to download, and can require considerable storage space locally, especially when the SSL client is an embedded device with limited storage capacity. The Online Certificate Status Protocol (OCSP), specified in RFC 2560, addresses these problems. Unfortunately, this is not yet a widely accepted standard protocol, nor is it likely to become so anytime soon. Additionally, the only version that is widely deployed has serious security issues (see Chapter 3 for more information). OpenSSL has only added OCSP support in Version 0.9.7, and few CAs even offer it as a service. Other authorities have facilities for incremental updates to CRLs, allowing for minimal download times, but that solution still requires space on the client, or some sort of caching server.

These solutions all require the CA’s server to be highly available if clients are to have up-to-the-minute information. Some clients will be deployed in environments where a constant link to the CA is not possible. In addition, the need to query the CA can add significant latency to connection times that can be intolerable to the end user.

Another problem is that there is no standard delivery mechanism specified for CRLs. As a result, OpenSSL in particular does not provide a simple way to access CRL information, not even from VeriSign, currently the most popular CA. One common method of CRL (and certificate) distribution is using the Lightweight Directory Access Protocol (LDAP). LDAP provides a hierarchical structure for storing such information and fits nicely for PKI distribution.

Due to the many problems surrounding CRLs, it becomes even more important to take whatever measures are feasible to ensure that SSL private keys are not stolen. At the very least, you should put intrusion detection systems in place to detect compromises of your private key so that you can report the compromise to the CA quickly.

CRLs aren’t useful if a client isn’t performing adequate validation of server certificates to begin with. Often, they don’t. Certainly, for SSL to work at all, the client must be able to extract the public key from a presented certificate, and the server must have a private key that corresponds with that public key. However, there is no mechanism to force further validation. As a result, man-in-the-middle attacks are often feasible.

First, developers must decide which Certification Authorities should be trusted, and must locate the certificates associated with each of those authorities. That’s more effort than most developers are willing to exert. As a result, many applications using SSL are at the mercy of man-in-the-middle attacks.

Second, even those applications that install CA certificates and use them to validate server certificates often fail to perform adequate checking on the contents of the certificate. As a result, such systems are susceptible to man-in-the-middle attacks in which the attacker gets his hands on credentials that will look legitimate to the client, such as a stolen set of credentials in which the certificate is signed by the CA that has not yet appeared on any CRLs.

The best solution for thwarting this problem depends on the authentication needs of the client. Many applications can expect that they will only legitimately talk to a small set of servers. In such a case, you can check appropriate fields in the certificate against a white list of valid server names. For example, you might allow any certificate signed by VeriSign in which the fully qualified domain name mentioned in the certificate ends with “yourcompany.com”. Another option is to hardcode a list of known server certificates. However, this is a far more difficult solution to manage if you ever wish to add servers.

Additionally, if you do not wish to trust the authentication mechanisms of the established CAs, you could consider running your own CA, which we discuss in Chapter 3 (of course, we are assuming you control both the client and server code in such a situation). In environments where you expect that anyone can set up their own server, and thus managing DNS space or your own Certification Authority is not feasible, then the best you can do is ensure that the DNS address for the server that the client tried to contact is the same as the one presented in the certificate. If that is true, and the certificate was signed by a valid CA, everything should be fine if the certificate was not stolen or fraudulently obtained.

In the SSL protocol, both the client and the server need to generate random data for keys and other secrets. The data must be generated in such a way that a knowledgeable attacker cannot guess anything about it. SSL implementations usually generate such data using a pseudorandom number generator (PRNG). PRNGs are deterministic algorithms that produce a series of random-looking numbers. Classical PRNGs are not suitable for use in security-critical situations. Instead, SSL implementations use “cryptographic” PRNGs, which work in security-critical situations, as long as they are “seeded” properly.

A seed is a piece of data fed to the PRNG to get it going. Given a single, known seed at startup, the PRNG should produce a predictable set of outputs. That is, if you seed the PRNG and ask for three random numbers, reseed with the same value, and then ask for three more random numbers, the first three numbers and the second three numbers should be identical.

The seed itself must be a random number, but it can’t just be a cryptographically random number. It must be truly unguessable to keep the PRNG outputs unguessable. Entropy is a measurement of how much unguessable information actually exists in data from the point of view of an attacker who might be able to make reasonable guesses about the state of the machine on which the number is stored. If a single bit is just as likely to be a 0 as a 1, then it is one bit of entropy. If you have 128 bits of data, it can have up to 128 bits of entropy. However, it may have as little as 0 bits of entropy—as would be the case if the data’s value is public knowledge. The work an attacker must do to guess a piece of data is directly related to how much entropy there is in the data. If the data has 4 bits of entropy, then the attacker has a 1 in 2⁴ chance (1 in 16) chance of guessing right the first time. Additionally, within 16 guesses, the attacker will have tried the right value (On average, he will find the right value in 8 guesses). If the data has 128 bits of entropy in it, then the attacker will need, on average 2¹²⁷ guesses to find the seed, which is such a large number as to be infeasible for all practical purposes. In practice, if you’re using 128-bit keys, it’s desirable to use a seed with 128 bits of entropy or more. Anything less than 64 bits of entropy can probably be broken quickly by an organization with a modest hardware budget.

To illustrate, in 1996, Ian Goldberg and David Wagner found a problem with the way Netscape was seeding its PRNG in its implementation of SSLv2. Netscape was using three inputs hashed with the MD5 message digest algorithm, the time of day, the process ID, and the parent process ID. None of these values is particularly random. At most, their PRNG seed could have had 47 bits of entropy. A clever attacker could decrease that substantially. Indeed, in practice, Goldberg and Wagner were able to compromise real SSL sessions within 25 seconds.

If you try to use OpenSSL without bothering to seed the random number generator, the library will complain. However, the library has no real way to know whether the seed you give it contains enough entropy. Therefore, you must have some idea how to get entropy. There are certainly hardware devices that do a good job of collecting it, including most of the cryptographic accelerator cards. However, in many cases hardware is impractical, because your software will be deployed across a large number of clients, most of whom will have no access to such devices.

Many software tricks are commonly employed for collecting entropy on a machine. They tend to work by indirectly measuring random information in external events that affect the machine. You should never need to worry about those actual techniques. Instead, use a package that harvests entropy for you. Many Unix-based operating systems now come with a random device, which provides entropy harvested by the operating system. On other Unix systems, you can use tools such as EGADS (http://www.securesw.com/egads/), which is a portable entropy collection system.^[7] EGADS also works on Windows systems.

If you’re interested in the entropy harvesting techniques behind random devices and tools like EGADS, see Chapter 10 of the book Building Secure Software by John Viega and Gary McGraw (Addison-Wesley).

While Version 3 of the SSL protocol and TLS are believed to be reasonably secure if used properly,^[8] SSLv2 (Version 2) had fundamental design problems that led to wide-ranging changes in subsequent versions (Version 1 was never publicly deployed). For this reason, you should not support Version 2 of the protocol, just to ensure that an attacker does not launch a network attack that causes the client and server to settle upon the insecure version of the protocol. All you need to do is intercept the connection request and send a response that makes it look like a v3 server does not exist. The client will then try to connect using Version 2 of the protocol.

Unfortunately, people commonly configure their clients and servers to handle both versions of the protocol. Don’t do that. Support only SSLv3 and TLS, to whatever degree possible. Note that clients can’t really support TLS only, because TLS implementations are supposed to be able to speak SSLv3. If you wish to use only TLS in a client, you must connect then terminate the connection if the server chooses SSLv3.

As we mentioned when discussing different types of cryptographic algorithms, you should also avoid small key lengths and, to a lesser degree, algorithms that aren’t well regarded. 40-bit keys are never secure and neither is 56-bit DES. Nonetheless, it’s common to see servers that support only these weak keys, due to old U.S. export regulations that no longer apply.

As for individual algorithm choices in SSL, RC4 and 3DES are both excellent solutions. RC4 is much faster, and 3DES is more conservative. Soon, TLS will be standardizing on AES, at which time this will be widely regarded as a good choice.

Note that the server generally picks a cipher based on a list of supported ciphers that the client presents. We recommend supporting only strong ciphers in the server, where feasible. In other cases, make sure to prefer the strongest cipher the client offers. We discuss cipher selection in detail in Chapter 5.

SSL works well with TCP/IP. However, it doesn’t work at all with transport layer protocols that are not connection-oriented, such as UDP and IPX. There’s not really a way to make it work for such protocols, either. Secure encryption of protocols in which order and reliability are not ensured is a challenge, and is outside the scope of SSL. We do outline solutions for encrypting UDP traffic in Chapter 6.

Let’s say that Alice and Bob are communicating over SSL. Alice may receive a message from Bob that she would like to show to Charlie, and she would like to prove that she received the message from Bob. If that was possible, the message would be non-repudiated, meaning that Bob cannot deny that he sent the message. For example, Alice may receive a receipt for a product, and wish to demonstrate that she purchased the product for tax purposes.

SSL has no support for non-repudiation. However, it is simple to add on top of SSL, if both Alice and Bob have well-established certificates. In such a case, they can sign each message before SSL-encrypting it. Of course, in such a situation, if Bob wishes to have a message he can repudiate, he just attaches an invalid signature. In such a case, Alice should refuse further communications.

In Chapter 10, we discuss how to sign encrypted messages using S/MIME. This same technique can be used for sending messages over SSL by signing the data before sending it. Alternatively, S/MIME messages could simply be sent over an SSL connection to achieve the same result.

Sometimes SSL fails to secure an application because of a fundamental security flaw in the application itself, not because of any actual problem in SSL’s design. That is, SSL doesn’t protect against buffer overflows, race conditions, protocol errors, or any other design or implementation flaws in the application that uses SSL.

Even though there are many common risks when deploying SSL, those risks are often minor compared to the gaping holes in software design and implementation. Attackers will tend to target the weakest link, and SSL is often not the weakest link.

Developers should thoroughly educate themselves on building secure software. For administrators deploying other people’s software, try to use well-regarded software if you have any option whatsoever.

SSL can protect data in transit on a live connection, but it provides no facilities for protecting data before it is sent, or after it arrives at its destination. Additionally, if there is no active connection, SSL can do nothing. For any other data security needs, other solutions are necessary.

Let’s say that we want to run SSL-enabled POP3 on the standard port for this (995). If we already have the unencrypted POP3 server running on port 110, we simply put Stunnel on port 995, and tell it to forward connections to port 110 on the loopback interface (so that unencrypted data isn’t sent over your local network, just to come back onto the current machine). When SSL-enabled POP3 clients connect to port 995, Stunnel will negotiate the connection, connect itself to the POP3 port, then start decrypting data. When it has data to pass on to the POP3 server, it does so. Similarly, when the POP3 server responds to a client request, it talks with the Stunnel proxy, which encrypts the response, and passes it on to the client. See Figure 1-5 for a graphical overview of the process.

Figure 1-5. Stunnel proxies

To use Stunnel on the server side, you must install a valid server certificate and private key. An appropriate Certification Authority should sign the certificate. You can generate your own credentials using OpenSSL. That process is covered in Chapter 3.

These server credentials will need to be made available to Stunnel. Often, the correct location of these credentials will be hardcoded into the Stunnel binary. If not, you can specify their location on the command line.

Assuming the POP3 server is already running, here is how you would run Stunnel from the command line to implement the above scenario (assuming that you’re running as root, which is necessary for binding to low ports on Unix machines):

# stunnel -d 995 -r 127.0.0.1:110

The -d flag specifies that Stunnel should run as a proxy in daemon mode on the specified port (you can also specify the IP address on which to bind; the default is all IPs on the machine). The -r flag specifies the location of the service to which Stunnel will proxy. In this case, we specifically mention the loopback address to avoid exposing unencrypted traffic to other hosts on the same local network. Optionally, we could hide the port from external eyes using a firewall.

The location of the certificate file can be specified with the -p flag, if necessary. If your machine’s services file contains entries for the POP3 and the Secure POP3 protocol, you can also run Stunnel like this:

# stunnel -d pop3s -r 127.0.0.1:pop3

You can also run Stunnel from inetd. However, this is generally not desirable, because you forego the efficiency benefits of session caching. If you’re running on Windows, Stunnel is available as a precompiled binary, and can be easily launched from a DOS-style batch file. See the Stunnel FAQ (http://www.stunnel.org/faq) for more details.

Unfortunately, Stunnel can’t protect all the services you might want to run. First, it can protect only TCP connections, not UDP connections. Second, it can’t really protect protocols like FTP that use out-of-band connections. The FTP daemon can bind to arbitrary ports, and there’s no good way to have Stunnel detect it. Also, note that some clients that support SSL-enabled versions of a protocol will expect to negotiate SSL as an option. In such cases, the client won’t be able to communicate with the Stunnel proxy, unless it goes through an SSL proxy on the client end as well.

Since Stunnel will proxy to whatever address you tell it to use, you can certainly proxy to services running on other machines. You can use this ability to offload the cost of establishing SSL connections to a machine by itself, providing a cost-effective way of accelerating SSL. In such a scenario, the unencrypted server should be connected only to the SSL proxy by a crossover cable, and should be connected to no other machines. That way, the unencrypted data won’t be visible to other machines on your network, even if they are compromised. If you have a load balancer, you can handle even more SSL connections by installing additional proxies (see Figure 1-6). For most applications, a single server is sufficient to handle the unencrypted load.

Figure 1-6. Load balancing with Stunnel for cryptographic acceleration

The biggest problem with using Stunnel as a proxy is that IP header information that would normally be available to the server isn’t. In particular, the server may log IP addresses with each transaction. Since the server is actually talking to the proxy, from the server’s point of view, every single connection will appear to come from the proxy’s IP address. Stunnel provides a limited solution to this problem. If the secure port is on a Linux machine, then the Stunnel process can be configured to rewrite the IP headers, thus providing transparent proxying. Simply adding the -T flag to the command line does this. For transparent proxying to work this way, the client’s default route to the unencrypted server must go through the proxy machine, and the route cannot go through the loopback interface.

Stunnel can be configured to log connections to a file by specifying the -o flag and a filename. That at least allows you to get information about connecting IP addresses (which should never be used for security purposes anyway, since they are easy to forge), even when transparent proxying is not an option.

Stunnel can also be used to connect clients that are SSL-unaware with servers that do speak the protocol. Setting up a client-side proxy is a bit more work than setting up a server-side proxy because, while clients are usually authenticated using some sort of password mechanism, servers are authenticated primarily using cryptographic certificates. You can set up the client not to authenticate, but if you do so, be warned that man-in-the-middle attacks will be easy to perform. Unauthenticating client proxies only buys you security against the most naive eavesdropping attacks, but is still better than no protection at all.

Let’s start with a case in which we are not yet validating certificates. Let’s say that we’d like to connect to Amazon.com’s SSL-enabled web server, running on port 443 on www.amazon.com. First, we can interactively test the connection by running Stunnel in client mode (specified by the -c flag):

$ stunnel -c -r www.amazon.com:443

Stunnel silently connects. We type in an HTTP request, and get back the appropriate response. For example:

GET /
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>302 Found</TITLE>
</HEAD><BODY>
<H1>Found</H1>
The document has moved <A HREF="http://www.amazon.com/">here</A>.<P>
</BODY></HTML>

After sending its response, the server closes the connection.

As you can see, we can talk with the SSL-enabled web server running on Amazon.com, yet the SSL handling is completely transparent from our point of view.

Running Stunnel in interactive mode is useful for the purposes of debugging. However, interactive mode is not practical for use with arbitrary clients. Let’s say we wish to point an SSL-unaware POP3 client at an SSL-enabled POP3 server running on mail.example.com. On the machine running the client, we would like to set up a proxy that only accepts connections from the local machine, and then makes connections on behalf of the local machine to the SSL-enabled server. We can easily do that with the following command:

# stunnel -c -r mail.example.com:pop3s -d 127.0.0.1:pop3

This command sets up a proxy on the local machine that does what we want it to. Now we can simply point our mail client to our loopback interface, and we will magically connect to the intended SSL-enabled POP3 server (assuming no man-in-the-middle attacks).

Note that the above command will work only if you have permission to bind to the POP3 port locally. If that is an issue, and your POP client can connect to servers on arbitrary ports, the problem is easy to fix. Otherwise, you’ll need to grant the proxy process root privileges, or find a new client. Root privileges pose a big risk, because there may be an obscure security bug in Stunnel that would allow data passing through the proxy to gain root privileges. If you do choose to grant the proxy root privileges, on most operating systems you should probably run the proxy as root, and then use the -s flag to specify a username to switch to after the port is bound. You might consider making the binary setuid—but you shouldn’t, because you would then let any user bind to privileged ports as long as he can run the Stunnel binary.

As we mentioned previously, you should always have client proxies perform certificate validation. To use certificate validation, you must specify where on the client machine valid CA certificates live, and you must specify the level of validation you want. We recommend maximum validation (level 3), and we think you should completely stay away from level 1, since it offers no real validation. Here’s an extension of the above example that takes into account certificate validation:

# stunnel -c -r mail.example.com:pop3s -d 127.0.0.1:pop3 -A /etc/ca_certs -v 2

The file /etc/ca_certs stores a list of trusted CA certificates (see Chapter 3 for more information on obtaining such certificates). Unfortunately, Stunnel doesn’t support validation based on domain-name matching. If you wish to restrict valid servers to a small set (usually a very good idea), you can use validation level 3 (the maximum), and place the known certificates in a directory of their own. The certificate’s filename must be the hash value of the certificate’s subjected (see the -hash option to the x509 command in Chapter 2 to find out how to generate this value), with a “.0” file extension. Additionally, you use the -a flag to specify where valid server certificates live. For example:

# stunnel -c -r mail.example.com:pop3s -d 127.0.0.1:pop3 -A /etc/ca_certs -a 
/etc/server_certs -v 3

Again, we talk more about certificate formats in Chapter 3.

As with server-side SSL proxies, there are some situations in which client-side use of Stunnel isn’t appropriate. Once again, it doesn’t make sense to use Stunnel in a UDP-based environment or with a protocol that makes out-of-band connections. Additionally, some servers that support SSL expect to negotiate whether or not to use it. These servers won’t understand a connection that is encrypted with SSL from start to finish. Such negotiation is especially popular with SSL-enabled SMTP servers.

Stunnel has support for negotiating some of the more common protocols. To use that support, invoke Stunnel in the same way as in the previous client-side example, but add the -n argument, which takes a single argument (the name of the protocol). Currently, SSL supports SMTP, POP3, and NNTP. For example, to connect to a secure SMTP server over SSL, use the command:

# stunnel -c -r mail.example.com:smtp -d 127.0.0.1:smtp -A /etc/ca_certs -a /etc/
server_certs -v 3 -n smtp

Unfortunately, as of this writing, Stunnel doesn’t support any other protocols for which SSL is a negotiated option, most notably SSL-TELNET.

The command-line tool provides a large number of options for each of its many commands. Remembering the option names, their defaults if they’re not specified, and even to include them with a command to obtain the desired result can be difficult, if not downright frustrating at times. The task of managing options is made considerably simpler using configuration files.

OpenSSL includes a default configuration file that is normally used unless an alternate one is specified. The settings in the default configuration are all quite reasonable, but it can often be useful to replace them with settings that are better tailored to your own needs. The location of the default configuration file varies greatly, depending on the operating system that you’re using and how OpenSSL was built and installed. So, unfortunately, we can’t point you to any one specific location to find it. Although it is not at all intuitive, the command-line tool will tell you where the default configuration file is located if you issue the ca command without any options. Any errors that are issued due to the lack of options may be safely ignored.

Unfortunately, only three of the many commands supported by the command-line tool make any use of the configuration file. On the bright side, the three commands that do use it are perhaps the most complex of all of the supported commands, and accept the greatest number of options to control their behavior. The commands that do support the configuration file are ca, req, and x509 (we discuss these commands below).

An OpenSSL configuration file is organized in sections. Each section contains a set of keys, and each key has an associated value. Sections and keys are both named and case-sensitive. A configuration file is parsed from top to bottom with sections delimited by a line containing the name of the section surrounded by square brackets. The other lines contain key and value pairs that belong to the most recently parsed section delimiter. In addition, an optional global section that is unnamed occurs before the first named section in the file. Keys are separated from their associated value by an equals sign (=).

For the most part, whitespace is insignificant. Comments may begin anywhere on a line with a hash mark (#), and they end at the end of the line on which they begin. Key and section names may not contain whitespace, but they may be surrounded by it. Leading and trailing whitespace is stripped from a value, but any whitespace in the middle is significant. Example 2-1 shows an excerpt from the default OpenSSL configuration file.

[ ca ]
default_ca      = CA_default            # The default ca section
 
####################################################################
[ CA_default ]
 
dir             = ./demoCA              # Where everything is kept
certs           = $dir/certs            # Where the issued certs are kept
crl_dir         = $dir/crl              # Where the issued crl are kept
database        = $dir/index.txt        # database index file
new_certs_dir   = $dir/newcerts         # default place for new certs
 
certificate     = $dir/cacert.pem       # The CA certificate
serial          = $dir/serial           # The current serial number
crl             = $dir/crl.pem          # The current CRL
private_key     = $dir/private/cakey.pem# The private key
RANDFILE        = $dir/private/.rand    # private random number file
 
x509_extensions = usr_cert              # The extentions to add to the cert
 
# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crl_extensions        = crl_ext
 
default_days    = 365                   # how long to certify for
default_crl_days= 30                    # how long before next CRL
default_md      = md5                   # which md to use
preserve        = no                    # keep passed DN ordering
 
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy          = policy_match

In the example, you’ll notice the use of $dir. Used in a value, a key name preceded by a dollar sign is known as a macro, and is replaced with the value for that key. Only macros using keys that are defined within the same section or in the global section will be expanded. Additionally, the key must be defined before you use it as a macro in a value, because the macro is expanded as the configuration file parses rather than when the value is used. Macros are particularly useful when you have a number of values referencing the same path in a filename.

Although only a few commands currently make any use of a configuration file, other commands may be modified in the future to take advantage of them. Each command that currently uses the configuration file reads its base configuration information from a section that shares the name of the command. Other sections that are not named after a command may exist, and quite frequently, they do. Many keys’ values are interpreted as the name of a section to use for finding more keys. We’ll see frequent examples of this as we examine the commands that do use the configuration file in detail.

The following examples illustrate the use of the message digest commands:

The following examples illustrate the use of the symmetric cipher commands:

The Appendix gives a complete list of algorithms used to perform symmetric encryption.

Diffie-Hellman is used for key agreement . In simple terms, key agreement is the exchange of information over an insecure medium that allows each of the two parties in a conversation to compute a value that is typically used as the key for a symmetric cipher. By itself, Diffie-Hellman cannot be used for encryption or authentication; it only provides secrecy. Because the exchange of information takes place over an insecure medium, it should never be used by itself. Some means of authenticating the parties in the conversation should also be used.

Diffie-Hellman works by first creating a set of parameters that are agreed upon by both parties in the conversation. The parameters, consisting of a randomly chosen prime number and a generator value that is typically specified as either 2 or 5, are public and can be either agreed upon before the conversation begins or exchanged as part of the conversation. Using the agreed-upon parameters, each party computes a public and private key. As its name implies, the private key is never shared with anyone. The parties exchange their public keys, and then each party can compute the shared secret using their private key and the peer’s public key.

The command-line tool provides a command for generating Diffie-Hellman parameters, but the only method for generating keys is deprecated, and should not be used. OpenSSL 0.9.5 added the dhparam command, and in doing so, deprecated the two commands dh and gendh, which were capable of generating Diffie-Hellman parameters and keys, respectively. As of this writing, the two deprecated commands are still accessible in OpenSSL 0.9.7, but because they’re deprecated, we’ll pretend that they do not exist, because they’re likely to be completely removed from the next release of OpenSSL. Unfortunately, the new dhparam command does not support the generation of Diffie-Hellman keys, but it is likely that future versions will add support for it.

As its name implies, the Digital Signature Algorithm (DSA) is used for creating and verifying digital signatures. It provides authentication, but cannot be used for encryption or secrecy. DSA is frequently used in combination with Diffie-Hellman. Two parties in a conversation can exchange DSA public keys before the conversation begins (or during the conversation using certificates, as we’ll explain in Chapter 3) and use the DSA keys to authenticate the communication of Diffie-Hellman parameters and keys. Combining Diffie-Hellman with DSA provides authentication and secrecy, and by using the shared secret resulting from the Diffie-Hellman exchange as a key, a symmetric cipher can then be used for encryption.

Just like Diffie-Hellman, DSA also requires parameters from which keys are generated. There is no harm in making the parameters used to generate a key pair public, but there’s equally no compelling reason to do so. Only the private key that is generated must be kept private, as is implied by its name. The public key is the only thing that really needs to be shared with any party that wishes to verify the authenticity of anything signed with a private key.

Three commands are provided by the command-line tool for generating DSA parameters and keys, as well as for examining and manipulating them. The dsaparam command is used to generate and examine DSA parameters. Its function and options are not unlike those of the dhparam command. One major difference between the two is that the dsaparam command also provides an option to generate a private DSA key. The private key resulting from the dsaparam command will be unencrypted, which means that neither a password nor a passphrase will be required to decrypt and make use of it.

The gendsa command is used for generating private keys from a set of DSA parameters. By default, the generated private key will not be encrypted, but options are available that allow the key to be encrypted using any one of the DES, 3DES, or IDEA ciphers. No options are provided for specifying the password or passphrase to use for encryption on the command line, so encrypted DSA private key generation cannot be easily automated.

Both the dsaparam and gendsa commands are capable of generating private keys, either encrypted or not, but neither of them has the capability for generating a public key, which is required in order for DSA to provide any utility. The dsa command provides the means by which a public key can be generated from a private key. It also allows changes to be made to the encryption on a private key. For private keys that are not encrypted, encryption can be added, and for private keys that are already encrypted, the password or passphrase can be changed, as well as the encryption cipher that’s used to encrypt it. It’s also possible to remove the encryption on a private key with this command.

RSA is the most popular public key algorithm currently in use, despite the fact that it was encumbered by patent restrictions until the patent expired in September of 2000. It is named after its creators, Ron Rivest, Adi Shamir, and Leonard Adleman. One of the reasons that it is so popular is because it provides secrecy, authentication, and encryption all in one neat little package.

Unlike Diffie-Hellman and DSA, the RSA algorithm does not require parameters to be generated before keys can be generated, which simplifies the amount of work that is necessary to generate keys, and authenticate and encrypt communications. The command-line tool provides three commands for generating, examining, manipulating, and using RSA keys.

OpenSSL’s genrsa command is used to generate a new RSA private key. Generation of an RSA private key involves finding two large prime numbers, each approximately half the length of the key. A typical key size for RSA is 1,024. We don’t recommend that you use smaller key lengths or key lengths greater than 2,048 bits. By default, the generated private key will be unencrypted, but the command does have the ability to encrypt the resultant key using DES, 3DES, or IDEA.

The rsa command is used to manipulate and examine RSA keys and is the RSA version of the dsa command for DSA keys. It is capable of adding, modifying, and removing the encryption protecting an RSA private key. It is also capable of producing an RSA public key from a private key. The command can also be used to display information about a public or private key.

The rsautl command provides the ability to use an RSA key pair for encryption and signatures. Options are provided for encrypting and decrypting data, as well as for signing and verifying signatures. Remember that signing is normally performed on hashes, so this command is not useful for signing large amounts of data, or even more than 160 bits of data. In general, we do not recommend that you use this command at all for signing data. You should use the enc command instead. Additionally, encryption and decryption using RSA is slow, and for that reason, it should not be used on its own. Instead, it is commonly used to encrypt a key for a symmetric cipher. This is discussed in more detail in Chapter 8.

The following examples illustrate the use of the S/MIME commands:

A Certification Authority (CA) is an organization or company that issues certificates. By its very nature, a CA has a huge responsibility to ensure that the certificates it issues are legitimate. That is, the CA must ensure beyond all reasonable doubt that every certificate it issues contains a public key that was issued by the party that claims to have issued it. It must be able to produce acceptable proof for any certificate that it issues on demand. Otherwise, how can the CA itself be trusted?

There are two basic types of CAs. A private CA has the responsibility of issuing certificates only for members of its own organization, and is likewise trusted only by members of its own organization. A public CA, such as VeriSign or Thawte, has the responsibility of issuing certificates for any member of the public, and must be trusted by the public. The burden of proof varies depending on the type of CA that has issued a certificate and the type of certificate that is issued.

A CA must be trusted, and so for that trust to be extended, its certificate containing its public key must be widely distributed. For public CAs, their certificates are generally published so that anyone can obtain them. More commonly, the software that makes use of them, such as a web browser, is shipped containing them. Most often, the software allows certificates from other CAs to be added to its list of trusted certificates, thus facilitating the use of private CAs with off-the-shelf software.

A certificate that is issued by a CA can be used to issue and sign another certificate, if the issued certificate is created with the appropriate permissions to do so. In this way, certificates can be chained. At the root of the chain is the root CA’s certificate. Because it is at the root of the chain and there is no other authority to sign its certificate, the root CA signs its own certificate. Such a certificate is known as a self-signed certificate .

There is no way to digitally verify the authenticity of a self-signed certificate because the issuer and the subject are the same, which is why it has become a common practice to provide them with the software that uses them. When they’re included with an application, they are generally obtained by the software author through some physical means. For example, Thawte provides its root certificates on its web site, free and clear, but strongly advises anyone making use of them to confirm the certificate fingerprints with Thawte via telephone before using or distributing them.

To verify the authenticity and validity of a given certificate, each certificate in the chain must also be verified, from the certificate in question’s issuer all the way up to the root certificate. If any certificate in the chain is invalid, each certificate below it in the chain must also be considered invalid. Invalid certificates typically have either expired or been revoked (perhaps due to certificate theft). A certificate is also considered invalid if it has been tampered with and the signatures on the certificate don’t match with the ones that should have been used to sign it.

The decision whether to employ a certificate hierarchy more complex than a single root CA depends on many factors. These factors and their trade-offs are well beyond the scope of this book. Entire books have been devoted to PKI, and we strongly recommend that you consult one or more of them to assist you in making an informed decision. Again, we strongly recommend Planning for PKI by Russ Housley and Tim Polk.

The most widely accepted format for certificates is the X.509 format, first introduced in 1988. There are three versions of the format, known as X.509v1, X.509v2, and X.509v3. The most recent revision of the standard was introduced in 1996, and most, if not all, modern software now supports it. A large number of changes were made between X.509v1 and X.509v3, but perhaps one of the most significant features introduced in the X.509v3 standard is its support of extensions.

Version 3 extensions allow a certificate to contain additional fields beyond those defined by previous versions of the X.509 standard. The additional fields may be standard in X.509v3, such as the basicConstraints or keyUsage fields, or they may be completely nonstandard, perhaps recognized only by a single application. Each extension has a name for its field, a designation indicating whether the extension is critical , and a value to be associated with the extension field. When an extension is designated as critical, software that does not recognize the extension must reject the certificate as being invalid. If the extension is noncritical, it may be ignored.

The X.509v3 standard defines 14 extensions in an effort to consolidate the most common extensions implemented by third parties. One example is the permissible uses for a certificate—for instance, whether a certificate is allowed to sign another certificate, or is usable in an SSL Server. If each application were to create its own disparate extensions, the information in those extensions would be either unusable by other applications or significantly complicate the process of validating a certificate because it would need to recognize a virtually unlimited number of different extensions that all essentially mean the same thing.

Of the 14 standard extensions defined by X.509v3, only 4 are well-supported and in widespread use. Only one of them must be designated critical according to the standard, while the other three may or may not be. Since the majority of the standard extensions are not well supported, we won’t discuss them here. Later in this chapter, when we setup our own CA, we’ll be making use of some of the better-supported extensions, as appropriate.

The basicConstraints extension is a sequence that may contain two possible components: cA and pathLenConstraint. Without getting into the technical details of an X.509 certificate, a sequence can best be thought of as a container, which contains other components; it has no value of its own. The cA component is a boolean indicating whether the certificate may be used as a CA’s certificate. If the cA component is absent, OpenSSL will check the keyUsage extension to determine whether to allow the certificate to be used as a CA certificate. If the keyUsage extension is present and the keyCertSign bit is not set, the certificate may not be used as a CA certificate. The optional pathLenConstraint component is an integer that specifies the maximum number of certificates in the chain that may be used below this certificate. If the value is less than the number of certificates in the chain that have already been validated, this certificate must be rejected.

The keyUsage extension is a bit string that defines how a certificate can be used, and may or may not be designated critical. If the extension is present in the certificate, it should be marked critical. If it is designated critical, the information that it contains will always be used to determine valid usage. If the extension is absent or designated noncritical, the certificate should be treated as though all bits are set. Rather than individually explain what each bit means, it’s more useful to show which bits should be set for each of the common uses for a certificate, which we do in Table 3-1.

The extKeyUsage extension is a sequence of object identifiers that further defines which uses of the certificate are permissible, and may or may not be designated critical. As with the keyUsage extension, if this extension is present, it should be designated critical. If it is designated critical, the certificate must be used for one of the purposes defined by the extension. If it is designated noncritical, the information is advisory only and may be ignored. There are eight possible purposes defined for use with this extension, as summarized in Table 3-2.

It’s worth noting that neither the keyUsage nor the extKeyUsage extension is well-defined, and as such, their usage is subject to wide interpretation. In particular, how to treat the critical flag on either extension is not well-defined, but it would seem that in many existing software products, the extensions are largely ignored. In addition, various profiles (guidelines that dictate what certificates should contain) specify their usage differently. For instance, PKIX (the IETF Public Key Infrastructure working group) has obsoleted the three IPSec-related OIDs that may be present in an extKeyUsage sequence. Additionally, they are not implemented consistently across vendors. As a result of these problems, these two extensions are mostly useless. If you do use them, be sure that you are using them in a consistent manner with any existing software with which you’ll be interoperating.

The cRLDistributionPoints extension is a sequence that is used to communicate how the CA that issued the certificate makes its CRLs available. The standard indicates that this extension should be designated noncritical; however, it does advise CAs to include the information. Providing the location of the CRL that would contain this certificate’s serial number if it is revoked inside the certificate itself is perhaps the best possible way for software validating a certificate to obtain the information.

Once a certificate has been issued, it is generally put into production, where it will be distributed to many clients. If an attacker compromises the associated private key, he now has the ability to use the certificate even though it doesn’t belong to him. Assuming the proper owner is aware of the compromise, a new certificate with a new key pair should be obtained and put into use. In this situation there are two certificates for the same entity—both are technically valid, but one should not be trusted. The compromised certificate will eventually expire, but in the meantime, how will the world at large know not to trust it?

The answer lies in something called a certificate revocation list (CRL). A CRL contains a list of all of the revoked certificates a CA has issued that have yet to expire. When a certificate is revoked, the CA declares that the certificate should no longer be trusted.

Bandwidth is a significant concern when distributing CRLs, since clients need to have reasonably current revocation information in order to properly validate a certificate. In an ideal world, the client would get up-to-date revocation information as soon as the CA gets the information. Unfortunately, many CAs distribute CRLs only as a huge list. Downloading a huge list before validating each certificate could easily add unacceptable latency and place an undue load on the server when there are a lot of clients. As a result, CAs tend to update their CRLs regularly, but not immediately after they learn about key compromises. Included in the revocation list is the date and time that the next update will be published, so once an application has downloaded the list, it doesn’t need to do so again until the one it has expires. Clients are encouraged to cache the information (which can be infeasible if the client has limited storage space).

This scheme leaves a window of vulnerability in which the CA knows about a revoked certificate, yet the client does not find out about it immediately. If a CA publishes the list too frequently, it will require massive amounts of bandwidth in order to sustain the frequent demand for the list. On the other hand, if a CA publishes the list too infrequently, certificates that need to be revoked will still be considered valid until the next list is published. Each CA needs to strike a balance with the community that it’s serving to determine how frequently to publish its list.

One solution to this problem is for the CA to break up its CRLs into segments. To do this, the CA specifies ranges of certificate serial numbers that each CRL contains. For example, the CA could create a different CRL for each 1,000 serial numbers. Therefore, the first CRL would be for serial numbers 1 through 1,000; the second would be for serial numbers 1,001 through 2,000, and so on. This solution does require forethought and planning on the part of the CA, but it reduces the size of the CRLs that the CA issues. Another option is to use " delta CRLs,” where a CA periodically publishes incremental changes to its CRL list. Delta CRLs still require the client to cache CRL information or download everything anew each time a certificate needs to be validated.

Another problem with CRLs is that while there is a standard means to publish them formally, specified in RFC 2459, that mechanism is optional, and many of the more common public CAs, such as VeriSign, do not distribute their CRLs this way. There are also other standard methods for distributing CRLs, but the overall problem is that there isn’t just one, and so many software applications do not actually make use of CRLs. Of the various methods of distribution, LDAP is most commonly used as a repository for CRLs. Additionally, multiple applications on the same machine, or even on the local network, could be interested in the same data and require that it be queried from the CA multiple times within a short period.

The problems with the distribution of CRLs currently make them difficult to manage, and what’s worse, few applications even make the attempt. This essentially makes CRLs useless and leaves no way for a CA to revoke a certificate effectively once it’s been issued. Ideally, CAs need to standardize a method for CRL distribution, and both CAs and applications need to start making use of it.

Another potentially serious problem that has not been addressed is what happens when a root CA’s certificate needs to be revoked. A CRL is not suited to handle this, and neither are applications. The reason for this is that CRLs are issued by a parent (a CA) for its children, but a root CA has no parent. It is possible for a CA to revoke its own certificate as long as it still has its private key. For the purposes of signing a CRL containing its own certificate, the CA’s compromised key can still be trusted. Unfortunately, given the poor state of CRL handling in existing software in general, it’s not likely that this situation is handled very well, if at all.

A classic example demonstrating that CRLs are poorly supported is what happened in early 2001 when VeriSign issued two class 3 code-signing certificates to Microsoft Corporation. The problem was that Microsoft never requested these certificates—someone claiming to represent Microsoft did. VeriSign handled the situation in the appropriate manner and published the serial numbers of the certificates in a new CRL. What really demonstrated the flaws with CRLs was how Microsoft handled the situation. It quickly became clear that Microsoft’s software, while distributing VeriSign’s root certificates and using their services, did not check VeriSign’s CRLs. Microsoft issued a patch to deal with the problem of the revoked certificates, but the patch did nothing to fix the problem of their software not utilizing the CRLs at all. Had Microsoft’s software made proper use (or, arguably, any use at all) of CRLs, no patch would have been necessary, and the problem would have ended with VeriSign’s publication of its CRL (minus the inherent window of vulnerability).

It could be argued that if a major software company like Microsoft can’t handle CRLs properly, how can smaller software companies and individual software developers be expected to handle them properly? While the argument may very well be faulty in a number of respects, it is still a question worth asking, and in truth, the answer, at least for right now, is not one that we would all like to hear. PKI is still relatively immature, and much work needs to be done to remedy not only the issues that we’ve discussed here, but others that we leave as an exercise for the reader to explore as well. While CRLs may not be the ultimate answer to revoking a certificate, for the time being, they are the most widely implemented means by which to do so. It’s worth taking the time to ensure that your software is capable of dealing with the technology and provides for a reasonably safe and pleasant experience for your users.

To complicate matters more, the standard CRL specification has changed over time, and both the old format (Version 1) and the new format (Version 2) are actively used. OpenSSL supports both Version 1 and Version 2 CRLs, but there is much software still in common use that does not yet support Version 2, and certainly old legacy applications that are no longer being developed or supported never will, even though they continue to be used. The major addition that Version 2 offers is extensions. The standard defines four extensions that are used primarily to indicate when a certificate was revoked, why a certificate was revoked, and how to handle a certificate that has been revoked.

The fourth standard extension is used in indirect CRLs. An indirect CRL is one that is not necessarily issued by a CA, but instead by a third party. Such a CRL can contain certificates from multiple CAs. The extension, then, is used to indicate which CA issued the certificate that has been revoked. Currently, indirect CRLs are not very common, because CRLs in Version 2 format are not widely supported.

The Online Certificate Status Protocol (OCSP), formally specified in RFC 2560, is a relatively new addition to PKI. Its primary aim is to address some of the distribution problems that have traditionally plagued CRLs.

Using OCSP, an application makes a connection to an OCSP responder and requests the status of a certificate by passing the certificate’s serial number. The responder replies “good,” “revoked,” or “unknown.” A “good” response indicates that the certificate is valid, so far as the responder knows. This does not necessarily mean that the certificate was ever issued, just that is hasn’t been revoked. A “revoked” response indicates that the certificate has been issued and that it has indeed been revoked. An “unknown” response indicates that the responder doesn’t know anything about the certificate. A typical reason for this response could be that the certificate was issued by a CA that is unknown to the responder.

An OCSP responder is typically operated by a CA or by a trusted third party that is authorized by the CAs for which it provides information. The client must trust the OCSP responder in a manner similar to a root CA. More importantly, there is only one way to revoke an OCSP’s trusted status, and it’s not pretty. If an OCSP responder is compromised, every client that makes use of that responder must be reconfigured manually either not to trust it or to use a new certificate that can be trusted.

A client’s request includes information about the issuer of the certificate it is requesting status information for, so it is possible for a single OCSP responder to provide certificate revocation information for more than a single CA. Unfortunately, one of the problems of OCSP responders when run by a third party is that the information they’re serving can become stale. At the very least, a delay often occurs between the time when a CA revokes a certificate and when the responder receives the information from the CA, particularly if the responder is relying on CRLs published by its serviceable CAs to supply its information.

Currently, OCSP is not nearly as widely recognized or implemented as CRLs are, so unless you know that all your users will have an OCSP server available, it is generally best to use the technology to supplement CRLs rather than replace them completely.

Three of the more significant problems that OCSP introduces are the potential for denial of service, replay, and man-in-the-middle attacks. Most servers are vulnerable to denial of service attacks to some extent, but the nature of the service, the amount of information transferred, and the way requests are handled help determine just how vulnerable a given server is to such an attack. The details of denial of service attacks are beyond the scope of this book; however, OCSP responders are typically more susceptible to them than other common services such as HTTP.

The OCSP Version 1 specification allows responders to pre-produce signed responses in an effort to reduce the load on the responder required by signing definitive responses. Allowing for pre-produced signed responses opens the door for replay attacks. Man-in-the-middle attacks are possible because error responses are not signed, although this type of attack could more accurately be considered a denial of service attack. Perhaps what’s most disturbing about the aforementioned vulnerabilities is the fact that each one is noted in the RFC, yet nothing was done when formalizing the standard to prevent them.

There are only a handful of public OCSP responders available at the time of this writing, as listed by www.OpenValidation.org . The small number of responders is a clear indication that OCSP is not widely deployed. While OCSP is an attempt at resolving the problems of CRLs, we feel that the additional problems it creates, at least in its current state, outweigh the problems that it solves. Certainly, it cannot be reasonably considered as a replacement for CRLs. In its defense, there was an IETF draft submitted in March of 2001 for Version 2 of the protocol, which addresses some of the issues, but this has not yet completed the standards process.

S/MIME email relies on personal certificates (as opposed to certificates granted to an organization), which VeriSign calls a Class 1 Digital ID . It is the easiest kind of certificate to obtain, and is available for a modest price, but it is limited to email security only. You can get a Class 1 Digital ID that works with Netscape Messenger, or you can get one intended to work with Microsoft Outlook express. If you use a different application to read and write your email, you should consult with that application’s vendor to find out whether it interoperates with either of these certificate types.

The first step in obtaining a personal certificate is to visit VeriSign’s web site at http://www.verisign.com/ and follow the links from the main page to “Secure E-Mail”, which is listed under “Home & Home Office” products, to the Digital ID enrollment form. We won’t outline all of the links here, not only because they’re subject to change, but because there’s a wealth of information on the site that is well worth reading, including information on how to make use of the certificate once it has been issued. Once you have filled out and submitted the enrollment form, VeriSign will send an automated email to the address you provided with instructions on how to “pick up” the certificate.

The first set of questions on the enrollment form is self-explanatory. The first and last name that you enter will be how your Digital ID is listed in VeriSign’s directory service. The email address that you enter should be the one that you will be using with the Digital ID. It becomes the certificate’s distinguished name. It is also listed alongside your first and last name in the directory. VeriSign will also use the address to verify its validity by sending an automated email to that address with instructions on how to retrieve the certificate that has been issued.

Next, VeriSign will request a challenge phrase, which will be used to protect the certificate. The phrase will be available to you and VeriSign. You should not share it with anyone else! VeriSign will use the phrase to verify that you are the owner of the certificate when you request that it be revoked, renewed, or replaced. Be sure to choose a phrase that you’ll be able to remember, but one that will not be easily guessed by someone that knows you well.

VeriSign chooses a default key length for the certificate and issues it to you based upon the information from your browser. You shouldn’t need to change the key length that is selected for you unless you’re using something other than Netscape or Microsoft products to access your email, in which case the documentation for your email software or the vendor of the software should have advised you on the proper setting to choose.

If you’re using Microsoft Internet Explorer, your certificate will be unprotected by default. That is, once you install it in your email software, you will not be required to enter any password or passphrase to gain access to it. If you opt to keep your certificate unprotected in this manner, you must make every assurance that the private key for your certificate is not compromised. It is generally not a good idea to leave your certificate unprotected, so VeriSign offers two methods of protecting it. One step up from the default of low security is medium security, which requires your approval each time the private key is accessed. With medium security, you still are not required to enter a password or passphrase to unlock the private key. High security requires you to enter a password or passphrase to unlock the key each time it is accessed.

Remember that anybody gaining access to your private key will be able to use your certificate to masquerade as you. When an email is signed with your private key, people are going to trust it, and this can have disastrous effects if your key is compromised. Anyone with access to your private key will also be able to decrypt email that has been encrypted with your public key. Sure, your certificate can be revoked, but as we discussed earlier, revoking a certificate doesn’t have any effect if its revocation status is not being checked. With this in mind, particularly for mobile users, we strongly recommend that you choose high security.

Finally, you must read and accept VeriSign’s subscriber agreement and privacy policy. If you’re using Microsoft Internet Explorer and you checked the checkbox for securing your certificate, a dialog will be presented to you to select the security level that you wish to apply to the certificate. Within an hour or so, you will receive an email from VeriSign at the address that you entered into the enrollment form containing instructions on how to “pick up” your certificate from VeriSign. Included in the email are a URL and a PIN, both of which will be required to get the certificate from VeriSign. You should use the same machine and browser to retrieve the certificate as you did to request it.

That’s all there is to it! Once you’ve retrieved your certificate from VeriSign, follow the directions presented on VeriSign’s site to use the certificate in either Netscape or Microsoft Internet Explorer. Again, if you’re using other software to access your email, follow the vender’s directions to enable the certificate. Now you’re ready to start sending and receiving secure email!

VeriSign offers code-signing certificates for use by software developers and software vendors. The purpose of such a certificate is to sign code that users download off the Internet. By signing your code, users can be assured that the code has not been tampered with or corrupted since it was digitally signed with your certificate. In the online world where people are not only becoming increasingly aware of security issues, but also worry about viruses and worms, signing your code provides a certain assurance to your users that they are getting the software that they’re expecting to get.

Obtaining a code-signing certificate is not nearly as quick and easy as obtaining a personal certificate. They are also considerably more expensive, but then again, they’re not really intended for everyday individual users. At the time that this text was written, VeriSign offered six different types of code-signing certificates for various types of programs. You must be sure to get the proper certificate for the code that you wish to sign because the different types of certificates may not work properly with other types of code. For example, Microsoft Authenticode certificates only work for Microsoft’s Internet Explorer browser. For Netscape browsers, you need to get a Netscape Object Signing certificate. The available types of code-signing certificates are listed as part of the process of obtaining a code-signing certificate. Choosing a type is the first step in obtaining a code-signing certificate.

The type of code-signing certificate determines the specific requirements for making the request to VeriSign to obtain it. For a Microsoft Authenticode Digital ID, for example, much of the work is automated through Microsoft’s Internet Explorer, while a Sun Java Signing Digital ID requires you to generate a certificate request using Sun’s Java tools to be submitted along with the request. For each type of certificate, VeriSign supplies full instructions on what code-signing-certificate-dependent information is needed and how to go about obtaining and supplying it to VeriSign.

While each type of code-signing certificate has its own specific requirements for making the request, there are common requirements that must be met as well. Most of the requirements are self-explanatory, such as contact and payment information. Each certificate must also have information about who owns the certificate. Such information includes the name of the company or organization and the location from which it does business. For example, a company doing business from the United States would be required to supply the city and state in which it is located.

There is, of course, also the very important need for the CA, VeriSign in this case, to verify that they’d be issuing the certificate to someone who should legitimately have it. The quickest and easiest way for VeriSign to verify this information is with a Dun & Bradstreet D-U-N-S number. Supplying this information is optional, but the alternatives require more time and effort both on your part and VeriSign’s. If you do not have or do not want to use a D-U-N-S number, you can optionally mail or fax copies of your business license, articles of incorporation, or partnership papers along with your request for a code-signing certificate.

Once your request, including any appropriate documentation, has been submitted, VeriSign takes it under review. If everything is in order, a code-signing certificate is issued and instructions on how to retrieve the certificate so that you may distribute and use it are provided. Unlike a personal certificate, the request for a code-signing certificate is reviewed and verified by an actual living human being, and so is not made immediately available. Depending on VeriSign’s workload, it may take several days for the certificate to be issued, although VeriSign expedites requests for an additional fee.

The process of obtaining a certificate for use in securing a web site, which VeriSign calls a secure server certificate, is similar to the process for obtaining a certificate for a code-signing certificate. Much of the same information is required, although there are some differences worth noting. Obviously, one of the primary differences is in the types of certificates offered. While code-signing certificates differ based on the type of code that will be signed (Netscape plug-ins versus Java applets, for example), secure server certificates are one of either 40-bit or 128-bit SSL certificates. That is, web site certificates explicitly restrict the size of the symmetric keys that should be used with the certificate. We recommend you stick with 128-bit certificates, since 40-bit symmetric keys are widely regarded as unacceptably weak.

No matter which server software you plan to use, you must follow its instructions on how to generate a Certificate Signing Request (CSR). Due to the wide variety of servers available today, it is not practical for us to provide instructions on how to do this here. VeriSign has instructions for many of the more popular servers available on its web site. The CSR you generate will also generate a key pair. While you must submit the CSR to VeriSign to have the certificate issued, you should keep the private key to yourself. It should not be sent to VeriSign or to anybody else.

As with code-signing certificates, you must also provide acceptable proof to VeriSign that you have a right to the certificate you are requesting. The options for providing this proof are the same—provide either a D-U-N-S number or a copy of one of the aforementioned acceptable documents. Additionally, a secure server certificate is bound to a domain name. VeriSign will issue certificates only to the registered owner of a domain. This means that if the domain is owned by a corporate entity, you must be an employee of that company.

Once your request, including any appropriate documentation, has been submitted, VeriSign takes it under review. If everything is in order, a secure server certificate is issued and the certificate is emailed to the technical contact that was provided when the request was submitted. As with code-signing certificates, an actual living human being reviews the information, so it may take several days for the certificate to be issued, depending on VeriSign’s workload. Expedited processing is also available for an additional fee.

The first step in setting up a CA with the OpenSSL command-line tool is creating an environment for it to run in. Several files and directories must be created. The easiest way to set everything up is from the command line, using your favorite text editor to create the necessary files. For our example CA, we’ll be using the bash shell on a Unix system. Whether the system is Linux or FreeBSD or some other variety of Unix doesn’t matter; the instructions will be the same. There will be some variation for Windows-based systems.

First, we must choose a location for all of our CA’s files to reside. For our example, we use /opt/exampleca as the root directory for our CA, but you may choose any location you like on your system. All of our CA’s files, including issued certificates and CRLs, will be contained within this directory. Keeping the files together makes it easier to find any of the files used by our CA and to set up multiple CAs.

Within the CA’s root directory, we need to create two subdirectories. We’ll name them certs and private . The subdirectory certs will be used to keep copies of all of the certificates that we issue with our CA. The subdirectory private will be used to keep a copy of the CA certificate’s private key. For the most part, the majority of the files that the CA uses are visible to anyone on the system. In fact, many of the files are supposed to be distributed to the public, or at least to anyone who makes any use of the certificates issued by our CA. The one notable exception is the CA certificate’s private key. The private key should never be disclosed to anyone not authorized to issue a certificate or CRL from our CA.

A good CA needs to protect its private key as best it can. 2,048 bits are the bare minimum length for a CA key. The private key should be stored in hardware, or at least on a machine that is never put on a network (CSRs would arrive via the sneaker net).

Besides key generation, we will create three files that our CA infrastructure will need. The first file is used to keep track of the last serial number that was used to issue a certificate. It’s important that no two certificates ever be issued with the same serial number from the same CA. We’ll call this file serial and initialize it to contain the number 1. OpenSSL is somewhat quirky about how it handles this file. It expects the value to be in hex, and it must contain at least two digits, so we must pad the value by prepending a zero to it. The second file is a database of sorts that keeps track of the certificates that have been issued by the CA. Since no certificates have been issued at this point and OpenSSL requires that the file exist, we’ll simply create an empty file. We’ll call this file index.txt (see Example 3-1).

# mkdir /opt/exampleca
# cd /opt/exampleca
# mkdir certs private
# chmod g-rwx,o-rwx private
# echo '01' > serial
# touch index.txt

One more file still needs to be created, but it is significantly more complex than the first two files that we’ve already created. It is a configuration file that will be used by the OpenSSL command-line tool to obtain information about how to issue certificates. We could conceivably skip creating this file and use the OpenSSL defaults, which are actually quite sane, but by using a configuration file, we save ourselves some work in issuing commands to OpenSSL. We briefly discussed configuration files and their use with the command-line tool in the Chapter 2. Now it’s time to actually create a configuration file of our own and put it to use.

The OpenSSL command for the CA functions is aptly named ca , and so the first section that we’re interested in is named ca. For our purposes, this section is quite simple, containing only a single key: default_ca . The value is the name of a section containing the configuration for the default CA. OpenSSL allows for multiple CA configurations in the same configuration file. If the name of a configuration to use is not specified, OpenSSL uses the name paired with the default_ca key. The default can be overridden on the command line with the name option.

Example 3-2 shows the configuration file for our CA. We’ve already explained what the files and directories we’ve created are for, so the first set of keys in the example should be clear; we’re simply telling OpenSSL where we’ve decided to place the files and directories that it needs to use. The three keys, default_crl_days , default_days , and default_md , correspond to the command-line crldays, days, and md options, and may be overridden by using them.

The default_crl_days key specifies the number of days between CRLs. You may wish to use default_crl_hours instead if you plan to publish CRLs more than once a day. This setting computes the nextUpdate field of the CRL when it is generated. The default_days key specifies the number of days an issued certificate will be valid. The default_md specifies the message digest algorithm that will be used to sign issued certificates and CRLs. Possible legal values for this key include md5, sha1, and mdc2.

The policy key specifies the name of a section that will be used for the default policy. It may be overridden on the command line with the policy option. A policy definition is a set of keys with the same name as the fields in a certificate’s distinguished name. For each key or field, there are three legal values: match, supplied, or optional. A value of match means that the field by that name in a certificate request must match the same field in the CA’s certificate. A value of supplied means that the certificate request must contain the field. A value of optional means that the field is not required in the certificate request.

By default, when a certificate is issued, OpenSSL orders the DN (distinguished name) fields in the same order as they appear in the policy definition being used. Any fields that are present in the certificate request but not present in the policy definition are omitted from the issued certificate. This behavior can be changed by using the preserveDN option or by setting the preserve key to yes in the CA definition section. When this option is set, all of the fields in the certificate request are included in the issued certificate, and they remain in the same order as they were in the certificate request. Ordinarily, you should not need to enable this option unless you’re dealing with older versions of Microsoft Internet Explorer, which require the fields in the issued certificate to match the certificate request. If you’re dealing with very old versions of Microsoft Internet Explorer, you may also need to enable the “MSIE hack” either by using the msie_hack option or by setting the msie_hack key to yes in the CA definition section.

The x509_extensions key specifies the name of a section that will contain the extensions to be added to each certificate issued by our CA. If this key is absent, OpenSSL creates an X.509v1 certificate, but if it is present, even if it is empty, an X.509v3 certificate is created. The only extension that we’ve included in our example is the basicConstraints extension, and we’ve set its cA component to false so that the certificates issued by our CA, in turn, may not be used as CA certificates. The certificate chain stops with certificates that we issue. Example 3-2 shows the configuration file.

 [ ca ]
default_ca = exampleca
 
[ exampleca ]
dir              = /opt/exampleca
certificate      = $dir/cacert.pem
database         = $dir/index.txt
new_certs_dir    = $dir/certs
private_key      = $dir/private/cakey.pem
serial           = $dir/serial
 
default_crl_days = 7
default_days     = 365
default_md       = md5
 
policy           = exampleca_policy
x509_extensions  = certificate_extensions
 
[ exampleca_policy ]
commonName             = supplied
stateOrProvinceName    = supplied
countryName            = supplied
emailAddress           = supplied
organizationName       = supplied
organizationalUnitName = optional
 
[ certificate_extensions ]
basicConstraints = CA:false

Now that we’ve created a configuration file, we need to tell OpenSSL where to find it. By default, OpenSSL uses a system-wide configuration file. Its location is determined by your particular installation, but common locations are /usr/local/ssl/lib/openssl.cnf or /usr/share/ssl/openssl.cnf. Since we’ve created our own configuration file solely for the use of our CA, we do not want to use the system-wide configuration file. There are two ways to tell OpenSSL where to find our configuration file: using the environment variable OPENSSL_CONF, or specifying the filename with the config option on the command line. Since we will issue a sizable number of commands that should make use of our configuration file, the easiest way for us to tell OpenSSL about it is through the environment (see Example 3-3).

# OPENSSL_CONF=/opt/exampleca/openssl.cnf
# export OPENSSL_CONF

Before we can begin issuing certificates with our CA, it needs a certificate of its own with which to sign the certificates that it issues. This certificate will also be used to sign any CRLs that are published. Any certificate that has the authority to sign certificates and CRLs will do. By this definition, a certificate from another CA or a self-signed root certificate will work. For our purposes, we should create our own self-signed root certificate to do the job.

The first thing that we need to do is add some more information to our configuration file. Example 3-4 shows the newly added information. Note that we’ll be using the command-line tool’s req command, so we’ll start by adding a new section by the same name. Since we will use only this configuration file for our CA, and since we will use only the command-line tool’s req command this one time, we’ll put all of the necessary information that OpenSSL allows in the configuration file rather than typing it out on the command line. It’s a little more work to do it this way, but it is the only way to specify X.509v3 extensions, and it also allows us to keep a record of how the self-signed root certificate was created.

 [ req ]
default_bits       = 2048
default_keyfile    = /opt/exampleca/private/cakey.pem
default_md         = md5
 
prompt             = no
distinguished_name = root_ca_distinguished_name
 
x509_extensions    = root_ca_extensions
 
 [ root_ca_distinguished_name ]
commonName          = Example CA
stateOrProvinceName = Virginia
countryName         = US
emailAddress        = ca@exampleca.org
organizationName    = Root Certification Authority
 
[ root_ca_extensions ]
basicConstraints = CA:true

The default_bits key in the req section tells OpenSSL to generate a private key for the certificate with a length of 2,048 bits. If we don’t specify this, the default is to use 512 bits. A key length of 2,048 bits provides significantly more protection than 512, and for a self-signed root certificate, it’s best to use all of the protection afforded to us. With the vast computing power that is affordable today, the speed penalty for using a 2,048-bit key over a 512-bit key is well worth the trade-off in protection, since the security of this one key directly impacts the security of all keys issued by our CA.

The default_keyfile key in the req section tells OpenSSL where to write the newly generated private key. Note that we’re specifying the same directory for output as we specified earlier in the ca section as the location of the private key for the certificate. We can’t use the $dir “macro” here because the dir key is private to the ca section, so we need to type out the full path again.

The default_md key in the req section tells OpenSSL which message digest algorithm to use to sign the key. Since we specified MD5 as the algorithm to use when signing new certificates and CRLs, we’ll use the same algorithm here for consistency. The SHA1 algorithm is actually a stronger algorithm and would be preferable, but for the sake of this example, we’ve chosen MD5 because it is more widely used and all but guaranteed to be supported by any software that could possibly be using our certificates. If you will be using only software that you know supports SHA1 with your certificates, we recommend that you use SHA1 instead of MD5.

The prompt and distinguished_name keys determine how OpenSSL gets the information it needs to fill in the certificate’s distinguished name. By setting prompt to no, we’re telling OpenSSL that it should get the information from the section named by the distinguished_name key. The default is to prompt for the information, so we must explicitly turn prompting off here. The keys in the distinguished_name section that we’ve defined by the name of root_ca_distinguished_name are the names of the fields making up the distinguished name, and the values are the values that we want placed in the certificate for each field. We’ve included only the distinguished name fields that we previously configured as required and omitted the one optional field.

Finally, the x509_extensions key specifies the name of a section that contains the extensions that we want included in the certificate. The keys in the section we’ve named root_ca_extensions are the names of the extension fields that we want filled in, and the values are the values we want them filled in with. We discussed the basicConstraints key earlier in this chapter. We’ve set the cA component of the extension to true, indicating that this certificate is permitted to act as a CA to sign both certificates and CRLs.

Now that we have the configuration set up for generating our self-signed root certificate, it’s time to actually create the certificate and generate a new key pair to go along with it. The options required on the command line are minimal because we’ve specified most of the options that we want in the configuration file. From the root directory of the CA, /opt/exampleca, or whatever you’ve used on your system, execute the following command. Make sure that you’ve set the OPENSSL_CONF environment variable first so that OpenSSL can find your configuration file!

# openssl req -x509 -newkey rsa -out cacert.pem -outform PEM

When you run the command, OpenSSL prompts you twice to enter a passphrase to encrypt your private key. Remember that this private key is a very important key, so choose your passphrase accordingly. If this key is compromised, the integrity of your CA is compromised, which essentially means that any certificates issued, whether they were issued before the key was compromised or after, can no longer be trusted. The key will be encrypted with 3DES, using a key derived from your passphrase. Example 3-5 shows the results of the command we just generated followed by a textual dump of the resulting certificate. Although your certificate will be different because your public and private key will be different from ours, your output should look similar.

# openssl req -x509 -newkey rsa -out cacert.pem -outform PEM
 
Using configuration from /opt/exampleca/openssl.cnf
Generating a 1024 bit RSA private key
..................................................++++++
.........++++++
writing new private key to '/opt/exampleca/private/cakey.pem'
Enter PEM pass phrase:
Verifying password - Enter PEM pass phrase:
-----
# openssl x509 -in cacert.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 0 (0x0)
        Signature Algorithm: md5WithRSAEncryption
        Issuer: CN=Example CA, ST=Virginia, C=US/Email=ca@exampleca.org, 
O=Root 
Certificate Authority
        Validity
            Not Before: Jan 13 10:24:19 2002 GMT
            Not After : Jan 13 10:24:19 2003 GMT
        Subject: CN=Example CA, ST=Virginia, C=US/Email=ca@exampleca.org, 
O=Root 
Certificate Authority
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (1024 bit)
                Modulus (1024 bit):
                    00:cb:4f:55:6c:a4:2c:8a:f4:21:44:ec:fc:ca:9f:
                    ca:c7:43:2f:14:7d:07:1a:05:e7:3f:08:6c:ee:88:
                    30:ef:5b:24:6c:90:59:a2:81:af:99:bc:f6:94:96:
                    ab:48:53:98:b3:13:b2:42:aa:01:31:7d:59:0d:9a:
                    99:dc:95:b8:c2:0a:fc:b5:d0:d1:7a:5c:db:87:a3:
                    e0:db:8a:3f:c3:10:40:b5:d5:e9:5f:58:8d:fd:f1:
                    06:65:e2:73:7a:17:7f:98:ac:6f:b5:be:56:e1:5f:
                    16:2b:43:02:60:d8:80:b7:7e:0e:d4:48:3e:6a:c9:
                    2d:a6:02:3d:b0:e1:ac:fc:3d
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:TRUE
    Signature Algorithm: md5WithRSAEncryption
        2e:54:2c:cf:d8:1a:d0:bc:bb:9d:eb:3e:2f:fa:8b:7b:21:ef:
        4f:30:0e:93:6c:85:26:8d:c2:62:69:49:7b:55:26:09:6a:ea:
        00:bc:a0:03:ab:5b:45:8a:71:eb:39:46:6c:50:29:4b:00:ff:
        19:a1:e8:a2:4a:75:07:79:50:f0:38:6d:d2:20:09:63:48:75:
        67:6b:59:41:74:ae:63:69:13:4e:27:6b:5d:7e:55:6a:7b:3c:
        86:c8:b2:c5:15:01:e3:68:08:ec:3c:8a:00:68:43:ce:43:f0:
        76:e2:e2:97:ad:88:08:bf:87:ec:ba:d1:db:fa:c4:91:fb:b6:
        33:95

You’ll notice in Example 3-5’s output that when OpenSSL displays a DN in a shortened form, it uses a nonstandard representation that can be somewhat confusing. In this example, we see C=US/Email=ca@exampleca.org as an example of this representation. What’s confusing here is the slash separating the two fields. The reason for this is that the Email and O fields are nonstandard in a DN. OpenSSL lists the standard fields first and the nonstandard fields second, separating them with a slash.

Everything is now set up for our CA, and it’s time to take it out for a test drive by issuing a certificate. To do that, we need a certificate request. It’s also a good idea to know how to create a certificate request that your CA will be able to use. Unless you plan to create both the certificate requests and certificates for anybody you’ll be issuing a certificate to, you’ll probably need to be able to tell someone how to give you a certificate request that you can use. Either way, you’ll still need to know how to do it yourself.

To create a certificate request, start with a clean shell without the OPENSSL_CONF environment variable set so that the default configuration file is used. We don’t want to use our custom configuration file to do this, as that configuration file is intended for use only by the CA, and generating a certificate request is not at all a function of a CA.

The command to generate a certificate request is similar to the command we used to create our self-signed root certificate. We use the command-line tool’s req command, but we’ll need to specify some extra parameters. The operation will be much more interactive, prompting for information to fill in the certificate request’s distinguished name. Example 3-6 shows the output from generating a certificate request.

# openssl req -newkey rsa:1024 -keyout testkey.pem -keyform PEM
 -out testreq.pem -outform PEM
Using configuration from /usr/share/ssl/openssl.cnf
Generating a 1024 bit RSA private key
.........++++++
.........++++++
writing new private key to 'testkey.pem'
Enter PEM pass phrase:
Verifying password - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Virginia
Locality Name (eg, city) []:Manassas
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Test Request     
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:www.exampleca.org
Email Address []:ca@exampleca.org   
 
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:cloud noon sundry presto madrid baker
An optional company name []:Examples-R-Us, Inc.

The result of this command is the creation of two files: testreq.pem and testkey.pem. The former, testreq.pem, contains the certificate request as shown in Example 3-7, and testkey.pem contains the private key that matches the public key embedded in the certificate request. As part of the process to generate a certificate request, a new key pair was also generated. The first passphrase that is prompted for is the passphrase used to encrypt the private key. The challenge phrase is stored in the certificate request, and is otherwise ignored by OpenSSL. Some CAs may make use of it, however.

# openssl req -in testreq.pem -text -noout
Using configuration from /usr/share/ssl/openssl.cnf
Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject: C=US, ST=Virginia, L=Manassas, O=Test Request, 
CN=www.exampleca.org/Email=ca@exampleca.org
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (1024 bit)
                Modulus (1024 bit):
                    00:d8:a5:1b:c6:b6:e4:75:bf:f3:e3:ce:29:1d:ab:
                    e2:5b:0d:bb:2e:94:de:52:a1:20:51:b1:77:d9:42:
                    a3:6c:26:1f:c3:3e:58:8f:91:b1:b3:ed:bd:7c:62:
                    1c:71:05:3b:47:ff:1a:de:98:f3:b4:a6:91:fd:91:
                    26:db:41:76:85:b5:10:3f:c2:10:04:26:4f:bc:03:
                    39:ff:b9:42:d0:d3:2a:89:db:91:8e:75:6d:f5:71:
                    ec:96:e8:d6:03:29:8e:fe:20:3f:5d:d8:cb:14:5e:
                    e5:64:fc:be:fa:d1:27:42:b6:72:eb:b4:16:16:71:
                    77:d3:0e:8c:cc:87:16:fc:41
                Exponent: 65537 (0x10001)
        Attributes:
            unstructuredName         :drowssap egnellahc
            challengePassword        :drowssap egnellahc
    Signature Algorithm: md5WithRSAEncryption
        25:aa:ca:78:64:fa:29:46:cf:dc:df:d9:95:dd:48:24:bf:4f:
        7b:7e:f4:09:76:96:c4:c5:b1:10:9b:64:95:19:30:8d:cd:d0:
        da:ac:b2:21:5e:34:e6:be:7b:41:52:2c:b3:e7:d4:dc:99:e5:
        a0:c2:46:12:9f:ef:99:0e:03:89:c1:f9:db:0d:0d:21:1b:e2:
        da:4e:23:ef:c1:aa:1b:24:b5:ce:53:a1:05:08:6e:4a:85:78:
        6e:71:ef:bc:36:48:5c:3e:ee:b1:bb:28:f4:31:df:23:a9:89:
        96:35:1b:b4:01:f9:63:4d:46:b4:ed:5d:be:1d:28:50:1c:86:
        43:5e

With a certificate request now in hand, we can use our CA to issue a certificate. For the sake of convenience in this example, the certificate request that we’ll be using, testreq.pem, which we just created, should be in the CA’s root directory. Make sure that the OPENSSL_CONF variable is set to the CA’s configuration file, and issue the command to generate a certificate, as shown in Example 3-8.

# openssl ca -in testreq.pem
Using configuration from /opt/exampleca/openssl.cnf
Enter PEM pass phrase:
Check that the request matches the signature
Signature ok
The Subjects Distinguished Name is as follows
countryName           :PRINTABLE:'US'
stateOrProvinceName   :PRINTABLE:'Virginia'
localityName          :PRINTABLE:'Manassas'
organizationName      :PRINTABLE:'Test Request'
commonName            :PRINTABLE:'www.exampleca.org'
emailAddress          :IA5STRING:'ca@exampleca.org'
Certificate is to be certified until Jan 14 04:31:25 2003 GMT (365 days)
Sign the certificate? [y/n]:y
 
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
        Signature Algorithm: md5WithRSAEncryption
        Issuer: CN=Example CA, ST=Virginia, C=US/Email=ca@exampleca.org, 
O=Root 
Certificate Authority
        Validity
            Not Before: Jan 14 04:58:29 2002 GMT
            Not After : Jan 14 04:58:29 2003 GMT
        Subject: CN=www.exampleca.org, ST=Virginia, 
C=US/Email=ca@exampleca.org, O=Test Request
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (1024 bit)
                Modulus (1024 bit):
                    00:d8:a5:1b:c6:b6:e4:75:bf:f3:e3:ce:29:1d:ab:
                    e2:5b:0d:bb:2e:94:de:52:a1:20:51:b1:77:d9:42:
                    a3:6c:26:1f:c3:3e:58:8f:91:b1:b3:ed:bd:7c:62:
                    1c:71:05:3b:47:ff:1a:de:98:f3:b4:a6:91:fd:91:
                    26:db:41:76:85:b5:10:3f:c2:10:04:26:4f:bc:03:
                    39:ff:b9:42:d0:d3:2a:89:db:91:8e:75:6d:f5:71:
                    ec:96:e8:d6:03:29:8e:fe:20:3f:5d:d8:cb:14:5e:
                    e5:64:fc:be:fa:d1:27:42:b6:72:eb:b4:16:16:71:
                    77:d3:0e:8c:cc:87:16:fc:41
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
    Signature Algorithm: md5WithRSAEncryption
        13:33:75:8e:a4:05:9b:76:de:0b:d0:98:b8:86:2a:95:5a:13:
        0b:14:c7:48:83:f3:95:0e:3e:bf:76:04:f7:ab:ae:cc:cd:76:
        ae:32:77:ea:8c:96:60:28:52:4e:89:c5:ed:85:68:47:68:95:
        74:53:9f:dc:64:95:62:1a:b0:21:09:76:75:14:25:d4:fd:17:
        de:f9:87:7f:d5:dc:e4:41:1e:ad:f6:7b:2d:bf:a6:8a:cd:65:
        60:3b:71:74:bc:4d:0d:94:5a:22:c4:35:de:b0:19:46:f3:c1:
        bb:c5:e0:d4:f7:a2:92:65:ec:40:4c:cc:d4:b7:a3:84:bd:a9:
        b0:86
-----BEGIN CERTIFICATE-----
MIICcjCCAdugAwIBAgIBATANBgkqhkiG9w0BAQQFADB7MRMwEQYDVQQDEwpFeGFt
cGxlIENBMREwDwYDVQQIEwhWaXJnaW5pYTELMAkGA1UEBhMCVVMxHzAdBgkqhkiG
9w0BCQEWEGNhQGV4YW1wbGVjYS5vcmcxIzAhBgNVBAoTGlJvb3QgQ2VydGlmaWNh
dGUgQXV0aG9yaXR5MB4XDTAyMDExNDA0NTgyOVoXDTAzMDExNDA0NTgyOVowdDEa
MBgGA1UEAxMRd3d3LmV4YW1wbGVjYS5vcmcxETAPBgNVBAgTCFZpcmdpbmlhMQsw
CQYDVQQGEwJVUzEfMB0GCSqGSIb3DQEJARYQY2FAZXhhbXBsZWNhLm9yZzEVMBMG
A1UEChMMVGVzdCBSZXF1ZXN0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDY
pRvGtuR1v/Pjzikdq+JbDbsulN5SoSBRsXfZQqNsJh/DPliPkbGz7b18YhxxBTtH
/xremPO0ppH9kSbbQXaFtRA/whAEJk+8Azn/uULQ0yqJ25GOdW31ceyW6NYDKY7+
ID9d2MsUXuVk/L760SdCtnLrtBYWcXfTDozMhxb8QQIDAQABow0wCzAJBgNVHRME
AjAAMA0GCSqGSIb3DQEBBAUAA4GBABMzdY6kBZt23gvQmLiGKpVaEwsUx0iD85UO
Pr92BPerrszNdq4yd+qMlmAoUk6Jxe2FaEdolXRTn9xklWIasCEJdnUUJdT9F975
h3/V3ORBHq32ey2/porNZWA7cXS8TQ2UWiLENd6wGUbzwbvF4NT3opJl7EBMzNS3
o4S9qbCG
-----END CERTIFICATE-----
Data Base Updated

The first thing that happens is OpenSSL asks for a passphrase. The passphrase that it is looking for is not the passphrase for the certificate request, but the passphrase for the CA’s private key. The private key will be used to sign the new certificate. After displaying the subject’s distinguished name, OpenSSL prompts you for confirmation to sign the certificate. Since certificate requests are likely to come from people needing certificates from you, you should check to be sure that the information they’ve provided in their certificate requests is correct before issuing the certificate. The next and final prompt is to confirm whether the certificates should be committed to the CA’s database. Finally, the new certificate will be written to stdout, and the command is finished.

The confirmation prompts that are issued can be suppressed and automatically answered in the affirmative by adding the batch option. This is useful if you’re building a wrapper around the OpenSSL command-line tool, or if you’ve already manually verified the information in the request and you don’t want to be prompted. It’s also possible to issue multiple certificates for multiple certificate requests all with one command. For example, suppose you have three certificate requests that need to have certificates issued. The infiles option can be used instead of the in option; the list of files to be processed follows immediately after it. If you use the infiles option, be aware that it must be the last option specified; everything after it is treated as an input filename.

The resulting certificate is also written to the directory that we specified in our configuration file with the new_certs_dir key. It’s written out in PEM format and given a filename composed of the certificate’s serial number and an extension of .pem. The output of the certificate to stdout when it is created can be suppressed by using the notext option. Using the out option, the name of a file to write the certificate to can be specified. We recommend that you also use the notext option if you use the out option. The result will be a file containing a certificate that is identical to the one written to the new_certs_dir directory, /opt/exampleca/certs in our example. If you use the out option, it’ll save you having to search through all of the certificates that you’ve issued and pick the highest numbered among them to pass on to their owners.

After the command has completed and the certificate has been issued, you should see a new file in the subdirectory certs that we created. This file is the certificate that was issued. You should also be able to see that information was added to the file index.txt, OpenSSL’s CA database. Finally, you should be able to see that the serial number in the file serial was incremented. When you look at the text dump of the certificate that was created, you’ll notice that it was assigned a serial number of “1”, the number that we used to seed the serial number file.

The first certificate that we issued with our CA was simply a test certificate to make sure that the CA is working properly. We can see that the certificate was issued properly, but it’s a certificate that we don’t actually want anybody to be able to use, so we will need to revoke the certificate. This provides us with an excellent opportunity to find out how certificate revocation works using OpenSSL’s CA command.

Revoking a certificate is a simple process. All you need is a copy of the certificate to be revoked. Even if you don’t keep a copy of all of the certificates that you’ve issued, the CA infrastructure we created does. We can obtain a copy of the certificate that way, but it’s much easier to keep a copy of your own and name the file something meaningful since the CA simply names the file containing the certificates it issues with each certificate’s serial number. Using the command in the example from the last section to create the test certificate, we didn’t keep a copy for ourselves, but we issued only a single certificate, so it’s quite easy to get a copy of the certificate file. We’ll make a copy of that certificate file in the CA’s root directory and call it testcert.pem. Then we’ll use that file as the certificate required by the revoke option to the ca command (see Example 3-9).

# cp certs/01.pem testcert.pem
# openssl ca -revoke testcert.pem
Using configuration from /opt/exampleca/openssl.cnf
Enter PEM pass phrase:
Revoking Certificate 01.
Data Base Updated

Once again, the command-line tool prompts us for a passphrase. The passphrase it is looking for is the passphrase that protects the CA’s private key. Although the key is not actually used for any signing as part of the certificate revocation process, it is required to validate the certificate as the CA’s own and as a security measure to ensure that only someone authorized to use the CA can revoke a certificate that it has issued.

No change is made to the certificate at all. In fact, the only noticeable change is to the CA’s database to indicate that the certificate has been revoked. Remember that once a certificate has been issued, it cannot be modified. It’s presumably out in the wild and there’s no way to ensure that every copy of the certificate in existence can be updated. This is where CRLs become relevant. We’ve revoked the first certificate that we issued with our CA, but the only entity that is aware of the revocation is the CA itself. By itself, this doesn’t do anybody any good. Anybody that might be using the certificate also needs to know that the certificate has been revoked, so we need to issue a CRL.

When we issue our first CRL, we set the initial policy for how frequently we’ll be issuing CRLs. In our configuration, we’ve indicated that we’ll issue them once a week. When a CRL is made available, it contains a field that indicates the next time a new one will be published. In other words, each CRL is given an expiration date, and a new one must be obtained once the current one expires. Whether there are any new certificate revocations, a new CRL should be generated when the old one expires.

While CRLs should be published on a regularly scheduled periodic basis, it is also possible to generate and publish CRLs when a new one is needed. In fact, it’s good practice to do so. Consider that not all software may cache the CRLs that it retrieves, particularly if they’re retrieved automatically. It’s also possible that the current CRL was most likely not retrieved by everyone that may be using certificates issued by your CA. Therefore, it’s best to make the information as current as possible rather than waiting until the current CRL expires, especially if the period between issuance is large.

Issuing a CRL before a new one is due means there are possibly two or more CRLs from your CA in distribution, but that’s fine. CRLs usually have only data added to them, and any time data is removed, it’s because the revoked certificate has expired, and thus its revocation status is irrelevant. Some CAs may opt to keep even expired certificates in their CRLs. While this may not be a bad idea for a short period of time after a certificate expires, it’s generally not a good idea to keep the information indefinitely; otherwise, the CRL could grow to be quite large and make distribution of it more costly in terms of both time and bandwidth.

Without any further ado, let’s issue our first CRL. This is done by issuing a simple ca command using the gencrl option, along with an out option to specify the name of the file to write the resultant CRL to (see Example 3-10). OpenSSL prompts us for the passphrase protecting the CA’s private key, which it will use to sign the CRL that it generates.

The command completes without writing anything to stdout indicating success, but if there is a problem, an appropriate error message will be written. We can see that the command completed successfully by noting that the file we specified with the -out option has been written. With that file, we can use the command-line tool’s crl command to investigate the details of the CRL that we just generated.

# openssl ca -gencrl -out exampleca.crl
Using configuration from /opt/exampleca/openssl.cnf
Enter PEM pass phrase:
# openssl crl -in exampleca.crl -text -noout
Certificate Revocation List (CRL):
        Version 1 (0x0)
        Signature Algorithm: md5WithRSAEncryption
        Issuer: /CN=Example CA/ST=Virginia/C=US/Email=ca@exampleca.org/O=Root 
Certificate Authority
        Last Update: Jan 14 05:42:08 2002 GMT
        Next Update: Jan 21 05:42:08 2002 GMT
Revoked Certificates:
    Serial Number: 01
        Revocation Date: Jan 14 05:16:43 2002 GMT
    Signature Algorithm: md5WithRSAEncryption
        32:73:3b:e5:b4:f6:2d:57:58:15:e8:87:05:23:27:c3:5d:e5:
        10:a0:5d:1d:09:68:27:b8:8c:70:5c:5d:4a:0d:07:ff:63:09:
        2d:df:61:13:7b:ea:5a:49:74:3b:0a:e9:2b:2d:92:3e:4d:c6:
        f4:4f:18:fa:c9:9e:f7:bb:92:b5:ed:46:14:a1:c2:25:5d:3f:
        9d:5a:b4:c9:63:5f:06:fc:04:22:0b:80:aa:fd:77:a5:16:9d:
        36:47:f7:e9:5b:95:16:ff:bb:e6:db:98:3c:2a:aa:bd:4f:91:
        eb:20:86:44:09:7f:ef:62:69:ef:db:1e:79:7e:24:70:72:34:
        cf:1e
# openssl crl -in exampleca.crl -noout -CAfile cacert.pem
verify OK

When we get a text dump of the CRL, we can see the algorithm that was used to sign it, the CA that issued it, when it was issued, when the next list will be issued, and a list of all of the certificates that it contains. We can also use the crl command to verify the signature on the CRL. Doing so requires us to have a copy of the certificate that was used to sign it.

We can see in Example 3-10 that the version of the CRL that was generated was Version 1. By default, this is what OpenSSL will produce unless the crl_extensions key is specified in the configuration file in the ca section. We strongly recommend that you produce only Version 1 CRLs, unless you can be sure all of the software you’re using with your certificates supports Version 2. If it’s important that the software that supports Version 2 CRLs get them, you can produce both Version 1 and Version 2 lists.

Note that you are essentially on your own when it comes to publishing CRLs. One reasonable solution is to make CRLs available to all via secure HTTP.

The static locking mechanism requires the application to provde two callback functions. In addition to providing an implementation for the functions, the application must tell OpenSSL about them so that it knows to call them when appropriate. The first callback function is used either to acquire or release a lock, and is defined like this:

void locking_function(int mode, int n, const char *file, int line);

The next callback function is used to get a unique identifier for the calling thread. For example, GetCurrentThreadId on Windows will do just that. For reasons that will soon become clear, it is important the value returned from this function be consistent across calls for the same thread, but different for each thread within the same process. The return value from the function should be the unique identifier. The function is defined like this:

unsigned long id_function(void);

Example 4-1 introduces two new OpenSSL library functions: CRYPTO_set_id_callback and CRYPTO_set_locking_callback . These two functions are used to tell OpenSSL about the callbacks that we’ve implemented for the static locking mechanism. We can either pass a pointer to a function to install a callback or NULL to remove a callback.

int THREAD_setup(void);
int THREAD_cleanup(void);
 
#if defined(WIN32)
    #define MUTEX_TYPE HANDLE
    #define MUTEX_SETUP(x)   (x) = CreateMutex(NULL, FALSE, NULL)
    #define MUTEX_CLEANUP(x) CloseHandle(x)
    #define MUTEX_LOCK(x)    WaitForSingleObject((x), INFINITE)
    #define MUTEX_UNLOCK(x)  ReleaseMutex(x)
    #define THREAD_ID        GetCurrentThreadId(  )
#elif _POSIX_THREADS
    /* _POSIX_THREADS is normally defined in unistd.h if pthreads are available
       on your platform. */
    #define MUTEX_TYPE       pthread_mutex_t
    #define MUTEX_SETUP(x)   pthread_mutex_init(&(x), NULL)
    #define MUTEX_CLEANUP(x) pthread_mutex_destroy(&(x))
    #define MUTEX_LOCK(x)    pthread_mutex_lock(&(x))
    #define MUTEX_UNLOCK(x)  pthread_mutex_unlock(&(x))
    #define THREAD_ID        pthread_self(  )
#else
    #error You must define mutex operations appropriate for your platform!
#endif
 
/* This array will store all of the mutexes available to OpenSSL. */
static MUTEX_TYPE mutex_buf[] = NULL;
 
static void locking_function(int mode, int n, const char * file, int line)
{
    if (mode & CRYPTO_LOCK)
        MUTEX_LOCK(mutex_buf[n]);
    else
        MUTEX_UNLOCK(mutex_buf[n]);
}
 
static unsigned long id_function(void)
{
    return ((unsigned long)THREAD_ID);
}
 
int THREAD_setup(void)
{
    int i;
 
    mutex_buf = (MUTEX_TYPE *)malloc(CRYPTO_num_locks(  ) * sizeof(MUTEX_TYPE));
    if (!mutex_buf)
        return 0;
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_SETUP(mutex_buf[i]);
    CRYPTO_set_id_callback(id_function);
    CRYPTO_set_locking_callback(locking_function);
    return 1;
}
 
int THREAD_cleanup(void)
{
    int i;
 
    if (!mutex_buf)
        return 0;
    CRYPTO_set_id_callback(NULL);
    CRYPTO_set_locking_callback(NULL);
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_CLEANUP(mutex_buf[i]);
    free(mutex_buf);
    mutex_buf = NULL;
    return 1;
}

To use these static locking functions, we need to make one function call before our program starts threads or calls OpenSSL functions, and we must call THREAD_setup , which returns 1 normally and 0 if it is unable to allocate the memory required to hold the mutexes. In our example code, we do make a potentially unsafe assumption that the initialization of each individual mutex will succeed. You may wish to add additional error handling code to your programs. Once we’ve called THREAD_setup and it returns successfully, we can safely make calls into OpenSSL from multiple threads. After our program’s threads are finished, or if we are done using OpenSSL, we should call THREAD_cleanup to reclaim any memory used for the mutex structures.

The dynamic locking mechanism requires a data structure (CRYPTO_dynlock_value ) and three callback functions. The structure is meant to hold the data necessary for the mutex, and the three functions correspond to the operations for creation, locking/unlocking, and destruction. As with the static locking mechanism, we must also tell OpenSSL about the callback functions so that it knows to call them when appropriate.

The first thing that we must do is define the CRYPTO_dynlock_value structure. We’ll be building on the static locking support that we built in Example 4-1, so we can use the same platform-dependent macros that we defined already. For our purposes, this structure will be quite simple, containing only one member:

struct CRYPTO_dynlock_value
{
    MUTEX_TYPE mutex;
};

The first callback function that we need to define is used to create a new mutex that OpenSSL will be able to use to protect a data structure. Memory must be allocated for the structure, and the structure should have any necessary initialization performed on it. The newly created and initialized mutex should be returned in a released state from the function. The callback is defined like this:

struct CRYPTO_dynlock_value *dyn_create_function(const char *file, int line);

The next callback function is used for acquiring or releasing a mutex. It behaves almost identically to the corresponding static locking mechanism’s callback, which performs the same operation. It is defined like this:

void dyn_lock_function(int mode, struct CRYPTO_dynlock_value
                      *mutex, const char *file, int line);

The third and final callback function is used to destroy a mutex that OpenSSL no longer requires. It should perform any platform-dependent destruction of the mutex and free any memory that was allocated for the CRYPTO_dynlock_value structure. It is defined like this:

void dyn_destroy_function(struct CRYPTO_dynlock_value *mutex,
                         const char *file, int line);

Using the static locking mechanism’s code from Example 4-1, we can easily build a dynamic locking mechanism implementation. Example 4-2 shows an implementation of the three dynamic locking callback functions. It also includes new versions of the THREAD_setup and THREAD_cleanup functions extended to support the dynamic locking mechanism in addition to the static locking mechanism. The modifications to these two functions are simply to make the appropriate OpenSSL library calls to install and remove the dynamic locking callback functions.

struct CRYPTO_dynlock_value
{
    MUTEX_TYPE mutex;
};
 
static struct CRYPTO_dynlock_value * dyn_create_function(const char *file,
                                                         int line)
{
    struct CRYPTO_dynlock_value *value;
 
    value = (struct CRYPTO_dynlock_value *)malloc(sizeof(
                                                  struct CRYPTO_dynlock_value));
    if (!value)
        return NULL;
    MUTEX_SETUP(value->mutex);
    return value;
}
 
static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value *l,
                              const char *file, int line)
{
    if (mode & CRYPTO_LOCK)
        MUTEX_LOCK(l->mutex);
    else
        MUTEX_UNLOCK(l->mutex);
}
 
static void dyn_destroy_function(struct CRYPTO_dynlock_value *l,
                                 const char *file, int line)
{
    MUTEX_CLEANUP(l->mutexp);
    free(l);
}
 
int THREAD_setup(void)
{
    int i;
 
    mutex_buf = (MUTEX_TYPE *)malloc(CRYPTO_num_locks(  ) * sizeof(MUTEX_TYPE));
    if (!mutex_buf)
        return 0;
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_SETUP(mutex_buf[i]);
    CRYPTO_set_id_callback(id_function);
    CRYPTO_set_locking_callback(locking_function);
 
    /* The following three CRYPTO_... functions are the OpenSSL functions
       for registering the callbacks we implemented above */
    CRYPTO_set_dynlock_create_callback(dyn_create_function);
    CRYPTO_set_dynlock_lock_callback(dyn_lock_function);
    CRYPTO_set_dynlock_destroy_callback(dyn_destroy_function);
 
    return 1;
}
 
int THREAD_cleanup(void)
{
    int i;
 
    if (!mutex_buf)
        return 0;
    CRYPTO_set_id_callback(NULL);
    CRYPTO_set_locking_callback(NULL);
    CRYPTO_set_dynlock_create_callback(NULL);
    CRYPTO_set_dynlock_lock_callback(NULL);
    CRYPTO_set_dynlock_destroy_callback(NULL);
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_CLEANUP(mutex_buf[i]);
    free(mutex_buf);
    mutex_buf = NULL;
    return 1;
}

When an error occurs in the OpenSSL library, a significant amount of information is logged. Some of the information can be useful in attempting to recover from an error automatically, but much of it is for debugging and reporting the error to a user.

The ERR package provides six basic functions that are useful for obtaining information from the error queue. Each function always retrieves the oldest information from the queue so that errors are returned in the order that they were generated. The most basic piece of information that is logged is an error code, which describes the error that occurred. The error code is a 32-bit integer that has meaning only to OpenSSL. That is, OpenSSL defines its own unique error codes for any error condition that it could possibly encounter. It does not rely on error codes defined by any other library, including the standard C runtime. For each of the six basic functions, this error code is the return value from the function. If there is no error in the queue, the return from any of them will be 0, which also tells us that 0 is never a valid error code.

This first function retrieves only the error code from the error queue. It also removes that error report from the queue, so the next call will retrieve the next error that occurred or possibly 0 if there are no more errors in the queue:

unsigned long ERR_get_error(void);

The second function also retrieves only the error code from the error queue, but it does not remove the error report from the queue, so the next call will retrieve the same error:

unsigned long ERR_peek_error(void);

The third function builds on the information returned by ERR_get_error and ERR_peek_error. In addition to returning the error code, it also returns the name of the source file and source line number that generated the error. Like ERR_get_error, it also removes the error report from the queue:

unsigned long ERR_get_error_line(const char **file, int *line);

The fourth function returns the same information as ERR_get_error_line, but like ERR_peek_error, it does not remove the error report from the queue. Its arguments and their meanings are identical to ERR_get_error_line:

unsigned long ERR_peek_error_line(const char **file, int *line);

The fifth function builds on the information returned by ERR_get_error_line and ERR_peek_error_line. In addition to returning the error code, source filename, and line number, it also returns extra data and a set of flags that indicate how that data should be treated. The extra data and flags are supplied when the error is generated. Like ERR_get_error and ERR_get_error_line, this function also removes the error report from the queue:

unsigned long ERR_get_error_line_data(const char **file, int *line,
                                      const char **data, int *flags);

The sixth function returns the same information as ERR_get_error_line_data, but like ERR_peek_error and ERR_peek_error_line, it does not remove the error report from the queue. Its arguments and their meanings are identical to ERR_get_error_line_data:

unsigned long ERR_peek_error_line_data(const char **file, int *line,
                const char **data, int *flags);

ERR_get_error_line_data and ERR_peek_error_line_data both retrieve the optional piece of data that can be associated with an error report. This optional piece of data can be anything, but most frequently, it’s a string. Stored along with the data is a bit mask of flags that describe the data so that it can be dealt with appropriately by the error handling package. If the flag ERR_TXT_MALLOCED is set, the memory for the data will be freed by a call to OpenSSL’s OPENSSL_free function. If the flag ERR_TXT_STRING is set, the data is safe to be interpreted as a C-style string.

Note that the file and data information that can be obtained from the queue is returned as a pointer to the information on the queue. It is not a copy, so you should not attempt to modify the data. In the case of the file information, it is usually a constant string from the _ _FILE_ _ preprocessor macro. For the data information, if you need to store the information for any reason, you should make a copy and not store the pointer that is returned. When you use the “get” family of functions to obtain this data, the data remains valid for a short period, but you should be sure to make a copy before any other error handler function is called if you need to preserve it. Example 4-3 demonstrates how to print out the error information that is in the calling thread’s error queue.

void print_errors(void)
{
    int           flags, line;
    char          *data, *file;
    unsigned long code;
 
    code = ERR_get_error_line_data(&file, &line, &data, &flags);
    while (code)
    {
        printf("error code: %lu in %s line %d.\n", code, file, line);
        if (data && (flags & ERR_TXT_STRING))
            printf("error data: %s\n", data);
        code = ERR_get_error_line_data(&file, &line, &data, &flags);
    }
}

There is one last queue-manipulation function that we’ll discuss here: the function for clearing the error queue. It will delete all errors currently in the queue. In general, there is no need to call this function unless we are trying to reset the error status for the current thread and don’t care about any other errors that are on the queue. There is no way to recover the previous errors once it’s been called, so use it judiciously:

void ERR_clear_error(void);

In some cases, the most appropriate way to handle an error condition is to display or log the error so that the user of your application can take the necessary steps to resolve the error. To do that, it’s best to display a human-readable error message rather than an error code. The error handling package provides standard error messages for its error codes for just this purpose, but before they can be used, they must be loaded.

There are two sets of error messages: one for the errors generated by libcrypto , and one for the errors generated by libssl. The function ERR_load_crypto_strings loads the errors generated by libcrypto, and the function ERR_load_SSL_strings loads the errors generated by libssl. There is an additional function, SSL_load_error_strings , which will load both sets of error messages.

Once the error strings are loaded, ERR_error_string and ERR_error_string_n can be used to translate an error code into an error message that is more meaningful to humans. Particularly in a multithreaded application, ERR_error_string should never be used. It is always best to use ERR_error_string_n. Both functions always return a pointer to the start of the buffer into which the translated error message was written.

char *ERR_error_string(unsigned long e, char *buf);

char *ERR_error_string_n(unsigned long e, char *buf, size_t len);

The resultant error message is formatted into a colon-separated list of fields. The first field is always the word “error”, and the second field is always the error code represented in hexadecimal. The third field is the name of the package that generated the error, such as “BIO routines” or “bignum routines”. The fourth field is the name of the function that generated the error, and the fifth field is the reason why the error was generated. The function name is taken from an internal table that is actually rather small, and may very likely be represented as func(<code>), in which code is a number representing the function.

To get information about an error, ERR_get_error_line_data and ERR_error_string should be used. Armed with all of the information from these two functions, we can emit rather detailed error information. The OpenSSL library provides us with two functions that ease this process for us, however. ERR_print_errors will produce an error listing and write it to a BIO. ERR_print_errors_fp will produce an error listing and write it to a standard C runtime FILE object. The error listings are produced by iterating through each error report in the error queue and removing them as it goes. For each error report, ERR_get_error_line_data and ERR_error_string are used to obtain the information necessary to produce the listing:

void ERR_print_errors(BIO *bp);

void ERR_print_errors_fp(FILE *fp);

A common concern of developers is the handling of errors produced by a library when using threaded code, and rightly so. With a few exceptions that can be easily avoided, OpenSSL’s error handling is completely thread-safe. Each thread is assigned its own error queue, which is one of the reasons why the id_function callback that we described earlier in the chapter must return a different identifier for each thread. Each error queue will contain only errors that were caused by that thread. This is convenient for threaded applications because the programmer doesn’t need to do anything special to handle errors correctly.

By creating a separate error queue for each thread, it would seem that all the bases are covered for error handling, but that’s not entirely true. OpenSSL does not use thread-local storage for the error queues, and so there is no way for each queue to be automatically destroyed when a thread terminates. Thread-local storage is a great feature to have in a multithreaded environment, but unfortunately, it is not supported on all platforms. The bottom line is that the application is responsible for destroying a thread’s error queue when a thread terminates because OpenSSL has no way of knowing on its own when a thread has terminated.

OpenSSL provides a function to destroy a thread’s error queue called ERR_remove_state . It should be called by a thread just before it terminates, or it may be called by another thread within the process after the thread has terminated. The function requires a single argument that is the identifier of the thread as it would be returned by the id_function callback that we described earlier in the chapter.

Until now, we have overlooked the implications of loading the strings for error processing. These strings do take up memory, and it isn’t always appropriate to load them. It should be mentioned that all of the error handling routines work properly without the strings loaded. The translated error messages will merely have internal OpenSSL codes inserted instead of the more meaningful strings. If we do choose to load the error strings, we should also be sure to free them when they’re no longer needed by calling ERR_free_strings. For most applications, this should happen after the program is done making calls into the OpenSSL library.

A BIO that is used for reading is known as a source BIO, and a sink BIO is one that is used for writing. A source/sink BIO is attached to a concrete input/output medium such as a file, a socket, or memory. Only a single source/sink BIO may exist in a chain. It is possible to conceive of situations in which it might be useful to have more than one, particularly for writing, but the source/sink types of BIOs provided by OpenSSL do not currently allow for more than one source/sink BIO to exist in a chain.

OpenSSL provides nine source/sink types of BIOs that can be used with BIO_new and BIO_set. A function is provided for each that simply returns a BIO_METHOD object suitable for passing to BIO_new or BIO_set. Most of the source/sink types of BIOs require additional setup work beyond just creating a BIO with the appropriate BIO_METHOD. We’ll cover only the four most commonly used types in any detail here due to space limitations and the huge number of individual functions that are available to operate on them in various ways.

/* Create a read/write BIO */
bio = BIO_new(BIO_s_mem(  ));
 
/* Create a read-only BIO using an allocated buffer */
buffer = malloc(4096);
bio = BIO_new_mem_buf(buffer, 4096);
 
/* Create a read-only BIO using a C-style string */
bio = BIO_new_mem_buf("This is a read-only buffer.", -1);
 
/* Get a pointer to a memory BIO's memory segment */
BIO_get_mem_ptr(bio, &buffer);
 
/* Prevent a memory BIO from destroying its memory segment when it is destroyed
 */
BIO_set_close(bio, BIO_NOCLOSE);

/* Create a buffered file BIO with an existing FILE object that will be closed
   when the BIO is destroyed. */
file = fopen("filename.ext", "r+");
bio = BIO_new(BIO_s_file(  ));
BIO_set_file(bio, file, BIO_CLOSE);
 
/* Create an unbuffered file BIO with an existing file descriptor that will not
   be closed when the BIO is destroyed. */
fd = open("filename.ext", O_RDWR);
bio = BIO_new(BIO_s_fd(  ));
BIO_set_fd(bio, fd, BIO_NOCLOSE);
 
/* Create a buffered file BIO with a new FILE object owned by the BIO */
bio = BIO_new_file("filename.ext", "w");
 
/* Create an unbuffered file BIO with an existing file descriptor that will be
   closed when the BIO is destroyed. */
fd = open("filename.ext", O_RDONLY);
bio = BIO_new_fd(fd, BIO_CLOSE);

/* Create a socket BIO attached to an already existing socket descriptor.  The
   socket descriptor will not be closed when the BIO is destroyed. */
bio = BIO_new(BIO_s_socket(  ));
BIO_set_fd(bio, sd, BIO_NOCLOSE);
 
/* Create a socket BIO attached to an already existing socket descriptor.  The
   socket descriptor will be closed when the BIO is destroyed. */
bio = BIO_new_socket(sd, BIO_CLOSE);
 
/* Create a socket BIO to establish a connection to a remote host. */
bio = BIO_new(BIO_s_connect(  ));
BIO_set_conn_hostname(bio, "www.ora.com");
BIO_set_conn_port(bio, "http");
BIO_do_connect(bio);
 
/* Create a socket BIO to listen for an incoming connection. */
bio = BIO_new(BIO_s_accept(  ));
BIO_set_accept_port(bio, "https");
BIO_do_accept(bio); /* place the underlying socket into listening mode */
for (;;)
{
    BIO_do_accept(bio); /* wait for a new connection */
    new_bio = BIO_pop(bio);
    /* new_bio now behaves like a BIO_s_socket(  ) BIO */
}

a = BIO_new(BIO_s_bio(  ));
BIO_set_write_buf_size(a, 4096);
b = BIO_new(BIO_s_bio(  ));
BIO_set_write_buf_size(b, 4096);
BIO_make_bio_pair(a, b);
 
BIO_new_bio_pair(&a, 8192, &b, 8192);
 
c = BIO_new(BIO_s_bio(  ));
BIO_set_write_buf_size(c, 1024);
BIO_destroy_bio_pair(a); /* disconnect a from b */
BIO_make_bio_pair(a, c);

A filter BIO by itself provides no utility. It must be chained with a source/sink BIO and possibly other filter BIOs to be useful. The ability to chain filters with other BIOs is perhaps the most powerful feature of OpenSSL’s BIO package, and it provides a great deal of flexibility. A filter BIO often performs some kind of translation of data before writing to or after reading from a concrete medium, such as a file or socket.

Creating BIO chains is reasonably simple and straightforward; however, care must be taken to keep track of the BIO that is at the end of the chain so that the chain can be manipulated and destroyed safely. If you destroy a BIO that is in the middle of a chain without first removing it from the chain, it’s a safe bet that your program will crash shortly thereafter. As we mentioned earlier, the BIO package is one of OpenSSL’s lower-level packages, and as such, little error checking is done. This places the burden on the programmer to be sure that any operations performed on a BIO chain are both legal and error-free.

When creating a chain, you must also ensure that you create the chain in the proper order. For example, if you use filters that perform base64 conversion and encryption, you would probably want to perform base64 encoding after encryption, not before. It’s also important to ensure that your source/sink BIO is at the end of the chain. If it’s not, none of the filters in the chain will be used.

The interface for creating a filter BIO is similar to creating source/sink BIO. BIO_new is used to create a new BIO with the appropriate BIO_METHOD object. Filter BIOs are provided by OpenSSL for performing encryption and decryption, base64 encoding and decoding, computing message digests, and buffering. There are a handful of others as well, but they are of limited use, since they are either platform-specific or meant for testing the BIO package.

The function shown in Example 4-8 can be used to write data to a file using the BIO package. What’s interesting about the function is that it creates a chain of four BIOs. The result is that the data written to the file is encrypted and base64 encoded with the base64 encoding performed after the data is encrypted. The data is first encrypted using outer triple CBC DES and the specified key. The encrypted data is then base64-encoded before it is written to the file through an in-memory buffer. The in-memory buffer is used because triple CBC DES is a block cipher, and the two filters cooperate to ensure that the cipher’s blocks are filled and padded properly. Chapter 6 discusses symmetric ciphers in detail.

int write_data(const char *filename, char *out, int len, unsigned char *key)
{
    int total, written;
    BIO *cipher, *b64, *buffer, *file;
 
    /* Create a buffered file BIO for writing */
    file = BIO_new_file(filename, "w");
    if (!file)
        return 0;
 
    /* Create a buffering filter BIO to buffer writes to the file */
    buffer = BIO_new(BIO_f_buffer(  ));
 
    /* Create a base64 encoding filter BIO */
    b64 = BIO_new(BIO_f_base64(  ));
 
    /* Create the cipher filter BIO and set the key.  The last parameter of
       BIO_set_cipher is 1 for encryption and 0 for decryption */
    cipher = BIO_new(BIO_f_cipher(  ));
    BIO_set_cipher(cipher, EVP_des_ede3_cbc(  ), key, NULL, 1);
 
    /* Assemble the BIO chain to be in the order cipher-b64-buffer-file */
    BIO_push(cipher, b64);
    BIO_push(b64, buffer);
    BIO_push(buffer, file);
 
    /* This loop writes the data to the file.  It checks for errors as if the
       underlying file were non-blocking */
    for (total = 0;  total < len;  total += written)
    {
        if ((written = BIO_write(cipher, out + total, len - total)) <= 0)
        {
            if (BIO_should_retry(cipher))
            {
                written = 0;
                continue;
            }
            break;
        }
    }
 
    /* Ensure all of our data is pushed all the way to the file */
    BIO_flush(cipher);
 
    /* We now need to free the BIO chain. A call to BIO_free_all(cipher) would
       accomplish this, but we'll first remove b64 from the chain for
       demonstration purposes. */
    BIO_pop(b64);
 
    /* At this point the b64 BIO is isolated and the chain is cipher-buffer-file.
       The following frees all of that memory */
    BIO_free(b64);
    BIO_free_all(cipher);
}

A common security pitfall is the incorrect seeding of the OpenSSL PRNG. There are functions that seed the generator easily enough, but the problems occur when a developer uses some predictable data for the seed. While the internal routines can quantify the amount of “seed” data, they can do nothing to determine the quality of that data (i.e., how much entropy the data contains). We’ve stated that the seed is an important value, but we haven’t explicitly looked at why this is so. For example, when using a session key to secure a connection, the basis for security is both the encryption algorithm used to encrypt the messages and the inability of the attacker to simply guess the session key. If an insecure seed is used, the PRNG output is predictable. If the output is predictable, the keys generated are predictable; thus the security of even a correctly designed application will be compromised. Clearly, a lot depends on the PRNG’s output and as such, OpenSSL provides several functions for manipulating it. It’s important to understand how to use these functions so that security can be assured.

The function RAND_add seeds the PRNG with the specified data, considering only the specified number of bytes to be entropic. For example, suppose the buffer contained a pointer to the current time as returned by the standard C runtime function, time. The buffer size would be four bytes, but only a single byte of that could be reasonably considered entropic because the high bytes don’t change frequently and are extremely predictable. The current time by itself is never a good source of entropy; we’ve only used it here for clarity.

void RAND_add(const void *buf, int num, double entropy);

Like RAND_add, the function RAND_seed seeds the PRNG with the specified data, but considers it to contain pure entropy. In fact, the default implementation of RAND_seed is simply to call RAND_add using the number of bytes in the buffer as the amount of entropy contained in the buffer’s data.

void RAND_seed(const void *buf, int num);

Two additional functions are provided for use on Windows systems. They’re not the best sources of entropy, but lacking a better source, they’re better than what most programmers would typically use or devise on their own. In general, it’s a good idea to avoid using either of these two functions unless there is no other entropy source available, especially if your application is running on a machine that ordinarily has no user interaction, such as a server. They’re intended to be a last resort, and you should treat them as such.

int  RAND_event(UINT iMsg, WPARAM wParam, LPARAM lParam);

RAND_event should be called from message handling functions and pass each message’s identifier and parameters. The current implementation uses only the WM_KEYDOWN and WM_MOUSEMOVE messages for gathering entropy.

void RAND_screen(void);

RAND_screen can be called periodically to gather entropy as well. The function will take a snapshot of the contents of the screen, generate a hash for each scan-line, and use the hash value as entropy. This function should not be called too frequently for a couple of reasons. One reason is that the screen won’t change much, which can lead to predictability. The other reason is that the function is not particularly fast.

A common misuse of the PRNG seeding functions is to use a static string as the seed buffer. Most often, this is done for no reason other than to silence OpenSSL because it will generate warning messages whenever the PRNG is not seeded and an attempt to use it is made. Another bad idea is to use an uninitialized memory segment, assuming its contents will be unpredictable enough. There are plenty of other examples of how not to seed the PRNG, but rather than enumerate them all here, we’ll concentrate on the right way. A good rule of thumb to determine whether you’re seeding the PRNG correctly is this: if you’re not seeding it with data from a service whose explicit purpose is to gather entropy, you’re not seeding the PRNG correctly.

On many Unix systems, /dev/random is available as an entropy-gathering service. On systems that provide such a device, there is usually another device, /dev/urandom. The reason for this is that the /dev/random device will block if there is not enough entropy available to produce the output requested. The /dev/urandom device, on the other hand, will use a cryptographic PRNG to assure that it never blocks. It’s actually most accurate to say that /dev/random produces entropy and that /dev/urandom produces pseudorandom numbers.

The OpenSSL package provides a function, RAND_load_file , which will seed the PRNG with the contents of a file up to the number of bytes specified, or its entirety if the limit is specified as -1. It is expected that the file read will contain pure entropy. Since OpenSSL has no way of knowing whether the file actually does contain pure entropy, it assumes that the file does; OpenSSL leaves it to the programmer. Example 4-9 shows some example uses of this function and its counterpart, RAND_write_file . On systems that do have /dev/random available, seeding the PRNG with RAND_load_file from /dev/random is the best thing to do. Be sure to limit the number of bytes read from /dev/random to some reasonable value, though! If you specify -1 to read the entire file, RAND_load_file will read data forever and never return.

The RAND_write_file function will write 1,024 bytes of random bytes obtained from the PRNG to the specified file. The bytes written are not purely entropic, but they can be safely used to seed an unseeded PRNG in the absence of a better entropy source. This can be particularly useful for a server that starts running immediately when a system boots up because /dev/random will not have much entropy available when the system first boots. Example 4-9 demonstrates various methods of employing RAND_load_file and RAND_write_file.

int RAND_load_file(const char *filename, long bytes);
int RAND_write_file(const char *filename);
 
/* Read 1024 bytes from /dev/random and seed the PRNG with it */
RAND_load_file("/dev/random", 1024);
 
/* Write a seed file */
RAND_write_file("prngseed.dat");
 
/* Read the seed file in its entirety and print the number of bytes obtained */
nb = RAND_load_file("prngseed.dat", -1);
printf("Seeded the PRNG with %d byte(s) of data from prngseed.dat.\n", nb);

When you write seed data to a file with RAND_write_file, you must be sure that you’re writing the file to a secure location. On a Unix system, this means the file should be owned by the user ID of the application, and all access to group members and other users should be disallowed. Additionally, the directory in which the file resides and all parent directories should have only write access enabled for the directory owner. On a Windows system, the file should be owned by the Administrator and allow no permissions to any other users.

One final point worth mentioning is that OpenSSL will try to seed the PRNG transparently with /dev/urandom on systems that have it available. While this is better than nothing, it’s a good idea to go ahead and read better entropy from /dev/random, unless there is a compelling reason not to. On systems that don’t have /dev/urandom, the PRNG will not be seeded at all, and you must make sure that you seed it properly before you attempt to use the PRNG or any other part of OpenSSL that utilizes the PRNG. For systems that have /dev/random, Example 4-10 demonstrates how to use it to seed the OpenSSL PRNG.

int seed_prng(int bytes)
{
    if (!RAND_load_file("/dev/random", bytes))
        return 0;
    return 1;
}

We’ve discussed /dev/random and /dev/urandom as entropy sources at some length, but what about systems that don’t have these services available? Many operating systems do not provide them, including Windows. Obtaining entropy on such systems can be problematic, but luckily, there is a solution. Several third-party packages are available for various platforms that perform entropy-gathering services. One of the more full-featured and portable solutions available is EGADS (Entropy Gathering and Distribution System). It’s licensed under the BSD license, which means that it’s free and the source code is available. You can obtain a copy of EGADS from http://www.securesw.com/egads/.

As we mentioned, there are other entropy solutions available in addition to EGADS. EGD is an entropy-gathering daemon that is written in Perl by Brian Warner and is available from http://egd.sourceforge.net/. Because it is written in Perl, it requires a Perl interpreter to be installed. It provides a Unix domain socket interface for clients to obtain entropy. It does not support Windows at all. PRNGD is another popular entropy-gathering daemon written by Lutz Jänicke. It provides an EGD-compatible interface for clients to obtain entropy from it; like EGD itself, Windows is not supported. Because neither EGD nor PRNGD support Windows, we’ll concentrate primarily on EGADS, which does support Windows. Where appropriate, we will also discuss EGD and PRNGD together, because all three use the same interface.

Before we can use EGADS to obtain entropy, we must first initialize it. This is done with a simple call to egads_init. Once the library is initialized, we can use the function egads_entropy to obtain entropy. Like /dev/random on systems that make it available, egads_entropy will block until enough entropy is available to satisfy the request. Example 4-11 shows how to use EGADS to seed OpenSSL’s PRNG.

int seed_prng(int bytes)
{
    int       error;
    char      *buf;
    prngctx_t ctx;
 
    egads_init(&ctx, NULL, NULL, &error);
    if (error)
        return 0;
 
    buf = (char *)malloc(bytes);
    egads_entropy(&ctx, buf, bytes, &error);
    if (!error)
        RAND_seed(buf, bytes);
    free(buf);
 
    egads_destroy(&ctx);
    return (!error);
}

EGADS, EGD, and PRNGD all provide a Unix domain socket that allows clients to obtain entropy. EGD defines a simple protocol for clients to communicate with that both EGADS and PRNGD have mimicked. Many cryptographic applications, such as GnuPG and OpenSSH, provide support for obtaining entropy from a daemon using the EGD protocol. OpenSSL also provides support for seeding its PRNG using the EGD protocol.

OpenSSL provides two functions for communicating with a server that speaks the EGD protocol. Version 0.9.7 of OpenSSL adds a third. In addition, Version 0.9.7 will attempt to automatically connect to four different commonly used names for EGD sockets in the following order: /var/run/egd-pool, /dev/egd-pool, /etc/egd-pool, and /etc/entropy.

RAND_egd attempts to connect to the specified Unix domain socket. If the connection is successful, 255 bytes of entropy will be requested from the server. The data returned will be passed in a call to RAND_add to seed the PRNG. RAND_egd is actually a wrapper around the next function, RAND_egd_bytes .

int RAND_egd(const char *path);

RAND_egd_bytes will attempt to connect to the specified Unix domain socket. If the connection is successful, the specified number of bytes of entropy will be requested from the server. The data returned will be passed in a call to RAND_add to seed the PRNG. Both RAND_egd and RAND_egd_bytes will return the number of bytes obtained from the EGD server if they’re successful. If an error occurred connecting to the daemon, they’ll both return -1.

int RAND_egd_bytes(const char *path, int bytes);

Version 0.9.7 of OpenSSL adds the function RAND_query_egd_bytes to make a query for data from an EGD server without automatically feeding the returned data into OpenSSL’s PRNG via RAND_add. It attempts to connect to the specified Unix domain socket and obtain the specified number of bytes. The data that is returned from the EGD server is copied into the specified buffer. If the buffer is specified as NULL, the function works just like RAND_egd_bytes and passes the returned data to RAND_add to seed the PRNG. It returns the number of bytes received on success; otherwise, it returns -1 if an error occurs.

int RAND_query_egd_bytes(const char *path, unsigned char *buf, int bytes);

Example 4-12 demonstrates how to use the RAND functions to access an EGD socket and seed the PRNG with the entropy that is obtained from a running entropy-gathering server, whether it’s EGADS, EGD, PRNGD, or another server that provides an EGD socket interface.

#ifndef DEVRANDOM_EGD
#define DEVRANDOM_EGD "/var/run/egd-pool", "/dev/egd-pool", "/etc/egd-pool", \
                      "/etc/entropy"
#endif
 
int seed_prng(int bytes)
{
    int  i;
    char *names[] = { DEVRANDOM_EGD, NULL };
 
    for (i = 0;  names[i];  i++)
        if (RAND_egd(names[i]) != -1)  /* RAND_egd_bytes(names[i], 255) */
            return 1;
    return 0;
}

To use the BIGNUM package in your programs, you’ll need to include the header file openssl/bn.h. Before we can use a BIGNUM , we must first initialize it. The BN package provides support for both statically allocated and dynamically allocated BIGNUMs. The function BN_new will allocate a new BIGNUM and initialize it for use. The function BN_init will initialize a statically allocated BIGNUM. When you’re done using a BIGNUM, you should always be sure to destroy it, even if it is allocated on the stack, because internally, a BIGNUM dynamically allocates memory. The function BN_free will destroy a BIGNUM. Example 4-13 shows some examples of how these three functions are used.

BIGNUM static_bn, *dynamic_bn;
 
/* Initialize a statically allocated BIGNUM */
BN_init(&static_bn);
 
/* Allocate an initialize a new BIGNUM */
dynamic_bn = BN_new(  );
 
/* Destroy the two BIGNUMs that we just created */
BN_free(dynamic_bn);
BN_free(&static_bn);

A BIGNUM is implemented as an opaque structure that contains dynamically allocated memory. The functions provided to operate on a BIGNUM allow the programmer to remain blissfully unaware of what’s going on for the most part, but it is important that the programmer does understand that there is much more going on internally. One such instance is when the value of one BIGNUM needs to be assigned to another BIGNUM. The natural inclination is to perform a shallow copy of the structure, but this should never be done! Deep copies must be performed, and the BN package provides functions for doing just that. Example 4-14 demonstrates the right way and the wrong way to make a copy of a BIGNUM.

BIGNUM a, b *c;
 
/* First, the wrong way to copy a BIGNUM */
a = b;
*c = b;
 
/* Now the right way to copy a BIGNUM */
BN_copy(&a,&b);    /* Copies b to a */
c = BN_dup(&b);    /* Creates c and initializes it to the same value as b */

It is important that we copy BIGNUMs properly. If we don’t, we’re likely to experience unpredictable behavior or crashes. When programming with BIGNUMs, there will likely be situations in which you’ll need to make copies, so it’s best to learn this lesson early.

Another operation that is similar in nature is the comparison of two BIGNUM s. We cannot simply compare two BIGNUMs using normal C comparison operators like =, <, or >. Instead, we must use the BN_cmp function to compare two BIGNUMs. This function will compare the two values, a and b, and return -1 if a is less than b, 0 if they are equal, and 1 if a is greater than b. The function BN_ucmp will perform the same type of comparison on the absolute values of a and b.

It may be useful to convert a BIGNUM into a flat binary representation for the purpose of storing it in a file or sending it over a socket connection to a peer. Since a BIGNUM contains pointers to internal, dynamically allocated memory, it is not a flat structure, so we must convert it to one before we can send it anywhere. Conversely, we must be able to convert a flat representation of a BIGNUM back into a BIGNUM structure that the BN package can use. Two functions are provided for performing these respective operations. The first, BN_bn2bin , will convert a BIGNUM to a flat binary representation in big-endian form. The second, BN_bin2bn , performs the inverse operation, converting a flat binary representation of a big-endian number into a BIGNUM.

Before converting a BIGNUM into a flat binary representation, we need to know the number of bytes of memory that will be required to hold the converted data. It’s also important to know how big the binary representation is before converting it back into a BIGNUM. The number of bytes required to represent a BIGNUM in flat binary form can be discovered using the BN_num_bytes function. Example 4-15 demonstrates converting between BIGNUM and flat binary representations.

/* Converting from BIGNUM to binary */
len = BN_num_bytes(num);
buf = (unsigned char *)malloc(len);
len = BN_bn2bin(num, buf);
 
/* Converting from binary to BIGNUM */
BN_bin2bn(buf, len, num);
num = BN_bin2bn(buf, len, NULL);

When BN_bn2bin performs its conversion, it will return the number of bytes that were written out into the supplied buffer. If an error occurs in the conversion, the return will be 0. When BN_bin2bn performs its conversion, the result is placed into the BIGNUM specified as the third argument, overwriting any value that it may have previously held. If the third argument is specified as NULL, a new BIGNUM is created and initialized with the value from the binary representation. In either case, the BN_bin2bn will always return a pointer to the BIGNUM that received the value or NULL if an error occurred during the conversion.

Binary-encoded numbers are fine when we want to transfer the data over a medium that supports it—for example, a binary file or a socket. However, for circumstances in which we need a text-based representation, such as printing the number on the screen, it is inadequate. We can always base64-encode the data before emitting it, but the BN package provides more intuitive methods.

The function BN_bn2hex converts a BIGNUM into a hexadecimal representation stored in a C-style string. The C-style string is allocated dynamically using OPENSSL_malloc , which must then be freed by the caller using OPENSSL_free .

char *BN_bn2hex(const BIGNUM *num);

The function BN_bn2dec converts a BIGNUM into a decimal representation stored in a C-style string. The C-style string is allocated dynamically using OPENSSL_malloc, which must then be freed by the caller using OPENSSL_free.

char *BN_bn2dec(const BIGNUM *num);

The function BN_hex2bn converts a hexadecimal representation of a number stored in a C-style string to a BIGNUM. The resulting value is stored in the supplied BIGNUM, or a new BIGNUM is created with BN_new if the BIGNUM is supplied as NULL.

int BN_hex2bn(BIGNUM **num, const char *str);

The function BN_dec2bn converts a decimal representation of a number stored in a C-style string to a BIGNUM. The resulting value is stored in the supplied BIGNUM, or a new BIGNUM is created with BN_new if the BIGNUM is supplied as NULL.

int BN_dec2bn(BIGNUM **num, const char *str);

With few exceptions, the majority of the remainder of the functions that make up the BN package all perform mathematical operations such as addition and multiplication. Most of the functions require at least two BIGNUM operands and store their result in a third BIGNUM. It is often safe to use one of the operands to store the result, but it isn’t always, so you should exercise care in doing so. Consult Table 4-1, which lists the most common arithmetic functions, if you’re not sure. Unless otherwise noted in the table, the BIGNUM that will receive the result of the operation may not be the same as any of the operands. Each of the functions returns nonzero or zero, indicating success or failure, respectively.

Many of the functions in Table 4-1 are shown as accepting an argument labeled “ctx”. This argument is a pointer to a BN_CTX structure. This argument may not be specified as NULL, but instead should be a context structure returned by BN_CTX_new. The purpose of the context structure is to store temporary values used by many of the arithmetic operations. Storing the temporary values in a context increases the performance of the various functions. When a context structure is no longer needed, it should be destroyed using BN_CTX_free .

One of the functions provided by the BN package that is most import to public key cryptography is BN_generate_prime . As its name implies, the function generates prime numbers, but more importantly, it generates pseudorandom primes. In other words, it repeatedly chooses numbers at random until one of the choices it makes is a prime number. Such a function can be quite useful for other applications as well, which is one of the reasons why we’ve chosen to pay so much attention to it in this chapter. Another reason is because its parameter list is rather large and complex, which can make using the function seem to be a daunting task.

BIGNUM *BN_generate_prime(BIGNUM *ret, int bits, int safe, BIGNUM *add,
               BIGNUM *rem, void (*callback)(int, int, void *), void *cb_arg);

If one is used, the callback function should accept three arguments and return no value. The third argument to the callback function is always the cb_arg argument to BN_generate_prime. The first argument passed to the callback function is a status code indicating which phase of the prime generation has just completed. The status code will always be 0, 1, or 2. The meaning of the second argument depends on the status code. When the status code is 0, it indicates that a potential prime has been found, but it has not yet been tested to ensure that it conforms to the criteria specified in the call to BN_generate_prime. The callback can be called with a status code of 0 many times, and each time the second argument will contain a counter of the number of primes that have been found so far, not including the current one. When the status code is 1, the second argument indicates the number of Miller-Rabin probabilistic primality tests that have been completed. Finally, when the status code is 2, a conforming prime has been found, and the second argument indicates the number of candidates that were tested before it. Example 4-16 demonstrates how to use the BN_generate_prime function with a callback for displaying the status of the process.

static void prime_status(int code, int arg, void *cb_arg)
{
    if (code == 0)
        printf("\n  * Found potential prime #%d ...", (arg + 1));
    else if (code == 1 && arg && !(arg % 10))
        printf(".");
    else
        printf("\n Got one!\n");
}
 
BIGNUM *generate_prime(int bits, int safe)
{
    char   *str;
    BIGNUM *prime;
 
    printf("Searching for a %sprime %d bits in size ...", (safe ? "safe " : ""),
           bits);
 
    prime = BN_generate_prime(NULL, bits, safe, NULL, NULL, prime_status, NULL);
    if (!prime)
        return NULL;
 
    str = BN_bn2dec(prime);
    if (str)
    {
        printf("Found prime: %s\n", str);
        OPENSSL_free(str);
    }
 
    return prime;
}

We will be using two very simple applications: a client and server in which the server simply echoes data from the client to the console. Our goal is to augment our two applications so that they can perform their tasks in a hostile environment. In other words, we’ll implement each program to strictly authenticate connecting peers. As we walk the path to creating SSL-enabled versions of our programs, we discuss the choices that a developer must make at each stage.

Before moving forward, let’s look at our sample applications. There a total of four files: common.h, common.c, client.c, and server.c. The code for each is presented in Example 5-1 through Example 5-4. Also, we use the code presented in Example 4-2 so we can use multiple threads. For Unix systems, we’ll continue to use POSIX threads.

Starting with common.h in Example 5-1, Lines 1-5 include relevant headers from OpenSSL. Right now, we don’t make use of anything from a few of these headers, but we soon will, so they are included. Lines 22-24 define the strings for the client and server machines as well as the server’s listening port. In addition, the header contains some definitions for convenient error handling and for threading in a platform-independent manner similar to the definitions set forth in Chapter 4’s threading examples.

  1  #include <openssl/bio.h>
  2  #include <openssl/err.h>
  3  #include <openssl/rand.h>
  4  #include <openssl/ssl.h>
  5  #include <openssl/x509v3.h>
  6  
  7  #ifndef WIN32
  8  #include <pthread.h>
  9  #define THREAD_CC
 10  #define THREAD_TYPE                    pthread_t
 11  #define THREAD_CREATE(tid, entry, arg) pthread_create(&(tid), NULL, \
 12                                                        (entry), (arg))
 13  #else
 14  #include <windows.h>
 15  #define THREAD_CC                      _  _cdecl
 16  #define THREAD_TYPE                    DWORD
 17  #define THREAD_CREATE(tid, entry, arg) do { _beginthread((entry), 0, (arg));\
 18                                              (tid) = GetCurrentThreadId(  );   \
 19                                         } while (0)
 20  #endif
 21   
 22  #define PORT            "6001"
 23  #define SERVER          "splat.zork.org"
 24  #define CLIENT          "shell.zork.org"
 25   
 26  #define int_error(msg)  handle_error(__FILE__, __LINE_  _, msg)
 27  void handle_error(const char *file, int lineno, const char *msg);
 28   
 29  void init_OpenSSL(void);

Example 5-2, the file common.c , defines our error reporting function handle_error . The error handling in our example applications is a bit draconian, and you’ll most likely want to handle errors in your own applications in a much more user-friendly manner. In general, it’s not appropriate to handle all possible errors by bringing the application to such an abrupt halt.

The file common.c also defines a function that will perform common initialization such as setting up OpenSSL for multithreading, initializing the library, and loading error strings. The call to SSL_load_error_strings loads the associated data for error codes so that when errors occur and we print the error stack, we get human-readable information about what went wrong. Since loading these diagnostic strings does take memory, there are circumstances in which we would not want to make this call, such as when we’re developing applications for embedded systems or other machines with limited memory. Generally, it’s a good idea to load the strings since it makes the job of decoding error messages much easier.

As we build in SSL support, the file common.c will hold the implementations of functions used by both our client and server, and common.h will receive the prototypes.

  1  #include "common.h"
  2   
  3  void handle_error(const char *file, int lineno, const char *msg)
  4  {
  5      fprintf(stderr, "** %s:%i %s\n", file, lineno, msg);
  6      ERR_print_errors_fp(stderr);
  7      exit(-1);
  8  }
  9   
 10  void init_OpenSSL(void)
 11  {
 12      if (!THREAD_setup() || !SSL_init_library(  ))
 13      {
 14          fprintf(stderr, "** OpenSSL initialization failed!\n");
 15          exit(-1);
 16      }
 17      SSL_load_error_strings(  );
 18  }

The bulk of our client application is in Example 5-3, client.c. At a high level, it creates a connection to the server on port 6001, as specified in common.h. Once a connection is established, it reads data from stdin until EOF is reached. As data is read and an internal buffer is filled, the data is sent over the connection to the server. Note that at this point, although we’re using OpenSSL for socket communications, we have not yet enabled the use of the SSL protocol.

Lines 27-29 create a new BIO object with a BIO_METHOD returned from BIO_s_connect ; the call to BIO_new_connect is a simplified function to accomplish this task. As long as no error occurs, lines 31-32 do the work of actually making the TCP connection and checking for errors. When a connection is successfully established, do_client_loop is called, which continually reads data from stdin and writes that data out to the socket. If an error while writing occurs or an EOF is received while reading from the console, the function exits and the program terminates.

  1  #include "common.h"
  2   
  3  void do_client_loop(BIO *conn)
  4  {
  5      int  err, nwritten;
  6      char buf[80];
  7   
  8      for (;;)
  9      {
 10          if (!fgets(buf, sizeof(buf), stdin))
 11              break;
 12          for (nwritten = 0;  nwritten < sizeof(buf);  nwritten += err)
 13          {
 14              err = BIO_write(conn, buf + nwritten, strlen(buf) - nwritten);
 15              if (err <= 0)
 16                  return;
 17          }
 18      }
 19  }
 20   
 21  int main(int argc, char *argv[])
 22  {
 23      BIO  *conn;
 24   
 25      init_OpenSSL(  );
 26   
 27      conn = BIO_new_connect(SERVER ":" PORT);
 28      if (!conn)
 29          int_error("Error creating connection BIO");
 30   
 31      if (BIO_do_connect(conn) <= 0)
 32          int_error("Error connecting to remote machine");
 33   
 34      fprintf(stderr, "Connection opened\n");
 35      do_client_loop(conn);
 36      fprintf(stderr, "Connection closed\n");
 37   
 38      BIO_free(conn);
 39      return 0;
 40  }

The server application in Example 5-4, server.c, differs from the client program in a few ways. After making the call to our common initialization function (line 44), it creates a different kind of BIO, one based on the BIO_METHOD returned from BIO_s_accept . This type of BIO creates a server socket that can accept remote connections. In lines 50-51, a call to BIO_do_accept binds the socket to port 6001; subsequent calls to BIO_do_accept will block and wait for a remote connection. The loop in lines 53-60 blocks until a connection is made. When a connection is made, a new thread to handle the new connection is spawned, which then calls do_server_loop with the connected socket’s BIO. The function do_server_loop simply reads data from the socket and writes it back out to stdout. If any errors occur here, the function returns and the thread is terminated. As a note, we call ERR_remove_state on line 33 to make sure any memory used by the error queue for the thread is freed.

  1  #include "common.h"
  2   
  3  void do_server_loop(BIO *conn)
  4  {
  5      int  done, err, nread;
  6      char buf[80];
  7   
  8      do
  9      {
 10          for (nread = 0;  nread < sizeof(buf);  nread += err)
 11          {
 12               err = BIO_read(conn, buf + nread, sizeof(buf) - nread);
 13               if (err <= 0)
 14                   break;
 15          }
 16          fwrite(buf, 1, nread, stdout);
 17      }
 18      while (err > 0);
 19  }
 20   
 21  void THREAD_CC server_thread(void *arg)
 22  {
 23      BIO *client = (BIO *)arg;
 24   
 25  #ifndef WIN32
 26      pthread_detach(pthread_self(  ));
 27  #endif
 28      fprintf(stderr, "Connection opened.\n");
 29      do_server_loop(client);
 30      fprintf(stderr, "Connection closed.\n");
 31  
 32      BIO_free(client);
 33      ERR_remove_state(0);
 34  #ifdef WIN32
 35      _endthread(  );
 36  #endif
 37  }
 38   
 39  int main(int argc, char *argv[])
 40  {
 41      BIO         *acc, *client;
 42      THREAD_TYPE tid;
 43   
 44      init_OpenSSL(  );
 45   
 46      acc = BIO_new_accept(PORT);
 47      if (!acc)
 48          int_error("Error creating server socket");
 49    
 50      if (BIO_do_accept(acc) <= 0)
 51          int_error("Error binding server socket");
 52    
 53      for (;;)
 54      {
 55          if (BIO_do_accept(acc) <= 0)
 56              int_error("Error accepting connection");
 57   
 58          client = BIO_pop(acc);
 59          THREAD_CREATE(tid, server_thread, client);
 60      }
 61   
 62      BIO_free(acc);
 63      return 0;
 64  }

Now that we understand the sample application, we’re ready to take the steps necessary to secure the communications with SSL.

In order for SSL connections to be secure, we must select a secure version of the protocol and provide accurate certificate information for the peer to validate. Since this is our first introduction to the SSL API, we will cover the background information about the structures and functions we need to accomplish our task.

int passwd_cb(char * buf, int size, int flag, void *userdata);

  1  #include "common.h"
  2   
  3  #define CERTFILE "client.pem"
  4  SSL_CTX *setup_client_ctx(void)
  5  {
  6      SSL_CTX *ctx;
  7  
  8      ctx = SSL_CTX_new(SSLv23_method(  ));
  9      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 10          int_error("Error loading certificate from file");
 11      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 12          int_error("Error loading private key from file");  
 13      return ctx;
 14  }
 15  
 16  int do_client_loop(SSL *ssl)
 17  {
 18      int  err, nwritten;
 19      char buf[80];
 20   
 21      for (;;)
 22      {
 23          if (!fgets(buf, sizeof(buf), stdin))
 24              break;
 25          for (nwritten = 0;  nwritten < sizeof(buf);  nwritten += err)
 26          {
 27              err = SSL_write(ssl, buf + nwritten, strlen(buf) - nwritten);
 28              if (err <= 0)
 29                  return 0;
 30          }
 31      }
 32      return 1;
 33  }
 34   
 35  int main(int argc, char *argv[])
 36  {
 37      BIO     *conn;
 38      SSL     *ssl;
 39      SSL_CTX *ctx;
 40  
 41      init_OpenSSL(  );
 42      seed_prng(  );
 43  
 44      ctx = setup_client_ctx(  );
 45  
 46      conn = BIO_new_connect(SERVER ":" PORT);
 47      if (!conn)
 48          int_error("Error creating connection BIO");
 49   
 50      if (BIO_do_connect(conn) <= 0)
 51          int_error("Error connecting to remote machine");
 52   
 53      if (!(ssl = SSL_new(ctx)))
 54          int_error("Error creating an SSL context");
 55      SSL_set_bio(ssl, conn, conn);
 56      if (SSL_connect(ssl) <= 0)
 57          int_error("Error connecting SSL object");
 58  
 59      fprintf(stderr, "SSL Connection opened\n");
 60      if (do_client_loop(ssl))
 61          SSL_shutdown(ssl);
 62      else
 63          SSL_clear(ssl);
 64      fprintf(stderr, "SSL Connection closed\n");
 65  
 66      SSL_free(ssl);
 67      SSL_CTX_free(ctx);
 68      return 0;
 69  }

$ openssl req -newkey rsa:1024 -sha1 -keyout rootkey.pem -out rootreq.pem
$ openssl x509 -req -in rootreq.pem -sha1 -extfile myopenssl.cnf \
>  -extensions v3_ca -signkey rootkey.pem -out rootcert.pem 
$ cat rootcert.pem rootkey.pem > root.pem
$ openssl x509 -subject -issuer -noout -in root.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA

$ openssl req -newkey rsa:1024 -sha1 -keyout serverCAkey.pem -out \
>  serverCAreq.pem
$ openssl x509 -req -in serverCAreq.pem -sha1 -extfile \
>  myopenssl.cnf -extensions v3_ca -CA root.pem -CAkey root.pem \
>  -CAcreateserial -out serverCAcert.pem
$ cat serverCAcert.pem serverCAkey.pem rootcert.pem > serverCA.pem
$ openssl x509 -subject -issuer -noout -in serverCA.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/OU=Server Division/CN=Server CA
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA

$ openssl req -newkey rsa:1024 -sha1 -keyout serverkey.pem -out \
>  serverreq.pem
$ openssl x509 -req -in serverreq.pem -sha1 -extfile myopenssl.cnf \
>  -extensions usr_cert -CA serverCA.pem -CAkey serverCA.pem \
>  -CAcreateserial -out servercert.pem
$ cat servercert.pem serverkey.pem serverCAcert.pem rootcert.pem > \
>  server.pem
$ openssl x509 -subject -issuer -noout -in server.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=splat.zork.org
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/OU=Server Division/CN=Server CA

$ openssl req -newkey rsa:1024 -sha1 -keyout clientkey.pem -out \
>  clientreq.pem
$ openssl x509 -req -in clientreq.pem -sha1 -extfile myopenssl.cnf \
>  -extensions usr_cert -CA root.pem -CAkey root.pem \
>  -CAcreateserial -out clientcert.pem
$ cat clientcert.pem clientkey.pem rootcert.pem > client.pem
$ openssl x509 -subject -issuer -noout -in client.pem
subject= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=shell.zork.org
issuer= /C=US/ST=VA/L=Fairfax/O=Zork.org/CN=Root CA

$ openssl dhparam -check -text -5 512 -out dh512.pem
$ openssl dhparam -check -text -5 1024 -out dh1024.pem

  1  #include "common.h"
  2   
  3  #define CERTFILE "server.pem"
  4  SSL_CTX *setup_server_ctx(void)
  5  {
  6      SSL_CTX *ctx;
  7  
  8      ctx = SSL_CTX_new(SSLv23_method(  ));
  9      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 10          int_error("Error loading certificate from file");
 11      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 12          int_error("Error loading private key from file");
 13      return ctx;
 14  }
 15  
 16  int do_server_loop(SSL *ssl)
 17  {
 18      int  err, nread;
 19      char buf[80];
 20   
 21      do
 22      {
 23          for (nread = 0;  nread < sizeof(buf);  nread += err)
 24          {
 25              err = SSL_read(ssl, buf + nread, sizeof(buf) - nread);
 26              if (err <= 0)
 27                  break;
 28          }
 29          fwrite(buf, 1, nread, stdout);
 30      }
 31      while (err > 0);
 32      return (SSL_get_shutdown(ssl) & SSL_RECEIVED_SHUTDOWN) ? 1 : 0;
 33  }
 34   
 35  void THREAD_CC server_thread(void *arg)
 36  {
 37      SSL *ssl = (SSL *)arg;
 38  
 39  #ifndef WIN32
 40      pthread_detach(pthread_self(  ));
 41  #endif
 42      if (SSL_accept(ssl) <= 0)
 43          int_error("Error accepting SSL connection");
 44      fprintf(stderr, "SSL Connection opened\n");
 45      if (do_server_loop(ssl))
 46          SSL_shutdown(ssl);
 47      else
 48          SSL_clear(ssl);
 49      fprintf(stderr, "SSL Connection closed\n");
 50      SSL_free(ssl);
 51  
 52  ERR_remove_state(0);
 53  
 54  #ifdef WIN32
 55      _endthread(  );
 56  #endif
 57  }
 58   
 59  int main(int argc, char *argv[])
 60  {
 61      BIO         *acc, *client;
 62      SSL         *ssl;
 63      SSL_CTX     *ctx;
 64      THREAD_TYPE tid;
 65   
 66      init_OpenSSL(  );
 67      seed_prng(  );
 68  
 69      ctx = setup_server_ctx(  );
 70  
 71      acc = BIO_new_accept(PORT);
 72      if (!acc)
 73          int_error("Error creating server socket");
 74   
 75      if (BIO_do_accept(acc) <= 0)
 76          int_error("Error binding server socket");
 77   
 78      for (;;)
 79      {
 80          if (BIO_do_accept(acc) <= 0)
 81              int_error("Error accepting connection");
 82   
 83          client = BIO_pop(acc);
 84          if (!(ssl = SSL_new(ctx)))
 85              int_error("Error creating SSL context");
 86  
 87          SSL_set_bio(ssl, client, client);
 88          THREAD_create(tid, server_thread, ssl);
 89      }
 90      
 91      SSL_CTX_free(ctx);
 92      BIO_free(acc);
 93      return 0;
 94  }

The security of an SSL-enabled program is compromised by failure to verify peer certificates properly. In this step, we’ll look into the various API calls that deal with trusted certificates, certificate chain verification, CRL usage, and post-connection verification.

int SSL_CTX_load_verify_locations(SSL_CTX *ctx, const char *CAfile,
                                    const char *CApath);

int verify_callback(int ok, X509_STORE_CTX *store)
{
    char data[256];
 
    if (!ok)
    {
        X509 *cert = X509_STORE_CTX_get_current_cert(store);
        int  depth = X509_STORE_CTX_get_error_depth(store);
        int  err = X509_STORE_CTX_get_error(store);
 
        fprintf(stderr, "-Error with certificate at depth: %i\n", depth);
        X509_NAME_oneline(X509_get_issuer_name(cert), data, 256);
        fprintf(stderr, "  issuer   = %s\n", data);
        X509_NAME_oneline(X509_get_subject_name(cert), data, 256);
        fprintf(stderr, "  subject  = %s\n", data);
        fprintf(stderr, "  err %i:%s\n", err, X509_verify_cert_error_string(err));
    }
 
    return ok;
}

X509_STORE *SSL_CTX_get_cert_store(SSL_CTX *ctx);

long post_connection_check(SSL *ssl, char *host)
{
    X509      *cert;
    X509_NAME *subj;
    char      data[256];
    int       extcount;
    int       ok = 0;
 
    /* Checking the return from SSL_get_peer_certificate here is not strictly
     * necessary.  With our example programs, it is not possible for it to return
     * NULL.  However, it is good form to check the return since it can return NULL
     * if the examples are modified to enable anonymous ciphers or for the server
     * to not require a client certificate.
     */
    if (!(cert = SSL_get_peer_certificate(ssl)) || !host)
        goto err_occured;
    if ((extcount = X509_get_ext_count(cert)) > 0)
    {
        int i;
 
        for (i = 0;  i < extcount;  i++)
        {
            char              *extstr;
            X509_EXTENSION    *ext;
 
            ext = X509_get_ext(cert, i);
            extstr = OBJ_nid2sn(OBJ_obj2nid(X509_EXTENSION_get_object(ext)));
 
            if (!strcmp(extstr, "subjectAltName"))
            {
                int                  j;
                unsigned char        *data;
                STACK_OF(CONF_VALUE) *val;
                CONF_VALUE           *nval;
                X509V3_EXT_METHOD    *meth;
 
                if (!(meth = X509V3_EXT_get(ext)))
                    break;
                data = ext->value->data;
 
                val = meth->i2v(meth, 
                                meth->d2i(NULL, &data, ext->value->length),
                                NULL);
                for (j = 0;  j < sk_CONF_VALUE_num(val);  j++)
                {
                    nval = sk_CONF_VALUE_value(val, j);
                    if (!strcmp(nval->name, "DNS") && !strcmp(nval->value, host))
                    {
                        ok = 1;
                        break;
                    }
                }
            }
            if (ok)
                break;
        }
    }
 
    if (!ok && (subj = X509_get_subject_name(cert)) &&
        X509_NAME_get_text_by_NID(subj, NID_commonName, data, 256) > 0)
    {
        data[255] = 0;
        if (strcasecmp(data, host) != 0)
            goto err_occured;
    }
 
    X509_free(cert);
    return SSL_get_verify_result(ssl);
 
err_occured:
    if (cert)
        X509_free(cert);
    return X509_V_ERR_APPLICATION_VERIFICATION;
}

  1  #include "common.h"
  2   
  3  #define CAFILE "root.pem"
  4  #define CADIR NULL
  5  #define CERTFILE "client.pem"
  6  SSL_CTX *setup_client_ctx(void)
  7  {
  8      SSL_CTX *ctx;
  9   
 10      ctx = SSL_CTX_new(SSLv23_method(  ));
 11      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 12          int_error("Error loading CA file and/or directory");
 13      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 14          int_error("Error loading default CA file and/or directory");
 15      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 16          int_error("Error loading certificate from file");
 17      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 18          int_error("Error loading private key from file");
 19      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
 20      SSL_CTX_set_verify_depth(ctx, 4);
 21      return ctx;
 22  }
 23   
 24  int do_client_loop(SSL *ssl)
 25  {
 26      int  err, nwritten;
 27      char buf[80];
 28   
 29      for (;;)
 30      {
 31          if (!fgets(buf, sizeof(buf), stdin))
 32              break;
 33          for (nwritten = 0;  nwritten < sizeof(buf);  nwritten += err)
 34          {
 35              err = SSL_write(ssl, buf + nwritten, strlen(buf) - nwritten);
 36              if (err <= 0)
 37                  return 0;
 38          }
 39      }
 40      return 1;
 41  }
 42   
 43  int main(int argc, char *argv[])
 44  {
 45      BIO     *conn;
 46      SSL     *ssl;
 47      SSL_CTX *ctx;
 48      long    err;
 49  
 50      init_OpenSSL(  );
 51      seed_prng(  );
 52   
 53      ctx = setup_client_ctx(  );
 54   
 55      conn = BIO_new_connect(SERVER ":" PORT);
 56      if (!conn)
 57          int_error("Error creating connection BIO");
 58   
 59      if (BIO_do_connect(conn) <= 0)
 60          int_error("Error connecting to remote machine");
 61   
 62      ssl = SSL_new(ctx);
 63      SSL_set_bio(ssl, conn, conn);
 64      if (SSL_connect(ssl) <= 0)
 65          int_error("Error connecting SSL object");
 66      if ((err = post_connection_check(ssl, SERVER)) != X509_V_OK)
 67      {
 68          fprintf(stderr, "-Error: peer certificate: %s\n",
 69                  X509_verify_cert_error_string(err));
 70          int_error("Error checking SSL object after connection");
 71      }
 72      fprintf(stderr, "SSL Connection opened\n");
 73      if (do_client_loop(ssl))
 74          SSL_shutdown(ssl);
 75      else
 76          SSL_clear(ssl);
 77      fprintf(stderr, "SSL Connection closed\n");
 78   
 79      SSL_free(ssl);
 80      SSL_CTX_free(ctx);
 81      return 0;
 82  }

  1  #include "common.h"
  2   
  3  #define CAFILE "root.pem"
  4  #define CADIR NULL
  5  #define CERTFILE "server.pem"
  6  SSL_CTX *setup_server_ctx(void)
  7  {
  8      SSL_CTX *ctx;
  9   
 10      ctx = SSL_CTX_new(SSLv23_method(  ));
 11      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 12          int_error("Error loading CA file and/or directory");
 13      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 14          int_error("Error loading default CA file and/or directory");
 15      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 16          int_error("Error loading certificate from file");
 17      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 18          int_error("Error loading private key from file");
 19      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT,
 20                         verify_callback);
 21      SSL_CTX_set_verify_depth(ctx, 4);
 22      return ctx;
 23  }
 24   
 25  int do_server_loop(SSL *ssl)
 26  {
 27      int  err, nread;
 28      char buf[80];
 29   
 30      for (;;)
 31      {
 32          for (nread = 0;  nread < sizeof(buf);  nread += err)
 33          {
 34              err = SSL_read(ssl, buf + nread, sizeof(buf) - nread);
 35              if (err <= 0)
 36                  break;
 37          }
 38          fwrite(buf, 1, nread, stdout);
 39      }
 40      return (SSL_get_shutdown(ssl) & SSL_RECEIVED_SHUTDOWN) ? 1 : 0;
 41  }
 42   
 43  void THREAD_CC server_thread(void *arg)
 44  {
 45      SSL *ssl = (SSL *)arg;
 46  long err;
 47   
 48  #ifndef WIN32
 49      pthread_detach(pthread_self(  ));
 50  #endif
 51      if (SSL_accept(ssl) <= 0)
 52          int_error("Error accepting SSL connection");
 53      if ((err = post_connection_check(ssl, CLIENT)) != X509_V_OK)
 54      {
 55          fprintf(stderr, "-Error: peer certificate: %s\n",
 56                  X509_verify_cert_error_string(err));
 57          int_error("Error checking SSL object after connection");
 58      }
 59      fprintf(stderr, "SSL Connection opened\n");
 60      if (do_server_loop(ssl))
 61          SSL_shutdown(ssl);
 62      else
 63          SSL_clear(ssl);
 64      fprintf(stderr, "SSL Connection closed\n");
 65      SSL_free(ssl);
 66  ERR_remove_state(0)
 67  #ifdef WIN32
 68      _endthread(  );
 69  #endif
 70  }
 71   
 72  int main(int argc, char *argv[])
 73  {
 74      BIO     *acc, *client;
 75      SSL     *ssl;
 76      SSL_CTX *ctx;
 77   THREAD_TYPE tid;
 78  
 79      init_OpenSSL(  );
 80      seed_prng(  );
 81   
 82      ctx = setup_server_ctx(  );
 83   
 84      acc = BIO_new_accept(PORT);
 85      if (!acc)
 86          int_error("Error creating server socket");
 87   
 88      if (BIO_do_accept(acc) <= 0)
 89          int_error("Error binding server socket");
 90   
 91      for (;;)
 92      {
 93          if (BIO_do_accept(acc) <= 0)
 94              int_error("Error accepting connection");
 95   
 96          client = BIO_pop(acc);
 97          if (!(ssl = SSL_new(ctx)))
 98          int_error("Error creating SSL context");
 99          SSL_set_accept_state(ssl);
100          SSL_set_bio(ssl, client, client);
101          THREAD_create(tid, server_thread, ssl);
102      }
103   
104      SSL_CTX_free(ctx);
105      BIO_free(acc);
106      return 0;
107  }

We have not yet discussed a few important points about SSL connections. For instance, we mentioned that SSLv2 shouldn’t be used, yet our example was created with the SSLv23_method that still allows for this insecure protocol version. Beyond protocol limitation, OpenSSL provides for many workarounds for known bugs in other SSL implementations. While these bugs don’t affect security, our application may lose interoperability if we do not account for them.

Additionally, we need to delve into the selection of cipher suites . A cipher suite is a combination of lower-level algorithms that an SSL connection uses to do authentication, key exchange, and stream encryption. Suite selection is important because OpenSSL supports some algorithms for compatibility that we want to exclude for security reasons. Similarly, some of the cipher suites that are secure require an application to provide callbacks in order to be utilized. Learning how to do these things properly and extending our example for this final step will be the topic of this section.

DH *tmp_dh_callback(SSL *ssl, int is_export, int keylength);

  1  include "common.h"
  2   
  3  #define CIPHER_LIST "ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH"
  4  #define CAFILE "root.pem"
  5  #define CADIR NULL
  6  #define CERTFILE "client.pem"
  7  SSL_CTX *setup_client_ctx(void)
  8  {
  9      SSL_CTX *ctx;
 10   
 11      ctx = SSL_CTX_new(SSLv23_method(  ));
 12      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 13          int_error("Error loading CA file and/or directory");
 14      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 15          int_error("Error loading default CA file and/or directory");
 16      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 17          int_error("Error loading certificate from file");
 18      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 19          int_error("Error loading private key from file");
 20      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
 21      SSL_CTX_set_verify_depth(ctx, 4);
 22      SSL_CTX_set_options(ctx, SSL_OP_ALL|SSL_OP_NO_SSLv2);
 23      if (SSL_CTX_set_cipher_list(ctx, CIPHER_LIST) != 1)
 24          int_error("Error setting cipher list (no valid ciphers)");
 25      return ctx;
 26  }

  1  #include "common.h"
  2   
  3  DH *dh512 = NULL;
  4  DH *dh1024 = NULL;
  5  
  6  void init_dhparams(void)
  7  {
  8      BIO *bio;
  9  
 10      bio = BIO_new_file("dh512.pem", "r");
 11      if (!bio)
 12          int_error("Error opening file dh512.pem");
 13      dh512 = PEM_read_bio_DHparams(bio, NULL, NULL, NULL);
 14      if (!dh512)
 15          int_error("Error reading DH parameters from dh512.pem");
 16      BIO_free(bio);
 17  
 18      bio = BIO_new_file("dh1024.pem", "r");
 19      if (!bio)
 20          int_error("Error opening file dh1024.pem");
 21      dh1024 = PEM_read_bio_DHparams(bio, NULL, NULL, NULL);
 22      if (!dh1024)
 23          int_error("Error reading DH parameters from dh1024.pem");
 24      BIO_free(bio);
 25  }
 26  
 27  DH *tmp_dh_callback(SSL *ssl, int is_export, int keylength)
 28  {
 29      DH *ret;
 30  
 31      if (!dh512 || !dh1024)
 32          init_dhparams(  );
 33  
 34      switch (keylength)
 35      {
 36          case 512:
 37              ret = dh512;
 38              break;
 39          case 1024:
 40          default: /* generating DH params is too costly to do on the fly */
 41              ret = dh1024;
 42              break;
 43      }
 44      return ret;
 45  }
 46  
 47  #define CIPHER_LIST "ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH"
 48  #define CAFILE "root.pem"
 49  #define CADIR NULL
 50  #define CERTFILE "server.pem"
 51  SSL_CTX *setup_server_ctx(void)
 52  {
 53      SSL_CTX *ctx;
 54   
 55      ctx = SSL_CTX_new(SSLv23_method(  ));
 56      if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1)
 57          int_error("Error loading CA file and/or directory");
 58      if (SSL_CTX_set_default_verify_paths(ctx) != 1)
 59          int_error("Error loading default CA file and/or directory");
 60      if (SSL_CTX_use_certificate_chain_file(ctx, CERTFILE) != 1)
 61          int_error("Error loading certificate from file");
 62      if (SSL_CTX_use_PrivateKey_file(ctx, CERTFILE, SSL_FILETYPE_PEM) != 1)
 63          int_error("Error loading private key from file");
 64      SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT,
 65                         verify_callback);
 66      SSL_CTX_set_verify_depth(ctx, 4);
 67      SSL_CTX_set_options(ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2 |
 68                               SSL_OP_SINGLE_DH_USE);
 69      SSL_CTX_set_tmp_dh_callback(ctx, tmp_dh_callback);
 70      if (SSL_CTX_set_cipher_list(ctx, CIPHER_LIST) != 1)
 71          int_error("Error setting cipher list (no valid ciphers)");
 72      return ctx;
 73  }

An SSL session is different from an SSL connection. A session refers to the set of parameters and encryption keys created by performing a handshake, whereas a connection is an active conversation that uses a session. In other words, the connection refers to the process of communication, and the session refers to the parameters by which that communication was set up. Understanding this, we can delve into some of the OpenSSL routines that provide for SSL session caching.

Since the majority of the computational expense associated with an SSL server is in setting up connections with clients and not in maintaining them, session caching can be a lifesaver when it comes to server load reduction. OpenSSL implements sessions as SSL_SESSION objects. While the majority of the work in implementing session caching falls on the server side, the client must also keep its previous sessions in order for the efficiency benefit to be realized.

From the server’s perspective, once it has established a connection, it merely needs to label the data and cache the session. This label, called the session ID context , is described in Section 5.2.1.2. After establishing a session, the server assigns a timeout value to it, which is simply a time after which the session will be discarded and another will need to be negotiated. The default behavior of a session caching server is to flush expired sessions automatically. From the client side, an SSL_SESSION object is received from the server. The client can then save this session and reuse it if it needs to make another connection. For instance, a web browser may need to make several connections to display all of the information presented from an SSL-enabled web server. By keeping this session information around, the client connects quickly, and the load on the server is reduced, since it need not negotiate a new session for each connection.

As we’ve said, the server determines the valid session ID context. When a client attempts to connect to a caching server, it has only one attempt at presenting the correct value. If incorrect, a normal handshake occurs, and a new session is created. If the client saves this newly created session, it will then have the correct session ID context for future connections.

The details associated with sessions, such as enabling a client and server, dealing with timeouts, and flushing old sessions, are discussed in the following sections. In addition, we will also outline a mechanism for server-side, on-disk storage of sessions.

ssl = SSL_new(ctx)
... setup underlying communications layer for ssl ...
... connect to host:port ...
if (saved session for host:port in cache)
  SSL_set_session(ssl, saved session)
  SSL_SESSION_free(saved session)
SSL_connect(ssl)
call post_connection_check(ssl, host) and check return value
... normal application code here ...
saved session = SSL_get1_session(ssl)
if (saved session != NULL)
  enter saved session into cache under host:port
SSL_shutdown(ssl)
SSL_free(ssl)

int new_session_cb(SSL *ctx, SSL_SESSION *session);

void remove_session_cb(SSL_CTX *ctx, SSL_SESSION *session);

SSL_SESSION *get_session_cb(SSL *ctx, unsigned char *id, int len, int *ref);

new_session_cb(  )
{
  acquire a lock for the file named according to the session id
  open file named according to session id
  encrypt and write the SSL_SESSION object to the file
  release the lock
}

remove_session_cb(  )
{
  acquire a lock for the file named according to the session id
  remove the file named according to the session id
  release the lock
}

get_session_cb(  )
{
  acquire a lock for the file named according to the session id in the 2nd arg
  read and decrypt contents of file and create a new SSL_SESSION object
  release the lock
  set the integer referenced by the fourth parameter to 0
  return the new session object
}

The motivation behind creating an SSL connection is to transfer data back and forth securely. Thus far, we’ve concentrated on making this connection very secure, but we’ve avoided the details on how data is transferred. In the earlier examples, we cheated on the I/O processing, since the client only writes to the SSL connection, and the server only reads from it. Most real-world applications do not operate on such a simple model. Since the calls we used to perform the reading and writing (SSL_read and SSL_write) closely mirror the system calls read and write, it seems as though I/O would not be significantly different from I/O in non-SSL applications. Unfortunately, this is far from the truth.

There are many subtleties in performing I/O correctly with OpenSSL. We will first look at SSL_read and SSL_write in more detail. Once we understand them, we will get into the detail of the differences between performing blocking and non-blocking I/O on SSL connections. By the end of this chapter, we will have built up our knowledge of the pitfalls with I/O in OpenSSL. This will enable us to avoid them when implementing real-world applications that require I/O paradigms that are more complex than the ones provided in our examples.

code = SSL_read(ssl, buf + offset, size - offset);
switch (SSL_get_error(ssl, code))
{
    case SSL_ERROR_NONE:
        /* update the offset value */
        offset += code;
        break;
    case SSL_ERROR_ZERO_RETURN:
        /* react to the SSL connection being closed */
        do_cleanup(ssl);
        break;
    case SSL_ERROR_WANT_READ:
        /* handle this in an application specific way to retry the SSL_read */
        break;
    case SSL_ERROR_WANT_WRITE:
        /* handle this in an application specific way to retry the SSL_read */
        break;
    default:
        /* an error occurred. shutdown the connection */
        shutdown_connection(ssl);
}

  1  #include <openssl/ssl.h>
  2  #include <openssl/err.h>
  3  #include <string.h>
  4   
  5  #define BUF_SIZE 80
  6   
  7  void data_transfer(SSL *A, SSL *B)
  8  {
  9      /* the buffers and the size variables */
 10      unsigned char A2B[BUF_SIZE];
 11      unsigned char B2A[BUF_SIZE];
 12      unsigned int A2B_len = 0;
 13      unsigned int B2A_len = 0;
 14      /* flags to mark that we have some data to write */
 15      unsigned int have_data_A2B = 0;
 16      unsigned int have_data_B2A = 0;
 17      /* flags set by check_availability(  ) that poll for I/O status */
 18      unsigned int can_read_A = 0;
 19      unsigned int can_read_B = 0;
 20      unsigned int can_write_A = 0;
 21      unsigned int can_write_B = 0;
 22      /* flags to mark all the combinations of why we're blocking */
 23      unsigned int read_waiton_write_A = 0;
 24      unsigned int read_waiton_write_B = 0;
 25      unsigned int read_waiton_read_A = 0;
 26      unsigned int read_waiton_read_B = 0;
 27      unsigned int write_waiton_write_A = 0;
 28      unsigned int write_waiton_write_B = 0;
 29      unsigned int write_waiton_read_A = 0;
 30      unsigned int write_waiton_read_B = 0;
 31      /* variable to hold return value of an I/O operation */
 32      int code;
 33   
 34      /* make the underlying I/O layer behind each SSL object non-blocking */
 35      set_non-blocking(A);
 36      set_non-blocking(B);
 37      SSL_set_mode(A, SSL_MODE_ENABLE_PARTIAL_WRITE|
 38                      SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); 
 39      SSL_set_mode(B, SSL_MODE_ENABLE_PARTIAL_WRITE| 
 40                      SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER);
 41   
 42      for (;;)
 43      {
 44          /* check I/O availability and set flags */
 45          check_availability(A, &can_read_A, &can_write_A,
 46                             B, &can_read_B, &can_write_B);
 47   
 48          /* this "if" statement reads data from A. it will only be entered if
 49           * the following conditions are all true:
 50           * 1. we're not in the middle of a write on A
 51           * 2. there's space left in the A to B buffer
 52           * 3. either we need to write to complete a previously blocked read
 53           *    and now A is available to write, or we can read from A
 54           *    regardless of whether we're blocking for availability to read.
 55           */
 56          if (!(write_waiton_read_A || write_waiton_write_A) &&
 57              (A2B_len != BUF_SIZE) &&
 58              (can_read_A || (can_write_A && read_waiton_write_A)))
 59          {
 60              /* clear the flags since we'll set them based on the I/O call's
 61               * return
 62               */
 63              read_waiton_read_A = 0;
 64              read_waiton_write_A = 0;
 65   
 66              /* read into the buffer after the current position */
 67              code = SSL_read(A, A2B + A2B_len, BUF_SIZE - A2B_len);
 68              switch (SSL_get_error(A, code))
 69              {
 70                  case SSL_ERROR_NONE:
 71                      /* no errors occured.  update the new length and make
 72                       * sure the "have data" flag is set.
 73                       */
 74                      A2B_len += code;
 75                      have_data_A2B = 1;
 76                      break;
 77                  case SSL_ERROR_ZERO_RETURN:
 78                      /* connection closed */
 79                      goto end;
 80                  case SSL_ERROR_WANT_READ:
 81                      /* we need to retry the read after A is available for
 82                       * reading
 83                       */
 84                      read_waiton_read_A = 1;
 85                      break;
 86                  case SSL_ERROR_WANT_WRITE:
 87                      /* we need to retry the read after A is available for
 88                       * writing
 89                       */
 90                      read_waiton_write_A = 1;
 91                      break;
 92                  default:
 93                      /* ERROR */
 94                      goto err;
 95              }
 96          }
 97   
 98          /* this "if" statement is roughly the same as the previous "if"
 99           * statement with A and B switched
100           */
101          if (!(write_waiton_read_B || write_waiton_write_B) &&
102              (B2A_len != BUF_SIZE) &&
103              (can_read_B || (can_write_B && read_waiton_write_B)))
104          {
105              read_waiton_read_B = 0;
106              read_waiton_write_B = 0;
107   
108              code = SSL_read(B, B2A + B2A_len, BUF_SIZE - B2A_len);
109              switch (SSL_get_error(B, code))
110              {
111                  case SSL_ERROR_NONE:
112                      B2A_len += code;
113                      have_data_B2A = 1;
114                      break;
115                  case SSL_ERROR_ZERO_RETURN:
116                      goto end;
117                  case SSL_ERROR_WANT_READ:
118                      read_waiton_read_B = 1;
119                      break;
120                  case SSL_ERROR_WANT_WRITE:
121                      read_waiton_write_B = 1;
122                      break;
123                  default:
124                      goto err;
125              }
126          }
127   
128          /* this "if" statement writes data to A. it will only be entered if
129           * the following conditions are all true:
130           * 1. we're not in the middle of a read on A
131           * 2. there's data in the A to B buffer
132           * 3. either we need to read to complete a previously blocked write
133           *    and now A is available to read, or we can write to A
134           *    regardless of whether we're blocking for availability to write
135           */
136          if (!(read_waiton_write_A || read_waiton_read_A) &&
137              have_data_B2A &&
138              (can_write_A || (can_read_A && write_waiton_read_A)))
139          {
140              /* clear the flags */
141              write_waiton_read_A = 0;
142              write_waiton_write_A = 0;
143   
144              /* perform the write from the start of the buffer */
145              code = SSL_write(A, B2A, B2A_len);
146              switch (SSL_get_error(A, code))
147              {
148                  case SSL_ERROR_NONE:
149                      /* no error occured. adjust the length of the B to A
150                       * buffer to be smaller by the number bytes written.  If
151                       * the buffer is empty, set the "have data" flags to 0,
152                       * or else, move the data from the middle of the buffer
153                       * to the front.
154                       */
155                      B2A_len -= code;
156                      if (!B2A_len)
157                          have_data_B2A = 0;
158                      else
159                          memmove(B2A, B2A + code, B2A_len);
160                      break;
161                  case SSL_ERROR_ZERO_RETURN:
162                      /* connection closed */
163                      goto end;
164                  case SSL_ERROR_WANT_READ:
165                      /* we need to retry the write after A is available for
166                       * reading
167                       */
168                      write_waiton_read_A = 1;
169                      break;
170                  case SSL_ERROR_WANT_WRITE:
171                      /* we need to retry the write after A is available for
172                       * writing
173                       */
174                      write_waiton_write_A = 1;
175                      break;
176                  default:
177                      /* ERROR */
178                      goto err;
179              }
180          }
181   
182          /* this "if" statement is roughly the same as the previous "if"
183           * statement with A and B switched
184           */
185          if (!(read_waiton_write_B || read_waiton_read_B) &&
186              have_data_A2B &&
187              (can_write_B || (can_read_B && write_waiton_read_B)))
188          {
189              write_waiton_read_B = 0;
190              write_waiton_write_B = 0;
191   
192              code = SSL_write(B, A2B, A2B_len);
193              switch (SSL_get_error(B, code))
194              {
195                  case SSL_ERROR_NONE:
196                      A2B_len -= code;
197                      if (!A2B_len)
198                          have_data_A2B = 0;
199                      else
200                          memmove(A2B, A2B + code, A2B_len);
201                      break;
202                  case SSL_ERROR_ZERO_RETURN:
203                      /* connection closed */
204                      goto end;
205                  case SSL_ERROR_WANT_READ:
206                      write_waiton_read_B = 1;
207                      break;
208                  case SSL_ERROR_WANT_WRITE:
209                      write_waiton_write_B = 1;
210                     break;
211                  default:
212                      /* ERROR */
213                      goto err;
214              }
215          }
216      }
217   
218  err:
219      /* if we errored, print then before exiting */
220      fprintf(stderr, "Error(s) occured\n");
221      ERR_print_errors_fp(stderr);
222  end:
223      /* close down the connections. set them back to blocking to simplify. */
224      set_blocking(A);
225      set_blocking(B);
226      SSL_shutdown(A);
227      SSL_shutdown(B);
228  }

An SSL renegotiation is essentially an SSL handshake during a connection. This causes client credentials to be reevaluated and a new session to be created. We’ve discussed the effects of renegotiation on the I/O and other aspects of a program’s implementation, but haven’t talked about why renegotiations are important.

Since renegotiations cause the creation of a new session, the session key is replaced. For long-lasting SSL connections or for those that transfer a high quantity of data, it is a good idea to replace the session key periodically. In general, the longer a session key exists, the more the likelihood of key compromise increases. Using renegotiations, we can replace the session key so that we don’t encrypt too much data with just one key.

SSL renegotiations can occur during normal application data transfer. When the new handshake is completed, both parties switch to using the new key. The function to request renegotiation on an SSL connection is SSL_renegotiate .

This function does not actually perform the new handshake when called; however, it sets a flag that causes a renegotiation request to be sent to the peer. The request is sent out on the next call to an I/O function on the SSL connection. An important point to consider is that the peer has the option of not renegotiating the connection. If the peer chooses not to respond to the request and to continue transferring data instead, no renegotiation will occur; unless the requestor checks for negotiation success, it may not have happened. This is especially important when making applications that will connect with other non-OpenSSL SSL implementations.

A renegotiation can also be done explicitly. In other words, we will send a request and not send or receive application data until the new handshake has completed successfully. While this will sometimes be the appropriate way to refresh session keys for long-term connections, it serves another important purpose. It allows us to upgrade client authentication from the server side. For instance, our example server above requires a valid client certificate to be presented in order for the initial connection to take place. This is overly restrictive since at connection time, we do not know the client’s intentions.

As an example, consider a simple protocol that allows clients to connect via SSL and send commands to the server. A subset of these commands will be reserved for administrators. Our goal is to allow anyone to connect to the server to run the general commands and allow only administrators to run reserved commands. If we require all users to present a certificate at connection time, we must issue a certificate to every possible client and special certificates to administrators. This can become cumbersome rather quickly. Alternatively, we can run two servers, one for normal users and another for administrators. This is also a suboptimal solution since it requires extra consumption of resources.

We can use renegotiations to make our job simpler. First, we will allow anyone to connect without a certificate. When a client connects, it sends its command. If it’s a reserved command, we require a renegotiation with the more stringent requirements on the client in place. If the client renegotiates successfully, we can accept the command, or otherwise discard it. Thus, we need to issue certificates only to administrators. This option is clearly better than the previous two since it allows us to determine what the client is attempting before putting stronger authentication requirements in place.

/* assume ssl is connected and error free up to here */
set_blocking(ssl); /* this is unnecessary if it is already blocking */
SSL_renegotiate(ssl);
SSL_do_handshake(ssl);
if (ssl->state != SSL_ST_OK)
    int_error("Failed to send renegotiation request");
ssl->state |= SSL_ST_ACCEPT;
SSL_do_handshake(ssl);
if (ssl->state != SSL_ST_OK)
    int_error("Failed to complete renegotiation");
/* our renegotiation is complete */

/* assume ctx is an SSL_CTX object that is setup to not have any verify
   options. */
int normal_user = 1;
int admin_user = 2;
SSL_CTX_set_session_id_context(ctx, &normal_user, sizeof(int));
/* perform rest of ctx setup, create an ssl object, and connect it */
/* normal SSL I/O operations and application code go here */
/* if we want to upgrade client privilege, we enter the following code block */
SSL_set_verify(ssl, SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT,
               verify_callback);
SSL_set_session_id_context(ssl, &admin_user, sizeof(int));
/* code fragment from Example 5-18 goes here.  the new session is made */
post_connection_check(ssl, host);
/* if everything is error-free, we have properly authenticated the client */

Only two types of symmetric ciphers exist that are well-respected and see any sort of widespread use: block ciphers and stream ciphers. Block ciphers are traditionally the most popular. They operate by breaking up data into fixed-size blocks, and then encrypting each block individually, in which the encryption algorithm is a reversible function of the input. Leftover data is traditionally padded so that the length of the plaintext is a multiple of the cipher’s block size. Stream ciphers are essentially just cryptographic pseudorandom number generators. They use a starting seed as a key to produce a stream of random bits known as the keystream . To encrypt data, one takes the plaintext and simply XORs it with the keystream. Stream ciphers don’t need to be padded per se, though they are usually padded to byte-boundaries, since implementations usually operate on a per-byte level instead of a per-bit level.

The best block ciphers are a far more conservative solution than stream ciphers because they are better studied. Yet stream ciphers tend to be far faster than block ciphers. For example, RC4, currently the most popular stream cipher, runs about 4 times faster than Blowfish, which is among the fastest available block ciphers, and runs almost 15 times faster than 3DES, which is a very conservative cipher choice. AES is faster than 3DES and has more security in terms of a longer key size, but it is still generally slower than even Blowfish.

Neither block ciphers nor stream ciphers can give us perfect security , in which an attacker can never recover a message as long as the communicating parties use the algorithm properly. For each type of cipher, the security is, at best, a function of the key length. It’s always possible to launch a brute-force attack, in which the attacker tries every possible key until the message properly decrypts. If the key length is long enough, the attack will take so long on average as to be infeasible in practice.

Even if there was no better attack on an individual cipher than brute force, there are other issues that plague naive use of both types of cipher. Stream ciphers have the problem that a one-bit flip of the ciphertext causes a one-bit flip in the decrypted plaintext. Obviously, stream ciphers need to be supplemented with data integrity checks. For such purposes, we recommend message authentication codes (MACs—see Chapter 8).

When used directly, block ciphers always encrypt a given block of data in the same way, and thus do not effectively conceal patterns in a stream of data. An attacker can keep a dictionary of known plaintext blocks to known ciphertext blocks, which can often be useful in deciphering real messages. Additionally, an attacker can easily substitute one ciphertext block for another, often with great success. There are ways to use ciphers that can solve these problems to some degree, which we discuss in the next section. Additionally, MACs can be used to thwart actual modification attacks.

Stream ciphers are subject to a similar, more serious problem. Once you start encrypting using a given key, you must continue to generate new data in the keystream, or generate and exchange a new key. If you start over using the same key, the security of the stream cipher is effectively lost. The solution is to never reuse keys when using a stream cipher. Don’t even use the same key across reboots.

OpenSSL implements four common modes for block ciphers. Each of these modes can be used with every block cipher in the library, with the exception of DESX, which is defined as having only a single mode of operation.

OpenSSL provides implementations of an array of algorithms that meet most needs. The only significant lack in some versions is an implementation of the new Advanced Encryption Standard (AES). AES is supported in the long-awaited 0.9.7 release.

In addition to the actual cipher algorithms that OpenSSL provides, the null cipher is also supported, which passes data through untouched. You can access this cipher using EVP_enc_null. It is primarily useful for testing the EVP interface, so you should generally avoid using it in production systems.

Before we can begin encrypting or decrypting, we must allocate and initialize a cipher context. The cipher context is a data structure that keeps track of all relevant state for the purposes of encrypting or decrypting data over a period of time. For example, we can have multiple streams of data encrypted in CBC mode. The cipher context will keep track of the key associated with each stream and the internal state that needs to be kept between messages for CBC mode. Additionally, when encrypting with a block-based cipher mode, the context object buffers data that doesn’t exactly align to the block size until more data arrives, or until the buffer is explicitly flushed, at which point the data is usually padded as appropriate.^[1]

The generic cipher context type is EVP_CIPHER_CTX . We can initialize one, whether it was allocated dynamically or statically, by calling EVP_CIPHER_CTX_init , like so:

EVP_CIPHER_CTX *x = (EVP_CIPHER_CTX *)malloc(sizeof(EVP_CIPHER_CTX));
EVP_CIPHER_CTX_init(x);

After allocating the context object and initializing it, we must set up the cipher context. At this point, we generally determine whether the particular context will be used for encrypting or decrypting. It is possible to set up a context to do both, but it’s a bad idea in any mode other than ECB mode, because race conditions can easily occur that will desynchronize communication. Essentially, the internal cipher state of the two communicating parties needs to remain synchronized at all times. If both parties send data at the same time, they will likely end up trying to decrypt the data they receive using an incorrect state.

When we set up a cipher context, we not only choose whether we’re encrypting or decrypting, but also do the following:

EVP_EncryptInit is the preferred method for setting up a cipher context for encryption. For decryption, it is EVP_DecryptInit . Both of these methods have the same signature, which includes four parameters.

int EVP_EncryptInit(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *type,
                    unsigned char *key, unsigned char *iv);
int EVP_DecryptInit(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *type,
                    unsigned char *key, unsigned char *iv);

Notice that the engine package prefers an extended API, EVP_EncryptInit_ex and EVP_DecryptInit_ex, which inserts a fifth argument before the key that is a pointer to the engine to use; it should be NULL when no hardware acceleration is being used. When Version 0.9.7 of OpenSSL is released, these versions of the calls will be the preferred API. When using engines, many calls can fail, so check error codes. We don’t do this because we don’t use the ENGINE API in our examples.

Let’s consider an example in which we try to encrypt using Blowfish with 128-bit keys in CBC mode. CBC mode can use an initialization vector, which is always the size of one block (in this case, 8 bytes). We will use the OpenSSL pseudorandom number generator to provide a randomly generated key; however, distributing that key is not really covered in this example. For now, we’ll assume that you will do it offline, perhaps by exchanging a disk, or reading the key over the phone (key exchange protocols are discussed in Chapter 8). To that end, we do print out the key to stdout in hexadecimal format. Note that doing this is not really the best idea for real applications. Example 6-1 shows how to do this.

#include <openssl/evp.h>

void select_random_key(char *key, int b)
{
    int i;

    RAND_bytes(key, b);
    for (i = 0;  i < b - 1;  i++)
        printf("%02X:", key[i]);
    printf("%02X\n", key[b - 1]);
}

void select_random_iv(char *iv, int b)
{
    RAND_pseudo_bytes(iv, b);
}

int setup_for_encryption(void)
{
    EVP_CIPHER_CTX ctx;
    char           key[EVP_MAX_KEY_LENGTH]; 
    char           iv[EVP_MAX_IV_LENGTH];

    if (!seed_prng(  ))
        return 0;
    select_random_key(key, EVP_MAX_KEY_LENGTH);
    select_random_iv(iv, EVP_MAX_IV_LENGTH);
    EVP_EncryptInit(&ctx, EVP_bf_cbc(  ), key, iv);
    return 1;
}

Note that multiple implementations of the seed_prng function are provided in Chapter 4. It returns 0 if the pseudorandom number generator cannot be seeded securely. We return an error status from our setup function in this case, so we don’t need to check the return value of RAND_pseudo_bytes when we call it. Also, you may want to use raw entropy. See Chapter 4 for more information.

Another thing to note is that the party decrypting the data will need to initialize its cipher context with the same initialization vector created here. Passing the initialization vector in the clear is OK, but it should probably be MAC’d so that the receiver can detect tampering. If NULL is passed in for an IV, an array filled with zeros is used. Note that IVs can be used in all modes except ECB. In ECB mode, you can still pass in an IV, but block ciphers will ignore it.

Setting up for decryption is generally easier, because we already know the key and the IV used. Example 6-2 shows how to set up for decryption under the same configuration.

#include <openssl/evp.h>

void setup_for_decryption(char *key, char *iv)
{
    EVP_CIPHER_CTX ctx;

    EVP_DecryptInit(&ctx, EVP_bf_cbc(  ), key, iv);
}

Subsequent calls to EVP_EncryptInit or EVP_DecryptInit will change the value of any non-null parameter as long as the cipher type parameter is set to NULL. Otherwise, the context is completely reinitialized. Additionally, the key and the IV can both be set to NULL on the first call to these functions, and set separately later. This is necessary when you specify a cipher and then change the key length from the default. Of course, you will need to at least provide a valid key before encryption begins.

Many ciphers take a variable key length, which can be easily set after initialization using the call EVP_CIPHER_CTX_set_key_length . For example, we can set the Blowfish key length to 64 bits, as follows:

EVP_EncryptInit(&ctx, EVP_bf_ecb(  ), NULL, NULL);
EVP_CIPHER_CTX_set_key_length(&ctx, 8);
EVP_EncryptInit(&ctx, NULL, key, NULL);

In this case, we set the key with a second call to EVP_EncryptInit after we specify the key length.

When using this functionality, make sure you only set the key length to a valid value for the cipher.

If we wish to check the default key length of a cipher object, we can use the call EVP_CIPHER_key_length . For example, the following will show us the default key length for Blowfish:

printf("%d\n", EVP_CIPHER_key_length(EVP_bf_ecb(  ));

We can also check to see the length of the keys a cipher context is using:

printf("%d\n", EVP_CIPHER_CTX_key_length(&ctx));

For other cipher parameters, OpenSSL provides a generic call, EVP_CIPHER_CTX_ctrl . Currently, this call can only set or query the effective key strength in RC2 or the number of rounds used in RC5.

int EVP_CIPHER_CTX_ctrl(EVP_CIPHER_CTX *ctx, int type, int arg, void *ptr);

For example, to query the effective key bits in an RC2 cipher context, storing the result in a variable called kb:

EVP_CIPHER_CTX_ctrl(&ctx, EVP_CTRL_GET_RC2_KEY_BITS, 0, &kb);

And to set the effective key strength of RC2 to 64 bits:

EVP_CIPHER_CTX_ctrl(&ctx, EVP_CTRL_SET_RC2_KEY_BITS, 64, NULL);

Setting and querying RC5 rounds works the same way. Remember from our previous discussion that OpenSSL is limited to 8, 12, or 16 rounds for RC5.

Another desirable option to set in a cipher context is whether padding is used. Without padding, the size of the ciphertext will always be the same size as the plaintext. On the downside, the length of the data encrypted must be an exact multiple of the block size. With padding, any length in bytes is feasible, but the resulting ciphertext can be up to a block longer than the plaintext. Unfortunately, OpenSSL versions through 0.9.6c do not allow padding to be disabled. This changes in Version 0.9.7, which has a function called EVP_CIPHER_CTX_set_padding that takes a pointer to a cipher context, and an integer that represents a Boolean value (0 for no padding, 1 for padding).

Once the c ipher context is initialized, there are two steps to encryption with the EVP interface: updating and finalization. When you have data to encrypt, you pass the data to the update function, along with a pointer to where you’d like any output to go. There may or may not be output as the result of an update. If the cipher can encrypt one or more entire blocks of data, it will do so. Any leftover data will be buffered and processed either during the next call to the update function or during the call to finalize. When calling finalize, any leftover data is padded and encrypted. If there is no leftover data, a block of pad is encrypted.^[2] As with updating, you must tell the routine where to store resulting data.

The update function is EVP_EncryptUpdate .

int EVP_EncryptUpdate(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl,
                      unsigned char *in, int inl);

When using a block-based cipher mode (ECB or CBC), the amount of output written can be both larger and smaller than the length of the input, due to internal buffering and padding. If you’re using a cipher with 8-byte (64-bit) blocks, the output could be up to 7 bytes smaller than the input, or up to 7 bytes larger.^[3] If you are encrypting incrementally with a single key, and are producing packets of data, this is good to keep in mind. If, instead, you’re encrypting to a single buffer, you will always avoid overflow by making the output buffer an entire block bigger than the input buffer (the extra block may fill with padding). Optionally, you can manually keep track of exactly how much data will be output as a function of how much data was input.

The finalization function is EVP_EncryptFinal .

int EVP_EncryptFinal(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl);

Currently, this function always outputs. However, in the forthcoming 0.9.7 release it will not place anything in the output buffer at all if padding is turned off. In such a case, if there is any buffered data, the function returns an error. Additionally,Version 0.9.7 adds an EVP_EncryptFinal_ex call that should be used when a context has been initialized by EVP_EncryptInit_ex.

Example 6-3 shows the implementation of a function that takes a pointer to an initialized EVP cipher context, a buffer to encrypt, an associated length, and a pointer to an integer. The function then encrypts the data 100 bytes at a time into a heap-allocated buffer, which is the function’s return value. The length of the resulting ciphertext is passed back in the address specified by the final parameter.

#include <openssl/evp.h>

char *encrypt_example(EVP_CIPHER_CTX *ctx, char *data, int inl, int *rb)
{
    char *ret;
    int  i, tmp, ol;

    ol = 0;
    ret = (char *)malloc(inl + EVP_CIPHER_CTX_block_size(ctx));
    for (i = 0;  i < inl / 100;  i++)
    {
        EVP_EncryptUpdate(ctx, &ret[ol], &tmp, &data[ol], 100);
        ol += tmp;
    }
    if (inl % 100)
    {
        EVP_EncryptUpdate(ctx, &ret[ol], &tmp, &data[ol], inl%100);
        ol += tmp;
    }
    EVP_EncryptFinal(ctx, &ret[ol], &tmp);
    *rb = ol + tmp;
    return ret;
}

Factoring in the block length (done by calling EVP_CIPHER_CTX_block_size) is unnecessary when using a stream cipher, or when using the CFB or OFB cipher modes, since there is no padding in those cases. As a result, the cipher can output encrypted data as needed, without having to buffer any plaintext.^[4]

The above example works well when we can afford to encrypt everything into a single buffer before processing the ciphertext. It doesn’t work so well if we need to deal with ciphertext incrementally. For example, we might wish to send blocks of data as quickly as possible, and not wait for all data to be processed. Example 6-4 shows a solution for such a scenario. Data to be encrypted is sent to incremental_encrypt as needed. When there’s data to be sent, incremental_encrypt calls incremental_send , which is simply a stub, but it can place those blocks currently encrypted on the network. When all data to be encrypted has been passed to incremental_encrypt, then incremental_finish is called.

#include <openssl/evp.h>

int incremental_encrypt(EVP_CIPHER_CTX *ctx, char *data, int inl)
{
    char *buf;
    int  ol;
    int  bl = EVP_CIPHER_CTX_block_size(ctx);

    /* Up to the block size - 1 chars can be buffered up.  Add that to the length
     * of the input, and then we can easily determine the maximum number of
     * blocks output will take by integer divison with the block size.
     */
    buf = (char *)malloc((inl + bl - 1) / bl * bl);
    EVP_EncryptUpdate(ctx, buf, &ol, data, inl);
    if (ol)
        incremental_send(buf, ol);
    /* incremental_send must copy if it wants to store. */
    free(buf);
    return ol;
}

/* Also returns the number of bytes written. */
int incremental_finish(EVP_CIPHER_CTX *ctx)
{
    char *buf;
    int  ol;
  
    buf = (char *)malloc(EVP_CIPHER_CTX_block_size(ctx));
    EVP_EncryptFinal(ctx, buf, &ol);
    if (ol)
        incremental_send(buf, ol);
    free(buf);
    return ol;
}

Note that the number of bytes written by EVP_EncryptFinal should always be 8 when using 64-bit blocks and when padding is enabled.

As expected, the decryption API looks similar to the encryption API. After cipher initialization, two methods are involved, EVP_DecryptUpdate and EVP_DecryptFinal . You can pass as much data as you want into EVP_DecryptUpdate.

When using a block-based mode (ECB or CBC), you can pass in partial blocks of text, but EVP_DecryptUpdate will output only whole blocks. The rest will be stored until there is more data to be processed, or until EVP_DecryptFinal is called. Moreover, if the context’s cached ciphertext plus the length of the new ciphertext is exactly block-aligned, the entire final block will be held in the context instead of getting output. With CFB and OFB modes and with stream ciphers, there’s no padding, so the size of the resulting ciphertext will be equal to the size of the plaintext.

If using a block-based mode, EVP_DecryptFinal first checks to see if the padding on the last block is in the right format. If not, the function returns 0, signifying failure. Otherwise, it flushes any remaining data to the buffer passed in as the second argument, and writes the number of bytes flushed into the third argument, a reference parameter. For other modes, this call does nothing.

Example 6-5 shows a simple function that decrypts a block of encrypted text into a dynamically allocated buffer and returns it. This function can be used to incrementally decrypt, and thus requires EVP_DecryptFinal to be called when a block-based mode is used. Of course, EVP_DecryptInit should always be called before passing a context to this function.

char *decrypt_example(EVP_CIPHER_CTX *ctx, char *ct, int inl)
{
    /* We're going to null-terminate the plaintext under the assumption it's
     * non-null terminated ASCII text.  The null can be ignored otherwise.
     */
    char *pt = (char *)malloc(inl + EVP_CIPHER_CTX_block_size(ctx) + 1);
    int  ol;
  
    EVP_DecryptUpdate(ctx, pt, &ol, ct, inl);
    if (!ol) /* there's no block to decrypt */
    {
        free(pt);
        return NULL;
    }
    pt[ol] = 0;
    return pt;
}

As is the case with encryption, factoring in the cipher’s block length is strictly only necessary when using a block-based cipher mode.

Using the above function, and the ones we previously developed, let’s look at Example 6-6, which encrypts a 15-byte string using Blowfish in CBC mode, then passes it into decrypt_example in two parts. Note, we use the macro EVP_MAX_BLOCK_LENGTH, which exists only in 0.9.7 and later. If you are using an earlier version of OpenSSL, you can define this macro to the value “64”, which is the largest possible block size in 0.9.6c and earlier.

int main(int argc, char *argv[])
{
    EVP_CIPHER_CTX ctx;
    char           key[EVP_MAX_KEY_LENGTH]; 
    char           iv[EVP_MAX_IV_LENGTH];
    char           *ct, *out;
    char           final[EVP_MAX_BLOCK_LENGTH];
    char           str[] = "123456789abcdef";
    int            i;

    if (!seed_prng(  ))
    {
        printf("Fatal Error!  Unable to seed the PRNG!\n");
        abort(  );
    }

    select_random_key(key, EVP_MAX_KEY_LENGTH);
    select_random_iv(iv, EVP_MAX_IV_LENGTH);

    EVP_EncryptInit(&ctx, EVP_bf_cbc(  ), key, iv);
    ct = encrypt_example(&ctx, str, strlen(str), &i);
    printf("Ciphertext is %d bytes.\n", i);

    EVP_DecryptInit(&ctx, EVP_bf_cbc(  ), key, iv);
    out = decrypt_example(&ctx, ct, 8); 
    printf("Decrypted: >>%s<<\n", out);
    out = decrypt_example(&ctx, ct + 8, 8); 
    printf("Decrypted: >>%s<<\n", out);
    if (!EVP_DecryptFinal(&ctx, final, &i))
    {
        printf("Padding incorrect.\n");
        abort(  );
    }
    final[i] = 0;
    printf("Decrypted: >>%s<<\n", final);
}

If we run this example, the first time we try to output decrypted plaintext, we will see nothing, even though we fed the decryption routine a full block of data. Note that when we feed the remainder of the data, we are passing in eight bytes, because the encryption routine padded the data to a block-aligned length. At that point, one block will be output, and the second block will be held in reserve until there is more data, or until EVP_DecryptFinal is called.

If we were to change the cipher to RC4, the above example would compile, but give slightly incorrect output. The length of the encrypted text would be 15 bytes, not 16, due to the lack of padding. As a result, passing in 16 bytes of ciphertext to the decrypt routine will cause a block of garbage to be decrypted. Changing the third argument in the second call to decrypt_example to seven fixes the problem.

It is almost never desirable to use ECB mode for encryption. Schneier recommends it for encrypting other keys or other situations in which the data is short and random. However, this advice applies only when the key length is no larger than the cipher block length. Additionally, if you wish to include an integrity check alongside your data (which is almost always a good idea), ECB again becomes undesirable.

Another occasion when people think to use ECB is when encrypting datagrams to be sent over a UDP connection. The problem is that packets may show up out of order, or not at all. All basic cipher modes besides ECB require an ordered, reliable stream of data.

CBC mode is much better suited to handling UDP traffic. A single key can be used to encrypt all data, but each packet gets initialized with a randomly chosen IV, which can be sent alongside the encrypted data.

Counter mode is slightly better suited than CBC for encrypting UDP traffic. One advantage of counter mode over OFB and other modes that simulate stream ciphers is that it can inherently survive data loss—the current state of the counter can be passed in the clear each time a packet is sent. Another major advantage of counter mode is that it allows for parallelization, which is not supported by any of the default modes, except for ECB (a CBC-based approach could parallelize at the packet level, but could not parallelize the processing of data within a single packet). Another feature unique to OFB is that most of the work can be done offline. That is, you can generate a keystream before there is even data available to encrypt, while you have spare CPU cycles.

Additionally, because counter mode essentially supports arbitrarily jumping around a data stream, it can enable file encryption where random access to the data is still possible. Moreover, in theory, counter mode should be able to handle the UDP encryption problem without a rekey for every packet. However, current limitations of the OpenSSL library make that goal difficult to achieve, although the forthcoming 0.9.7 release will fix the problem.

OpenSSL currently doesn’t support counter mode, but it is simple to implement yourself. Counter mode effectively turns a block cipher into a stream cipher. For each block of plaintext, we encrypt a counter of exactly one block in length, which gets XOR’d with the corresponding block of plaintext. Then, the counter is incremented in some way. Increasing the counter by one or using a PRNG to increment the counter both work. The PRNG doesn’t even need to be cryptographically secure, though it must not repeat items before enumerating all possible values.

We can prevent the attacker from seeing the counter if need be. Let’s say that, in addition to an agreed-upon key, we share a second secret of the same length. Additionally, a sequence number is sent in the clear with each packet. To generate the counter used to encrypt the first data block in that packet, we concatenate the second shared secret with the sequence number, and cryptographically hash that. We show how to use cryptographic hashes in Chapter 7.

Counter mode is easily implemented by keeping track of a counter that is encrypted in ECB mode once for each block. In our UDP example, each packet would have several blocks. Only one counter should need to be sent per packet, because the receiving end should be able to recreate the counter values for the subsequent blocks.

Example 6-7 shows an implementation of counter mode that will work for any cipher in ECB mode. There is just one function, counter_encrypt_or_decrypt , as encryption and decryption are identical in counter mode.

int counter_encrypt_or_decrypt(EVP_CIPHER_CTX *ctx, char *pt, char *ct, int len,
                               unsigned char *counter);

In this example, the counter must be the same size as the block size of the cipher we’re using (we query the cipher’s block size to specify that value). Cipher block sizes are usually bigger than 32 bits, so it is best to represent the counter as an array of unsigned bytes. This function also modifies the counter in place as it processes blocks. To increment the counter, we simply increment the leftmost byte until it rolls over, at which point we increment the byte next to it, and so on. Note that it’s extremely important not to reuse counter values with a single key, not even across a reboot. If there is any chance of reusing a counter, be sure to change the key.

The next function will return -1 if it determines that the cipher is not in ECB mode (we need to use this mode in order to implement counter mode; do not take this example as an endorsement of ECB in general). This is accomplished by calling EVP_CIPHER_CTX_mode , which returns a number. If that number is equal to the constant EVP_CIPH_ECB_MODE, then we know the cipher was initialized improperly. See the documentation on the book’s web site for a list of other valid mode constants.

Note that the code in Example 6-7 should not be used without also using a MAC to provide data integrity, as discussed in Chapter 7.

int counter_encrypt_or_decrypt(EVP_CIPHER_CTX *ctx, char *pt, char *ct, int len,
              unsigned char *counter)
{
    int  i, j, where = 0, num, bl = EVP_CIPHER_CTX_block_size(ctx);
    char encr_ctrs[len + bl]; /* Encrypted counters. */

    if (EVP_CIPHER_CTX_mode(ctx) != EVP_CIPH_ECB_MODE)
        return -1;
    /* <= is correct, so that we handle any possible non-aligned data. */
    for (i = 0;  i <= len / bl;  i++)
    {
        /* Encrypt the current counter. */
        EVP_EncryptUpdate(ctx, &encr_ctrs[where], &num, counter, bl);
        where += num;
        /* Increment the counter. Remember it's an array of single characters */
        for (j = 0;  j < bl / sizeof(char);  j++)
        {
            if (++counter[j])
                break;
        }
    }
    /* XOR the key stream with the first buffer, placing the results in the
     * second buffer.
     */
    for (i = 0;  i < len;  i++)
        ct[i] = pt[i] ^ encr_ctrs[i];
    return 1; /* Success. */
}

As we discussed, the above example requires the state of the counter to be kept externally. Another option is to make a COUNTER_CTX data type that could hold a pointer to the underlying cipher context and the current state of the counter. However, this less-abstract API makes it easier to use in a situation in which the counter may need to be reset explicitly after desynchronization, such as when dealing with UDP traffic.

OpenSSL has direct support only for HMAC, but there are several other kinds of MACs that are easily implemented. Some of the simplest and most useful are based on block ciphers. A large part of why HMAC is so popular is that it uses a cryptographic one-way hash as its underlying cryptographic primitive. A one-way hash was advantageous in the days when it was difficult to export strong cryptography, since true one-way functions were not restricted in any way.

However, MACs based on block ciphers can be compelling. First, such constructions can be faster, because HMAC must perform two hash operations. Second, those looking to keep total code size small will appreciate being able to reuse block cipher code (unless you specifically add the algorithm, the hash function won’t get linked in if you don’t use it). This advantage is especially appealing to people who wish to push cryptography into hardware.

HMAC does have provable security properties, but so do many cipher-based MACs, such as CBC-MAC, UMAC, and XCBC-MAC. In all cases, security proofs rely on an assumption of security in the underlying cryptographic primitive. For example, HMAC is secure, assuming that the underlying hash algorithm used with it is secure, and UMAC and XCBC-MAC are secure, assuming that the underlying block cipher is secure. It’s smart to keep security assumptions to a minimum. For example, if you use a block cipher and a hash function, a break of either is likely to break the entire system. In such a case, your system is only as secure as its weakest link.

CBC-MAC is certainly unencumbered by patents, and XCBC-MAC and XOR-MAC are probably unencumbered. Some of the theoretical work UMAC is based upon might actually be covered by patent, so use it with caution.

#ifndef CBC_MAC_H_  _
#define CBC_MAC_H_  _

#include <openssl/evp.h>
#include <stdlib.h>


typedef struct CBCMAC_CTX_st
{
    EVP_CIPHER_CTX cctx;
    char           cbcstate[CBCMAC_MAX_BYTES];
    char           workspace[CBCMAC_MAX_BYTES];
    short          worklen;
} CBCMAC_CTX;

int CBCMAC_Init(CBCMAC_CTX *mctx, EVP_CIPHER *c, const unsigned char *k);
int CBCMAC_Update(CBCMAC_CTX *mctx, const char *data, int len);
int CBCMAC_Final(CBCMAC_CTX *mctx, unsigned char *out, int *outl);
int CBCMAC(EVP_CIPHER *c, const char *key, int key_len, 
       unsigned char *str, int sz, unsigned char *out, int *outlen);

#endif

#include "cbcmac.h"

int CBCMAC_Init(CBCMAC_CTX *mctx, EVP_CIPHER *c, const unsigned char *k)
{
    int i, bl;

    EVP_EncryptInit(&(mctx->cctx), c, (unsigned char *)k, 0);
    if (EVP_CIPHER_CTX_mode(&(mctx->cctx)) != EVP_CIPH_ECB_MODE) 
        return -1;
    mctx->worklen = 0;
    bl = EVP_CIPHER_CTX_block_size(&(mctx->cctx));
    for (i = 0;  i < bl;  i++)
        mctx->cbcstate[i] = 0;
    return 0;
}

/* We hand implement CBC-mode because of the requirements for the last block,
 * and to avoid dynamic memory allocation.
 */
int CBCMAC_Update(CBCMAC_CTX *mctx, const char *data, int len)
{
    int bl, i, n = 0, outl;

    bl = EVP_CIPHER_CTX_block_size(&(mctx->cctx));

    if (mctx->worklen)
    {
        n = bl - mctx->worklen;
        if (n > len) /* Not enough bytes passed in to fill block buffer. */
        {
            for (i = 0;  i < len;  i++)
                mctx->workspace[mctx->worklen + i] = data[i];
            mctx->worklen += len;
            return 0;
        }
        else
        {
            for (i = 0;  i < n;  i++)
                mctx->workspace[mctx->worklen + i] = data[i] ^ mctx->cbcstate[i];
            EVP_EncryptUpdate(&(mctx->cctx), mctx->cbcstate, &outl, 
                             mctx->workspace, bl);
        }
    }
    while (n < len)
    {
        for (i = 0;  i < bl;  i++)
            mctx->workspace[i] = data[n + i] ^ mctx->cbcstate[i];
        n = n + bl;
        EVP_EncryptUpdate(&(mctx->cctx), mctx->cbcstate, &outl, 
                         mctx->workspace, bl);
    }
    mctx->worklen = len - n;
    for (i = 0;  i < mctx->worklen;  i++)
        mctx->workspace[i] = data[n + i];
    return 0;
}

int CBCMAC_Final(CBCMAC_CTX *mctx, unsigned char *out, int *outl)
{
    int i, bl = EVP_CIPHER_CTX_block_size(&(mctx->cctx));

    /* Pad with null bytes if necessary. In reality, we just copy in the
     * CBC state, since x ^ 0 = x. 
     */
    if (mctx->worklen)
    {
        for (i = mctx->worklen;  i < bl;  i++)
            mctx->workspace[i] = mctx->cbcstate[i];
        EVP_EncryptUpdate(&(mctx->cctx), out, outl, mctx->workspace, bl);
    }
    else
    {
        for (i = 0;  i < bl;  i++)
            out[i] = mctx->cbcstate[i];
        *outl = bl;
    }
    return 0;
}

int CBCMAC(EVP_CIPHER *c, const char *key, int key_len, unsigned char *str,
       int sz, unsigned char *out, int *outlen)
{
    CBCMAC_CTX x;
    int        e;

    if ((e = CBCMAC_Init(&x, c, key)))
        return e;
    if ((e = CBCMAC_Update(&x, str, sz)))
        return e;
    return CBCMAC_Final(&x, out, outlen);
}

#ifndef XCBC_MAC_H_  _
#define XCBC_MAC_H_  _

#include <openssl/evp.h>
#include <stdlib.h>

#define XCBC_MAX_BYTES   32

typedef struct XCMAC_CTX_st 
{
    EVP_CIPHER_CTX cctx;
    char           dk1[XCBC_MAX_BYTES];
    char           dk2[XCBC_MAX_BYTES];
    char           dk3[XCBC_MAX_BYTES];
    char           cbcstate[XCBC_MAX_BYTES];
    char           workspace[XCBC_MAX_BYTES];
    short          worklen;
    short          started;
} XCMAC_CTX;

int XCMAC_Init(XCMAC_CTX *mctx, EVP_CIPHER *c, const unsigned char *k);
int XCMAC_Update(XCMAC_CTX *mctx, const char *data, int len);
int XCMAC_Final(XCMAC_CTX *mctx, unsigned char *out, int *outl);
int XCMAC(EVP_CIPHER *c, const char *key, unsigned char *str, int sz,
          unsigned char *out, int *outlen);

#endif

#include "xcbcmac.h"

/* These are recommended by Rogaway. */
static char g1[XCBC_MAX_BYTES] = 
{
    0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
    0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
    0x01, 0x01, 0x01, 0x01, 0x01, 0x01 
};

static char g2[XCBC_MAX_BYTES] = 
{
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02,
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02,
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02 
};

static char g3[XCBC_MAX_BYTES] = 
{
    0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03,
    0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03,
    0x03, 0x03, 0x03, 0x03, 0x03, 0x03 
};

/* This is the extra plaintext for when generating the second half of a key  
 * when the block size is half the key length.
 */
static char g4[XCBC_MAX_BYTES] = 
{
    0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04,
    0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04,
    0x04, 0x04, 0x04, 0x04, 0x04, 0x04
};

int XCMAC_Init(XCMAC_CTX *mctx, EVP_CIPHER *c, const unsigned char *k)
{ 
    EVP_CIPHER_CTX tctx;
    int            i, outl, bl, kl;

    EVP_EncryptInit(&tctx, c, (unsigned char *)k, 0);

    kl = EVP_CIPHER_CTX_key_length(&tctx);
    bl = EVP_CIPHER_CTX_block_size(&tctx);

    if (kl != bl && bl * 2 != kl)
        return -1;
    EVP_EncryptUpdate(&tctx, mctx->dk1, &outl, g1, bl);

    if (kl != bl)
        EVP_EncryptUpdate(&tctx, &(mctx->dk1[bl]), &outl, g4, bl);
    EVP_EncryptUpdate(&tctx, mctx->dk2, &outl, g2, bl);
    EVP_EncryptUpdate(&tctx, mctx->dk3, &outl, g3, bl);
  
    EVP_EncryptInit(&(mctx->cctx), c, mctx->dk1, 0);

    if (EVP_CIPHER_CTX_mode(&(mctx->cctx)) != EVP_CIPH_ECB_MODE) 
        return -2;

    mctx->worklen = 0;
    mctx->started = 0;
    for (i = 0;  i < bl;  i++) 
        mctx->cbcstate[i] = 0;
    return 0;
}

int XCMAC_Update(XCMAC_CTX *mctx, const char *data, int len) 
{
    int bl, i, n = 0, outl;

    if (!len) 
        return 0;

    bl = EVP_CIPHER_CTX_block_size(&(mctx->cctx));
    for (i = 0;  i < len;  i++) 
    {
        if (!mctx->worklen && mctx->started) 
            EVP_EncryptUpdate(&(mctx->cctx), mctx->cbcstate, &outl,
                             mctx->workspace, bl);
        else 
            mctx->started = 1;
        mctx->workspace[mctx->worklen] = data[n++] ^ mctx->cbcstate[mctx->worklen];
        mctx->worklen++;
        mctx->worklen %= bl;
    }
    return 0;
}

int XCMAC_Final(XCMAC_CTX *mctx, unsigned char *out, int *outl) 
{
    int i, bl = EVP_CIPHER_CTX_block_size(&(mctx->cctx));

    if (!mctx->started) 
        return -1;
    if (mctx->worklen) 
    {
        /* Pad and XOR with K2, then encrypt */
        mctx->workspace[mctx->worklen] = 0x90 ^ mctx->cbcstate[mctx->worklen];
        for (i = mctx->worklen + 1;  i < bl;  i++) 
            mctx->workspace[i] = mctx->cbcstate[mctx->worklen]; /* ^ 0 */
        for (i = 0;  i < bl;  i++) 
            mctx->workspace[i] ^= mctx->dk2[i];
    } 
    else 
    {
        /* XOR with K3, then encrypt. */
        for (i = 0;  i < bl;  i++) 
            mctx->workspace[i] ^= mctx->dk3[i];
    }
    EVP_EncryptUpdate(&(mctx->cctx), out, outl, mctx->workspace, bl);
    return 0;
}

int XCMAC(EVP_CIPHER *c, const char *key, unsigned char *str, int sz, 
      unsigned char *out, int *outlen) 
{
    XCMAC_CTX x;
    int       e;

    if ((e = XCMAC_Init(&x, c, key))) 
        return e; 
    if ((e = XCMAC_Update(&x, str, sz))) 
        return e; 
    return XCMAC_Final(&x, out, outlen);
}

The low-level interface to Diffie-Hellman provided by OpenSSL consists of a structure of the type DH and a set of functions that operate on that structure. The DH structure and functions are made accessible by including the openssl/dh.h header file. The DH structure itself contains many data members that are of little or no interest to us, but four members are important, as shown in the following abbreviated DH structure definition:

typedef struct dh_st
{
    BIGNUM *p;
    BIGNUM *g;
    BIGNUM *pub_key;
    BIGNUM *priv_key;
} DH;

The p and g members, known as Diffie-Hellman parameters, are public values that must be shared between the two parties using the algorithm to create a shared secret. Because they’re public values, no harm will come from a potential attacker discovering them, which means that they can be agreed upon beforehand or exchanged over an insecure medium. Typically, one side of the conversation generates the parameters and shares them with the peer.

The p member is a prime number that is randomly generated. For temporary keys, it is typically at least 512 bits in length, whereas persistent keys should more appropriately be at least 1,024 bits. These sizes correspond to the notion of ephemeral and static keys presented in Chapter 5. The prime that will be used for p is generated such that (p-1)/2 is also prime. Such a prime is known as a safe or strong prime . The g member, also known as the generator , is usually a small number greater than one. OpenSSL functions best when this number is either two or five. A value of two is sometimes used for performance reasons, but keep in mind that faster key generation also means that an attacker can break the algorithm faster. In general, we recommend that a value of five be used.

Using the two public parameters, p and g, each pair chooses a random large integer for the priv_key member. A value for the pub_key member is computed from the priv_key member and shared with the peer. It is important that only the value of the pub_key member be shared. The value of the priv_key member should never be shared.

Using the values of priv_key and the peer’s pub_key, each peer can independently compute the shared secret. The shared secret is suitable for use as the key for a symmetric cipher. The entire exchange between the peers can be done over an insecure medium. Even if someone captures the parameter and key exchange, the attacker will not be able to determine the shared secret.

Diffie-Hellman requires that both parties involved in the key exchange use the same parameters to generate public keys. This means the parameters either need to be agreed upon and exchanged before the conversation begins, or the parameters must be generated and exchanged as part of the key exchange. Either way, the parameters must first be generated by one party and given to the other, or perhaps generated by a third party and given to both. For the purposes of this discussion, we will assume that the generation of the parameters will be done as part of the key exchange process, although this is often not desirable because it can take a significant amount of time to generate the parameters.

The participants in the key agreement must first agree which party will be responsible for generating the parameters that they’ll both use. In a client/server scenario, the parameters are usually generated by the server. Often, the server will generate the parameters when it starts up or retrieve them from a file that contains already generated parameters, and use the same parameters for each client that connects. It is also common for both the client and server to have a copy of the parameters built into the respective applications.

OpenSSL provides the function DH_generate_parameters , which will create a new DH object that is initialized with fresh values for p and g. The generation of the parameters generates a value only for p. The value of g is specified by the caller and is not chosen randomly by OpenSSL. The value of g should be a small number greater than one, usually either two or five.

DH *DH_generate_parameters(int prime_len, int generator,
                            void (*callback)(int, int, void *), void *cb_arg);

Using the generation function alone can be dangerous. While the generation function does have validity checks for the prime that it generates, it could generate a prime that is not suitable for use with the algorithm. For this reason, the function DH_check should always be used to ensure that the generated prime is suitable.

int DH_check(DH *dh, int *codes);

If the function encounters an error unrelated to the validity of the generated prime, the return will be zero; otherwise, it will be nonzero. When the function returns successfully, the codes argument will contain a bit mask that indicates whether the parameters are suitable for use or not. If none of the bits is set, the parameters should be considered suitable for use. If any of the following bits are set, the parameters may not be suitable for use. In most cases, the parameters should be thrown away, and new ones should be generated.

Once the parameters have been generated, they can be transmitted to the peer. The details of how the data is sent depend on the medium that is being used for the exchange. To transmit the parameters over a TCP connection, the BIGNUM functions BN_bn2bin and BN_bin2bn are obvious candidates.

The party that generates the parameters calls DH_generate_parameters to obtain a DH object. The party that is receiving the parameters must also obtain a DH object. This is easily done by calling the function DH_new , which will allocate and initialize a new DH object. The parameters that are received from the peer can then be directly assigned to the DH object’s p and g data members, using the appropriate BIGNUM functions.

When we’re done with a DH object, we must be sure to destroy it by calling the function DH_free , and passing the pointer returned by either DH_generate_parameters or DH_new as the only argument.

Now that parameters have been generated and received by the two peers, each peer must generate a key pair and exchange their public keys. Remember that the private key must not be shared at all. Once this is done, each peer can independently compute the shared secret, and the algorithm will have done its job. With authenticated Diffie-Hellman, the public/private key pairs can persist beyond usage for a single key-agreement. In these cases, we must be wary of a special class of attack against Diffie-Hellman, which is discussed at the end of this section.

OpenSSL provides the function DH_generate_key for generating public and private keys. It requires as its only argument a DH object that has the parameters, p and g, filled in. If the keys are generated successfully, the return from the function will be nonzero. If an error occurs, the return will be zero.

Once the keys have been generated successfully, each peer must exchange their public key with the other peer. The details of how to exchange the value of the public key varies depending on the medium that is being used, but in a typical case in which the communication is taking place over an established TCP connection, the functions BN_bn2bin and BN_bin2bn will once again work for the exchange of the DH object’s pub_key data member.

With the parameters and public key now exchanged, each party in the exchange can use his own private key and the peer’s public key to compute the shared secret using the function DH_compute_key .

int DH_compute_key(unsigned char *secret, BIGNUM *pub_key, DH *dh);

After the shared secret is computed, the DH object is no longer needed unless more secrets will be generated and exchanged. It can be safely destroyed using the DH_free function.

In certain cases, Diffie-Hellman can be subject to a type of attack known as a small-subgroup attack . This attack results in a reduction of the computational complexity of brute-forcing the peer’s private key value. Essentially, a small-subgroup attack can result in the victim’s private key being discovered. There are several different methods of protecting Diffie-Hellman against this type of attack. The simplest method is to use ephemeral keying. If both parties stick to ephemeral keying and use a separate method of authentication, small-subgroup attacks are thwarted. This isn’t always feasible, however, mostly due to computational expense. If static keys will be used, two simple mathematical checks can be performed on the public key received from a peer to ensure these attacks aren’t possible. If the key passes both tests, it’s safe to use. The first test verifies that the supplied key is greater than 1 and less than the value of the p parameter. The second test computes y ^q mod p, in which y is the key to test and q is another large prime. If the result of this operation is 1, the key is safe; otherwise, it is not. The q parameter is not generated by OpenSSL even though there is a placeholder for it in the DH structure. An algorithm for generating q can be found in RFC 2631. If you’re interested in the other methods or more detailed information on the attack, we recommend that you read RFC 2785.

When we began our discussion of Diffie-Hellman, we mentioned that it provides key agreement and authentication. Use of the authentication features of this protocol is not very common; thus, pairing Diffie-Hellman with another algorithm for authentication is often done. The threat is that mistakenly leaving out authentication can lead to susceptibility to man-in-the-middle attacks. To execute such an attack, the attacker sits in between two hosts that are trying to communicate and intercepts all of the messages. For example, suppose that Alice and Bob plan to use Diffie-Hellman to make a shared secret. Charlie could intercept all messages from Alice to Bob and all messages from Bob to Alice. From this position, Charlie can agree upon a key with Alice and a different key with Bob. When the attacker receives a message from Alice, he decrypts it with the key he negotiated with her and reads the message. He can then encrypt the message using the key he negotiated with Bob and pass it along to him. Alice and Bob will believe that they’re communicating securely. They’ll be completely unaware that Charlie is eavesdropping and worse, possibly even altering their messages, inserting forged messages, or not passing the messages along at all.

To alleviate this problem, Diffie-Hellman should always be used with some method of authentication, most commonly from another algorithm. This is accomplished by authenticating the messages containing public values for the Diffie-Hellman agreement. Using signatures, each party would exchange their public keys to use for signing before the conversation begins, and then sign the public value before sending it. The details will be explained in the following section.

Similar to the low-level interface to Diffie-Hellman, the low-level interface to DSA provided by OpenSSL consists of a DSA structure and a set of functions that operate on that structure. The DSA structure and functions are made accessible by including the openssl/dsa.h header file. The DSA structure itself contains many data members that are of little or no interest to us, but five members are important, as shown in the following abbreviated DSA structure definition:

typedef struct dsa_st
{
    BIGNUM *p;
    BIGNUM *q;
    BIGNUM *g;
    BIGNUM *pub_key;
    BIGNUM *priv_key;
} DSA;

The p, q, and g members, known as DSA parameters, are public values that must be generated before a key pair can be generated. Because they’re public values, no harm will come if a potential attacker discovers them. The same parameters can be safely used to generate multiple keys. In fact, RFC 2459 specifies a mechanism in which DSA parameters for a certificate can be inherited from the certificate of the issuer. Using parameter inheritance not only reduces the size of certificates, it also enforces the sharing of parameters.

The p member is a prime number that is randomly generated. Initially, the proposed standard fixed the length of the prime at 512 bits. Due to much criticism, this was later changed to allow a range between 512 and 1,024 bits. The length of the prime must be a multiple of 64 bits, however. OpenSSL does not enforce the 1,024-bit upper bound, but it’s not a good idea to use a prime larger than 1,024 bits—many programs may not be able to use the keys that result from such a large prime. The q member is a prime factor of p-1. The value of q must also be exactly 160 bits in length. The g member is the result of a mathematical expression involving a randomly chosen integer, as well as p and q.

Using the three public parameters (p, q, and g), the public and private keys can be computed. Public parameters used to compute the keys are required to generate or verify a digital signature. The parameters must therefore be exchanged along with the public key in order for the public key to be useful. Of course, the private key should never be distributed.

Like Diffie-Hellman, DSA requires generation of parameters before keys can be generated. Once a set of parameters is generated, many keys can be created from it. This does not mean that only people with keys created from the same parameters can interoperate; it simply means that in order to do so, all of the parameter values must be included as part of the public key.

The interface for generating DSA parameters is similar to the interface for generating Diffie-Hellman parameters. The function DSA_generate_parameters will create a new DSA object that is initialized with fresh values for p, q, and g. Generation of DSA parameters is more complex than Diffie-Hellman parameter generation, although it is typically much faster, especially for large key lengths.

DSA * DSA_generate_parameters(int bits, unsigned char *seed, int seed_len,
                              int *counter_ret, unsigned long *h_ret,
                                void (*callback)(int, int, void *), void *cb_arg);