Chapter 9. OS Fingerprinting

OS fingerprinting is the science of determining the operating systems in use on a remote network. Fingerprinting is one of the first steps in an attack. Most vulnerabilities are dependent on the target OS, so fingerprinting is a vital skill. Although you can never fingerprint with 100% accuracy, the science is evolving to approach that level.

When might you need OS fingerprinting? If a remote company hires you to perform vulnerability testing, it is better if they do not provide you with detailed knowledge of their network. Before taking a company tour to inspect their security architecture, the first phase of any security audit should be a “blind” intrusion attempt from the Internet. You start the way an attacker does: gathering information on an occult target before attacking. This also applies when doing an audit of your own networks. In this chapter, we demonstrate simple and advanced techniques for OS fingerprinting. We also show technologies that have automated the fingerprinting process, including the tools Nmap, p0f, Xprobe, and RING.

Telnet Session Negotiation

Telnet session negotiation (TSN) is the simplest way to determine a remote OS. All it requires is that you telnet to the server. It is surprising how many systems have telnet running for no reason. Worse, many networks respond with a banner that gives the exact OS version! Although this method is not elegant, it is nevertheless effective. TSN should be the first thing you check in fingerprinting.

It is worth noting that this weakness is rampant among software makers and is not limited to operating systems. For example, NTMail, a popular POP3 mail server from Gordano, returns the exact version of the software to anyone passing by on the Internet. Simply telnet to the default POP3 port (port 110) on a server running NTMail, and you learn the exact version (and even the owner’s key!). This access was provided so that Gordano could troubleshoot and also track piracy of their software. However, with the information it provides, a cracker can do a quick search for exploits for that version (such as the denial-of-service vulnerability affecting early versions of NTMail) and attack with ease. TSN is a classic method, but it is becoming less effective as administrators are learning to turn off their banners (except in programs such as NTMail, where you can’t).

TCP Stack Fingerprinting

TCP stack fingerprinting involves hurling a variety of packet probes at a target and predicting the remote OS by comparing changes in responses against a database. Nmap, by Fyodor of Insecure.org, is considered the best tool for the job. Nmap runs on Linux and Windows and can craft custom-fragmented packets.

Nmap Test

Let’s try downloading Nmap (http://www.insecure.org/nmap) and using it against a remote host, with the following command:

nmap -v -sS -O ###.com

In this case, we’re scanning a remote host running a pre-release version of Windows .NET Server RC2, so it’s going to be tough to accurately fingerprint.

Host ###.com (xxx.xx.xx.xx) appears to be up ... good.
Initiating SYN half-open stealth scan against ###.com (xxx.xx.xx.xx)
Adding TCP port 88 (state open).
Adding TCP port 17 (state open).
Adding TCP port 389 (state open).
Adding TCP port 9 (state open).
Adding TCP port 19 (state open).
Adding TCP port 1068 (state open).
Adding TCP port 636 (state open).
Adding TCP port 593 (state open).
Adding TCP port 1067 (state open).
Adding TCP port 53 (state open).
Adding TCP port 13 (state open).
Adding TCP port 464 (state open).
Adding TCP port 445 (state open).
Adding TCP port 135 (state open).
Adding TCP port 5000 (state open).
Adding TCP port 7 (state open).
Adding TCP port 1026 (state open).
Adding TCP port 3389 (state open).
The SYN scan took 0 seconds to scan 1523 ports.
For OSScan assuming that port 7 is open and port 1 is closed and neither are firewalled
Interesting ports on ###.com (xxx.xx.xx.xx):
(The 1505 ports scanned but not shown below are in state: closed)
Port       State       Service
7/tcp      open        echo
9/tcp      open        discard
13/tcp     open        daytime
17/tcp     open        qotd
19/tcp     open        chargen
53/tcp     open        domain
88/tcp     open        kerberos-sec
135/tcp    open        loc-srv
389/tcp    open        ldap
445/tcp    open        microsoft-ds
464/tcp    open        kpasswd5
593/tcp    open        http-rpc-epmap
636/tcp    open        ldapssl
1026/tcp   open        nterm
1067/tcp   open        instl_boots
1068/tcp   open        instl_bootc
3389/tcp   open        msrdp
5000/tcp   open        fics
TCP Sequence Prediction: Class=random positive increments
Difficulty=14410 (Worthy challenge)
Sequence numbers: 3AD7953F 3AD8570E 3AD97977 3ADA2100 3ADB1400 3ADB9658
Remote operating system guess: Windows 2000 RC1 through final release
Nmap run completed -- 1 IP address (1 host up) scanned in 1 second

Nmap was impressively close, but not quite correct. The challenge was a little unfair, though, since the OS is a pre-release version. We used this example to emphasize the fact that TCP stack fingerprinting is based on an empirical database that must be regularly updated.

Nmap Techniques

Fyodor has written a classic paper (listed in the references at the end of this chapter) that delves into the intricacies of the Nmap fingerprinting engine. Nmap uses the following techniques:

FIN probe: Sends a FIN packet to an open port and looks for a response. The correct RFC 793 behavior is to not respond, but incorrect implementations such as MS Windows send a RESET back.
BOGUS flag probe: First used by the Queso scanner, this sets an undefined flag in the TCP header of a SYN packet to help identify an OS.
TCP ISN sampling: Used to find patterns in the initial sequence numbers (ISNs) chosen by TCP implementations when responding to connection requests.
DF bit: Operating systems that set the IP “don’t fragment” bit give clues that can narrow down their identity.
TCP initial window: By checking the window size on returned packets, you can often identify the OS.
ACK value: Various OS implementations use distinct values for the ACK field.
ICMP error message quenching: Operating systems that correctly follow RFC 1812 limit the rate at which various error messages are sent. You can assay this implementation by sending many packets to a random high UDP port and counting the number of unreachables received.
ICMP message quoting: For a port-unreachable message, most OSs send only the required IP header + eight bytes back. However, Solaris sends back more than this standard, and Linux sends back even more than Solaris. This technique allows Nmap to recognize Linux and Solaris hosts even if they don’t have any ports listening.
ICMP error message echoing integrity: Nmap assays ICMP errors to detect subtle, OS-dependent changes.
Type of service: Changes in the type-of-service (TOS) value packets sent back in ICMP port-unreachable messages give clues about the remote OS.
Fragmentation handling: Uses variations in how different OSs handle overlapping IP fragments.
TCP options: Options vary by OS implementation, which can be useful in fingerprinting.
Exploit chronology: Perhaps the most elegant of all fingerprinting methods, this technique involves launching sequential denial-of-service attacks in increasing chronology (not recommended). After each attack, simply ping the target to see if it has crashed. When you finally crash the target, you will likely have narrowed the OS down to the granularity of a single service pack or hotfix.

Defeating Nmap

There have been attempts to provide fingerprinting countermeasures. One example is IP Personality (http://ippersonality.sourceforge.net), a Linux netfilter module that allows you to vary the IP stack behavior in response to particular attack probes. The patch allows you to emulate the behavior of any system listed in Nmap’s list of OS fingerprints. In essence, each variety of probe elicits a different “personality” from the module, resulting in a different response. Some features can even be applied to routed traffic and thus fool scans directed to machines that are behind the router.

Note that Nmap assumes that if a port is open, the service associated with that port number is up—not always a useful assumption. For example, some port monitoring programs hold ports open in an attempt to fool scanners and keep the connection open so they can spy on the attacker.

Special-Purpose Tools

It is worth noting that there are also special-purpose tools that have been designed to work on individual services. One example of this is used in IDENT fingerprinting. The Identification Protocol (IDENT) provides a means to determine the identity of a user of a particular TCP connection. Given a TCP port number pair, IDENT returns a character string that identifies the owner of that connection on the server’s system.

IDENT is a connection-based application on TCP. An IDENT server listens for TCP connections on TCP port 113. Once a connection is established, the IDENT server reads a line of data that specifies the connection of interest. If it exists, the system-dependent user identifier of the connection of interest is sent as the reply. The server may shut down the connection or continue to read and respond to multiple queries.

If you connect to a host’s IDENT server, you can determine its type, version, and (occasionally) compilation date. By matching this against an empirical database, you can often predict the target OS. An example of a tool to automate this process is identfp, a Perl tool written by F0bic of Synergy.net.

Passive Fingerprinting

Nmap launches fragmented packets against a target, also known as active fingerprinting . In contrast, passive fingerprinting uses a sniffer to quietly map a network without sending any packets.

Passive fingerprinting works because TCP/IP flag settings are specific to various operating system stacks. These settings vary from one TCP stack implementation to another and include the following:

Initial TTL (8 bits)
Window size (16 bits)
Maximum segment size (16 bits)
“Don’t fragment” flag (1 bit)
sackOK option (1 bit)
nop option (1 bit)
Window scaling option (8 bits)
Initial packet size (16 bits)

When combined, these flag settings provide a unique, 67-bit signature for every system. p0f (the passive OS fingerprinting tool) is an example of a passive fingerprinting tool (http://www.stearns.org/p0f/).

p0f performs passive OS fingerprinting based on information from a remote host when it establishes a connection to your system. This works because incoming packets often contain enough information to determine the source OS. Unlike active scanners such as Nmap, p0f can fingerprint without sending anything to the source host. The real advantage is that the source host (i.e., an attacker) is not aware that you are fingerprinting his machine. So even if he is well firewalled, his outgoing packets can betray the name and version of his OS.

p0f was written for Linux, but using cygwin you can run it on almost any version of Windows. The cygwin environment emulates a Unix environment on top of your Windows machine. It is available for free from http://www.cygwin.com. p0f also needs the WinPcap drivers to be installed. These are also free and are available from http://winpcap.polito.it.

Once these are installed, make sure to place p0f.fp in your /etc directory in the cygwin environment or in the current directory. p0f has the following syntax:

p0f [ -f file ] [ -i device ] [ -o file ] [ -s file ] [ -vKUtq ] 

 -f file   read fingerprint information from file
 -i device read packets from device
 -s file   read packets from file
 -o file   write output to file (best with -vt)
 -v        verbose mode
 -U        do not display unknown signatures
 -K        do not display known signatures
 -q        be quiet (do not display banners)
 -t        add timestamps

Verbose mode gives you information on the source and destination IP addresses and source and destination ports.

p0f relies on a database of known OS fingerprints. This database is stored in a file in the /etc directory called p0f.fp. Each entry in this file is a description of the unique TCP parameters specific to the first SYN packet sent by a remote party while establishing a connection.

These unique TCP parameters include window size (wss), maximum segment size (mss), the “don’t fragment” flag (DF), window scaling (wscale), the sackOK flag, the nop flag, initial time-to-live (TTL), and SYN packet size (as declared).

The format for the fingerprints is as follows:

wwww:ttt:mmm:D:W:S:N:I:OS Description

with the following composition:

wwww - window size
ttt  - time-to-live
mmm  - maximum segment size
D    - don't fragment flag  (0=unset, 1=set) 
W    - window scaling (-1=not present, other=value)
S    - sackOK flag (0=unset, 1=set)
N    - nop flag (0=unset, 1=set)
I    - packet size (-1=irrelevant)

The following are example OS fingerprint signatures used in the p0f database, based on empirical data:

31072:64:3884:1:0:1:1:-1:Linux 2.2.12-20 (RH 6.1)
512:64:1460:0:0:0:0:44:Linux 2.0.35 - 2.0.38
32120:64:1460:1:0:1:1:60:Linux 2.2.9 - 2.2.18
16384:64:1460:1:0:0:0:44:FreeBSD 4.0-STABLE, 3.2-RELEASE
8760:64:1460:1:0:0:0:-1:Solaris 2.6 (2)
9140:255:9140:1:0:0:0:-1:Solaris 2.6 (sunsite)
49152:64:1460:0:0:0:0:44:IRIX 6.5 / 6.4
8760:255:1460:1:0:0:0:44:Solaris 2.6 or 2.7 (1)
8192:128:1460:1:0:0:0:44:Windows NT 4.0 (1)
8192:128:1460:1:0:1:1:48:Windows 9x (1)
8192:128:536:1:0:1:1:48:Windows 9x (2)
2144:64:536:1:0:1:1:60:Windows 9x (4)
16384:128:1460:1:0:1:1:48:Windows 2000 (1)

Now, let’s run p0f and examine a sample of its output:

>p0f

p0f: passive os fingerprinting utility, version 1.8.3

(C) Michal Zalewski <lcamtuf@gis.net>, William Stearns <wstearns@pobox.

p0f: file: '/etc/p0f.fp', 207 fprints, iface: '\', rule: 'all'.

208.239.76.103: UNKNOWN [64240:116:1380:1:-1:1:1:48].
207.161.10.186 [22 hops]: Windows NT 5.0 (2)
211.28.55.225 [21 hops]: Windows XP Pro, Windows 2000 Pro
68.58.136.227 [17 hops]: Windows NT 4.0 (1) *
80.133.65.39: UNKNOWN [65535:118:1440:1:0:1:1:52].
209.195.250.214 [19 hops]: Windows NT 5.0 (2)
213.7.50.19 [20 hops]: Windows NT 5.0 (2)
142.177.114.37 [19 hops]: Windows 2000 Pro (2128)
142.177.114.37 [19 hops]: Windows 2000 Pro (2128)
66.231.192.134 [14 hops]: Windows NT 5.0 (2)
208.239.76.97: UNKNOWN [64240:116:1380:1:-1:1:1:48].
12.230.149.236 [17 hops]: Windows XP Pro, Windows 2000 Pro
208.239.76.97: UNKNOWN [64240:116:1380:1:-1:1:1:48].
12.226.219.102 [19 hops]: Windows 9x (1) *
68.0.210.22 [17 hops]: Windows 2000 (9)
208.239.76.97: UNKNOWN [64240:116:1380:1:-1:1:1:48].
64.65.61.213 [17 hops]: Linux 2.2.9 - 2.2.18
206.169.77.31 [19 hops]: Windows XP Pro, Windows 2000 Pro
206.169.77.31 [19 hops]: Windows XP Pro, Windows 2000 Pro
208.239.76.97: UNKNOWN [64240:116:1380:1:-1:1:1:48].
133.11.36.25 [19 hops]: Linux 2.4.2 - 2.4.14 (1)
208.239.76.97: UNKNOWN [64240:116:1380:1:-1:1:1:48].
64.72.132.72 [11 hops]: Linux 2.4.2 - 2.4.14 (1)

p0f does a good job of fingerprinting most known operating systems. The main advantage of p0f is that it does not alert the source host that you are fingerprinting it. As you can see from the above output, p0f also reports the TCP parameters of each unknown OS, so that you can test new platforms and add your own rules to the database file.

The only thing you have to do yourself is determine the initial TTL of a packet. It’s usually equal to the first power of 2 greater than the TTL you’re seeing, assuming your remote party is not too far away (i.e., traceroute shows less than 25 hops). If you get a TTL of 55 in a fingerprint returned by p0f, the initial TTL was probably 64.

p0f Version 2 also introduced numerous improvements. Notable features of Version 2 include the SYN+ACK and RST+ fingerprinting modes, for silently identifying systems you connect to in the usual way (such as via a web browser) or even systems to which you cannot connect at all.

Another notable feature of p0f Version 2 is masquerade detection, implemented by using the -M flag. Masquerade detection calculates a score based on known operating systems signatures. The scoring system is as follows:

Differences in OS fingerprints for the same IP

-3 if the same OS
+4 if different signature for the same OS genre
+6 if different OS genres

NAT and firewall flags set

+4 if Network Address Translation (NAT) flags differ for the same signature
+4 if firewall (fw) flags differ for the same signature
+1 for each NAT and fw flag if signatures differ (maximum 4)

Link type differences

+4 if media type differs

Distance differences

+1 if host distance differs

Time from the previous occurence

/2 if more than half the cache size of the previous occurrence

Fuzzy Operating System Fingerprinting

Fyodor Yarochkin and Ofir Arkin have developed and enhanced Xprobe, an ICMP-based OS fingerprint scanner. Until recently, most tools for remote active OS fingerprinting used a static algorithm signature database to perform a match between the results they received from a targeted machine and known operating system fingerprints. This process has traditionally used strict signature matching to identify the remote operating system. However, in newer versions of Xprobe, the authors aggregate different remote active OS fingerprinting methods in order to identify the type of a remote operating system with a high precision rating that uses a “fuzzy” approach.

Tip

Nmap, with its osscan_guess option, actually implemented this feature before Xprobe did.

Obstacles to Fingerprinting

The fuzzy approach is designed to address several problems in the traditional strict decision-tree algorithms used by most active OS fingerprinting tools. For example, issues of network topology and of the fingerprinting process itself can both degrade the accuracy of the strict signature-matching technique.

A packet might be affected in different ways while in transit. First, a networking or filtering device might change one or several field values within the packet. For example, a packet-shaping device might alter time-to-live values, discard packets with malformed checksums, or calculate checksums for zero-checksum packets such as UDP packets. In addition, a router or firewall might spoof responses for a targeted system it protects; firewalls, for example, can spoof ICMP query replies. Also, a scrubber application may be present between the sending system and the target system, cleaning certain fields in the packet and thwarting fingerprinting.

Network firewalls or load-balancing devices can also cause bogus results by dropping or rerouting certain packets. Similarly, a TCP/IP stack that can be tuned by the user (for example, with the sysctl command on BSDs or the ndd command on Solaris) causes strict signature matching to fail. Finally, if a remote active OS fingerprinting tool utilizes malformed packets to produce its results, a properly configured intrusion detection system will alert the target.

Fuzzy Solution to Operating System Fingerprinting

In order to address these problems, the Xprobe authors revised the tool to use a fuzzy matching system to correlate received results with a known fingerprints signature database. They chose a matrix-based fingerprint-matching approach using existing OCR (optical character recognition) systems as their engine. This strategy employs a simple matrix representation of the scan results and subsequent calculation of “matches” by summing scores for each “signature” (OS). The program does this by reading the Xprobe configuration file, which holds the fingerprints signature database, and looking for the fingerprint and OS_ID entries. Once the fingerprinting test is executed, the program examines the packet(s) received as a result of the fingerprinting test and calculates a score for each possible OS.

The score value can take one of the following values:

  YES(3)
  PROBABLY_YES(2)
  PROBABLY_NO(1)
  NO(0)

Each test module assigns the appropriate score value according to the scheme implemented with the module. Thus, by using different score values, Xprobe introduces a degree of “fuzziness” to the solution. Once the tests are completed, each OS column is summed for a total score. The top-scoring OS is chosen as the final result. This method uses simple probability, since the highest score given for an OS (or OSs) is the most likely to produce an accurate match.

TCP/IP Timeout Detection

Another technology for OS detection is embodied in the tool known as RING. RING is a patch that you apply against Nmap to add temporal response fingerprinting. RING uses OS-specific variations in SYN/ACK timeout and regeneration cycles to fingerprint a remote operating system. As discussed in Chapter 6, TCP is a connected-mode, reliable protocol. As a result, hosts react to unanswered segments by regenerating them after an adapted timeout.

As described by the Intranode Research Team, segment regeneration may occur in various states of the TCP transition diagram. For example, the SYN_RCVD state is reached at the very beginning of a tentative TCP connection. If no ACK segment is received before the timeout expires, the system generates a new SYN/ACK segment. However, in some cases, simply regenerating one segment will not permit the connection process to continue. In this situation, the TCP/IP protocol dictates that the responding host assume the network is congested. The responding host will then network-pause, regenerate more segments, and so on, in a cycle.

RING uses this TCP timeout feature to detect a remote OS. Since TCP timeout values and regeneration cycles are loosely specified in RFCs, most OSs use their own parameters. Even OSs that share the same IP stack technology might have slightly different timeout values.

Thus, RING forces timeouts and then measures delays between successive SYN/ACK resends (and before optional resets). These results are compared to an empirical reference suite in order to identify the remote OS.

A typical fingerprinting session occurs as follows:

RING sends a SYN segment to an open port of the target, in the same manner as a normal TCP connection.
The target shifts from the LISTEN state to the SYN_RCVD state while sending back a SYN/ACK segment.
RING ignores the SYN/ACK segment and does not send the normally awaited ACK segment.
According to the TCP state transition diagram, the target remains in the SYN_RCVD state while reinjecting SYN/ACK segments from time to time. RING measures the times between these segments.

References

RFC 1413. “Identification Protocol,” February 1993.
“P0F—The Passive OS Fingerprinting Tool.” (http://www.stearns.org/p0f)
The new p0f: 2.0.2 (C) Copyright 2000-2003, by Michal Zalewski. (http://lcamtuf.coredump.cx/p0f.shtml)
“Examining Advanced Remote OS Detection Methods/Concepts using Perl,” by F0bic. (http://www.low-level.net)
“Nmap Remote OS Detection” by Fyodor, http://www.insecure.org. April 1999.
“ICMP Usage in Scanning,” by Ofir Arkin.(http://www.sys-security.com)
“Xprobe v2.0: A `Fuzzy’ Approach to Remote Active Operating System Fingerprinting,” by Fyodor Yarochkin and Ofir Arkin.
“New Tool and Technique for Remote Operating System Fingerprinting,” by Franck Veysset, Olivier Courtay, and Olivier Heen. Intranode Research Team.