Preface

Adversaries routinely target networks for gain. As I prepare this third edition of Network Security Assessment, the demand for incident response expertise is also increasing. Although software vendors have worked to improve the security of their products over the past decade, system complexity and attack surfaces have grown, and if anything, the overall integrity of the Internet has degraded.

Attacker tactics have become increasingly refined, combining intricate exploitation of software defects, social engineering, and physical attack tactics to target high-value assets. To make matters worse, many technologies deployed to protect networks have been proven ineffective. Google Project Zero¹ team member Tavis Ormandy has publicized severe remotely exploitable flaws within many security products.²

As stakes increase, so does the value of research output. Security researchers are financially incentivized to disclose zero-day vulnerabilities to third parties and brokers, who in turn share the findings with their customers, and in some cases, responsibly notify product vendors. There exists a growing gap by which the number of severe defects known only to privileged groups (e.g., governments and organized criminals) increases each day.

A knee-jerk reaction is to prosecute hackers and curb the proliferation of their tools. The adversaries we face, however, along with the tactics they adopt, are nothing but a symptom of a serious problem: the products we use are unfit for purpose. Product safety is an afterthought for many technology companies, and the challenges we face today a manifestation of this.

To aggravate things further, governments have militarized the Internet and eroded the integrity of cryptosystems used to protect data.³ As security professionals, we must advocate defense in depth to mitigate risks that will likely always exist, and work hard to ensure that our networks are a safe place to do commerce, store data, and communicate with one another. Life for us all would be very different without the Internet and the freedoms it provides.

Overview

This book tackles a single area of computer security in detail—undertaking network-based penetration testing in a structured manner. The methodology I present describes how determined attackers scour Internet-based networks in search of vulnerable components and how you can perform similar exercises to assess your environment.

Assessment is the first step any organization should take to manage its risk. By testing your networks in the same way that a determined adversary does, you proactively identify weaknesses within them. In this book, I pair offensive content with bulleted checklists of countermeasures to help you devise a clear technical strategy and fortify your environment accordingly.

Audience

This book assumes that you have familiarity with networking protocols and Unix-based operating system administration. If you are an experienced network engineer or security consultant, you should be comfortable with the contents of each chapter. To get the most out of this book, you should be familiar with:

OSI Layer 2 network operation (primarily ARP and 802.1Q VLAN tagging)
The IPv4 protocol suite, including TCP, UDP, and ICMP
The operation of popular network protocols (e.g., FTP, SMTP, and HTTP)
Basic runtime memory layout and Intel x86 processor registers
Cryptographic primitives (e.g., Diffie-Hellman and RSA key exchange)
Common web application flaws (XSS, CSRF, command injection, etc.)
Configuring and building Unix-based tools in your environment

Organization

This book consists of 15 chapters and 3 appendixes. At the end of each chapter is a checklist summarizing the threats and techniques described, along with recommended countermeasures. The appendixes provide reference material, including listings of TCP and UDP ports you might encounter during testing. Here is a brief description of each chapter and appendix:

Chapter 1, Introduction to Network Security Assessment, discusses the rationale behind network security assessment and introduces information assurance as a process, not a product.
Chapter 2, Assessment Workflow and Tools, covers the tools that make up a professional security consultant’s attack platform, along with assessment tactics that should be adopted.
Chapter 3, Vulnerabilities and Adversaries, categorizes vulnerabilities in software via taxonomy, along with low-level descriptions of vulnerability classes and adversary types.
Chapter 4, Internet Network Discovery, describes the Internet-based tactics that a potential attacker adopts to map your network—from open web searches to DNS sweeping and querying of mail servers.
Chapter 5, Local Network Discovery, defines the steps taken to perform local area network discovery and sniffing, along with circumvention of 802.1Q and 802.1X security features.
Chapter 6, IP Network Scanning, discusses popular network scanning techniques and their relevant applications. It also lists tools that support such scanning types. IDS evasion and low-level packet analysis techniques are also covered.
Chapter 7, Assessing Common Network Services, details the approaches used to test services found running across many operating platforms. Protocols covered within this chapter include SSH, FTP, Kerberos, SNMP, and VNC.
Chapter 8, Assessing Microsoft Services, covers testing of Microsoft services found in enterprise environments (NetBIOS, SMB Direct, RPC, and RDP).
Chapter 9, Assessing Mail Services, details assessment of SMTP, POP3, and IMAP services that transport email. Often, these services can fall afoul to information-leak and brute-force attacks, and in some cases, remote code execution.
Chapter 10, Assessing VPN Services, covers network-based testing of IPsec and PPTP services that provide secure network access and confidentiality of data in-transit.
Chapter 11, Assessing TLS Services, details assessment of TLS protocols and features that provide secure access to web, mail, and other network services.
Chapter 12, Web Application Architecture, defines web application server components and describes the way by which they interact with one another, including protocols and data formats.
Chapter 13, Assessing Web Servers, covers the assessment of web server software, including Microsoft IIS, Apache HTTP Server, and Nginx.
Chapter 14, Assessing Web Application Frameworks, details the tactics used to uncover flaws within frameworks, including Apache Struts, Rails, Django, Microsoft ASP.NET, and PHP.
Chapter 15, Assessing Data Stores, covers remote assessment of database servers (e.g., Oracle Database, Microsoft SQL Server, and MySQL), storage protocols, and distributed key-value stores found within larger systems.
Appendix A, Common Ports and Message Types, contains TCP, UDP, and ICMP listings, and details respective assessment chapters.
Appendix B, Sources of Vulnerability Information, lists public sources of vulnerability and exploit information so that you can devise matrixes to quickly identify areas of potential risk when assessing exposed services.
Appendix C, Unsafe TLS Cipher Suites, details vulnerable cipher suites supported by TLS that should be disabled and avoided where possible.

Use of RFC and CVE References

Throughout the book, references are made to particular IETF Request for Comments (RFC) drafts,⁴ documents, and MITRE Common Vulnerabilities and Exposures (CVE) entries.⁵ Published RFCs define the inner workings and mechanics of protocols including SMTP, FTP, TLS, HTTP, and IKE. The MITRE CVE list is a dictionary of publicly known information security vulnerabilities, and individual entries (formatted by year, along with a unique identifier) allow us to track particular flaws.

Vulnerabilities Covered in This Book

This book describes vulnerabilities that are exploited by both unauthenticated and authenticated users against network services in particular. Examples of tactics that are largely out of scope include local privilege escalation, denial of service conditions, and breaches performed with local network access (including man-in-the-middle attacks).

Vulnerabilities with CVE references dated 2008 and prior are not covered in this title. Previous editions of this book were published in 2004 and 2007; they detail older vulnerabilities in server packages including Microsoft IIS, Apache, and OpenSSL.

A number of less common server packages are not covered for the sake of brevity. During testing, you should manually search the NIST National Vulnerability Database (NVD)⁶ to investigate known issues in the services you have identified.

Recognized Assessment Standards

This book has been written in line with recognized penetration testing standards, including NIST SP 800-115, NSA IAM, CESG CHECK, CREST, Tiger Scheme, The Cyber Scheme, PCI DSS, and PTES. You can use the material within this book to prepare for infrastructure and web application testing exams across these accreditation bodies.

NIST SP 800-115

In 2008, the US National Institute of Standards and Technology (NIST) released special publication 800-115,⁷ which is a technical guide for security testing. PCI DSS materials refer to the document as an example of industry-accepted best practice. SP 800-115 describes the assessment process at a high level, along with low-level tests that should be undertaken against systems.

NSA IAM

The US National Security Agency (NSA) published the INFOSEC Assessment Methodology (IAM) framework to help consultants and security professionals outside of the NSA provide assessment services to clients. The IAM framework defines three levels of assessment relative to testing of computer networks:

Assessment (level 1): This level involves cooperative high-level discovery of the target organization, including policies, procedures, and details of information flow within systems. No hands-on network or system testing is undertaken at this level.
Evaluation (level 2): Evaluation is a hands-on cooperative process, involving network scanning, use of penetration testing tools, and the application of specific technical expertise.
Red Team (level 3): A red team assessment is a noncooperative external test of the target network, involving penetration testing to simulate an appropriate adversary. Red team assessment involves full qualification of vulnerabilities.

This book describes technical vulnerability scanning and penetration testing techniques used within levels 2 and 3 of the IAM framework.

CESG CHECK

The UK Government Communications Headquarters (GCHQ) has an information assurance arm known as the Communications and Electronics Security Group (CESG). In the same way that the NSA IAM framework enables security consultants outside of government to provide assessment services, CESG operates a program known as CHECK⁸ to evaluate and accredit testing teams within the UK.

Unlike the NSA IAM, which covers many aspects of information security (including review of security policy, antivirus, backups, and disaster recovery), CHECK squarely tackles network security assessment. A second program is the CESG Listed Adviser Scheme (CLAS), which covers information security in a broader sense by addressing ISO/IEC 27001, security policy creation, and auditing.

Consultants navigate a CESG-approved assault course (primarily those maintained by CREST and the Tiger Scheme) to demonstrate ability and achieve accreditation. The CESG CHECK notes list the following examples of technical competence:

Use of DNS information retrieval tools for both single and multiple records, including an understanding of DNS record structure relating to target hosts
Use of ICMP, TCP, and UDP network mapping and probing tools
Demonstration of TCP service banner grabbing
Information retrieval using SNMP, including an understanding of MIB structure relating to target system configuration and network routes
Understanding of common weaknesses in routers and switches relating to Telnet, HTTP, SNMP, and TFTP access and configuration

The following are Unix-specific competencies:

User enumeration (via finger, rusers, rwho, and SMTP techniques)
Enumeration of RPC services and demonstration of security implications
Identification of Network File System (NFS) weaknesses
Testing for weaknesses within r-services (rsh, rexec, and rlogin)
Detection of insecure X Windows servers
Identification of weaknesses within web, FTP, and Samba services

Here are Windows-specific competencies:

Assessment of NetBIOS, SMB, and RPC services to enumerate users, groups, shares, domains, password policies, and associated weaknesses
Username and password grinding via SMB and RPC services
Demonstrating the presence of known flaws within Microsoft IIS and SQL Server

This book documents assessment tactics across these disciplines, along with supporting information to help you gain a sound understanding of the vulnerabilities. Although the CHECK program assesses the methodologies of consultants who wish to perform UK government security testing work, security teams and organizations elsewhere should be aware of the framework.

CESG Recognized Qualifications

Within the UK, a number of bodies provide both training and examination through CESG-approved assault courses. Qualifications provided by these organizations are recognized by CESG as being CHECK equivalent, as follows:

CREST: CREST is a nonprofit organization that regulates the penetration testing industry by providing accreditation through its certified infrastructure tester and certified web application tester programs. Through partnership with CESG, CREST-certified tester qualifications confer CHECK team leader status, and so many organizations use this syllabus to accredit testing team members.
Tiger Scheme: A second examining body that partners with government and industry is the Tiger Scheme. Accreditation levels are associate, qualified, and senior, which are recognized by CESG and can be used to secure CHECK team member and team leader status.
The Cyber Scheme: The Cyber Scheme Team Member (CSTM) certification is recognized by CESG as CHECK team member equivalent. The organization provides both training and accreditation through its approved partners.

PCI DSS

The Payment Card Industry Security Standards Council (PCI SSC) maintains the PCI Data Security Standard (PCI DSS), which requires payment processors, merchants, and those using payment card data to abide to specific control objectives, including:

Build and maintain a secure network
Protect cardholder data
Maintain a vulnerability management program
Implement strong access control measures
Regularly monitor and test networks
Maintain an information security policy

PCI DSS version 3.1 is the current standard. Within the document, there are two requirements for vulnerability scanning and penetration testing by payment processors and merchants:

Requirement 11.2: Mandates quarterly internal and external vulnerability scanning. A PCI SSC Approved Scanning Vendor (ASV) must be engaged to perform external testing, however ASV accreditation is not required for internal testing purposes.
Requirement 11.3: Requires performance of annual internal and external penetration testing by a qualified resource adhering to industry-accepted best practices (i.e., NIST SP 800-115).

This book is written in line with NIST SP 800-115 and other published standards, so you can use the methodology to perform internal and external testing and fulfill PCI DSS requirement 11.3 in particular.

PTES

PCI SSC recognizes the Penetration Testing Execution Standard (PTES)⁹ as a reference framework for testing, which consists of seven sections. The PTES site includes detailed material across the sections, as follows:

Preengagement interactions
Intelligence gathering
Threat modeling
Vulnerability analysis
Exploitation
Post-exploitation
Reporting

Mirror Site for Tools Mentioned in This Book

URLs for tools in this book are listed so that you can browse the latest files and papers on each respective site. If you are worried about Trojan horses or other malicious content within these executables, they have been virus checked and are available via the book’s website. You will likely encounter a safety warning when trying to access this page, they are hacking tools after all!

Using Code Examples

Supplemental material (code examples, exercises, etc.) is available for download at http://examples.oreilly.com/9780596006112/tools/.

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Network Security Assessment by Chris McNab (O’Reilly). Copyright 2017 Chris McNab, 978-1-491-91095-5.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic: Indicates commands, example email addresses, passwords, error messages, filenames, emphasis, and the first use of technical terms
Constant width: Indicates IP addresses and command-line examples
Constant width bold italic: Indicates replaceable text
Constant width bold: Indicates user input

Note

This icon signifies a tip, suggestion, or general note.

Warning

This icon indicates a warning or caution.

O’Reilly Safari

Note

Safari (formerly Safari Books Online) is a membership-based training and reference platform for enterprise, government, educators, and individuals.

Members have access to thousands of books, training videos, Learning Paths, interactive tutorials, and curated playlists from over 250 publishers, including O’Reilly Media, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and Course Technology, among others.

For more information, please visit http://oreilly.com/safari.

Comments and Questions

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)

There’s a web page for this book that lists errata, examples, and any additional information. You can access this page at http://bit.ly/network-security-assessment-3e.

To comment or ask technical questions about this book, send email to bookquestions@oreilly.com.

For more information about books, conferences, Resource Centers, and the O’Reilly Network, see the O’Reilly website.

Acknowledgments

Throughout my career, many have provided invaluable assistance, and most know who they are. My late friend Barnaby Jack helped me secure a job in 2009 that changed my life for the better. I miss him dearly, and the Jägermeister doesn’t taste the same.

Thanks to a Myers–Briggs INTP personality type, I can be quiet and a challenge to deal with at times. I don’t mean to be, and thus deeply appreciate those who have put up with me over the years, particularly the girlfriends and my family.

I extend my gratitude to the O’Reilly Media team for their continued support, patience, and unshakable faith. This is an important book to maintain, and with their help it will eventually make the world a safer place.

Technical Reviewers and Contributors

Computer systems have become so nebulous that I had to call on many subject-matter experts to cover flaws across different technologies. It would simply not have been possible to put this material together without the help of the following talented individuals: Car Bauer, Michael Collins, Daniel Cuthbert, Benjamin Delpy, David Fitzgerald, Rob Fuller, Chris Gates, Dane Goodwin, Robert Hurlbut, David Litchfield, HD Moore, Ivan Ristić, Tom Ritter, Andrew Ruef, and Frank Thornton.

¹ For an overview of Project Zero, see Andy Greenberg’s “‘Meet Project Zero,’ Google’s Secret Team of Bug-Hunting Hackers”, Wired, July 15, 2014.

² Ormandy described flaws affecting FireEye and Sophos products in “FireEye Exploitation: Project Zero’s Vulnerability of the Beast” and “Sophail: Applied Attacks Against Sophos Antivirus”, respectively. In addition, he tweeted about a Kaspersky exploit, and identified issues within Symantec and Trend Micro.

³ See Daniel J. Bernstein’s “Making Sure Crypto Stays Insecure” and watch Matthew Green’s TEDx talk “Why the NSA Is Breaking Our Encryption—And Why We Should Care”.

⁴ See the IETF RFC website.

⁵ See the MITRE CVE website.

⁶ To perform a keyword search, see http://bit.ly/2bfCqgR.

⁷ Karen Scarfone et al., “Technical Guide to Information Security Testing and Assessment”, National Institute of Standards and Technology, September 2008.

⁸ See “CHECK Fundamental Principles”, National Cyber Security Centre, October 23, 2015.

⁹ For details on this standard, see http://www.pentest-standard.org.

Previous Chapter

section

Next Chapter