Understanding the capabilities of a specific software or hardware product may provide a starting point for investigating vulnerabilities that could exist in that product. Conducting vulnerability research is not easy, nor is it a one-click task. Hence, it requires a strong knowledge base with different factors to carry out security-analysis:

• Programming skills: This is a fundamental factor for ethical hackers. Learning the basic concepts and structures that exist with any programming language should grant the tester an advantage when finding vulnerabilities. Apart from basic knowledge of programming languages, you must be prepared to deal with the advanced concepts of processors, system memory, buffers, pointers, data types, registers, and caches. These concepts are implementable in almost any programming language, such as C/C++, Python, Perl, and Assembly.

• Reverse-engineering: This is another broad area for discovering the vulnerabilities that could exist in an electronic device, software, or system by analyzing its functions, structures, and operations. The purpose is to deduce code from a given system without any prior knowledge of its internal working; to examine it for error conditions, poorly-designed functions, and protocols; and to test the boundary conditions. There are several reasons to use your reverse-engineering skills, such as the removal of copyright protection from a software, security auditing, competitive technical intelligence, identification of patent infringement, interoperability, understanding the product workflow, and acquiring sensitive data. Reverse-engineering adds two layers of concept to examining the code of an application: source-code auditing and binary auditing. If you have access to the application source code, you can accomplish the security analysis through automated tools; or manually study the source in order to extract the conditions where a vulnerability can be triggered. On the other hand, binary auditing simplifies the task of reverse-engineering where the application exists without any source code. Disassemblers and decompilers are two generic types of tools that may assist the auditor with binary analysis. Disassemblers generate the assembly code from a complied binary program, while decompilers generate a high-level language code from a compiled binary program. However, dealing with either of these tools is quite challenging and requires a careful assessment.
• Instrumented tools: Instrumented tools, such as debuggers, data extractors, fuzzers, profilers, code coverage, flow analyzers, and memory monitors, play an important role in the vulnerability-discovery process, and provide a consistent environment for testing purposes. Explaining each of these tool categories is beyond the scope of this book. However, you may find several useful tools already present in Kali Linux. To keep track of the latest reverse-code-engineering tools, we strongly recommend that you visit the online library at http://www.woodmann.com/collaborative/tools/index.php/Category:RCE_Tools.
• Exploitability and payload construction: This is the final step in writing the Proof of Concept (PoC) code for a vulnerable element of an application, which could allow the penetration tester to execute custom commands on the target machine. We apply our knowledge of vulnerable applications from the reverse-engineering stage to polish shellcode with an encoding mechanism, in order to avoid bad characters that may result in the termination of the exploit process.

Depending on the type and classification of vulnerability discovered, it is very important to follow the specific strategy that may allow you to execute an arbitrary code or command on the target system. As a professional penetration tester, you will always be looking for loopholes that will result in getting shell access to your target operating system. Thus, we will demonstrate a few scenarios with the Metasploit framework in a later section of this chapter, which will show these tools and techniques.