Table of Contents for
Python: Penetration Testing for Developers

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Python: Penetration Testing for Developers by Dave Mound Published by Packt Publishing, 2016
  1. Cover
  2. Table of Contents
  3. Python: Penetration Testing for Developers
  4. Python: Penetration Testing for Developers
  5. Python: Penetration Testing for Developers
  6. Credits
  7. Preface
  8. What you need for this learning path
  9. Who this learning path is for
  10. Reader feedback
  11. Customer support
  12. 1. Module 1
  13. 1. Understanding the Penetration Testing Methodology
  14. Understanding what penetration testing is not
  15. Assessment methodologies
  16. The penetration testing execution standard
  17. Penetration testing tools
  18. Summary
  19. 2. The Basics of Python Scripting
  20. Python – the good and the bad
  21. A Python interactive interpreter versus a script
  22. Environmental variables and PATH
  23. Understanding dynamically typed languages
  24. The first Python script
  25. Developing scripts and identifying errors
  26. Python formatting
  27. Python variables
  28. Operators
  29. Compound statements
  30. Functions
  31. The Python style guide
  32. Arguments and options
  33. Your first assessor script
  34. Summary
  35. 3. Identifying Targets with Nmap, Scapy, and Python
  36. Understanding Nmap
  37. Nmap libraries for Python
  38. The Scapy library for Python
  39. Summary
  40. 4. Executing Credential Attacks with Python
  41. Identifying the target
  42. Creating targeted usernames
  43. Testing for users using SMTP VRFY
  44. Summary
  45. 5. Exploiting Services with Python
  46. Understanding the chaining of exploits
  47. Automating the exploit train with Python
  48. Summary
  49. 6. Assessing Web Applications with Python
  50. Identifying hidden files and directories with Python
  51. Credential attacks with Burp Suite
  52. Using twill to walk through the source
  53. Understanding when to use Python for web assessments
  54. Summary
  55. 7. Cracking the Perimeter with Python
  56. Understanding the link between accounts and services
  57. Cracking inboxes with Burp Suite
  58. Identifying the attack path
  59. Gaining access through websites
  60. Summary
  61. 8. Exploit Development with Python, Metasploit, and Immunity
  62. Understanding the Windows memory structure
  63. Understanding memory addresses and endianness
  64. Understanding the manipulation of the stack
  65. Understanding immunity
  66. Understanding basic buffer overflow
  67. Writing a basic buffer overflow exploit
  68. Understanding stack adjustments
  69. Understanding the purpose of local exploits
  70. Understanding other exploit scripts
  71. Reversing Metasploit modules
  72. Understanding protection mechanisms
  73. Summary
  74. 9. Automating Reports and Tasks with Python
  75. Understanding how to create a Python class
  76. Summary
  77. 10. Adding Permanency to Python Tools
  78. Understanding the difference between multithreading and multiprocessing
  79. Building industry-standard tools
  80. Summary
  81. 2. Module 2
  82. 1. Python with Penetration Testing and Networking
  83. Approaches to pentesting
  84. Introducing Python scripting
  85. Understanding the tests and tools you'll need
  86. Learning the common testing platforms with Python
  87. Network sockets
  88. Server socket methods
  89. Client socket methods
  90. General socket methods
  91. Moving on to the practical
  92. Summary
  93. 2. Scanning Pentesting
  94. What are the services running on the target machine?
  95. Summary
  96. 3. Sniffing and Penetration Testing
  97. Implementing a network sniffer using Python
  98. Learning about packet crafting
  99. Introducing ARP spoofing and implementing it using Python
  100. Testing the security system using custom packet crafting and injection
  101. Summary
  102. 4. Wireless Pentesting
  103. Wireless attacks
  104. Summary
  105. 5. Foot Printing of a Web Server and a Web Application
  106. Introducing information gathering
  107. Information gathering of a website from SmartWhois by the parser BeautifulSoup
  108. Banner grabbing of a website
  109. Hardening of a web server
  110. Summary
  111. 6. Client-side and DDoS Attacks
  112. Tampering with the client-side parameter with Python
  113. Effects of parameter tampering on business
  114. Introducing DoS and DDoS
  115. Summary
  116. 7. Pentesting of SQLI and XSS
  117. Types of SQL injections
  118. Understanding the SQL injection attack by a Python script
  119. Learning about Cross-Site scripting
  120. Summary
  121. 3. Module 3
  122. 1. Gathering Open Source Intelligence
  123. Gathering information using the Shodan API
  124. Scripting a Google+ API search
  125. Downloading profile pictures using the Google+ API
  126. Harvesting additional results from the Google+ API using pagination
  127. Getting screenshots of websites with QtWebKit
  128. Screenshots based on a port list
  129. Spidering websites
  130. 2. Enumeration
  131. Performing a ping sweep with Scapy
  132. Scanning with Scapy
  133. Checking username validity
  134. Brute forcing usernames
  135. Enumerating files
  136. Brute forcing passwords
  137. Generating e-mail addresses from names
  138. Finding e-mail addresses from web pages
  139. Finding comments in source code
  140. 3. Vulnerability Identification
  141. Automated URL-based Directory Traversal
  142. Automated URL-based Cross-site scripting
  143. Automated parameter-based Cross-site scripting
  144. Automated fuzzing
  145. jQuery checking
  146. Header-based Cross-site scripting
  147. Shellshock checking
  148. 4. SQL Injection
  149. Checking jitter
  150. Identifying URL-based SQLi
  151. Exploiting Boolean SQLi
  152. Exploiting Blind SQL Injection
  153. Encoding payloads
  154. 5. Web Header Manipulation
  155. Testing HTTP methods
  156. Fingerprinting servers through HTTP headers
  157. Testing for insecure headers
  158. Brute forcing login through the Authorization header
  159. Testing for clickjacking vulnerabilities
  160. Identifying alternative sites by spoofing user agents
  161. Testing for insecure cookie flags
  162. Session fixation through a cookie injection
  163. 6. Image Analysis and Manipulation
  164. Hiding a message using LSB steganography
  165. Extracting messages hidden in LSB
  166. Hiding text in images
  167. Extracting text from images
  168. Enabling command and control using steganography
  169. 7. Encryption and Encoding
  170. Generating an MD5 hash
  171. Generating an SHA 1/128/256 hash
  172. Implementing SHA and MD5 hashes together
  173. Implementing SHA in a real-world scenario
  174. Generating a Bcrypt hash
  175. Cracking an MD5 hash
  176. Encoding with Base64
  177. Encoding with ROT13
  178. Cracking a substitution cipher
  179. Cracking the Atbash cipher
  180. Attacking one-time pad reuse
  181. Predicting a linear congruential generator
  182. Identifying hashes
  183. 8. Payloads and Shells
  184. Extracting data through HTTP requests
  185. Creating an HTTP C2
  186. Creating an FTP C2
  187. Creating an Twitter C2
  188. Creating a simple Netcat shell
  189. 9. Reporting
  190. Converting Nmap XML to CSV
  191. Extracting links from a URL to Maltego
  192. Extracting e-mails to Maltego
  193. Parsing Sslscan into CSV
  194. Generating graphs using plot.ly
  195. A. Bibliography
  196. Index

Understanding the Windows memory structure

The Windows Operating System (OS) memory structure has a number of sections that can be broken down into high level components. To understand how to write exploits and take advantages of poor programming practices, we first have to understand these sections. The following details break this information down into manageable chunks. The following figure provides a representative diagram of the Windows memory structure for an executable.

Understanding the Windows memory structure

Now, each of these components is important, but the pieces we use with most exploit writing are the stack and the heap.

Understanding the stack and the heap

The stack is used for short term local storage in an ordered manner. Each time a function is called, or a thread, a unique stack is assigned of a fixed size for that function or thread. Once the function or thread has finished the operations, the stack is destroyed.

The heap, on the other hand, is where global variables and values are assigned in a relatively disorganized manner. The heap is shared by applications and the areas of memory are actually managed by the application or process. Once the application terminates that specific region of memory is freed. In this example, we are attacking the stack, not the heap.

Tip

Keep in mind that the exploit examples here are often written in Perl, though you can easily convert the code to Python, as highlighted in Chapter 2, The Basics of Python Scripting.

To better understand the difference between the heap and the stack movement, see the following figure, which shows the adjustment as memory is allocated for global and local resources.

Understanding the stack and the heap

The stack builds up the data from bottom of the stack to the top. The growth goes from high memory addresses to low memory addresses. The heap is opposite of the stack as it grows in the other direction, toward the higher addresses.

To understand the way a program would be loaded onto the stack, we create a sample code snippet. With this code, you can see how the main function calls function1 and the local variables as they are placed onto the stack. Pay attention to the way that the program would normally flow with calls to function1 and how the data is placed on the stack.

int function1(int a, int b, int c)
{
    diffa - b - c;
    sum = a + b + c;
    return sum;
}
int main()
{
    return function1(argv[1], argv[2], argv[3]);
}
Understanding the stack and the heap

The code loaded on the stack would look similar to this, which highlights how the information components are presented. As you can see, the old Base Pointer is loaded on to the stack for storage and the new EBP is the old Stack Pointer value, since the top of the stack has shifted to its new location.

Items that are put onto the stack are pushed onto it, and items that are run or removed from the stack are popped off of it. A stack is a programmable concept known as a Last In First Out (LIFO) structure. Think of it as a stack of dishes; to effectively remove dishes you have to take them off the top by one or by sets, otherwise you risk breaking things. The safest way, of course, is one at a time, which takes longer, but it is traceable and effective. With an understanding of the most dynamic parts of the memory structure that we will be using to inject our code into, you need to understand the remaining areas of Windows memory that will function as the building blocks, which we will manipulate to get from injection to shell. Specifically, we are speaking of the program image and Dynamic Link Libraries (DLL).

Note

Remember, we are attempting to inject shellcode into the memory, which we will then use to gain access to the system through a solution such as a Meterpreter.

Understanding the program image and dynamic-link libraries

Simply put, the program image is where the actual executable is stored in memory. Portable Executable (PE) is the defined format for the executable, which contains the executable and the DLL. Within the program image component of the memory, the following items are defined.

  • PE header: This contains the definitions for the rest of the PE.
  • .text: This component contains the code segment or the executable instructions.
  • .rdata: This is the read-only data segment, which contains static constants rather than variables.
  • .data: When the executable is loaded into memory, this area contains the static variables after they have been initialized, the global variables and static local variables. This area is readable and writeable, but the size does not change at runtime, it is determined at execution.
  • .rsrc: This section is where the resources for the executable are stored. This includes the icons, menus, dialogs, version information, fonts, and so forth.

Note

Many penetration testers manipulate the .rsrc component of an executable to change the format of payloads so that it appears as something else. This is often done to change the way a malicious payload appears on a Universal Serial Bus (USB) drive. Think about when you do a USB drop when you change your payload from looking like an executable to a folder. Most people would want to see what is in the folder and would be more likely to double click a fake folder than a suspicious executable. Tools like resource tuner make the manipulation of this section of the PE very easy.

The final component to understand here for the PE is the DLL, which encompasses Microsoft's concept of shared libraries. DLLs are similar to executables, but they cannot be called directly, and instead they have to be called by an executable. At its core, the idea of DLLs is to provide a method for the capabilities to upgrade without requiring the entire program to be recompiled when OS is updated.

Because of this, many of the basic building blocks for system operations need to be referenced regardless of start-up cycle. This means that even if other components are going to be in different memory locations, many core DLLs will stay in the same referenced locations. Remember, programs require specific callable instructions and many of the foundational DLLs are loaded into the same regions of memory.

What you need to understand is that we will use these DLLs to find an instruction that is reliably put into the same location so that we can reference it. This means that across the systems and the reboots, the memory reference will work as long as the OS and Service Pack (SP) version are the same if you use OS DLLs. If you use DLLs that are completely native to the program, you will be able to use this exploit across OS versions. For this example, though, we are going to use OS DLLs. The discovered instruction will enable us to tell the system to jump to our shell code, and in turn, execute it.

The reason we have to do a reference code in DLL is because we will be unsure of the exact location that our code will be loaded into memory each time we initiate this attack, so we cannot tell the system our exact memory address to jump to. So, instead, we are going to load the stack with our code and then tell the program to jump to the top of it by referencing the position.

Remember that this may change each time we execute the program and/or each reboot. The stack memory addresses are served as required per program, and we are attempting to inject our code directly into this running function's stack. So, we have to take advantage of the known and repeatable target instruction sets. We will explain the exact process of this in detail, but for now, just know that we use DLLs known instruction sets to jump to our shell code. From this area of memory, the other components are less important for our exploitation techniques highlighted here, but you need to understand them as they are referenced in your debuggers.

Note

The PE can be better understood from the following two older articles, Peering Inside the PE: A Tour of the Win32 Portable Executable File Format, found here https://msdn.microsoft.com/en-us/magazine/ms809762.aspx, and An In-Depth Look into the Win32 Portable Executable File Format, found here https://msdn.microsoft.com/en-us/magazine/cc301805.aspx.

Understanding the process environment block

The Process Environment Block (PEB) is where nonkernel components of a running process are stored. Information that is needed by systems that should not have access to kernel components is stored in memory. Some Host Intrusion Prevention Systems (HIPS) monitor activities in this memory region to see if malicious activities are taking place. The PEB contains details related to the loaded DLLs, executables, access restrictions, and so on.

Understanding the thread environment block

A Thread Environment Block (TEB) is spawned for each thread that a process has established. The first thread is known as the primary thread and each thread after that has its own TEB. Each TEB share the memory allocations of the process that initiated them, but they can execute instructions in a manner that makes task completion more efficient. Since writeable access is required, this environment resides in the nonkernel block of the memory.

Kernel

This is the area of memory reserved for device drivers, the Hardware Access Layer (HAL), the cache and other components that programs do not need direct access to. The best way to understand the kernel is that this is the most critical component of the OS. All communication is brokered as necessary through OS features. The attacks we are highlighting here do not depend on a deep understanding of the kernel. Additionally, a deep understanding of the Windows kernel would take a book of its own. After defining the memory locations, we have to understand how data is addressed within it.