The most economical way to decrypt data—whether or not the algorithm is known—is to let the program itself perform the decryption in the course of its normal activities. We call this process self-decoding.

If you’ve ever stopped a malware program in a debugger and noticed a string in memory that you didn’t see when you ran strings, you have already used the self-decoding technique. If the previously hidden information is decoded at any point, it is easier to just stop the process and do the analysis than it is to try to determine the encoding mechanism used (and try to build a decoder).

Although self-decoding can be a cheap and effective way to decode content, it has its drawbacks. First, in order to identify every instance of decryption performed, you must isolate the decryption function and set a breakpoint directly after the decryption routine. More important, if the malware doesn’t happen to decrypt the information you are interested in (or you cannot figure out how to coax the malware into doing so), you are out of luck. For these reasons, it is important to use techniques that provide more control.

For simple ciphers and encoding methods, you can often use the standard functions available within a programming language. For example, Example 13-7 shows a small Python program that decodes a standard Base64-encoded string. Replace the example_string variable to decode the string of interest.

import string
import base64

example_string = 'VGhpcyBpcyBhIHRlc3Qgc3RyaW5n'
print base64.decodestring(example_string)

For simple encoding methods that lack standard functions, such as XOR encoding or Base64 encoding that uses a modified alphabet, often the easiest course of action is to just program or script the encoding function in the language of your choice. Example 13-8 shows an example of a Python function that implements a NULL-preserving XOR encoding, as described earlier in this chapter.

def null_preserving_xor(input_char,key_char):
    if (input_char == key_char or input_char == chr(0x00)):
        return input_char
    else:
        return chr(ord(input_char) ^ ord(key_char))

This function takes in two characters—an input character and a key character—and outputs the translated character. To convert a string or longer content using NULL-preserving single-byte XOR encoding, just send each input character with the same key character to this subroutine.

Base64 with a modified alphabet requires a similarly simple script. For example, Example 13-9 shows a small Python script that translates the custom Base64 characters to the standard Base64 characters, and then uses the standard decodestring function that is part of the Python base64 library.

import string
import base64

s = ""
custom = "9ZABCDEFGHIJKLMNOPQRSTUVWXYabcdefghijklmnopqrstuvwxyz012345678+/"
Base64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"

ciphertext = 'TEgobxZobxZgGFPkb2O='


for ch in ciphertext:
    if (ch in Base64):
        s = s + Base64[string.find(custom,str(ch))]
    elif (ch == '='):
        s += '='

result = base64.decodestring(s)

For standard cryptographic algorithms, it is best to use existing implementations that are available in code libraries. A Python-based cryptography library called PyCrypto (http://www.dlitz.net/software/pycrypto/) provides a wide variety of cryptographic functions. Similar libraries exist for different languages. Example 13-10 shows a sample Python program that performs decryption using the DES algorithm.

from Crypto.Cipher import DES
import sys

obj = DES.new('password',DES.MODE_ECB)
cfile = open('encrypted_file','r')
cbuf = cfile.read()
print obj.decrypt(cbuf)

Using the imported PyCrypto libraries, the script opens the encrypted file called encrypted_file and decrypts it with DES in Electronic Code Book (ECB) mode using the password password.

Block ciphers like DES can use different modes of encryption to apply a single key to an arbitrary length stream of plaintext, and the mode must be specified in the library call. The simplest mode is ECB mode, which applies the block cipher to each block of plaintext individually.

There are many possible variations available for scripting decoding algorithms. The preceding examples give you an idea of the types of options available for writing your own decoders.

Writing your own version of the attacker’s cryptographic algorithms is typically reserved for when a cipher is simple or sufficiently well defined (in the case of standard cryptography). A more difficult challenge is dealing with cases where the cryptography is too complex to emulate and is also nonstandard.

In self-decoding, while trying to get the malware to do the decryption, you limit yourself to letting the malware run as it normally would and stopping it at the right time. But there is no reason to limit yourself to the normal execution paths of the malware when you can direct it.

Once encoding or decoding routines are isolated and the parameters are understood, it is possible to fully exploit malware to decode any arbitrary content using instrumentation, thus effectively using the malware against itself.

Let’s return to the malware that produced the multiple large encrypted files from the earlier Custom Encoding section. Example 13-11 shows the function header plus the primary instructions that are a part of the encryption loop shown previously in Figure 13-14.

004011A9                 push    ebp
004011AA                 mov     ebp, esp
004011AC                 sub     esp, 14h
004011AF                 push    ebx
004011B0                 mov     [ebp+counter], 0
004011B7                 mov     [ebp+NumberOfBytesWritten], 0
...
004011F5 loc_4011F5:                     ; CODE XREF: encrypted_Write+46j
004011F5                 call    encrypt_Init
004011FA
004011FA loc_4011FA:                     ; CODE XREF: encrypted_Write+7Fj
004011FA                 mov     ecx, [ebp+counter]
004011FD                 cmp     ecx, [ebp+nNumberOfBytesToWrite]
00401200                 jnb     short loc_40122A
00401202                 mov     edx, [ebp+lpBuffer]
00401205                 add     edx, [ebp+counter]
00401208                 movsx   ebx, byte ptr [edx]
0040120B                 call    encrypt_Byte
00401210                 and     eax, 0FFh
00401215                 xor     ebx, eax
00401217                 mov     eax, [ebp+lpBuffer]
0040121A                 add     eax, [ebp+counter]
0040121D                 mov     [eax], bl
0040121F                 mov     ecx, [ebp+counter]
00401222                 add     ecx, 1
00401225                 mov     [ebp+counter], ecx
00401228                 jmp     short loc_4011FA
0040122A
0040122A loc_40122A:                     ; CODE XREF: encrypted_Write+57j
0040122A                 push    0       ; lpOverlapped
0040122C                 lea     edx, [ebp+NumberOfBytesWritten]

We know a couple of key pieces of information from our previous analysis:

We have an encrypted file, the malware itself, and the knowledge of how its encryption function works. Our high-level goal is to instrument the malware so that it takes the encrypted file and runs it through the same routine it used for encryption. (We are assuming based on the use of XOR that the function is reversible.) This high-level goal can be broken down into a series of tasks:

In order to implement the instrumentation to perform these high-level tasks, we will use the Immunity Debugger (ImmDbg), which was introduced in Chapter 9. ImmDbg allows Python scripts to be used to program the debugger. The ImmDbg script in Example 13-12 is a fairly generic sample that has been written to process the encrypted files that were found with the malware, thereby retrieving the plaintext.

import immlib

def main ():
    imm = immlib.Debugger()
    cfile = open("C:\\encrypted_file","rb") # Open encrypted file for read
    pfile = open("decrypted_file", "w")     # Open file for plaintext
    buffer = cfile.read()                   # Read encrypted file into buffer
    sz = len(buffer)                        # Get length of buffer
    membuf = imm.remoteVirtualAlloc(sz)     # Allocate memory within debugger
    imm.writeMemory(membuf,buffer)          # Copy into debugged process's memory

    imm.setReg("EIP", 0x004011A9)           # Start of function header
    imm.setBreakpoint(0x004011b7)           # After function header
    imm.Run()                               # Execute function header

    regs = imm.getRegs()
    imm.writeLong(regs["EBP"]+16, sz)       # Set NumberOfBytesToWrite stack variable
    imm.writeLong(regs["EBP"]+8, membuf)    # Set lpBuffer stack variable

    imm.setReg("EIP", 0x004011f5)           # Start of crypto
    imm.setBreakpoint(0x0040122a)           # End of crypto loop
    imm.Run()                               # Execute crypto loop

    output = imm.readMemory(membuf, sz)     # Read answer
    pfile.write(output)                     # Write answer

The script in Example 13-12 follows the high-level tasks closely. immlib is the Python library, and the immlib.Debugger call provides programmatic access to the debugger. The open calls open files for reading the encrypted files and writing the decrypted version. Note that the rb option on the open commands ensures that binary characters are interpreted correctly (without the b flag, binary characters can be evaluated as end-of-file characters, terminating the reading prematurely).

The imm.remoteVirtualAlloc command allocates memory within the malware process space inside the debugger. This is memory that can be directly referenced by the malware. The cfile.read command reads the encrypted file into a Python buffer, and then imm.writeMemory is used to copy the memory from the Python buffer into the memory of the process being debugged. The imm.getRegs function is used to get the current register values so that the EBP register can be used to locate the two key arguments: the memory buffer that is to be decrypted and its size. These arguments are set using the imm.writeLong function.

The actual running of the code is done in two stages as follows, and is guided by the setting of breakpoints using the imm.setBreakpoint calls, the setting of EIP using the imm.setReg("EIP",location) calls, and the imm.Run calls:

Finally, the same buffer is read out of the process memory into the Python memory (using imm.readMemory) and then output to a file (using pfile.write).

Actual use of this script requires a little preparation. The file to be decrypted must be in the expected location (C:\encrypted_file). In order to run the malware, you open it in ImmDbg. To run the script, you select the Run Python Script option from the ImmLib menu (or press ALT-F3) and select the file containing the Python script in Example 13-12. Once you run the file, the output file (decrypted_file) will show up in the ImmDbg base directory (which is C:\Program Files\Immunity Inc\Immunity Debugger), unless the path is specified explicitly.

In this example, the encryption function stood alone. It didn’t have any dependencies and was fairly straightforward. However, not all encoding functions are stand-alone. Some require initialization, possibly with a key. In some cases, this key may not even reside in the malware, but may be acquired from an outside source, such as over the network. In order to support decoding in these cases, it is necessary to first have the malware properly prepared.

Preparation may merely mean that the malware needs to start up in the normal fashion, if, for example, it uses an embedded password as a key. In other cases, it may be necessary to customize the external environment in order to get the decoding to work. For example, if the malware communicates using encryption seeded by a key the malware receives from the server, it may be necessary either to script the key-setup algorithm with the appropriate key material or to simulate the server sending the key.