Table of Contents for
The IDA Pro Book, 2nd Edition

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition The IDA Pro Book, 2nd Edition by Chris Eagle Published by No Starch Press, 2011
  1. Cover
  2. The IDA Pro Book
  3. PRAISE FOR THE FIRST EDITION OF THE IDA PRO BOOK
  4. Acknowledgments
  5. Introduction
  6. I. Introduction to IDA
  7. 1. Introduction to Disassembly
  8. The What of Disassembly
  9. The Why of Disassembly
  10. The How of Disassembly
  11. Summary
  12. 2. Reversing and Disassembly Tools
  13. Summary Tools
  14. Deep Inspection Tools
  15. Summary
  16. 3. IDA Pro Background
  17. Obtaining IDA Pro
  18. IDA Support Resources
  19. Your IDA Installation
  20. Thoughts on IDA’s User Interface
  21. Summary
  22. II. Basic IDA Usage
  23. 4. Getting Started with IDA
  24. IDA Database Files
  25. Introduction to the IDA Desktop
  26. Desktop Behavior During Initial Analysis
  27. IDA Desktop Tips and Tricks
  28. Reporting Bugs
  29. Summary
  30. 5. IDA Data Displays
  31. Secondary IDA Displays
  32. Tertiary IDA Displays
  33. Summary
  34. 6. Disassembly Navigation
  35. Stack Frames
  36. Searching the Database
  37. Summary
  38. 7. Disassembly Manipulation
  39. Commenting in IDA
  40. Basic Code Transformations
  41. Basic Data Transformations
  42. Summary
  43. 8. Datatypes and Data Structures
  44. Creating IDA Structures
  45. Using Structure Templates
  46. Importing New Structures
  47. Using Standard Structures
  48. IDA TIL Files
  49. C++ Reversing Primer
  50. Summary
  51. 9. Cross-References and Graphing
  52. IDA Graphing
  53. Summary
  54. 10. The Many Faces of IDA
  55. Using IDA’s Batch Mode
  56. Summary
  57. III. Advanced IDA Usage
  58. 11. Customizing IDA
  59. Additional IDA Configuration Options
  60. Summary
  61. 12. Library Recognition Using FLIRT Signatures
  62. Applying FLIRT Signatures
  63. Creating FLIRT Signature Files
  64. Summary
  65. 13. Extending IDA’s Knowledge
  66. Augmenting Predefined Comments with loadint
  67. Summary
  68. 14. Patching Binaries and Other IDA Limitations
  69. IDA Output Files and Patch Generation
  70. Summary
  71. IV. Extending IDA’s Capabilities
  72. 15. IDA Scripting
  73. The IDC Language
  74. Associating IDC Scripts with Hotkeys
  75. Useful IDC Functions
  76. IDC Scripting Examples
  77. IDAPython
  78. IDAPython Scripting Examples
  79. Summary
  80. 16. The IDA Software Development Kit
  81. The IDA Application Programming Interface
  82. Summary
  83. 17. The IDA Plug-in Architecture
  84. Building Your Plug-ins
  85. Installing Plug-ins
  86. Configuring Plug-ins
  87. Extending IDC
  88. Plug-in User Interface Options
  89. Scripted Plug-ins
  90. Summary
  91. 18. Binary Files and IDA Loader Modules
  92. Manually Loading a Windows PE File
  93. IDA Loader Modules
  94. Writing an IDA Loader Using the SDK
  95. Alternative Loader Strategies
  96. Writing a Scripted Loader
  97. Summary
  98. 19. IDA Processor Modules
  99. The Python Interpreter
  100. Writing a Processor Module Using the SDK
  101. Building Processor Modules
  102. Customizing Existing Processors
  103. Processor Module Architecture
  104. Scripting a Processor Module
  105. Summary
  106. V. Real-World Applications
  107. 20. Compiler Personalities
  108. RTTI Implementations
  109. Locating main
  110. Debug vs. Release Binaries
  111. Alternative Calling Conventions
  112. Summary
  113. 21. Obfuscated Code Analysis
  114. Anti–Dynamic Analysis Techniques
  115. Static De-obfuscation of Binaries Using IDA
  116. Virtual Machine-Based Obfuscation
  117. Summary
  118. 22. Vulnerability Analysis
  119. After-the-Fact Vulnerability Discovery with IDA
  120. IDA and the Exploit-Development Process
  121. Analyzing Shellcode
  122. Summary
  123. 23. Real-World IDA Plug-ins
  124. IDAPython
  125. collabREate
  126. ida-x86emu
  127. Class Informer
  128. MyNav
  129. IdaPdf
  130. Summary
  131. VI. The IDA Debugger
  132. 24. The IDA Debugger
  133. Basic Debugger Displays
  134. Process Control
  135. Automating Debugger Tasks
  136. Summary
  137. 25. Disassembler/Debugger Integration
  138. IDA Databases and the IDA Debugger
  139. Debugging Obfuscated Code
  140. IdaStealth
  141. Dealing with Exceptions
  142. Summary
  143. 26. Additional Debugger Features
  144. Debugging with Bochs
  145. Appcall
  146. Summary
  147. A. Using IDA Freeware 5.0
  148. Using IDA Freeware
  149. B. IDC/SDK Cross-Reference
  150. Index
  151. About the Author

Chapter 11. Customizing IDA

image with no caption

After spending some time with IDA, you may have developed some preferred settings that you wish to use as defaults every time you open a new database. Some of the options you have changed may already carry over from session to session, while other options seem to need resetting every time you load a new database. In this chapter we examine the various ways in which you can modify IDA’s behavior through configuration files and menu-accessible options. We also examine where IDA stores various configuration settings and discuss the difference between database-specific settings and global settings.

Configuration Files

Much of IDA’s default behavior is governed by settings contained in various configuration files. For the most part, configuration files are stored in the <IDADIR>/cfg directory, with one notable exception being the plug-ins configuration file, which resides at <IDADIR>/plugins/plugins.cfg (plugins.cfg will be covered in Chapter 17). While you may notice quite a few files in the main configuration directory, the majority of the files are used by processor modules and are applicable only when certain CPU types are being analyzed. The three principal configuration files are ida.cfg, idagui.cfg, and idatui.cfg. Options that apply to all versions of IDA are generally found in ida.cfg, while idagui.cfg and idatui.cfg contain options specific to the GUI versions and the text-mode versions of IDA, respectively.

The Main Configuration File: ida.cfg

IDA’s principal configuration file is ida.cfg. Early in the startup process, this file is read to assign default processor types for various file extensions and to tune IDA’s memory usage parameters. Once a processor type has been specified, the file is then read a second time to process additional configuration options. The options contained in ida.cfg apply to all versions of IDA regardless of the user interface that is being used.

General options of interest in ida.cfg include memory-tuning parameters (VPAGESIZE), whether backup files are created (CREATE_BACKUPS), and the name of the external graph viewer (GRAPH_VISUALIZER).

Occasionally when working with very large input fields, IDA may report that not enough memory is available to create a new database. In such cases, increasing the VPAGESIZE and then reopening the input file is usually sufficient to solve the problem.

A large number of options that control the format of disassembly lines are also contained in ida.cfg, including the default values for many of the options accessible via Options ▸ General. These include default values for the number of opcode bytes to display (OPCODE_BYTES), how far instructions should be indented (INDENTATION), whether the stack pointer offset should be displayed with each instruction (SHOW_SP), and the maximum number of cross-references to be displayed with a disassembly line (SHOW_XREFS). Additional options control the format of disassembly lines while in graph mode.

The global option specifying the maximum name length for named program locations (as opposed to stack variables) is contained in ida.cfg and is called MAX_NAMES_LENGTH. This option defaults to 15 characters and causes IDA to generate a warning message any time you enter a name longer than the current limit. The default length is kept small because some assemblers cannot handle names longer than 15 characters. If you do not plan to run an IDA-generated disassembly back through an assembler, then you may safely increase the limit.

The list of characters allowed in user-assigned names is governed by the NameChars options. By default this list allows alphanumeric characters and the four special characters _$?@. If IDA complains about the characters that you wish to use when you assign new names to locations or stack variables, then you may want to add additional characters to the NameChars set. For example, NameChars is the option to modify if you want to make the dot (.) character legal for use in IDA names. You should avoid the use of the semicolon, colon, comma, and space characters within names because they may lead to confusion, as these characters are typically considered delimiters for various disassembly line parts.

The last two options worth mentioning influence IDA’s behavior when parsing C header files (see Chapter 8). The C_HEADER_PATH option specifies a list of directories that IDA will search to resolve #include dependencies. By default, a common directory used by Microsoft’s Visual Studio is listed. If you use a different compiler or if your C header files are in a nonstandard location, you should consider editing this option. The C_PREDEFINED_MACROS option can be used to specify a default list of preprocessor macros that IDA will incorporate regardless of whether IDA has encountered them while parsing a C header file. This option offers a limited workaround facility for dealing with macros that may be defined in header files to which you do not have access.

The second half of ida.cfg contains options specific to various processor modules. The only documentation available for options in this section of the file comes in the form of the comments (if any) associated with each option. The processor-specific options specified in ida.cfg generally dictate the default settings in the Processor options section of IDA’s initial file-loading dialog.

The last step in processing ida.cfg is to search for a file named <IDADIR>/cfg/idauser.cfg. If present,[72] this file is treated as an extension of ida.cfg, and any options in the file will override corresponding options in ida.cfg. If you do not feel comfortable editing ida.cfg, then you should create idauser.cfg and add to it all of the options that you wish to override. In addition, idauser.cfg offers the easiest means for transferring your customized options from one version of IDA to another. For example, with idauser.cfg you do not need to re-edit ida.cfg each time you upgrade your copy of IDA. Instead, simply copy your existing idauser.cfg to your new IDA installation any time you upgrade.

The GUI Configuration File: idagui.cfg

Configuration items specific to the GUI version of IDA are located in their own file: <IDADIR>/cfg/idagui.cfg. This file is organized into roughly three sections: default GUI behaviors, keyboard hotkey mappings, and file extension configuration for the File ▸ Open dialog. In this section we discuss a few of the more interesting options. Consult idagui.cfg for the complete list of available options, which in most cases are accompanied by comments describing their purpose.

The Windows GUI version of IDA allows a secondary help file to be specified using the HELPFILE option. Any file specified here does not replace IDA’s primary help file. The intended purpose of this option is to provide access to supplemental information that may apply in specific reverse engineering situations. When a supplemental help file is specified, ctrl-F1 causes IDA to open the named file and search for a topic that matches the word under the cursor. If no match is found, then you are taken to the help file’s index. As an example, unless you count auto comments, IDA does not offer any help information regarding the instruction mnemonics in a disassembly. If you are analyzing an x86 binary, you might like to have an x86 instruction reference available on command. If you can locate a help file that happens to contain topics for each x86 instruction,[73] then help for any instruction is only a hotkey away. The only word of caution concerning supplemental help files is that IDA supports only the older WinHelp-style help files (.hlp). IDA does not support the use of compiled HTML help files (.chm) as secondary help files.

Note

Microsoft Windows Vista and later do not provide native support for 32-bit WinHelp files because the WinHlp32.exe file does not ship with these operating systems. Please refer to Microsoft Knowledge Base article 917607[74] for more information.

A common question asked about using IDA is “How can I patch binaries using IDA?” In a nutshell, the answer is “You can’t,” but we will put off discussing the details of this issue until Chapter 14. What you can do with IDA is patch the database to modify instructions or data in almost any way you see fit. Once we discuss scripting (Chapter 15), you will understand that modifying the database is not terribly difficult. But what if you are not interested in or not ready to learn IDA’s scripting language? IDA contains a database-patching menu that is not shown by default. The DISPLAY_PATCH_SUBMENU option is used to show or hide IDA’s patching menu, which shows up as Edit ▸ Patch Program. The options available on this menu are discussed in Chapter 14.

The single-line input box at the bottom of your IDA workspace is known as the IDA comand line. You can use the DISPLAY_COMMAND_LINE option to control whether this field is displayed or not. By default the command will be shown. If you are tight on screen space and you don’t anticipate the need to enter one-line scripts, then turning this feature off can help you regain a small amount of room in your IDA display. Note that this command line does not allow you to execute operating system commands as if you were entering them at a command prompt.

The hotkey configuration section of idagui.cfg is used to specify mappings between IDA actions and hotkey sequences. Hotkey reassignment is useful in many instances, including making additional commands available via hot-keys, changing default sequences to sequences that are easier to remember, or changing sequences that might conflict with other sequences in use by the operating system or your terminal application (useful primarily for the console version of IDA).

Virtually every option that IDA makes available through menu items or toolbar buttons is listed in this section. Unfortunately, the names of the commands tend not to match the text used on IDA’s menus, so it may take some effort to determine exactly which configuration file option maps to a specific menu option. For example, the Jump ▸ Jump to Problem command equates to the JumpQ option (which does happen to match its hotkey: ctrl-Q) in idagui.cfg. In addition, while many commands have matching comments to describe their purpose, many commands have no description at all, so you are left to determine the behavior of a command based on its name within the configuration file. A trick that may help you figure out what menu item a configuration file action is associated with is to search for the action in IDA’s help system. The results of such searches usually lead to the description of the action’s corresponding menu item.

The following lines represent example hotkey assignments in idagui.cfg:

"Abort"                =      0               // Abort IDA, don't save changes
"Quit"                 =      "Alt-X"         // Quit to DOS, save changes

The first line is the hotkey assignment for IDA’s Abort command, which in this case has no hotkey assignment. The unquoted value 0 indicates that no hotkey has been assigned to a command. The second line shows the hotkey assignment for IDA’s Quit action. Hotkey sequences are specified as a quoted string naming the key sequence. Numerous examples of hotkey assignments exist within idagui.cfg.

The final portion of idagui.cfg associates file type descriptions with their associated file extensions and specifies which file types will be listed in the Files of type drop-down list within the File ▸ Open dialog. A large number of file types are already described in the configuration file; however, if you find yourself frequently working with a file type that is not available, you may want to edit the file types list to add your file type to the list. The FILE_EXTENSIONS option describes all file associations known to IDA. The following line is an example of a typical file type association.

CLASS_JAVA,  "Java Class Files",                           "*.cla*;*.cls"

The line contains three comma-separated components: a name for the association (CLASS_JAVA), a description, and a filename pattern. Wildcards are allowed in the filename pattern, and multiple patterns can be specified by using a semicolon to separate them. A second type of file association allows several existing associations to be grouped into a single category. For example, the following line groups all associations whose names begin with EXE_ into a single association named EXE.

EXE,         "Executable Files",                           EXE_*

Note that the pattern specifier in this case is not quoted. We might define our own file association as follows:

IDA_BOOK,    "Ida Book Files",                             "*.book"

We can choose any name we like for the association as long as it is not already in use; however, simply adding a new association to the FILE_EXTENSIONS list is not sufficient to make that association appear in the File ▸ Open dialog. The DEFAULT_FILE_FILTER option lists the names of all associations that will appear in the File ▸ Open dialog. To complete the process and make our new association available, we would need to add IDA_BOOK to the DEFAULT_FILE_FILTER list.

Similar to the idauser.cfg file, the last line in idagui.cfg contains a directive to include a file named <IDADIR>/cfg/idauserg.cfg. If you do not feel comfortable editing idagui.cfg, then you should create idauserg.cfg and add to it all of the options that you wish to override.

The Console Configuration File: idatui.cfg

The analog to idagui.cfg for users of the console version of IDA is <IDADIR>/cfg/idatui.cfg. This file is very similar in layout and functionality to idagui.cfg. Among other things, hotkey specifications are made in the exact same manner as they are in idagui.cfg. Because the two files are so similar, we will detail only the differences here.

First, the options DISPLAY_PATCH_SUBMENU and DISPLAY_COMMAND_LINE are not available in the console version and are not included in idatui.cfg. The File ▸ Open dialog used in the console version is far simpler than the dialog used in the GUI version, so all of the file association commands available in idagui.cfg are missing in idatui.cfg.

On the other hand, a few options are available only for console versions of IDA. For example, you can use the NOVICE option to have IDA start in a beginner mode, in which it disables some of its more complex functionality in an attempt to be easier to learn. A notable difference in novice mode is the almost complete lack of subviews.

Console users are far more likely to rely on the use of hotkey sequences. To facilitate the automation of common hotkey sequences, console mode IDA provides a keyboard macro definition syntax. Several example macros can be found in idatui.cfg; however, the ideal location to place any macros that you develop is <IDADIR>/cfg/idausert.cfg (the console equivalent of idauserg.cfg). A sample macro contained in the default idatui.cfg might look like the following (in the actual idatui.cfg, this macro is commented out):

 MACRO   "Alt-H"        // this sample macro jumps to "start" label
  {
          "G"
          's' 't' 'a' 'r', 't'
          "Enter"
  }

Macro definitions are introduced with the MACRO keyword followed by the hotkey to be associated with the macro. The macro sequence itself is specified between braces as a sequence of key name strings or characters, which may in turn represent hotkey sequences themselves. The preceding example macro, activated using alt-H, opens the Jump to Address dialog using the G hotkey, enters the label start into the dialog one character at a time, and then closes the dialog using the enter key. Note that we could not use the syntax “start” to enter the name of the symbol, as this would be taken as the name of a hotkey and result in an error.

Note

Macros and novice mode are not available in the GUI version of IDA.

As a final note about configuration file options, it is important to know that if IDA encounters any errors while parsing its configuration files, it immediately terminates with an error message that attempts to describe the nature of the problem. It is not possible to start IDA until the error condition has been corrected.



[72] This file does not ship with IDA. Users must generate this file on their own if they wish IDA to find it.

[73] Pedram Amini swears by this WinHelp32 file: http://pedram.redhive.com/openrce/opcodes.hlp.