The process for building and installing an IDA processor module is very similar to the process for building plug-ins and loaders, with one major difference that, if not followed, can result in the inability of IDA to utilize your processor. Some minor differences in the build process include these:

When you build a Windows-based processor module, you are expected to utilize a custom MS-DOS stub supplied with the SDK (<SDKDIR>/module/stub). In order to use a custom MS-DOS stub, you must instruct your linker to use your stub rather than the default stub it would otherwise include. When using Windows-specific compilers, it is occasionally possible to specify alternate stubs through the use of module definition (.def ) files. Borland build tools (used by Hex-Rays) support the specification of alternate stubs using .def files. The SDK includes <SDKDIR>/module/idp.def for your use if you happen to be using Borland tools. The GNU and Microsoft linkers both support .def files (albeit with a slightly different syntax); however, neither supports the specification of alternate MS-DOS stubs, which clearly poses a problem if you are using one of these compilers.

Assuming for a moment that you do manage to build your processor module with the SDK-supplied custom MS-DOS stub, you must still insert the processor description comment into the processor binary. This is the purpose of the <SDKDIR>/bin/mkidp.exe utility. You may add a description to a processor using the following syntax to invoke mkidp:

$ mkidp module description

Here, module is the path to your processor module, while description is a textual description of your module in the following form:

Long module name:short module name

To add a description to our Python processor module, we might use the following command line:

$ ./mkidp procs/python.w32 "Python Bytecode:python"

The mkidp utility attempts to insert the supplied description into the named module at an offset of 128 bytes into the file, in space that lies between the MS-DOS stub and the PE header, assuming such space exists. If there is not enough space because the PE header is too close to the end of the MS-DOS stub, you will receive the following error message:

mkidp: too long processor description

Things become more dependent on your tools at this point, because processors built with the Microsoft linker will have enough space available to insert a description, while processors built using the GNU linker will not.

In order to clear up the confusion in our minds and allow us to use either Microsoft or GNU tools, we developed a utility that we call fix_proc, which is available in the Chapter 19 section of the book’s website. The fix_proc utility uses the same command-line syntax as mkidp, but it provides additional behavior that allows it to insert a processor description into processor modules built with most compilers. When fix_proc is executed, it replaces a processor’s existing MS-DOS stub with the stub supplied with the SDK (thus eliminating the need to use .def files in the build process). At the same time, fix_proc performs the necessary actions to relocate the processor’s PE headers to create sufficient space to hold the processor-description string, before ultimately inserting the description string into the proper location within the processor binary. We use fix_proc as a replacement for mkidp in performing the required postprocessing steps on processor modules.

Table 19-1 describes the features of processors based on the tools used to build them.

Only processors that have valid descriptions will be listed in the file-loading dialog. In other words, without a valid description field, it is not possible to select a processor module.

All of these differences in the build process require a few more modifications to the makefile presented in Example 17-1 than were required to build loader modules. Example 19-1 shows a makefile modified to build our example Python processor.

#Set this variable to point to your SDK directory
  IDA_SDK=../../

  PLATFORM=$(shell uname | cut -f 1 -d _)

  ifneq "$(PLATFORM)" "MINGW32"
  IDA=$(HOME)/ida
  endif

  #Set this variable to the desired name of your compiled processor
  PROC=python

  #Specify a description string for your processor, this is required
  #The syntax is <long name>:<short name>
 DESCRIPTION=Python Bytecode:python

  ifeq "$(PLATFORM)" "MINGW32"
  PLATFORM_CFLAGS=-D__NT__ -D__IDP__ -DWIN32 -Os -fno-rtti
  PLATFORM_LDFLAGS=-shared -s
  LIBDIR=$(shell find ../../ -type d | grep -E "(lib|lib/)gcc.w32")
  ifeq ($(strip $(LIBDIR)),)
  LIBDIR=../../lib/x86_win_gcc_32
  endif
  IDALIB=$(LIBDIR)/ida.a
  PROC_EXT=.w32

  else ifeq "$(PLATFORM)" "Linux"
  PLATFORM_CFLAGS=-D__LINUX__
  PLATFORM_LDFLAGS=-shared -s
  IDALIB=-lida
  IDADIR=-L$(IDA)
  PROC_EXT=.ilx

    else ifeq "$(PLATFORM)" "Darwin"
  PLATFORM_CFLAGS=-D__MAC__
  PLATFORM_LDFLAGS=-dynamiclib
  IDALIB=-lida
  IDADIR=-L$(IDA)/idaq.app/Contents/MacOs
  PROC_EXT=.imc
  endif

  #Platform specific compiler flags
  CFLAGS=-Wextra $(PLATFORM_CFLAGS)

  #Platform specific ld flags
  LDFLAGS=$(PLATFORM_LDFLAGS)

  #specify any additional libraries that you may need
  EXTRALIBS=

  # Destination directory for compiled plugins
  OUTDIR=$(IDA_SDK)bin/procs/

  # Postprocessing tool to add processor comment
 MKIDP=$(IDA_SDK)bin/fix_proc
  #MKIDP=$(IDA)bin/mkidp

  #list out the object files in your project here
  OBJS=     ana.o emu.o ins.o out.o reg.o

  BINARY=$(OUTDIR)$(PROC)$(PROC_EXT)

  all: $(OUTDIR) $(BINARY)

  clean:
          -@rm *.o
          -@rm $(BINARY)

  $(OUTDIR):
          -@mkdir -p $(OUTDIR)

  CC=g++
  INC=-I$(IDA_SDK)include/

  %.o: %.cpp
          $(CC) -c $(CFLAGS) $(INC) $< -o $@

  LD=g++
    ifeq "$(PLATFORM)" "MINGW32"
  #Windows processor's require post processing
  $(BINARY): $(OBJS)
          $(LD) $(LDFLAGS) -o $@ $(OBJS) $(IDALIB) $(EXTRALIBS)
       $(MKIDP) $(BINARY) "$(DESCRIPTION)"
  else
  $(BINARY): $(OBJS)
         $(LD) $(LDFLAGS) -o $@ $(OBJS) $(IDALIB) $(EXTRALIBS)
  endif

  #change python below to the name of your processor, make sure to add any
  #additional files that your processor is dependent on
  python.o: python.cpp
  ana.o: ana.cpp
  emu.o: emu.cpp
  ins.o: ins.cpp
  out.o: out.cpp
  reg.o: reg.cpp

In addition to the minor changes to account for different suffixes and default file locations for processors, the primary differences are the definition of a description string , the specification of a utility to insert description strings , and the addition of a build step to insert the description string in Windows processor modules .