Since binary protectors generally encrypt or obfuscate the physical body of a program, static analysis can be extremely difficult and, left to its own devises, will prove to be futile in many cases. Most reverse engineers who are attempting to unpack or break a protected binary will agree that a combination of dynamic analysis and static analysis must be used to gain access to the decrypted body of a binary.

A protected binary has to decrypt itself, or at least the portions of itself that are executing at runtime. Without any anti-debugging techniques, a reverse engineer can simply attach to the process of the protected program and set a breakpoint on the last instruction of the stub (assuming that the stub decrypts the entire executable).

Once the breakpoint is hit, the attacker can look at the code segment for where the protected binary lives and find its decrypted body. This would be extremely simple, and therefore it is very important for good binary protection to use as many techniques as possible to make debugging and dynamic analysis difficult for the reverse engineer. A protector like Maya goes to great lengths to protect the binary from both static and dynamic analysis.

Dynamic analysis is not limited to the ptrace syscall, although most debuggers are limited to it for the purpose of accessing and manipulating a process. Therefore, a binary protector should not be limited to protecting only against ptrace; ideally it will also be resistant to other forms of dynamic analysis, such as emulation and dynamic instrumentation (for example, Pin and DynamoRIO). We covered many anti-debugging techniques against ptrace analysis in previous chapters, but what about resistance to emulation?