Often, emulators are used to perform dynamic analysis and reverse engineering tasks on executables. One very good reason for this is that they allow the reverse engineer to easily instrument the control of the execution, and they also bypass a lot of typical anti-debugging techniques. There are many emulators being used out there—QEMU, BOCHS, and Chris Eagles' IDA X86 emulator plugin, to name some. So, countless anti-emulation techniques exist, but some of them are specific to each emulator's particular implementation.
This topic could expand into some very in-depth discussions and move in many directions, but I will keep it limited to my own experience. In my own experimentation with emulation and anti-emulation in the Maya protector, I have learned some generic techniques that should work against at least some emulators. The goal of our binary protector's anti-emulation is to be able to detect when it is being run in an emulator, and if this is true, it should halt the execution and exit.
This technique can be especially useful in application-level emulators that are somewhat OS agnostic and are unlikely to have implemented more than the basic system calls (read, write, open, mmap, and so on). If an emulator does not support a system call and also does not delegate the unsupported syscall to the kernel, it is very likely that it will posit an erroneous return value.
So, the binary protector could invoke a handful of less common syscalls and check whether the return value matches the expected value. A very similar technique would be to invoke certain interrupt handlers to see whether they behave correctly. In either case, we are looking for OS features that were not properly implemented by the emulator.
The chances of an emulator perfectly emulating CPU architectures are next to none. Therefore, it is common to look for certain inconsistencies between how the emulator behaves and how the CPU should behave. One such technique is to attempt writing to privileged instructions, such as debug registers (for example, db0 to db7) or control registers (for example, cr0 to cr4). The emulation detection code may have a stub of ASM code that attempts to write to cr0 and see whether it succeeds.