An illegal opcode, also called an undocumented instruction, is an instruction to a CPU that is not mentioned in any official documentation released by the CPU's designer or manufacturer, which nevertheless has an effect. Illegal opcodes were common on older CPUs designed during the 1970s, such as the MOS Technology 6502, Intel 8086, and the Zilog Z80. They exist as a side effect of the wiring of transistors in the CPU, and usually combine functions of the CPU that were not intended to be combined.
While most illegal instructions have useless or even highly undesirable effects (such as crashing the computer), a few might by accident do something that can be useful in certain situations. Such instructions were sometimes exploited in computer games of the 1970s and 1980s to speed up certain time-critical sections. Another common use of them was in the ongoing battle between copy protection implementations and cracking. Here, they were a form of security through obscurity, and their secrecy usually did not last very long.
A danger associated with the use of illegal instructions was that, given the fact that the manufacturer did not guarantee their existence and function, they might disappear or behave differently with any change of the CPU internals or any new revision of the CPU, rendering programs that use them incompatible with the newer revisions. For example, a number of older Apple II games do not work correctly on the newer Apple IIc, because the latter uses a newer CPU revision — 65C02 — that does away with illegal opcodes.
More recent CPUs, such as the 80186, 80286, 68000 and its descendants or the PowerPC processor, usually do not have illegal opcodes. Their manufacturers guarantee that the CPU will behave in a well-defined way when it finds an unknown opcode in the instruction stream: usually, this means triggering a certain exception or fault condition. The operating system's exception or fault handler will then usually terminate the application that caused the fault. Another, less common way of handling illegal instructions is by defining them to do nothing except taking up time and space (equivalent to the CPU's official NOP instruction); this method is used by the TMS9900 and 65C02 processor, among others.
Today, the details of these instructions are mainly of interest for exact emulation of older systems.
- Illegal opcodes on the 6502: http://www.ffd2.com/fridge/docs/6502-NMOS.extra.opcodes (see also How MOS 6502 Illegal Opcodes really work)
- Illegal opcodes on the Z80: http://www.myquest.nl/z80undocumented/