Manual typo for instruction encoding: REX.R vs REX.B
Hello! I believe that I've found a set of typos in the Intel software developer manuals. I didn't see a forum category for reporting these types of issues; and this board seemed to be the closest to what I was looking for. Let me know if this is off-topic and if there's a better place to report manual typos in the future.
The manual for REX prefixes specifies that the REX.R bit is used to extend "reg" operands, and the REX.B bit is used to extend "r/m" operands. The problem is that the description of various instructions says to use REX.R, even when the instruction uses an r/m operand.
Here's an excerpt from the manual entry for the NEG instruction (bolded part relevant):
In 64-bit mode, the instruction’s default operation size is 32 bits. Using a REX prefix in the form of REX.R permits access to additional registers (R8-R15). Using a REX prefix in the form of REX.W promotes operation to 64 bits. See the summary chart at the beginning of this section for encoding data and limits.
The NEG instruction only operates on r/m operands; my understanding is that "REX.R" here should read "REX.B".
You can confirm this by assembling a "neg r8" instruction using gcc which produces the bytes [0x49, 0xf7, 0xd8]. Here, 0x49 is the REX prefix with REX.B set and REX.R unset. If you instead do what the manual currently describes by setting only REX.R, the instruction no longer decodes as "neg r8".
This problem is not unique to the NEG instruction. Other instructions descriptions seem to have the same issue, including MUL and others. (Some instruction descriptions do correctly use REX.B for r/m operands, including that for RDRAND).