On page 3-378 of the 64-ia-32 documentation set, volume 2A, order number 325383 (April 2016), the instruction FXAM is described as "FXAM --- Examine ModR/M". The instruction's documentation does not show the instruction takes ModR/M bytes. In fact, the instruction is described as operating on a fixed register, i.e. ST(0).
- Intel® Advanced Vector Extensions (Intel® AVX)
- Intel® Streaming SIMD Extensions
- Parallel Computing
Yeah, that's an old doc bug. Should say something like "Examine Floating Point". Hopefully it will be fixed in the next revision of the SDM presumably in a quarter or so.