In the SDM, there is a part dealing with the the CET notrack prefix(3EH).
It says that in non-64-bit mode 3EH is honoured only if it is the last Group 2 prefix present. In 64-bit mode, 3EH is honoured if there is no 64H or 65H present.
I'm not sure if this also applies to hint-taken prefix(3EH).
In the Optimization Reference Manual, it says that an instruction has the taken hint if it has at least one instruction prefix 3EH. It doesn't say that this behaviour is 64-bit only, and this contradicts the following.
According to XED, the Group 2 prefixes are resolved like this:
In non-64-bit mode, only the last Group 2 prefix is honoured. If 2EH or 3EH is not the last Group 2 prefix, it is ignored. Here, 3E 2E leads to hint-not-taken unlike in 64-bit mode.
In 64-bit mode, if 64H or 65H is present 3EH(hint-taken or CET notrack) is completely ignored. If 64H or 65H is not present, 3EH(hint-taken) takes precedence over 2EH(hint-not-taken).
2EH(hint-not-taken) has effect only if
- There is no 64H or 65H following it.
- There is no effective 3EH(hint-taken) i.e. if there is no 64H or 65H, 3EH(hint-taken) should also be absent.
I wasn't able to deduce this behaviour only from the documents, nor could I think of a way how to test this behaviour. For instance, I was able to test that mandatory prefix 66H is ignored when mandatory prefix F2H or F3H is present regardless of order, although this is not documented.
I could appreciate it if anyone could provide related materials or test methods.
