The size attribute of the operand for branch instructions are closely related to "address_size". This makes branch instructions different from data processing instructions.
The Address_size attribute can be overridden by 67H prefix, a legacy created since 16-bit processors. That was important for the software transition for 32-bit processors so that it can handle 16-bit code and 32-bit code without mode transition.
The transition from 32-bit to 64-bit involves compatibility mode which behaves just like before, 67H prefix can override address_size attribute from 32-bit to 16 bit.
In 64-bit mode's flat addressing programming paradigm, there aren't many practical needs to change the address_size attribute. But 67H prefix is kind of built-in from hardware perspective. So you see it still affecting branch instructions, which is kind of analogous to appendicitis to most programmers on the application side.
Address_size can affect IP and produce different wrap-around behavior