Question about ACE implementation

stlw · ‎09-13-2026

I am trying to implement ACE ISA in the Bochs emulator according to latest spec published ACE_v1_Specification_public_1_16_2.pdf and validate the result against SDE sde-external-10.13.1-2026-07-28-win.

I was under assumption that SDE is always exactly matching future HW but seems like I see differences. Should I trust SDE or pseudocode published in document ?

ACE v1 — VCVT{BF8,HF8}2BF4S (FP8→FP4) NaN-handling: pseudocode vs. SDE execution

Instructions: VCVTBF82BF4S (E5M2→E2M1), VCVTHF82BF4S (E4M3→E2M1)
Spec reference: ACE v1 Specification, Section 16.4, fp8_e5m2_to_fp4_e2m1(i) / fp8_e4m3_to_fp4_e2m1(i)
Observed on: Intel SDE 10.13.1 (2026-07-28), Diamond Rapids/ACE functional model, run with -chip_check_disable 1 (these opcodes are not yet marked valid without that flag — worth flagging to whoever you ask, since this is emulated, not-yet-shipped-silicon behavior)

The pseudocode (as published)

DEFINE fp8_e5m2_to_fp4_e2m1(i):
  ...
  IF e_i == 0x1F:                        // Inf or NaN — E5M2 doesn't distinguish
      e_o, m_o = 0x3, 0x1                // clamp to FP4 max
  ELSE IF (e_i > exp_rebias+3) or (...): // overflow
      e_o, m_o = 0x3, 0x1
  ...

DEFINE fp8_e4m3_to_fp4_e2m1(i):
  ...
  IF e_i == 0xF and m_i == 0x7:          // NaN (0xF/0x7 is the single named case, "NaN (0x7F)")
      e_o, m_o = 0x3, 0x1                // clamp to FP4 max
  ELSE IF (e_i > exp_rebias+3) or (...): // overflow
      e_o, m_o = 0x3, 0x1
  ...

Both formats route Infinity and NaN into the same "clamp to max magnitude, sign preserved" result. FP4 (E2M1) has no Inf/NaN encoding of its own, so this is the only sane target representation either way.

What we observe instead

Infinity and ordinary overflow saturate exactly as documented. Genuine NaN inputs specifically (not Inf) instead produce a fixed negative zero (sign=1, exp=0, frac=0) — regardless of the input's own sign bit.

One representative case per kind, each shown as raw byte → S.E.M → per-spec/Bochs result → actual SDE result:

# Instr Input (hex) S.E.M Kind Spec / Bochs result SDE result

A	VCVTBF82BF4S	0x7C = 0 11111 00	S=0 E=0x1F M=0x0	+Infinity	0111 = 0x7 (+max)	0111 = 0x7	match
B	VCVTBF82BF4S	0x7D = 0 11111 01	S=0 E=0x1F M=0x1	+NaN	0111 = 0x7 (+max)	1000 = 0x8 (−0)	mismatch
C	VCVTHF82BF4S	0x7E = 0 1111 110	S=0 E=0xF M=0x6	large-finite (E4M3's exp=0xF, non-NaN mantissa)	0111 = 0x7 (+max)	0111 = 0x7	match
D	VCVTHF82BF4S	0x7F = 0 1111 111	S=0 E=0xF M=0x7	+NaN (spec's own named case, "0x7F")	0111 = 0x7 (+max)	1000 = 0x8 (−0)	mismatch

Key details for the experts:

The NaN trigger condition in the pseudocode is correct — I verified SDE's own behavior agrees with the spec on which inputs count as NaN (case C, frac=6, is not treated specially; case D, frac=7, is). So this isn't a decode/classification bug on either side.
Only the NaN branch's output value diverges. Infinity (case A) and non-NaN overflow both saturate correctly on real execution, matching e_o,m_o = 0x3,0x1 from the pseudocode exactly.
The SDE output sign is not input-dependent. Case D's input has S=0 (positive), yet the output is sign=1 (negative zero) — same fixed 0x8 nibble appears for negative-sign NaN inputs too (e.g. 0xFF → 0x8, 0xFD → 0x8). So it's not "flush-to-zero preserving sign," it's a constant.
Ruled out: we tested whether this could be an artifact of a "non-saturating base + missing saturate" implementation (i.e., maybe hardware runs the ordinary RNE-rebias arithmetic unconditionally and only clamps afterward, and the pseudocode's explicit NaN/overflow branches are documentation of a saturate step that isn't actually being applied). Running that base arithmetic (the spec's own rnex = i + 1 + fixup rounding trick) unconditionally on exp=0xF inputs produces the same wraparound garbage exponent for both frac=6 and frac=7 (e_o=-6 for both) — so it cannot explain why real hardware treats those two cases differently. That pointed us away from "saturate is missing" and toward "the NaN branch fires correctly but its documented result constant is wrong."

Question for the experts: is e_o,m_o = 0x3,0x1 in the NaN branch of fp8_e5m2_to_fp4_e2m1/fp8_e4m3_to_fp4_e2m1 a documentation error (should it instead be e_o,m_o = 0x0,0x0 with sign forced to 1, matching what Diamond Rapids/ACE's SDE model actually does), or is the SDE model itself not yet a validated reference for this instruction?