Skip to main content

Check thresholds for floating-point comparisons note

Here is an interactive walkthrough that trace a real subtraction through every stage of the pipeline.The widget lets you step through all seven stages for four different examples — including the subnormal case. Use the prev/next buttons to walk the pipeline, or pick a different example at the top.

Here is a plain-language summary of each stage:

Overview — the function computes max_val − data as a real fp16 subtraction, then checks whether that result fits within threshold. The three preconditions (all positive, all finite, max_val ≥ data) let us skip all sign handling and borrow recovery.

Unpack — fp16 only stores 10 mantissa bits, so we prepend the implicit leading bit (1 for normals, 0 for subnormals) to get a full 11-bit significand. Subnormal effective exponents are corrected from 0 to 1 so alignment arithmetic stays consistent.

Align — subtraction requires both significands to share the same binary point. We right-shift the smaller one (lo) by exp_diff positions. Bits that fall off the bottom are not discarded — they are captured into three 1-bit flags: guard (G), round (R), and sticky (S), which drive rounding later.

Subtract — a simple binary subtraction of the two aligned 11-bit values. Non-negative by the max_val ≥ data precondition, so no borrow handling is needed. An exact zero here short-circuits immediately to return 1.

Normalise — subtraction can produce leading zeros (e.g. 1.0000 − 0.1111 = 0.0001…). We count them, left-shift the result to restore the leading 1 at bit 10, and subtract the count from the exponent. If the exponent would go to zero or below we produce a subnormal and shift by hi_exp − 1 instead.

Round — the three captured bits implement round-to-nearest-even: increment the mantissa only when G=1 and either the result is closer to the upper neighbour (R or S set) or it is an exact tie landing on an odd value (LSB set).

Compare — the packed difference and threshold are both positive fp16, so their raw 16-bit unsigned values are numerically ordered. A single unsigned ≤ gives the final answer — no floating-point comparator needed.

Normalisation step​

Why normalisation is needed at all​

Subtraction causes cancellation. When two close values are subtracted — say 1.0000000001 − 0.1111111111 — many high-order bits cancel and the result has several leading zeros. That means the implicit leading 1 is no longer sitting at bit 10 where IEEE 754 expects it. Normalisation restores it there by left-shifting, and reduces the exponent by the same amount to keep the value unchanged.

The leading-zero count and the branch condition​

if (hi_exp <= lz)   // subnormal path
else                // normal path

lz counts how many positions the result needs to shift left. If lz ≥ hi_exp, the required shift would push the exponent to zero or below — which means the result is too small to represent as a normal number, so it becomes a subnormal instead.

The normal path — shift left by lz​

for (int i = 0; i < 11; i++) begin
    if (5'(i) < lz) begin
        diff11 = {diff11[9:0], G};   // shift left, G enters at LSB
        G      = R;
        R      = S;
        S      = 1'b0;
    end
end
res_exp  = hi_exp - lz;
res_mant = diff11[9:0];   // implicit bit[10] discarded

Each iteration shifts diff11 one position left. The critical detail is {diff11[9:0], G} — G is not discarded, it fills the vacated LSB. This matters because G was the first bit shifted out during alignment (step 2), so it carries real precision. Filling with a hard zero instead (the original bug) would silently corrupt the low-order mantissa bit. After the full lz shifts the leading 1 lands at bit 10, and bit 10 is then stripped when res_mant = diff11[9:0] — it becomes the implicit bit that fp16 does not store.

The GRS chain advances in lockstep: each shift moves G into the mantissa LSB so the chain shifts right: G←R, R←S, S←0. This keeps G/R/S accurately describing the rounding residual below the kept mantissa.

The subnormal path — shift left by hi_exp − 1​

sub_shift = (hi_exp >= 5'd1) ? (hi_exp - 5'd1) : 5'd0;

When the result is subnormal, res_exp is forced to 0 and the implicit leading bit drops from 1 to 0 — the significand is now 0.mantissa rather than 1.mantissa. The derivation of the shift amount:

Normal:    value = 1.M  × 2^(res_exp)      →  res_exp = hi_exp − lz
Subnormal: value = 0.M' × 2^(−14)         →  res_exp = 0 (forced)

To preserve the correct value we need 0.M' × 2^(−14) = diff × 2^(hi_exp−15−10), which solves to M' = diff × 2^(hi_exp−1). So the left-shift is hi_exp − 1, not hi_exp. Using hi_exp was Bug 2 — it over-shifted by one, doubling the mantissa and producing a value twice as large as the true result.