QuakeFloat8 (QF8): A Provably Optimal Log-Domain 8-Bit Format for ML Multiply-Accumulate

Authors: Noah Everett Date: February 2026

Abstract

QuakeFloat8 (QF8) is a block-scaled logarithmic 8-bit number format for machine learning in which multiplication reduces to 7-bit integer addition. The format uses a 1-bit sign, a 7-bit u3.4 fixed-point log₂ code per element, and a shared E8M0 block exponent per 32 elements. Key results:

1. Log-uniform quantization is provably minimax-optimal for NMSE over all source distributions (Theorem 2.1).
2. QF8 achieves +6.6 dB SQNR over IEEE FP8 E4M3 at the same effective storage cost.
3. The QF8 multiplier costs ~66 gates vs. ~231 for FP8 (3.5× cheaper).
4. TinyGPT-2 training with STE-based QAT matches FP32 validation loss.

All theorems have been corrected, numerically verified, and checked in Lean 4.

Structure

The paper is organized into the following sections:

1. Introduction and Motivation — The 8-bit frontier, the Quake insight (log₂ bit-pattern trick), and contributions.
2. The QF8 Format — Encoding scheme (u3.4 + E8M0), logarithmic spacing (16 levels/octave), multiplication as integer addition, Kulisch accumulation strategy, and BMD realizability.
3. Minimax Optimality — Proof that log-uniform quantization uniquely minimizes worst-case NMSE over all densities. The equalization property makes NMSE density-independent.
4. Product Error Decomposition — General formula for how quantization error propagates through multiplication. Under centroid condition: (1 − NMSE_prod) = (1 − NMSE_X)(1 − NMSE_Y). Extension to dimension-free dot product NMSE.
5. Exact MSRE — Closed-form MSRE = 2 − 2√r·ln(r)/(r−1), confirming ε²/12 and correcting prior ε²/2 error.
6. Separation Result — Log vs. uniform quantization MSRE ratio grows exponentially in distribution spread (346,000× at σ=2 for LogNormal).
7. Hardware Cost Analysis — Gate-level models: QF8-Medium MAC at 814 gates, 0.038 pJ/MAC. 128×128 systolic array at 50.5 TOPS/W.
8. Training Results — TinyGPT-2 QAT: QF8 val loss 2.5445 vs FP32 2.5450 vs FP8 2.5478.
9. Lean 4 Verification — All key theorems checked via exhaustive numerical grids + formal theorem statements.
10. Literature Positioning — Relationship to Johnson (2018) ELMA, OCP MX formats, NF4/QLoRA, and classical LNS.

Key Contributions

1. Formal optimality — Minimax NMSE theorem with Lean 4 verification. No prior 8-bit ML format has a proof of optimality.
2. Format design — u3.4 log code + E8M0 block scaling: 2× precision per octave of FP8 E4M3 at same storage cost (8.25 bits/element).
3. Hardware analysis — 3.5× cheaper multiplier than FP8; near-INT8 area and power with floating-point dynamic range.
4. Training validation — QF8 matches FP32 loss; outperforms FP8 E4M3.

Open Questions

• Scaling validation beyond TinyGPT-2 (GPT-2 Small/Medium)
• Gradient quantization: u4.3 variant for backward passes
• BMD decoder characterization conjecture (unproven)

References

1. Johnson (2018), "Rethinking Floating Point for Deep Learning," arXiv:1811.01721
2. Micikevicius et al. (2022), "FP8 Formats for Deep Learning," arXiv:2209.05433
3. Rouhani et al. (2023), "Microscaling Data Formats for Deep Learning," arXiv:2310.10537
4. Dettmers et al. (2023), "QLoRA," arXiv:2305.14314
5. Gustafson & Yonemoto (2017), "Beating Floating Point at its Own Game: Posit Arithmetic"
6. Horowitz (2014), "Computing's Energy Problem," ISSCC
7. Miyashita et al. (2016), "CNNs using Logarithmic Data Representation," arXiv:1603.01025