Dimension-Free Dot Product NMSE Under Log-Uniform Quantization

A matrix multiplication in a neural network is built from dot products, each summing thousands of weight × activation products. A natural worry: does quantization error accumulate as the dot product dimension grows? This corollary shows it does not — the NMSE (normalized mean squared error, the fraction of signal power lost to quantization) of a dot product is independent of its length.

This is a direct consequence of the product error decomposition, which shows how quantization error in individual products combines. It means QF8 (or any format using log-uniform quantization) can be used in arbitrarily large matrix multiplications without degradation.

Statement (Corollary)

For a dot product Z = Σᵢ₌₁ᵈ wᵢxᵢ where:

• Each (wᵢ, xᵢ) pair is independent
• Weights are iid and activations are iid
• Both quantizers satisfy the centroid condition (each codepoint is the conditional mean of its cell — true for well-designed quantizers)

Then:

$$\text{NMSE}_Z \approx \text{NMSE}_w + \text{NMSE}_x + O(1/N^4)$$

The NMSE does not grow with dimension d.

Proof Sketch

Under independence across dimensions, the expected squared error of the dot product is:

$$E[(Z - \hat{Z})^2] = d \cdot E[w^2 x^2](<n_w + n_x - n_w n_x>)$$

and the expected squared signal is:

$$E[Z^2] = d \cdot E[w^2 x^2]$$

The dimension d and the scale factor E[w²x²] cancel in the NMSE ratio, leaving only the per-element quantization quality.

Why This Matters

Absolute error does accumulate: the MSE of a d-dimensional dot product grows linearly with d. But NMSE — the relative measure — does not, because both the signal and the noise scale equally with d.

This is analogous to the law of large numbers: individual product errors are random and partially cancel in the sum. The NMSE ratio captures this cancellation automatically.

Practical implication: A QF8 matrix multiplication with d = 4096 has the same NMSE as one with d = 32. The +6.6 dB SQNR advantage over FP8 holds at any scale, which is confirmed by benchmarks showing consistent SQNR across matrix sizes from 16×32×16 to 128×256×128.