Log-Uniform Quantizer is Optimal for Multiplication

The minimax NMSE theorem shows that the log-uniform quantizer is the best single-operand quantizer. But neural networks don't just store values — they multiply them (weights × activations). This theorem extends the optimality result to the multiplication operation itself: log-uniform quantization also minimizes the worst-case error in products of quantized values.

Combined with the product error decomposition, this means QF8's log-uniform encoding is optimal not just for storage fidelity, but for the actual computation that neural networks perform.

Statement

Among all N-level quantizer pairs (Q_X, Q_Y) on [a,b], the log-uniform quantizer minimizes the worst-case product NMSE (normalized mean squared error — the fraction of signal power lost):

$$Q_X^* = Q_Y^* = Q_{\log} = \arg\min_{Q_X, Q_Y \in \mathcal{Q}_N} \max_{f_X, f_Y} \text{NMSE}_{\text{prod}}$$

The minimax value is:

$$2n^* - (n^*)^2 = 1 - (1 - n^*)^2$$

where n* = ε²/12 is the per-operand minimax NMSE.

Proof

The proof combines two properties:

1. Monotonicity of product error: Under the centroid condition, the product NMSE formula is NMSE_X + NMSE_Y − NMSE_X·NMSE_Y. The partial derivative ∂/∂NMSE_X = 1 − NMSE_Y > 0, so product NMSE is strictly increasing in each factor's individual NMSE. Reducing either operand's NMSE always helps.

2. Density-independence of log-uniform: The minimax theorem shows that Q_log achieves NMSE = n* for all densities. Any other quantizer Q_X has some density f_X* where NMSE(Q_X, f_X*) > n*. By monotonicity, this gives strictly worse product NMSE.

Specialization to Log-Uniform Quantizers

For log-uniform quantizers with geometric midpoints, MSRE = NMSE = ε²/12 for any density, and:

$$\text{NMSE}_{\text{prod}} = \frac{(R\ln 2)^2}{6N^2} + O(1/N^4) = 2\text{NMSE}^* + O(1/N^4)$$

The product NMSE is approximately twice the single-operand NMSE — a clean and tight bound.

Significance

This result closes the loop: log-uniform quantization is provably optimal for scalar representation (minimax NMSE), for pairwise multiplication (this theorem), and for dot products of arbitrary dimension. The entire multiply-accumulate pipeline benefits from the same format choice.