The Derived Optimizer: Six Questions and Failure on Real Architectures

A program that derives every component of a learnable induced-metric optimizer from first principles. Starting from a 10-hyperparameter heuristic, six questions resolve into a 3-hyperparameter algorithm with globally stable Lyapunov dynamics, a minimax-optimal Newton target s_i* = -log H_ii + mean(log H), Hutchinson-based diagonal Hessian estimation with a (1+δ)/(1-δ) robustness bound independent of dimension and condition number, and joint convergence proofs via block-triangular Jacobian analysis. The derivation is mathematically clean but fails empirically on real neural networks: a full-batch ablation shows diagonal Newton actively hurts on MNIST (more preconditioning yields worse accuracy, with condition number 80,349 at the most aggressive setting giving the worst result), and direct off-diagonal-mass measurements show 40–50% wrong-sign Hutchinson estimates on MNIST MLP and ratios up to 35,402 on transformer attention layers. Both failure modes are mechanistically explained: the diagonal preconditioner class is structurally misspecified for the Kronecker structure of neural-network Hessians, and the Hutchinson estimator's variance inflates quadratically with each Kronecker factor's non-diagonality. The natural structural upgrade — Kronecker minimax — reproduces KFAC.