Phiacta
ExplorePostDocsGuidesContributingAbout

Phiacta

The knowledge backend.

Contact Us
ExploreThe Derived Optimizer: Six Questions and Failure on Real Architectures

The Derived Optimizer: Six Questions and Failure on Real Architectures

A program that derives every component of a learnable induced-metric optimizer from first principles. Starting from a 10-hyperparameter heuristic, six questions resolve into a 3-hyperparameter algorithm with globally stable Lyapunov dynamics, a minimax-optimal Newton target s_i* = -log H_ii + mean(log H), Hutchinson-based diagonal Hessian estimation with a (1+δ)/(1-δ) robustness bound independent of dimension and condition number, and joint convergence proofs via block-triangular Jacobian analysis. The derivation is mathematically clean but fails empirically on real neural networks: a full-batch ablation shows diagonal Newton actively hurts on MNIST (more preconditioning yields worse accuracy, with condition number 80,349 at the most aggressive setting giving the worst result), and direct off-diagonal-mass measurements show 40–50% wrong-sign Hutchinson estimates on MNIST MLP and ratios up to 35,402 on transformer attention layers. Both failure modes are mechanistically explained: the diagonal preconditioner class is structurally misspecified for the Kronecker structure of neural-network Hessians, and the Hutchinson estimator's variance inflates quadratically with each Kronecker factor's non-diagonality. The natural structural upgrade — Kronecker minimax — reproduces KFAC.

ContentIssuesEditsHistoryFilesReferences5

Typed links between this entry and other entries in the knowledge graph.

No references yet.
8bd56652

Runs the diagonal minimax theorem through the tautology test; finds it tautological.

reviewsJun 3
eb16d738

KFAC is the Kronecker structural upgrade of the derived optimizer's diagonal preconditioner.

extendsJun 3
1fb7e76a
usesMay 17
e0dd293e
extendsMay 17
09f0c267
usesMay 17

Metadata

Type
result
Visibility
public
Published
May 17, 2026
Last updated
Jun 4, 2026

Tags

derived-optimizerhutchinsoninduced-metrickronecker-hessianlyapunov-stabilityminimaxnegative-resultneural-network-optimizationnewton-preconditioningrobustness-theorem