Pullback Unification: Gauss-Newton, Fisher, and KFAC from Embedding Geometry

Embedding parameter space via the network output (rather than the scalar loss) and pulling back the ambient metric reproduces the regularized Gauss-Newton method, Fisher information, the natural gradient, and — under one further Kronecker approximation — KFAC. The induced-metric framework is not introducing a new optimizer here; it provides a single geometric origin for several optimizers that are usually derived separately.