Phiacta

Induced-metric optimizer where the inverse metric γ⁻¹ = diag(exp(s)) is a learnable per-parameter diagonal, updated online each step. Each parameter gets its own scale factor exp(s_i), with mean-centering to avoid scale degeneracy with ξ. O(N) learnable state. Best peak accuracy on MNIST (97.99%).

Embedding parameter space via the network output (rather than the scalar loss) and pulling back the ambient metric reproduces the regularized Gauss-Newton method, Fisher information, the natural gradient, and — under one further Kronecker approximation — KFAC. The induced-metric framework is not introducing a new optimizer here; it provides a single geometric origin for several optimizers that are usually derived separately.

A meta-theorem template (the "master minimax") for optimality claims in the induced-metric optimizer family, paired with a "tautology test" that flags when such a claim is merely an algorithmic re-derivation of a known optimizer rather than a genuine optimality result. The project's diagonal minimax (derived optimizer) and Kronecker minimax (pullback unification → KFAC) are both run through the test and shown to be tautological — which reframes them as algorithmic derivations from embedding geometry, not novel optimality theorems.

A complete pointwise theory of when the induced metric can convert a non-convex loss into a geodesically convex one. The Riemannian Hessian is (Hess_g L)_ij = (H_ij - C_ij)/(1 + ξ‖∇L‖²_γ), where C_ij = Γ^k_ij(γ)·∇_k L is a curvature correction depending on ∇γ and ∇L but not on H. Three-level hierarchy: (1) constant γ gives C ∝ H — eigenvalue signs are preserved, saddles cannot be fixed; (2) scalar γ = e^s I gives trace-constrained C (tr(C) = 0 in 2D), which can fix asymmetric saddles but not symmetric ones; (3) diagonal γ = diag(e^{s_i}) gives unrestricted C via exponential anisotropy ratios e^{s_i - s_j}, enabling sign-flipping for all N. The diagonal-N theorem is constructively proved via a Gershgorin argument and formally verified in Lean 4 (zero sorry). A universal critical-point obstruction (C ∝ ∇L vanishes at saddles) precludes global geodesic convexity for any function with stationary points, but the geodesic convexity basin around any minimum can be dramatically enlarged beyond the Euclidean convexity basin (demonstrated on the quartic well, four disconnected basins merge into one). The optimal correction has anti-correlation structure (∇s_i with sign opposite to H_ii) which motivates the curvature-aware diagonal variant.

A program that derives every component of a learnable induced-metric optimizer from first principles. Starting from a 10-hyperparameter heuristic, six questions resolve into a 3-hyperparameter algorithm with globally stable Lyapunov dynamics, a minimax-optimal Newton target s_i* = -log H_ii + mean(log H), Hutchinson-based diagonal Hessian estimation with a (1+δ)/(1-δ) robustness bound independent of dimension and condition number, and joint convergence proofs via block-triangular Jacobian analysis. The derivation is mathematically clean but fails empirically on real neural networks: a full-batch ablation shows diagonal Newton actively hurts on MNIST (more preconditioning yields worse accuracy, with condition number 80,349 at the most aggressive setting giving the worst result), and direct off-diagonal-mass measurements show 40–50% wrong-sign Hutchinson estimates on MNIST MLP and ratios up to 35,402 on transformer attention layers. Both failure modes are mechanistically explained: the diagonal preconditioner class is structurally misspecified for the Kronecker structure of neural-network Hessians, and the Hutchinson estimator's variance inflates quadratically with each Kronecker factor's non-diagonality. The natural structural upgrade — Kronecker minimax — reproduces KFAC.

Induced-metric optimizer with non-zero off-diagonal blocks in the ambient metric: h = [[γI, a], [bᵀ, 1]]. The pullback becomes G = γI + ξ(alᵀ + lbᵀ + llᵀ) where l, a, b can be gradient, momentum, parameters, or zero. Inverse via rank-2 Woodbury or Sherman-Morrison. Setting a=b=l recovers the original metric.

The pullback metric $G = \gamma I + \xi\, \nabla L\, \nabla L^\top$ on parameter space, obtained by embedding $\theta \mapsto (\theta, L(\theta))$ into $\mathbb{R}^{N+1}$ with block-diagonal ambient metric $\mathrm{diag}(\gamma I_N, \xi)$. The Sherman-Morrison inverse gives a Riemannian gradient flow $\dot\theta = -\nabla L / (\gamma + \xi \|\nabla L\|^2)$, which is a smoothed variant of gradient clipping. This is the base construction underlying the learnable, off-diagonal, Newton-target, and output-embedding (Gauss-Newton / KFAC) variants tracked in the rest of the project.

The recent development of bootstrap methods based on semidefinite relaxations of positivity constraints has enabled rigorous two-sided bounds on local observables directly in the thermodynamic limit. However, these bounds inevitably become loose in symmetry broken phases, where local constraints are insufficient to capture long-range order. In this work, we identify the origin of this looseness as order parameter defects which are difficult to remove using local operators. We introduce a defect bootstrap framework that resolves this limitation by embedding the system into an auxiliary defect model equipped with ancilla degrees of freedom. This construction effectively enables local operators to remove order parameter defects, yielding tighter bounds in phases with spontaneous symmetry breaking. This approach can be applied broadly to pairwise-interacting local lattice models with discrete or continuous internal symmetries that satisfy a property we call defect diamagnetism, which requires that the ground state energy does not decrease upon adding any finite number of symmetry defects. Applying the method to the transverse field Ising models in 1D and 2D, we obtain significantly improved bounds on energy densities and spin correlation functions throughout the symmetry broken phase in 1D and deep within the phase in 2D. Our results demonstrate that physically motivated constraint sets can dramatically enhance the power of bootstrap methods for quantum many-body systems.

Induced-metric optimizer where the inverse metric γ⁻¹ = diag(exp(s)) is a learnable per-parameter diagonal, updated online each step. Each parameter gets its own scale factor exp(s_i), with mean-centering to avoid scale degeneracy with ξ. O(N) learnable state. Best peak accuracy on MNIST (97.99%).

Embedding parameter space via the network output (rather than the scalar loss) and pulling back the ambient metric reproduces the regularized Gauss-Newton method, Fisher information, the natural gradient, and — under one further Kronecker approximation — KFAC. The induced-metric framework is not introducing a new optimizer here; it provides a single geometric origin for several optimizers that are usually derived separately.

A meta-theorem template (the "master minimax") for optimality claims in the induced-metric optimizer family, paired with a "tautology test" that flags when such a claim is merely an algorithmic re-derivation of a known optimizer rather than a genuine optimality result. The project's diagonal minimax (derived optimizer) and Kronecker minimax (pullback unification → KFAC) are both run through the test and shown to be tautological — which reframes them as algorithmic derivations from embedding geometry, not novel optimality theorems.

A complete pointwise theory of when the induced metric can convert a non-convex loss into a geodesically convex one. The Riemannian Hessian is (Hess_g L)_ij = (H_ij - C_ij)/(1 + ξ‖∇L‖²_γ), where C_ij = Γ^k_ij(γ)·∇_k L is a curvature correction depending on ∇γ and ∇L but not on H. Three-level hierarchy: (1) constant γ gives C ∝ H — eigenvalue signs are preserved, saddles cannot be fixed; (2) scalar γ = e^s I gives trace-constrained C (tr(C) = 0 in 2D), which can fix asymmetric saddles but not symmetric ones; (3) diagonal γ = diag(e^{s_i}) gives unrestricted C via exponential anisotropy ratios e^{s_i - s_j}, enabling sign-flipping for all N. The diagonal-N theorem is constructively proved via a Gershgorin argument and formally verified in Lean 4 (zero sorry). A universal critical-point obstruction (C ∝ ∇L vanishes at saddles) precludes global geodesic convexity for any function with stationary points, but the geodesic convexity basin around any minimum can be dramatically enlarged beyond the Euclidean convexity basin (demonstrated on the quartic well, four disconnected basins merge into one). The optimal correction has anti-correlation structure (∇s_i with sign opposite to H_ii) which motivates the curvature-aware diagonal variant.

A program that derives every component of a learnable induced-metric optimizer from first principles. Starting from a 10-hyperparameter heuristic, six questions resolve into a 3-hyperparameter algorithm with globally stable Lyapunov dynamics, a minimax-optimal Newton target s_i* = -log H_ii + mean(log H), Hutchinson-based diagonal Hessian estimation with a (1+δ)/(1-δ) robustness bound independent of dimension and condition number, and joint convergence proofs via block-triangular Jacobian analysis. The derivation is mathematically clean but fails empirically on real neural networks: a full-batch ablation shows diagonal Newton actively hurts on MNIST (more preconditioning yields worse accuracy, with condition number 80,349 at the most aggressive setting giving the worst result), and direct off-diagonal-mass measurements show 40–50% wrong-sign Hutchinson estimates on MNIST MLP and ratios up to 35,402 on transformer attention layers. Both failure modes are mechanistically explained: the diagonal preconditioner class is structurally misspecified for the Kronecker structure of neural-network Hessians, and the Hutchinson estimator's variance inflates quadratically with each Kronecker factor's non-diagonality. The natural structural upgrade — Kronecker minimax — reproduces KFAC.

Induced-metric optimizer with non-zero off-diagonal blocks in the ambient metric: h = [[γI, a], [bᵀ, 1]]. The pullback becomes G = γI + ξ(alᵀ + lbᵀ + llᵀ) where l, a, b can be gradient, momentum, parameters, or zero. Inverse via rank-2 Woodbury or Sherman-Morrison. Setting a=b=l recovers the original metric.

The pullback metric $G = \gamma I + \xi\, \nabla L\, \nabla L^\top$ on parameter space, obtained by embedding $\theta \mapsto (\theta, L(\theta))$ into $\mathbb{R}^{N+1}$ with block-diagonal ambient metric $\mathrm{diag}(\gamma I_N, \xi)$. The Sherman-Morrison inverse gives a Riemannian gradient flow $\dot\theta = -\nabla L / (\gamma + \xi \|\nabla L\|^2)$, which is a smoothed variant of gradient clipping. This is the base construction underlying the learnable, off-diagonal, Newton-target, and output-embedding (Gauss-Newton / KFAC) variants tracked in the rest of the project.

The recent development of bootstrap methods based on semidefinite relaxations of positivity constraints has enabled rigorous two-sided bounds on local observables directly in the thermodynamic limit. However, these bounds inevitably become loose in symmetry broken phases, where local constraints are insufficient to capture long-range order. In this work, we identify the origin of this looseness as order parameter defects which are difficult to remove using local operators. We introduce a defect bootstrap framework that resolves this limitation by embedding the system into an auxiliary defect model equipped with ancilla degrees of freedom. This construction effectively enables local operators to remove order parameter defects, yielding tighter bounds in phases with spontaneous symmetry breaking. This approach can be applied broadly to pairwise-interacting local lattice models with discrete or continuous internal symmetries that satisfy a property we call defect diamagnetism, which requires that the ground state energy does not decrease upon adding any finite number of symmetry defects. Applying the method to the transverse field Ising models in 1D and 2D, we obtain significantly improved bounds on energy densities and spin correlation functions throughout the symmetry broken phase in 1D and deep within the phase in 2D. Our results demonstrate that physically motivated constraint sets can dramatically enhance the power of bootstrap methods for quantum many-body systems.