The application of differential-geometric methods to families of probability distributions, treating parameter spaces as manifolds equipped with information-theoretic metrics such as the Fisher–Rao metric and with dual affine connections induced by statistical divergences.

Principle

Model parametric statistical models as smooth manifolds where the Fisher information defines a Riemannian metric and canonical connections (exponential and mixture) capture dual coordinate systems; geometric quantities (geodesics, curvature) encode properties of estimation and inference.

Demonstration

For the family of univariate normal distributions parameterized by mean μ and variance σ^2, the Fisher–Rao metric yields a two-dimensional Riemannian metric; geodesics represent natural interpolations between distributions and the curvature affects the Cramér–Rao lower bound for unbiased estimators.

Misapplication

Applying naive Euclidean distances on raw parameters (e.g., treating mean and log-variance as orthonormal coordinates) or substituting arbitrary divergences for the Fisher metric without checking local quadratic approximations leads to metrics that misrepresent statistical distinguishability and can bias inference.

Consequence

A correct information-geometric formulation yields coordinate-invariant statements about efficiency and bias of estimators, natural gradient methods for optimization, and quantitative links between curvature and statistical difficulty.

Reversal

Invert the viewpoint by treating the parameter space as purely Euclidean and ignoring information structure; this recovers classical parameter-based analysis but loses invariance properties and the geometric interpretation of information.

Boundary

Applies primarily to smooth parametric families with differentiable likelihoods and identifiable parameters; extensions to infinite-dimensional or singular models require additional functional-analytic machinery and may fail when support changes with parameters.

Semantic Tension

Tension arises between viewing the object as a geometric manifold (focus on metrics, curvature) versus as a probabilistic object (focus on divergences and inferential meaning); different communities emphasize Riemannian versus affine/dually flat structures.

Synthesis

Information geometry unites differential geometry and statistical theory by equipping parameter families with metrics and connections derived from information measures, producing coordinate-free insights into estimation, inference, and optimization.