Riemannian batch normalization: A gyro approach (2025)
Open in webOpen in zoteroOpen pdf
1 Abstract
Normalization layers are crucial for deep learning, but their Euclidean formulations are inadequate for data on manifolds. On the other hand, many Riemannian manifolds in machine learning admit gyro-structures, enabling principled extensions of Euclidean neural networks to non-Euclidean domains. Inspired by this, we introduce GyroBN, a principled Riemannian batch normalization framework for gyrogroups. We establish two necessary conditions, namely \emph{pseudo-reduction} and \emph{gyroisometric gyrations}, that guarantee GyroBN with theoretical control over sample statistics, and show that these conditions hold for all known gyrogroups in machine learning. Our framework also incorporates several existing Riemannian normalization methods as special cases. We further instantiate GyroBN on seven representative geometries, including the Grassmannian, five constant curvature spaces, and the correlation manifold, and derive novel gyro and Riemannian structures to enable these instantiations. Experiments across these geometries demonstrate the effectiveness of GyroBN. The code is available at https://github.com/GitZH-Chen/GyroBN.git.
2 NOTES
I’m not gonna pretend I understood the whole paper. This whole Gyro stuff is not something I’m very familiar with. However, the authors seems to construct a Batch Normalization based on Gyrospaces which can be extended for SPD manifolds, Grassmanian Manifolds, and much more. Unfortunately I don’t think this is for me. Good paper thought.
