SPDNet

1 SPDNet

SPDNet is the foundational end-to-end Riemannian neural network that maps SPD matrices through dimensionality-reducing bilinear maps and eigenvalue nonlinearities before Euclidean classification.

Ref: @huangRiemannianNetworkSPD2016

Overview

SPDNet receives SPD matrices directly and preserves positive definiteness through its manifold-valued layers. Its principal components are bilinear mapping (BiMap), eigenvalue rectification (ReEig), and eigenvalue logarithm (LogEig), followed by ordinary Euclidean fully connected and softmax layers. BiMap performs manifold-to-manifold dimensionality reduction, ReEig introduces a ReLU-like nonlinearity, and LogEig maps the result into the flat Log-Euclidean domain. The semi-orthogonal BiMap weights live on compact Stiefel manifolds and are optimized with Riemannian matrix backpropagation. The paper evaluates emotion recognition (AFEW), skeleton-based action recognition (HDM05), and video face verification (PaSC).

Architecture

For an input $X_0\in {\mathrm{Sym}}_{d_0}^{+}$, an SPDNet with $i$ BiMap/ReEig blocks has the pipeline

$$X_0\to [\mathrm{BiMap}\to \mathrm{ReEig}{]}^i\to \mathrm{LogEig}\to \mathrm{vec}\to \mathrm{FC}\to \mathrm{softmax}.$$

The reported three-block model is

$$X_0\to f_b^{(1)}\to f_r^{(2)}\to f_b^{(3)}\to f_r^{(4)}\to f_b^{(5)}\to f_l^{(6)}\to f_f^{(7)}\to f_s^{(8)}.$$

BiMap

The BiMap layer applies the congruence transformation

$$X_k=f_b^{(k)}(X_{k-1};W_k)=W_k^{\top }{X}_{k-1}{W}_k,$$

where $X_{k-1}\in {\mathrm{Sym}}_{d_{k-1}}^{+}$ and $W_k\in {\mathbb{R}}^{d_{k-1}\times d_k}$ with $d_k<d_{k-1}$. If $W_k$ has full column rank, then for every $v\ne 0$,

$$v^{\top }{X}_{k}v=(W_{k}v{)}^{\top }{X}_{k-1}(W_{k}v)>0,$$

so BiMap maps ${\mathrm{Sym}}_{d_{k-1}}^{+}$ to ${\mathrm{Sym}}_{d_k}^{+}$. It is both a bilinear dimensionality-reduction layer and a learned manifold-to-manifold metric embedding. The semi-orthogonality constraint $W_k^{\top }{W}_k=I$ places the weights on the compact Stiefel manifold $\mathrm{St}(d_k,d_{k-1})$.

ReEig

The ReEig layer introduces nonlinearity by flooring small eigenvalues. For $X_{k-1}=U_{k-1}{\Sigma }_{k-1}{U}_{k-1}^{\top }$,

$$X_k=f_r^{(k)}(X_{k-1})=U_{k-1}\max (ϵI,{\Sigma }_{k-1})U_{k-1}^{\top },$$

with element-wise flooring to $ϵ$. It moves matrices away from the positive-semidefinite boundary and acts as an SPD analogue of ReLU.

LogEig and Classifier

The LogEig layer maps the final SPD representation into the Log-Euclidean tangent space:

$$X_k=f_l^{(k)}(X_{k-1})=\log (X_{k-1})=U_{k-1}\log ({\Sigma }_{k-1})U_{k-1}^{\top }.$$

The symmetric logarithmic matrix is vectorized and passed to a fully connected layer followed by softmax log loss.

Riemannian Matrix Backpropagation

For a composition $f=f^{(l)}\circ \cdots \circ f^{(1)}$, matrix backpropagation propagates both weight and data derivatives. Under the paper’s semi-orthogonal convention, the Euclidean BiMap gradient is

$${\nabla }_{W_k}{L}^{(k)}=2\frac{\partial L^{(k+1)}}{\partial X_k}{W}_k{X}_{k-1}.$$

Its tangential component and retracted SGD update on the Stiefel manifold are

$${\tilde{\nabla }}_{W_k}{L}^{(k)}={\nabla }_{W_k}{L}^{(k)}-{\nabla }_{W_k}{L}^{(k)}{W}_k^{\top }{W}_k,$$ $$W_k^{t+1}=\Gamma \left(W_k^t-\lambda {\tilde{\nabla }}_{W_k}{L}^{(k),t}\right).$$

For eigendecomposition-based layers, with $X=U\Sigma U^{\top }$,

$$dU=2U\left(P^{\top }\circ (U^{\top }dXU{)}_{\mathrm{sym}}\right),d\Sigma =(U^{\top }dXU{)}_{\mathrm{diag}},P_{ij}=\{\begin{array}{ll}\frac{1}{{\sigma }_i-{\sigma }_j},& i\ne j,\\ 0,& i=j.\end{array}$$

ReEig differentiates with a masking matrix $Q_{ii}=1$ when ${\Sigma }_{ii}>ϵ$ else $0$, and LogEig differentiates through $\log (\Sigma )$ with ${\Sigma }^{-1}$.

Model Parameters

Dataset	Input SPD size	SPDNet-3BiRe progression	Output
AFEW	$400\times 400$	$400\to 200\to 100\to 50$	FC to 7 expressions, softmax
HDM05	$93\times 93$	$93\to 70\to 50\to 30$	FC to 130 actions, softmax
PaSC	$401\times 401$	$401\to 200\to 100\to 50$	FC + softmax for verification

Shared settings: random semi-orthogonal BiMap initialization; ReEig threshold $ϵ=10^{-4}$; BiRe block counts evaluated: 0, 1, 2, 3.

Training Parameters

Setting	AFEW	HDM05	PaSC
Optimizer	SGD on Stiefel manifold (gradient + retraction)	Same	Same
Learning rate	$10^{-2}$, fixed	$10^{-2}$, fixed	$10^{-2}$, fixed
Batch size	30	30	30
Epochs	500	500	100
Loss	Softmax log loss	Softmax log loss	Softmax log loss
Time/epoch	≈2 min	≈4 min	≈15 min
Hardware	i7-2600K 3.40 GHz CPU, no GPU	Same	Same

Data protocols:

• AFEW: frames normalized to $20\times 20$; each video is a $400\times 400$ covariance; training videos segmented into 1,747 clips for augmentation; results on the validation set (test labels unavailable).
• HDM05: covariance of 31-joint 3-D coordinates ($93\times 93$); ten random evaluations, half sequences train / half test; ≈18,000 training clips per evaluation.
• PaSC: deep face features reduced by PCA to 400 dims, fused with the mean into $401\times 401$ SPD matrices; 280 training + 900 COX videos augmented to 12,529 clips; control and handheld protocols evaluated separately.

Results

Accuracy (%). Main comparison:

Method	AFEW	HDM05	PaSC control	PaSC handheld
STM-ExpLet	31.73	-	-	-
RSR-SPDML	30.12	$48.01\pm 3.38$	-	-
CDL	31.81	$41.74\pm 1.92$	78.29	70.41
SPDML-AIM	26.72	$47.25\pm 2.78$	65.47	59.03
SPDNet-0BiRe	26.32	$48.12\pm 3.15$	68.52	63.92
SPDNet-1BiRe	29.12	$55.26\pm 2.37$	71.75	65.81
SPDNet-2BiRe	31.54	$59.13\pm 1.78$	76.23	69.64
SPDNet-3BiRe	34.23	$61.45\pm 1.12$	80.12	72.83

ReEig-threshold ablation on AFEW: $ϵ=10^{-4}$ → 34.23; $ϵ=5\times 10^{-5}$ → 33.15; $ϵ=0$ → 32.35. Removing LogEig drops AFEW to 21.49 and HDM05 to 4.89.

EEG Evaluations in Later Papers

The original paper did not evaluate EEG; the following are third-party runs, not Huang and Van Gool’s experiments.

Source	Protocol	SPDNet result
Ju et al. (2022), MI-KU	Within-session 10-fold CV (S1 / S2); cross-session holdout S1→S2	57.88 (8.68) / 58.88 (8.68) / 60.41 (12.13)
Ju et al. (2022), BCI Competition IV Dataset 2a	Within-session CV (T / E); holdout T→E	65.91 (10.31) / 61.16 (10.50) / 55.67 (9.54)
Peng et al. (2023), IIIa + 2a	2-class, original train/test split; 4-class multi-subject	76.9 ± 17.1 (2-class, pooled); 45.75 ± 17.56 (4-class, pooled)
Carrara and Papadopoulo (2024), Phase-SPDNet, Zhou2016	Binary L/R; within-session 5-fold CV; C3/Cz/C4; 8-32 Hz; AUC	SPDNet 88.85 ± 8.05; coherence variants 62.89 / 75.07

Details per Source

Ju et al. (2022) - SPDNet Baseline on MI-KU and 2a

• Dataset parameters: MI-KU - 54 subjects, 20 selected motor-cortex channels of 62, binary MI, imagery 1–3.5 s, 1,000 Hz, 2 sessions. 2a - 9 subjects, 22 EEG channels, four classes, imagery 0–4 s, 250 Hz, 2 sessions. Trials assumed band-pass filtered (nine causal Chebyshev Type-II filters in 4-Hz bins over 4-40 Hz), centered, and scaled.
• Model parameters: classic SPDNet pipeline - spatial covariance per trial → BiMap → ReEig → LogEig → Euclidean classifier. Exact BiMap dimensions and ReEig threshold for the baseline are not tabulated in the paper (the paper’s own Tensor-CSPNet uses depthwise BiMaps with output dimension o and RBN; the SPDNet baseline uses the standard summing BiMap).
• Training: the paper-wide protocol is cross-entropy, initial lr 0.01 with decay, batch 28, maximum 60 epochs, early-stopping patience 15, Riemannian optimization on the Stiefel manifold (see Tensor-CSPNet note); baseline-specific values not separately reported.
• Results: see the summary table above; the paper notes SPDNet only exploits spatial patterns (no temporal dynamics), which the authors give as the reason it trails the tensor models.

Peng et al. (2023) - SPD-Net Baseline on IIIa and 2a

• Dataset parameters: 250 Hz, 8-30 Hz band-pass, binary left/right-hand MI (2-class) or all four classes (multi-subject). Original train/test splits: IIIa 45 trials/class per set for B1 and 30 for B2/B3; 2a 72 trials/class per set (C1-C9). The 4-class experiment merges all subjects’ signals into one training/testing set (merged-subject, not held-out-subject).
• Model parameters (2-class): five BiMap layers + five ReEig layers, reducing to a 6×6 SPD target: IIIa dims 60×50 → 50×40 → 40×30 → 30×20 → 20×6; 2a dims 22×19 → 19×16 → 16×13 → 13×10 → 10×6. ReEig threshold ε = 10⁻⁴. (The same 5-block structure is used by SPD-Mani-Net, with shrinkage γ = 0.1 replacing ReEig.)
• Model parameters (4-class): two BiMap + two ReEig layers: IIIa 60×40 → 40×10; 2a 22×15 → 15×10; ε = 10⁻⁴.
• Model parameters (toy experiment): three BiMap layers 30×25, 25×20, 20×10 + two ReEig; ε = 10⁻⁴ (shrinkage γ = 0.01 for SPD-Mani-Net).
• Training: SGD with Riemannian gradient + retraction on the Stiefel manifold; lr 10⁻²; batch 50; random semi-orthogonal BiMap initialization. Epoch count not reported. Classification after LogEig vectorization into a Euclidean network (this vectorize-then-Euclidean tail is the paper’s stated reason SPD-Net underperforms on EEG).
• Results: 2-class pooled mean 76.9 ± 17.1; 4-class pooled mean 45.75 ± 17.56 (kappa 0.28 ± 0.24). Both pooled over all 12 IIIa+2a subjects - not dataset-specific.

Carrara and Papadopoulo (2024), Phase-SPDNet - Standard SPDNet Baseline

• Dataset parameters: six MOABB MI datasets (BNCI2014001, BNCI2014004, Cho2017, Schirrmeister2017, Weibo2014, Zhou2016); binary left vs right hand; only 3 electrodes C3/Cz/C4; band-pass 8–32 Hz (overlap-add), artifact rejection, per-channel zero-mean/unit-std standardization, full epoch; within-session 5-fold CV per session; metric ROC-AUC. See Phase-SPDNet note for per-dataset epochs/rates.
• Model parameters: standard SPDNet but with the subspace dimension kept equal to the input dimension (rotation-only BiMap) so that GradCam++ explainability analyses are possible; halving the subspace consistently reduced standard-SPDNet performance. ReEig ε = 10⁻⁴; the used ReEig implementation has no trainable parameters.
• Training: MOABB 1.1.0 common config - 300 epochs, batch 64, validation split 0.1, sparse categorical cross-entropy, Adam lr 10⁻³ (+ RiemannAdam on the Stiefel manifold for BiMap), early stopping patience 75, ReduceLROnPlateau factor 0.5.
• Results: Zhou2016 88.85 ± 8.05 AUC (imag-coherence variant 62.89 ± 8.97; inst-coherence 75.07 ± 7.63). Under the same protocol Phase-SPDNet-OPT reaches 95.62 ± 2.36.

Notes

• Frpm A calibration method for non-positive definite covariance matrix in multivariate data analysis: eigenvalue flooring can be viewed as the projection $P_c(X)=\arg {\min }_{A\in \mathcal{D}}\Vert X-A\Vert$ onto the nearest calibrated positive-definite matrix,

$$P_c(X)=Q\text{diag}(\max ({\lambda }_1,c),\dots ,\max ({\lambda }_n,c))Q^{\top }.$$

This gives an additional justification for ReEig beyond a mere activation heuristic.