SPD MLR

1 SPD MLR

Ref: @chenRiemannianMultinomialLogistics

Code at: https://github.com/GitZH-Chen/SPDMLR.git

Overview

The paper develops Riemannian multinomial logistic regression (RMLR) for SPD manifolds under pullback Euclidean metrics (PEMs). It instantiates the framework with parameterized Log-Euclidean ($(\alpha ,\beta )$-LEM) and Log-Cholesky (power-deformed $(\theta )$-LCM) metrics and evaluates the classifier with SPDNet on Radar and HDM05 and with TSMNet+SPDDSMBN on Hinss2021.

The central change is architectural: retain SPD-valued representations after BiMap and ReEig, then replace the conventional LogEig + FC + softmax classifier with an intrinsic SPD MLR.

Architecture

For an SPD input $S$, class $k$ is represented by an SPD shift matrix $P_k$ and nonzero tangent normal matrix $A_k$. The corresponding SPD hyperplane is

$${\tilde{H}}_{A_k,P_k}=\left\{S\in {\mathcal{S}}_{++}^n:{⟨{\mathrm{Log}}_{P_k}(S),A_k⟩}_{P_k}=0\right\}.$$

The general Riemannian classifier is defined through signed geodesic distance to this hyperplane. For a PEM induced by a diffeomorphism $\varphi :{\mathcal{S}}_{++}^n\to {\mathcal{S}}^n$, the distance has a closed form and the classifier simplifies to

$$p(y=k\mid S)\propto \exp \left(⟨\varphi (S)-\varphi (P_k),{\varphi }_{\ast ,I}({\tilde{A}}_k)⟩\right),$$

where ${\tilde{A}}_k\in {\mathcal{S}}^n$ is a Euclidean symmetric parameter and $P_k\in {\mathcal{S}}_{++}^n$ is optimized on the SPD manifold.

Under $(\alpha ,\beta )$-LEM,

$$p(y=k\mid S)\propto \exp \left[{⟨\mathrm{mlog}(S)-\mathrm{mlog}(P_k),{\tilde{A}}_k⟩}^{(\alpha ,\beta )}\right],⟨V,W{⟩}^{(\alpha ,\beta )}=\alpha ⟨V,W⟩+\beta \mathrm{tr}(V)\mathrm{tr}(W).$$

Under power-deformed $(\theta )$-LCM,

$$p(y=k\mid S)\propto \exp \left[\frac{1}{\theta }⟨X,Y⟩\right],$$

where $X$ depends on the strictly lower and log-diagonal components of $\mathrm{Chol}(S^{\theta })$ and $\mathrm{Chol}(P_k^{\theta })$, and $Y=\lfloor {\tilde{A}}_k\rfloor +\frac{1}{2}D({\tilde{A}}_k)$. The power parameter interpolates standard LCM at $\theta =1$ toward an LEM-like metric as $\theta \to 0$.

For standard LEM, an SPD MLR optimized with LEM-based RSGD is proven equivalent to the usual matrix-logarithm + FC + softmax classifier optimized with Euclidean SGD. The experiments instead update SPD parameters using an AIM-based optimizer.

The two evaluated backbones:

• SPDNet: alternating BiMap and ReEig blocks, followed by SPD MLR instead of LogEig + FC + softmax.
• TSMNet+SPDDSMBN: temporal convolution → spatial convolution → BiMap → ReEig → SPDDSMBN → SPD MLR.

Model Parameters

Architectures are represented by $[d_0,d_1,\dots ,d_L]$, with the $i$th BiMap parameter having dimensions $d_i\times d_{i-1}$:

Dataset	Architectures
Radar	$[20,16,8]$; $[20,16,14,12,10,8]$
HDM05	$[93,30]$; $[93,70,30]$; $[93,70,50,30]$
Hinss2021	$[40,20]$

We adopt the deepest architectures, namely [20, 16, 14, 12, 10, 8] for the Radar dataset, [93, 70, 50, 30] for the HDM05 dataset, and [40, 20] for the Hinss2021 dataset.

Metric hyperparameters: $\alpha =1$; $\beta \in \{1,1/n,1/n^2,0,-1/n+ϵ,-1/n^2\}$; LCM deformation $\theta \in \{0.5,1,1.5\}$. Reported: Radar $(1,1)$-LEM and $(0.5)$-LCM; HDM05 $(0.5)$-LCM; Hinss2021 $(1.5)$-LCM. Each class introduces one SPD shift parameter $P_k$ and one symmetric normal parameter ${\tilde{A}}_k$.

Training Parameters

All experiments use cross-entropy and the Riemannian AMSGrad optimizer:

Setting	Radar / HDM05	Hinss2021
Learning rate	$10^{-2}$	$10^{-3}$
Batch size	30	50
Maximum epochs	200	50
Weight decay	not reported	$10^{-4}$

Hardware: Intel Core i9-7960X CPU, 32 GB RAM, NVIDIA GeForce RTX 2080 Ti.

Data: Radar - 3,000 synthetic radar signals, windows of length 20 → 3,000 $20\times 20$ covariances, 3 classes. HDM05 - 2,086 $93\times 93$ covariance instances, 117 classes. Hinss2021 - MOABB/MNE preprocessing: resample 250/256 Hz, 4-36 Hz filter, ≤3 s segments, $40\times 40$ covariances; inter-session and inter-subject. Radar/HDM05: tenfold experiments; Hinss2021: random 5% of sessions/subjects left out. Metric: accuracy (Radar/HDM05), balanced accuracy (Hinss2021).

Results

Radar accuracy (mean ± SD over ten folds):

Classifier	$[20,16,8]$	$[20,16,14,12,10,8]$
LogEig MLR	92.88 ± 1.05	93.47 ± 0.45
Gyro-AIM	94.53 ± 0.95	94.32 ± 0.94
$(1,0)$-LEM	93.55 ± 1.21	94.60 ± 0.70
$(1,1)$-LEM	95.64 ± 0.83	95.87 ± 0.58
$(0.5)$-LCM	94.59 ± 0.82	95.16 ± 0.67

HDM05 accuracy (mean ± SD):

Classifier	$[93,30]$	$[93,70,30]$	$[93,70,50,30]$
LogEig MLR	57.42 ± 1.31	60.69 ± 0.66	60.76 ± 0.80
$(0.5)$-LCM	65.66 ± 0.73	65.79 ± 0.63	65.71 ± 0.75

Hinss2021 balanced accuracy (mean ± SD):

Classifier	Inter-session	Inter-subject
LogEig MLR	53.83 ± 9.77	49.68 ± 7.88
$(1.5)$-LCM	56.43 ± 8.79	51.65 ± 5.90

Deformed LCM improves over LogEig MLR by 2.60 pp inter-session and 1.97 pp inter-subject. Training time (s/epoch, deepest architecture): LCM MLR is far cheaper than Gyro-AIM (e.g., HDM05 3.29 vs 31.64; on HDM05 with 117 class parameters, LEM/LCM need ≈1/9 of Gyro-AIM’s time).