SymNet

1 SymNet

SymNet: Simple Symmetric Positive Definite Manifold Network

SymNet is a lightweight SPD-manifold network that replaces Riemannian end-to-end filter optimization with unsupervised $(2\text{-D}{)}^2$ PCA and uses KDA for final discriminant learning.

Ref: @wangSymNetSimpleSymmetric2022

Code at: https://github.com/GitWR/SymNet

Overview

SymNet addresses the large within-class variability of covariance representations used in image-set classification. It combines SPD matrix mapping, a two-part nonlinear rectifier, optional manifold-aware pooling, a Log-Euclidean map, kernel discriminant analysis (KDA), and nearest-neighbor classification. Unlike SPDNet, its mapping filters are learned without labels by $(2\text{-D}{)}^2$ PCA rather than Riemannian matrix backpropagation. It evaluates YTC, ETH-80, Virus, MDSD, AFEW, and FPHA.

The major contributions:

1. SymNet utilizes the simple but efficient (2-D)2 PCA algorithm to conduct unsupervised filter learning, whereas SPDNet exploits the Riemannian matrix backpropagation computing to perform end-to-end training, which is more time consuming than ours.
2. On the tail of the network, SPDNet makes use of the classical fully connected layer to learn Euclidean feature representations. Instead, SymNet utilizes KDA (Kernel Discriminant Analysis) to perform discriminative subspace learning. As a result, training SymNet is very easy.
3. SPDNet introduces the nonlinear mapping scheme by tuning up some small eigenvalues with a ReLU-like function in the designed ReEig layer. However, the proposed SymNet additionally considers the impact of some negative elements of the SPD matrices on the discriminability of the learned features and designs a nonlinear activation function in the rectifying layer to adjust them to desired ones.
4. The conventional pooling operations are generalized to SymNet to further compress the learned SPD matrices. However, SPDNet does not take this into consideration.

Architecture

The two reported compositions are

$$\text{SymNet-v1:}{X}_0\to f_m\to f_r\to f_p\to f_{\log }\to \mathrm{KDA}\to \mathrm{NN},$$ $$\text{SymNet-v2:}{X}_0\to f_m\to f_r\to f_m\to f_r\to f_{\log }\to \mathrm{KDA}\to \mathrm{NN}.$$

SymNet-v1 contains one mapping/rectifying stage and optional pooling. SymNet-v2 contains two mapping/rectifying stages and uses no pooling in the final recommended configuration.

SPD Matrix Mapping

For $X_{k-1}\in {\mathrm{Sym}}_{d_{k-1}}^{+}$, each BiMap-like mapping filter computes

$$X_k^r=f_m(W_k^r,X_{k-1})=(W_k^r{)}^{\top }{X}_{k-1}{W}_k^r,W_k^r\in {\mathbb{R}}^{d_{k-1}\times d_k},$$

where $r=1,\dots ,m_k$ indexes the filters and $d_k\le d_{k-1}$. The output is SPD iff $W_k^r$ has full column rank.

(2-D)² PCA Filter Learning

Each image set $S_i=\{s_i^j{\}}_{j=1}^{n_i}$ is first represented by the covariance

$$X_i=\frac{1}{n_i-1}\sum _{j=1}^{n_i}(s_i^j-{\mu }_i)(s_i^j-{\mu }_i{)}^{\top }.$$

Treating the matrices $X_i$ as basic samples, SymNet computes the row-direction covariance

$$\Phi =\frac{1}{N-1}\sum _{i=1}^N(X_i-\bar{X}{)}^{\top }(X_i-\bar{X}),$$

and solves

$$\underset{M\in {\mathbb{R}}^{d_{k-1}\times P}}{\min }\sum _{i=1}^N{\Vert X_i-M{M}^{\top }{X}_i\Vert }_F^2,M^{\top }M=I_P.$$

The solution consists of the $P$ leading eigenvectors of $\Phi$. With $P=d_k{m}_k$, the leading-eigenvector matrix is divided into $m_k$ nonoverlapping groups of $d_k$ eigenvectors, giving the filters $W_k^r$. The filters are orthonormal, full-rank, and learned without labels or iterative backpropagation.

Two-part Rectifying Layer

The first nonlinearity treats selected negative off-diagonal elements:

$${\Theta }_{ij}=\{\begin{array}{ll}-\eta ,& i\ne j\text{and}{X}_{k-1}(i,j)\in (-\eta ,0],\\ X_{k-1}(i,j),& \text{otherwise}.\end{array}$$

This amplifies small negative correlations toward $-\eta$, with the stated goal of reducing within-class variation.

The second nonlinearity is a ReEig-type eigenvalue floor. Given $\Theta =U\Lambda U^{\top }$,

$$X_k=f_{r2}(\Theta )=U\max (ϵI,\Lambda )U^{\top }.$$

The first operation changes selected matrix entries; the second ensures positive definiteness and supplies a spectral nonlinearity.

Pooling Variants

SymNet’s SPD Matrix Pooling uses a $2\times 2$ patch:

• Tangent-space pooling applies $\exp ({\mathrm{Pool}}_{X\in \mathcal{R}}\log X)$, giving tangent-space max (TMaP) or mean (TMeP) pooling.
• SPD max pooling applies region-wise conventional max pooling directly on SPD matrices: $[P_{\mathcal{R}}{]}_{ij}={\max }_{X\in \mathcal{R}}{X}_{ij}$. The pooling window does not move along geodesics and may distort SPD geometry.
• SPD mean pooling uses the Fréchet mean under the Log-Euclidean metric,

$$P^{\ast }=\arg \underset{P\in {\mathrm{Sym}}_d^{+}}{\min }\sum _{h=1}^H{D}_{\mathrm{LEM}}^2(X_h,P),P^{\ast }=\exp \left(\frac{1}{H}\sum _{h=1}^H\log X_h\right).$$

The paper additionally formulates a lower-dimensional mean-pooling projection $M_k$ optimized by conjugate gradients on a Grassmann manifold.

Log-map and KDA Tail

The LogEig layer computes $X_{\ell }=\log (X_{\ell -1})$; the $m_{\ell }$ symmetric outputs are vectorized and concatenated into $V_i$. KDA maps them into an RKHS and solves

$${\alpha }_{\mathrm{opt}}=\arg \underset{\alpha }{\max }\frac{{\alpha }^{\top }KUK\alpha }{{\alpha }^{\top }KK\alpha },KUK\alpha =\lambda KK\alpha ,$$

retaining the $c-1$ largest generalized eigenvectors. Classification is nearest neighbor in the KDA subspace.

Model Parameters

Dataset	Input	Notes
YTC	$400\times 400$	Complexity example uses $d_m=60$ (v1) and $120\to 38$ (v2)
ETH-80	$400\times 400$	v1: $400\to 20$, 8 filters
Virus	$400\times 400$	-
MDSD	$400\times 400$	v1: $m=4$, $\eta =2.98$, $ϵ=10^{-3}$; v2: ${\eta }_{r1}=6.5$, ${\eta }_{r2}=3.0$
AFEW	$400\times 400$	-
FPHA	$63\times 63$	21 joints × 3-D coordinates

Recommended pooling: v1 uses TMeP on YTC/AFEW and TMaP on ETH-80/Virus/MDSD; v2 uses NP (no pooling) everywhere.

Training Parameters

Component	Setting
Mapping-filter training	Unsupervised $(2\text{-D}{)}^2$ PCA (closed form)
Tail training	KDA generalized eigendecomposition
Classifier	Nearest neighbor
End-to-end optimizer	None (no backpropagation)
Hardware	i7-9700 3.00 GHz CPU, 16 GB RAM (MATLAB 2019a)

Data protocols: YTC 3 train / 6 test videos per subject; ETH-80 five sets per category; Virus 3 train / 2 test sets; MDSD 7 train / 3 test; AFEW 1,746 training clips, validation set evaluation; FPHA 600 train / 575 test. Random selection repeated ten times and averaged for YTC/ETH-80/Virus/MDSD; images resized to $20\times 20$ grayscale ($400\times 400$ covariances).

Results

Experiment	SymNet-v1	SymNet-v2
MDSD without the first entry-wise activation $f_{r1}$	35.26	34.19
MDSD AUC	74.95	73.61
MDSD selected $\eta$	2.98	${\eta }_{r1}=6.5$, ${\eta }_{r2}=3.0$

• SymNet-v1/v2 outperformed SPDNet on ETH-80, YTC, Virus, and MDSD; SPDNet remained better on AFEW and FPHA.
• Pooling ablation: NP best for v2 on all five datasets; SPD mean > SPD max for v1 on ETH-80/Virus/MDSD.
• Component ablation: LogEig significantly improves all datasets; adding $f_{r1}$/‌$f_{r2}$ generally improves discrimination.
• Complexity: ≈$10^8$ operations for SymNet vs ≈$10^{10}$ for SPDNet; training time example 23.36 s vs 1668.06 s (from paper notes).

Notes and Caveats

• No pooling (NP) was the best SymNet-v2 configuration on every pooling-ablation dataset.
• The first rectifier changes selected negative off-diagonal elements before spectral rectification - unlike SPDNet’s ReEig, which changes eigenvalues only.
• Tangent-space pooling may distort local geometry through repeated log/exp mappings; pooling also becomes a bottleneck in deeper designs because second-stage matrices get too small.
• SymNet was never tested on EEG.