Riemannian Local Mechanism for SPD Neural Networks (2023)
Open in web
Open in zotero
Open pdf
1 Abstract
The Symmetric Positive Definite (SPD) matrices have received wide attention for data representation in many scientific areas. Although there are many different attempts to develop effective deep architectures for data processing on the Riemannian manifold of SPD matrices, very few solutions explicitly mine the local geometrical information in deep SPD feature representations. Given the great success of local mechanisms in Euclidean methods, we argue that it is of utmost importance to ensure the preservation of local geometric information in the SPD networks. We first analyse the convolution operator commonly used for capturing local information in Euclidean deep networks from the perspective of a higher level of abstraction afforded by category theory. Based on this analysis, we define the local information in the SPD manifold and design a multi-scale submanifold block for mining local geometry. Experiments involving multiple visual tasks validate the effectiveness of our approach. The supplement and source code can be found in https://github.com/GitZH-Chen/MSNet.git.
2 NOTES
In this paper the authors want to ensure that local properties of SPD matrices are well utilized when used in SPD networks. As can be seen in the following figure, they use a single block of Bimap+ReEig as SPD Backbone and their proposal is what follows: the Multi-scale Sub-manifold Block (MSNet). The Subsec layer does a sub-matrix selection, with the objective of enabling local manifold feature learning. This layer retrieves sub-matrices that are also SPD, but, for instance, if selecting all the principal sub-matrices of size from a SPD matrix, the total number of selected sub-matrices would be . Doing this for is clearly an ‘impossible’ task and certainly inefficient. The authors did not mention, but they must have done some tests, since for the FPHA dataset they had branches of submanifold of size and for SPD matrices obtained from the SPD backbone. The details for the MSNet configuration for each dataset are in the following table
Figure 1.
Table 1: Note that 100,80,50,25 means , , .
Quotes:
The Principles of Selecting Submanifolds (pg.4)
The isomorphic submanifolds in SPD manifolds are the sets of principal submatrices of the same size.
Multi-scale Mechanism (pg.4)
By extracting principal submatrices, we focus on the correlation among multiple local regions. In this way, we can capture the local semantic information in a statistical form.