Domain-Adversarial Training of Neural Networks (2017)
Open in webOpen in zoteroOpen pdf
1 Abstract
We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains.
2 NOTES
This is the classical paper on domain adaption on neural networks. It is interesting to see in the paper how much of mathematical proof is behind the model, something that has not been very common recently. The two main aspects of the model are: a second ‘branch’ that is devoted to try to identify the domain of the input (and maybe works as a GAN I guess) and the gradient reversal layer (GRL). The domain identification minimizes a loss based on the correctly distinction from source to target domain data. However, due to the GRL, the feature extraction effectively maximizes this loss, encouraging it to produce features that are indistinguishable across domain. Amazing paper.
