Deep local descriptors with domain adaptation

Abstract

Due to the different distributions of training and testing datasets, the performance of the trained model based on the training set can rarely achieve the most optimal. Inspired by the successful application of domain adaptation in the object recognition area, we apply domain adaptation methods to CNN based local feature descriptors based on their own traits. Different from previous domain adaptation methods that focus only on the fully connected layer, we apply maximum mean discrepancy (MMD) criterion to both the fully connected layer and the convolutional layer, which makes the primary local filters of CNN adaptive to the target dataset in an unsupervised manner. Extensive experiments on Photo Tour and HPatches dataset show that domain adaption is effective to local feature descriptors, and, more importantly, the convolutional layer adaption can further improve the performance of traditional domain adaptation.