Automatic image annotation based on an improved nearest neighbor technique with tag semantic extension model

Abstract

Nearest Neighbor method (KNN) is a typical method to solve the problem of automatic image annotation (AIA). However, traditional AIA methods based on KNN only consider the relationships among images and labels. In this paper, we propose an improved KNN image annotation method based on a tag semantic extension model (TSEM). Our approach uses the convolutional neural network (CNN) to extract image features and predicts image tags automatically via nearest features. Different from existing work, the proposed method considers correlations among images, correlations between images and labels and those among labels. Additionally, a label quantity prediction (LQP) model is proposed to predict the number of tags, which further improves the tag prediction accuracy. Comparison experiments were performed on three typical image datasets Corel5k, ESP game and laprtc12. Experimental results show that the average F1 of our model is 0.427, which outperforms the state-of-the-art KNN image annotation methods.