Incremental Teacher Model with Mixed Augmentations and Scheduled Pseudo-label Loss for Handwritten Text Recognition

Abstract

We propose a training framework for deep neural network-based handwritten text recognizers using both labeled and unlabeled data. The proposed framework is a semi-supervised learning (SSL) framework based on Mixed Augmentations and Scheduled Pseudo-Label loss. Mixed Augmentations provide weakly and strongly transformed variants from each original sample so that the pseudo-label loss is computed between these two variants. The Scheduled Pseudo-Label loss is used to gradually include the pseudo-label loss into the optimizer to avoid the negative effect of incorrect pseudo labels. First, a student model is pre-trained by labeled samples and used to initiate a teacher model. Subsequently, the teacher model predicts a pseudo label from every weakly transformed variant. On the other hand, the student model is trained using the Scheduled Pseudo-Label loss. Next, the teacher model is incrementally updated using the student model. Finally, it is used to evaluate. We term the framework Incremental Teacher Model. The proposed framework was applied to four architectures of distinct handwriting recognizers. For almost every architecture, the recognizer trained by our method outperforms those trained by well-known SSL methods, namely Mean Teacher, Pseudo-Labeling, and FixMatch, evaluated using different ratios of labeled training samples on the IAM handwriting database.