Good performance in supervised text classification is usually obtained with the use of large amounts of labeled training data. However, obtaining labeled data is often expensive and time-consuming. To overcome these limitations, researchers have developed Semi-Supervised learning (SSL) algorithms exploiting the use of unlabeled data, which are generally easy and free to access. With SSL, unlabeled and labeled data are combined to outperform Supervised-Learning algorithms. However, setting up SSL neural networks for text classification is cumbersome and frequently based on a trial and error process. We show that the hyperparameter configuration significantly impacts SSL performance, and the learning rate is the most influential parameter. Additionally, increasing model size also improves SSL performance, particularly when less pre-processing data are available. Interestingly, as opposed to feed-forward models, recurrent models generally reach a performance threshold as pre-processing data size increases. This article expands the knowledge on hyperparameters and model size in relation to SSL application in text classification. This work supports the use of SSL work in future NLP projects by optimizing model design and potentially lowering training time, particularly if time-restricted.
CITATION STYLE
Lien, H., Biermann, D., Palumbo, F., & Goodwin, M. (2022). An Exploration of Semi-supervised Text Classification. In Communications in Computer and Information Science (Vol. 1600 CCIS, pp. 477–488). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-08223-8_39
Mendeley helps you to discover research relevant for your work.