Towards a Federated Learning Approach for NLP Applications

Abstract

Traditional machine learning involves the collection of training data to a centralized location. This collected data is prone to misuse and data breach. Federated learning is a promising solution for reducing the possibility of misusing sensitive user data in machine learning systems. In recent years, there has been an increase in the adoption of federated learning in healthcare applications. On the other hand, personal data such as text messages and emails also contain highly sensitive data, typically used in natural language processing (NLP) applications. In this paper, we investigate the adoption of federated learning approach in the domain of NLP requiring sensitive data. For this purpose, we have developed a federated learning infrastructure that performs training on remote devices without the need to share data. We demonstrate the usability of this infrastructure for NLP by focusing on sentiment analysis. The results show that the federated learning approach trained a model with comparable test accuracy to the centralized approach. Therefore, federated learning is a viable alternative for developing NLP models to preserve the privacy of data.