On Applicability of Neural Language Models for Readability Assessment in Filipino

Abstract

In the field of automatic readability assessment (ARA), the current trend in the research community focuses on the use of large neural language models such as BERT as evidenced from its high performance in other downstream NLP tasks. In this study, we dissect the BERT model and applied it to readability assessment in a low-resource setting using a dataset in the Filipino language. Results show that extracting embeddings separately from various layers of BERT obtain relatively similar performance with models trained using a diverse set of handcrafted features and substantially better than using conventional transfer learning approach.