A comparative study on data crawling and extraction of climate change issues using machine learning technique

Abstract

Many different problems are triggered in different fields such as society, culture, and economy due to constant climate change problems. As time goes by, its influences are mounting and national attention is increasing. Therefore, it is necessary to understand various issues and improve policies on climate change. If it is possible to analyze information from media outlet data created on real time by using text mining technique, various climate change issues can be understood. In this comparative study, therefore will collect news article data related to climate change, identify issues utilizing text mining, and see complex information through the detailed analysis considering the characteristics of the text. We crawled news related to climate change issues and analyzed related keywords in terms of cause, result (phenomenon), and response. First, we extracted news related to climate change by using keyword-based document extraction method and Latent Dirichlet Allocation (LDA)-based document extraction method. In addition, we propose four related keyword analysis methods using Word2Vec, which is one of word embedding methods, and keyword frequency based method. Methods proposed in this comparative study are expected to be used in extracting and analyzing data on other specific issues not upcoming climate change issues.