EMP-SA: Ensemble model based market prediction using sentiment analysis

Abstract

Predicting stock market trend is an extremely complicated task and calls for extensive study and insights into the context at hand. Primary requirement for any investor is to assess this trend to help invest for maximizing his returns. The advances in Machine learning and data analytics in particular have changed the way investors can approach this matter. Sentiment analysis or Opinion mining can be carried out by taking into consideration public sentiments regarding the stock market conditions and to understand the ups and down of this most volatile sector. In this paper, public sentiments from Twitter along with news feed related to the stock market conditions for predicting the nature of market is considered to analyse the stock market trend. The data is collected from twitter and various news sites to generate a gross sentiment score regarding the market. The gross sentiment score is used to find a correlation between market price and sentiments to train the proposed models for prediction using Linear and robustness regression techniques such as Ordinary Least squares (OLS), RANSAC, Theil-Sen estimator, Huber Regression and Ridge regression. Ensemble method is used to achieve reliable and better prediction accuracy instead of a single method. Ensemble method combines models and carries out majority voting among them to produce one final model to increase prediction accuracy. The obtained results reveal that public opinion does make a significant impact on market behaviour with the prediction accuracy between 65-91% depending on the dataset.