Enhanced Word Embeddings with Sentiment Contextualized Vectors for Sentiment Analysis

Abstract

Deep Learning models generally use pre-trained word embeddings as features to deal with Natural Language Processing problems. However, because most of these word embeddings are context-based, they cannot learn the complex characteristics of words, which may lead to words with similar vector representations expressing different meanings. For sentiment analysis, this represents a problem. Since these word embeddings fail to capture words’ sentimental information, two words with similar vectors can have opposite sentiment polarities, thus degrading the performance of sentiment analysis. This paper addresses this problem and proposes a novel model named Continuous Sentiment Contextualized Vectors (CSCV). CSCV is designed to learn sentiment embeddings based on the Continuous Bag-of-Words (CBOW) model and sentiment lexicons for a given the word from its surrounding context words. The sentiment vectors obtained from this model are combined with existing pre-trained vectors. Experimental results show that: (1) CSCV can be combined with any pre-trained word-embeddings; (2) the nearest neighbors derived from the generated embeddings are more reasonable than other word embeddings because they are both semantically and sentimentally similar; (3) by using this approach, the accuracy can be improved on sentiment classification.