The Scoring Challenge of Emotional Intelligence Ability Tests: A Confirmatory Factor Analysis Approach to Model Substantive and Method Effects Using Raw Item Scores

Abstract

The internal structure of ability emotional intelligence (EI) tests at item level has been hardly studied, and if studied often the predicted structure did not show. In the present study, an a priori model for responses to EI ability items using Likert response scales with a Situational Judgement Test (SJT) format is investigated with confirmatory factor analysis. The model consists of (1) a target EI ability factor, (2) an acquiescence factor, which is a method factor induced by the Likert response scales, and (3) design-based error covariances, which are induced by the SJT format. It is investigated whether this a priori model can account for the observed associations between the raw item responses of the Components of Emotion Understanding Test-24 (CEUT-24). The CEUT-24 is a new test developed to assess emotion understanding, a key aspect of the EI ability construct, based on the componential emotion framework. The sample consisted of 1184 participants (15–22 years old) from four European countries (United Kingdom, Belgium, Germany, and Spain) speaking four different languages (English, Dutch, German and Spanish). Findings showed that the a priori model fitted the data well in all four languages. Furthermore, measurement invariance testing gave evidence for a well-fitting configural, metric, and partial scalar invariance model. The conclusion is that within a regular CFA framework using raw observed items responses, method factors (acquiescence response style and scenario induced variance) can be disentangled from the targeted EI ability factor.