A study of deterioration in classification models in real-time big data environment

Abstract

Big Data (BD) is participating in the current computing revolutions. Industries and organizations are utilizing their insights for Business Intelligence (BI). BD and Artificial Intelligence are one of the fundamental pillars of Industrial Revolution (IR) 4.0. IR 4.0 demands real time BD analytic for prediction and classification. Due to complex characteristics of BD (5 V’s), BD analytics is considered a difficult task in offline mood. However, in real time or online mood, BD analytic become more challenging and requires Online Classification Models. In real time mood, the nature of input streams (input data) and target classes (output class) are dependent and non-identically distributed, which cause deterioration in OCM. Therefore, it is necessary to identify and mitigate the causes of this deterioration in OCM and improve OCM performance in RTBDE. This study investigates some fundamental causes of deterioration of Online Classification Models and discusses some possible mitigation approaches. This study also presents some experimental results to show the deterioration in OCM due to real time big data environment. In the future, this study will propose a method to mitigate deterioration in Online Classification Models.