An adaptive k-NN classifier for medical treatment recommendation under concept drift

Abstract

In the real world, concept drift happens in various scenarios including medical treatment planing. Traditional approaches simply eliminate/dilute the effect of outdated samples on the prediction, leading to a less confident (based on fewer samples) prediction and a waste of undiscovered information contained in past samples. With the knowledge of how concepts change, outdated samples can be adapted for up-to-date prediction, which improves the confidence of prediction, especially for medical data sets of which the scale is relatively small. In this paper we present an adaptive k-NN classifier which can detect the occurrence of target concept drift and update past samples according to the knowledge of the drift for better prediction, and assess the performance over simulated and real-world categorical medical data sets. The experiment results show our classifier achieves better performance under concept drift.