Benchmarking Concept Drift Detectors for Online Machine Learning

Mahgoub, Mahmoud; Moharram, Hassan; Elkafrawy, Passent; Awad, Ahmed

Benchmarking Concept Drift Detectors for Online Machine Learning

Date

2022

Authors

Publisher

Springer, Cham

Abstract

Concept drift detection is an essential step to maintain the accuracy of online machine learning. The main task is to detect changes in data distribution that might cause changes in the decision bound aries for a classification algorithm. Upon drift detection, the classifica tion algorithm may reset its model or concurrently grow a new learning model. Over the past fifteen years, several drift detection methods have been proposed. Most of these methods have been implemented within the Massive Online Analysis (MOA). Moreover, a couple of studies have compared the drift detectors. However, such studies have merely focused on comparing the detection accuracy. Moreover, most of these studies are focused on synthetic data sets only. Additionally, these studies do not consider drift detectors not integrated into MOA. Furthermore, None of the studies have considered other metrics like resource consumption and runtime characteristics. These metrics are of utmost importance from an operational point of view. In this paper, we fill this gap. Namely, this paper evaluates the perfor mance of sixteen different drift detection methods using three different metrics: accuracy, runtime, and memory usage. To guarantee a fair com parison, MOA is used. Fourteen algorithms are implemented in MOA. We integrate two new algorithms (ADWIN++ and SDDM) into MOA.

Keywords

Online machine learning · Concept drifts · Benchmarking

Citation

Mahgoub, M., Moharram, H., Elkafrawy, P., Awad, A. (2023). Benchmarking Concept Drift Detectors for Online Machine Learning. In: Fournier-Viger, P., Hassan, A., Bellatreche, L. (eds) Model and Data Engineering. MEDI 2022. Lecture Notes in Computer Science, vol 13761. Springer, Cham.