Anomaly Detection in BI Smart Metrics

The goal of this project was to develop an anomaly detection pipeline for business intelligence systems in order to trigger a BI report distribution - via slack or e-mail - when an anomalous situation was detected or send an alert to predefined groups of managers.

The system was designed to collect data online from various BI reporting systems (e.g., Tableau ). The data was later modeled as time series and stored in InfluxDB, following data was sent to a stream processing engine called Kapacitor. Kapacitor was bridged with user-defined methods responsible for indicating values that could be considered suspicious or anomalous, using both statistical methods and machine learning models. When the user-defined functions detected anomalous value Kapacitor was triggering a REST API calls to a back-end engine responsible for the final report distribution and alerting. Historical data stored in the time series database (InfluxDB) was also used to periodically compute the aforementioned statistics and train machine learning models.