Scalable architecture for real-time streaming analytics

An application for Energy Management & Forecasting

Abstract

The SIDECO project, funded by MIUR has been aimed at the creation of a Big Data Analytics software platform in order to support strategic decisions of institutions, companies, professionals and communities, with the objective to provide innovative data analysis techniques applied to the energy management domain.Within the project, an architecture for real–time monitoring, analysis and correlation of sensor data received from heterogeneous streams has been proposed and implemented. In this paper, a real–time analysis problem and how the architecture (inspired by Lambda Architecture) deals with it, will be introduced. This architecture has been divided into four layers each of which is explained in detail. Finally, we show a case study of an application, implementing the proposed architecture, in the electric energy domain, where the framework layers have been implemented using existing Open Source tools or with ad–hoc Java written applications.