Modern Data Engineering for Real-Time Intelligence

Recent advances in technology and the ongoing push toward digital transformation have strongly affected the engineering of data- and intelligence-centric systems. The deployment of Cloud infrastructures and technologies has introduced elasticity and scalability capabilities that can significantly lower the price of computation for batch analysis. At the same time, the increasing availability of inexpensive sensors and the improvement of cheap networking technologies have opened the path toward the Internet of Things (IoT) and the connected world. For years, systems were built mainly around these two elements—the vertical and horizontal scalability of Cloud infrastructures, and the data generated by the connected world. These systems were capable of gathering information from multiple sources into huge centralized systems, where the data could be processed in larger volumes.