Big Stream Processing Systems: An Experimental Evaluation

Date
2019
Journal Title
Journal ISSN
Volume Title
Publisher
IEEE computer society
Abstract
As the world gets more instrumented and connected, we are witnessing a flood of digital data generated from various hardware (e.g., sensors) or software in the format of flowing streams of data. Real-time processing for such massive amounts of streaming data is a crucial requirement in several application domains including financial markets, surveillance systems, man ufacturing, smart cities, and scalable monitoring infrastructure. In the last few years, several big stream processing engines have been introduced to tackle this challenge. In this article, we present an extensive experimental study of five popular systems in this domain, namely, Apache Storm, Apache Flink, Apache Spark, Kafka Streams and Hazelcast Jet. We report and analyze the performance characteristics of these systems. In addition, we report a set of insights and important lessons that we have learned from conducting our experiments.
Description
Keywords
Big Stream Processing, Benchmarking
Citation
Sakr, S. et al. (2019) “Big Stream Processing Systems: An Experimental Evaluation,” in 2019 IEEE 35th International Conference on Data Engineering Workshops (ICDEW), pp. 53–60. .
Collections