Strider: A Hybrid Adaptive Distributed RDF Stream Processing Engine

Abstract : Real-time processing of data streams emanating from sensors is becoming a common task in Internet of Things scenarios. The key implementation goal consists in efficiently handling massive incoming data streams and supporting advanced data analytics services like anomaly detection. In an ongoing , industrial project, a 24/7 available stream processing engine usually faces dynamically changing data and workload characteristics. These changes impact the engine's performance and reliability. We propose Strider, a hybrid adaptive distributed RDF Stream Processing engine that optimizes logical query plan according to the state of data streams. Strider has been designed to guarantee important industrial properties such as scalability, high availability, fault tolerance, high throughput and acceptable latency. These guarantees are obtained by designing the engine's architecture with state-of-the-art Apache components such as Spark and Kafka. We highlight the efficiency (e.g., on a single machine machine, up to 60x gain on throughput compared to state-of-the-art systems, a throughput of 3.1 million triples/second on a 9 machines cluster, a major breakthrough in this system's category) of Strider on real-world and synthetic data sets.
Complete list of metadatas

Cited literature [25 references]  Display  Hide  Download

https://hal-upec-upem.archives-ouvertes.fr/hal-01740499
Contributor : Olivier Curé <>
Submitted on : Thursday, March 22, 2018 - 9:23:51 AM
Last modification on : Thursday, July 5, 2018 - 2:45:34 PM
Long-term archiving on : Thursday, September 13, 2018 - 8:38:27 AM

File

iswc-2017-camera.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01740499, version 1

Citation

Xiangnan Ren, Olivier Curé. Strider: A Hybrid Adaptive Distributed RDF Stream Processing Engine. ISWC, Oct 2017, Vienna, Austria. ⟨hal-01740499⟩

Share

Metrics

Record views

51

Files downloads

104