Real-time Analytics with Apache Flink and Druid
5 likes2,307 views
This document discusses using Apache Flink and Druid for real-time analytics. It describes Druid as an online analytical processing (OLAP) system that is column-oriented, distributed, and uses built-in data sharding based on time windows. It also introduces Tranquility, which helps ingest real-time data into Druid from systems like Kafka, Spark, and Flink. The document proposes a processing architecture using Kafka, Flink, Druid and Tranquility, with HDFS for replays, to enable real-time reporting with capabilities for replays from HDFS and Kafka.
1 of 18
Downloaded 46 times
![DATA STRUCTURES
Column
TOP PRIVATE DOMAIN
battle.net
battle.net
noxxic.com
noxxic.com
Strings to Integers
battle.net 5
noxxic.com 6
Encoded column data
[5, 5, 6, 6]](https://image.slidesharecdn.com/buzzwords-2016-160610201619/85/Real-time-Analytics-with-Apache-Flink-and-Druid-6-320.jpg)
![DATA STRUCTURES
Column Bitmap Indices
battle.net [1, 1, 0, 0]
noxxic.com [0, 0, 1, 1]
TOP PRIVATE DOMAIN
battle.net
battle.net
noxxic.com
noxxic.com](https://image.slidesharecdn.com/buzzwords-2016-160610201619/85/Real-time-Analytics-with-Apache-Flink-and-Druid-7-320.jpg)
More Related Content
What's hot (20)
Ad
Viewers also liked (20)
Ad
Similar to Real-time Analytics with Apache Flink and Druid (20)
Recently uploaded (20)
Real-time Analytics with Apache Flink and Druid
- 1. REAL-TIME ANALYTICS WITH APACHE FLINK AND DRUID Berlin Buzzwords 2016 Jan Graßegger - @gesundkrank
- 4. DRUID
- 5. DRUID ‣ Online Analytical Processing (OLAP) System ‣ Column-oriented ‣ Distributed ‣ Built-in data sharding based on time windows ‣ JSON query language
- 6. DATA STRUCTURES Column TOP PRIVATE DOMAIN battle.net battle.net noxxic.com noxxic.com Strings to Integers battle.net 5 noxxic.com 6 Encoded column data [5, 5, 6, 6]
- 7. DATA STRUCTURES Column Bitmap Indices battle.net [1, 1, 0, 0] noxxic.com [0, 0, 1, 1] TOP PRIVATE DOMAIN battle.net battle.net noxxic.com noxxic.com
- 9. APACHE FLINK
- 11. TRANQUILITY
- 12. TRANQUILITY ‣ Helps ingesting real-time data into Druid ‣ Provides adapters for Samza, Spark, Storm and Flink ‣ Standalone HTTP and Kafka applications
- 14. LAMBDA
- 15. KAPPA
- 16. Kafka Flink Druid Tranquility HDFS for replays PROCESSING
- 17. RESULTS ▸Kappa-like architecture that’s able to do replays from HDFS & Kafka ▸Added Flink sink to Tranquility ▸“Hacked“ replays into Tranquility ▸Real-Time Reporting
- 18. QUESTIONS?