SlideShare a Scribd company logo
Robust
Stream Processing
with
Apache Flink
Jamie Grier
@jamiegrier
jamie@data-artisans.com
Who am I?
‱ Director of Applications Engineering at data
Artisans
‱ Previously working on streaming computation at
Twitter, Gnip and Boulder Imaging
‱ Involved in various kinds of stream processing for
about a decade
‱ High-speed video, social media streaming, general
frameworks for stream processing
Overview
‱ What is Apache Flink?
‱ What is Stateful Stream Processing?
‱ Windowed computation over streams
‱ Robust Time Handling (Event Time vs Processing Time)
‱ Robust Failure Handling
‱ Robust Planned Downtime Handling
‱ Robust Reprocessing
What is
Apache Flink?
s an open source platform for distributed stream and batch da
What is
Apache Flink?
Stream Processing
Your
Code
Data Stream Data Stream
Stateful
Stream Processing
Your
Code
Data Stream Data Stream
State
More Complex
Example
Kafka
Files
Rabb
itMQ
Filter
Map
Join /
Sum
Influx
DB
C*
Distributed and Parallel
Deployment
Kafka
Files
Rabb
it
MQ
Filter
Pars
e
Join /
Sum
Influx
DB
C*
Robust Stream Processing
with Apache Flink
Code Example!
Windowing
Processing Time
vs
Event Time
Windowing in Processing
Time
0 1 2 34 56 7 8 9 0 1 2 3 4 5 6 7 8 9
Processing Time
Event Time
Windowing in Event
Time
0 1 2 34 56 7 8 9 0 1 2 3 4 5 6 7 8 9
Event Time
Processing Time = Errors!
Event Time = Accuracy
Failure Handling
Downtime Handling
Data Reprocessing
We’re Hiring!
http://data-artisans.com/careers
Flink Forward 2016, Berlin
Submission deadline: June 30, 2016
Early bird deadline: July 15, 2016
www.flink-forward.org
Questions?
Thanks!

More Related Content

What's hot (20)

PPTX
data Artisans Product Announcement
Flink Forward
 
PDF
Big Data Warsaw
Maximilian Michels
 
PDF
Kafka Summit NYC 2017 - The Source of Truth: Why the New York Times Stores Ev...
confluent
 
PDF
Javier Lopez_Mihail Vieru - Flink in Zalando's World of Microservices - Flink...
Flink Forward
 
PDF
Stream Processing with Apache Flink
C4Media
 
PPTX
Flink. Pure Streaming
Indizen Technologies
 
PDF
Stream processing with Apache Flink @ OfferUp
Bowen Li
 
PDF
A look at Flink 1.2
Stefan Richter
 
PPTX
Portable Streaming Pipelines with Apache Beam
confluent
 
PDF
Flink Forward Berlin 2017: Zohar Mizrahi - Python Streaming API
Flink Forward
 
PPTX
Flink Forward Berlin 2017: Till Rohrmann - From Apache Flink 1.3 to 1.4
Flink Forward
 
PDF
Deploying Confluent Platform for Production
confluent
 
PDF
Tips & Tricks for Apache KafkaÂź
confluent
 
PDF
Flink Forward San Francisco 2019: Apache Beam portability in the times of rea...
Flink Forward
 
PDF
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
Flink Forward
 
PDF
Flink Forward San Francisco 2019: Elastic Data Processing with Apache Flink a...
Flink Forward
 
PPTX
Berlin Apache Flink Meetup May 2015, Community Update
Robert Metzger
 
PDF
Using Spark at Vungle
Vungle
 
PPTX
RedisConf17 - Pain-free Pipelining
Redis Labs
 
PDF
Introduction to Apache Flink
datamantra
 
data Artisans Product Announcement
Flink Forward
 
Big Data Warsaw
Maximilian Michels
 
Kafka Summit NYC 2017 - The Source of Truth: Why the New York Times Stores Ev...
confluent
 
Javier Lopez_Mihail Vieru - Flink in Zalando's World of Microservices - Flink...
Flink Forward
 
Stream Processing with Apache Flink
C4Media
 
Flink. Pure Streaming
Indizen Technologies
 
Stream processing with Apache Flink @ OfferUp
Bowen Li
 
A look at Flink 1.2
Stefan Richter
 
Portable Streaming Pipelines with Apache Beam
confluent
 
Flink Forward Berlin 2017: Zohar Mizrahi - Python Streaming API
Flink Forward
 
Flink Forward Berlin 2017: Till Rohrmann - From Apache Flink 1.3 to 1.4
Flink Forward
 
Deploying Confluent Platform for Production
confluent
 
Tips & Tricks for Apache KafkaÂź
confluent
 
Flink Forward San Francisco 2019: Apache Beam portability in the times of rea...
Flink Forward
 
Virtual Flink Forward 2020: Production-Ready Flink and Hive Integration - wha...
Flink Forward
 
Flink Forward San Francisco 2019: Elastic Data Processing with Apache Flink a...
Flink Forward
 
Berlin Apache Flink Meetup May 2015, Community Update
Robert Metzger
 
Using Spark at Vungle
Vungle
 
RedisConf17 - Pain-free Pipelining
Redis Labs
 
Introduction to Apache Flink
datamantra
 

Similar to Robust Stream Processing with Apache Flink (20)

PDF
Jamie Grier - Robust Stream Processing with Apache Flink
Flink Forward
 
PPTX
Robust stream processing with Apache Flink
Aljoscha Krettek
 
PPTX
Flink System Overview
Timo Walther
 
PDF
Zurich Flink Meetup
Konstantinos Kloudas
 
PDF
Introduction to Stateful Stream Processing with Apache Flink.
Konstantinos Kloudas
 
PPTX
Data Stream Processing with Apache Flink
Fabian Hueske
 
PDF
Apache Flink: Better, Faster & Uncut - Piotr Nowojski, data Artisans
Evention
 
PDF
Apache flink
pranay kumar
 
PDF
Unified Stream and Batch Processing with Apache Flink
DataWorks Summit/Hadoop Summit
 
PPTX
Stream processing for the practitioner: Blueprints for common stream processi...
Aljoscha Krettek
 
PPTX
Apache Flink(tm) - A Next-Generation Stream Processor
Aljoscha Krettek
 
PPTX
2018-04 Kafka Summit London: Stephan Ewen - "Apache Flink and Apache Kafka fo...
Ververica
 
PDF
Large scale stream processing with Apache Flink
Nikolay Stoitsev
 
PPTX
Flink Forward Berlin 2018: Aljoscha Krettek & Till Rohrmann - Keynote: "A Yea...
Flink Forward
 
PPTX
The Past, Present, and Future of Apache Flink
Aljoscha Krettek
 
PDF
Making Sense of Apache Flink: A Fearless Introduction
HostedbyConfluent
 
PDF
Stream Processing with Apache Flink (Flink.tw Meetup 2016/07/19)
Apache Flink Taiwan User Group
 
PPTX
Apache Flink - A Stream Processing Engine
Aljoscha Krettek
 
PPTX
Flexible and Real-Time Stream Processing with Apache Flink
DataWorks Summit
 
PDF
Stateful stream processing with Apache Flink
Knoldus Inc.
 
Jamie Grier - Robust Stream Processing with Apache Flink
Flink Forward
 
Robust stream processing with Apache Flink
Aljoscha Krettek
 
Flink System Overview
Timo Walther
 
Zurich Flink Meetup
Konstantinos Kloudas
 
Introduction to Stateful Stream Processing with Apache Flink.
Konstantinos Kloudas
 
Data Stream Processing with Apache Flink
Fabian Hueske
 
Apache Flink: Better, Faster & Uncut - Piotr Nowojski, data Artisans
Evention
 
Apache flink
pranay kumar
 
Unified Stream and Batch Processing with Apache Flink
DataWorks Summit/Hadoop Summit
 
Stream processing for the practitioner: Blueprints for common stream processi...
Aljoscha Krettek
 
Apache Flink(tm) - A Next-Generation Stream Processor
Aljoscha Krettek
 
2018-04 Kafka Summit London: Stephan Ewen - "Apache Flink and Apache Kafka fo...
Ververica
 
Large scale stream processing with Apache Flink
Nikolay Stoitsev
 
Flink Forward Berlin 2018: Aljoscha Krettek & Till Rohrmann - Keynote: "A Yea...
Flink Forward
 
The Past, Present, and Future of Apache Flink
Aljoscha Krettek
 
Making Sense of Apache Flink: A Fearless Introduction
HostedbyConfluent
 
Stream Processing with Apache Flink (Flink.tw Meetup 2016/07/19)
Apache Flink Taiwan User Group
 
Apache Flink - A Stream Processing Engine
Aljoscha Krettek
 
Flexible and Real-Time Stream Processing with Apache Flink
DataWorks Summit
 
Stateful stream processing with Apache Flink
Knoldus Inc.
 
Ad

Recently uploaded (20)

PDF
Capcut Pro Crack For PC Latest Version {Fully Unlocked} 2025
hashhshs786
 
PDF
HiHelloHR – Simplify HR Operations for Modern Workplaces
HiHelloHR
 
PDF
Mobile CMMS Solutions Empowering the Frontline Workforce
CryotosCMMSSoftware
 
PDF
Salesforce CRM Services.VALiNTRY360
VALiNTRY360
 
PPTX
Writing Better Code - Helping Developers make Decisions.pptx
Lorraine Steyn
 
PPTX
An Introduction to ZAP by Checkmarx - Official Version
Simon Bennetts
 
PDF
vMix Pro 28.0.0.42 Download vMix Registration key Bundle
kulindacore
 
PPTX
Feb 2021 Cohesity first pitch presentation.pptx
enginsayin1
 
PDF
Build It, Buy It, or Already Got It? Make Smarter Martech Decisions
bbedford2
 
PPTX
Tally_Basic_Operations_Presentation.pptx
AditiBansal54083
 
PPTX
Platform for Enterprise Solution - Java EE5
abhishekoza1981
 
PDF
Why Businesses Are Switching to Open Source Alternatives to Crystal Reports.pdf
Varsha Nayak
 
PPTX
Engineering the Java Web Application (MVC)
abhishekoza1981
 
PDF
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
PDF
Unlock Efficiency with Insurance Policy Administration Systems
Insurance Tech Services
 
PDF
Alarm in Android-Scheduling Timed Tasks Using AlarmManager in Android.pdf
Nabin Dhakal
 
PDF
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
PDF
GetOnCRM Speeds Up Agentforce 3 Deployment for Enterprise AI Wins.pdf
GetOnCRM Solutions
 
DOCX
Import Data Form Excel to Tally Services
Tally xperts
 
PPTX
Why Businesses Are Switching to Open Source Alternatives to Crystal Reports.pptx
Varsha Nayak
 
Capcut Pro Crack For PC Latest Version {Fully Unlocked} 2025
hashhshs786
 
HiHelloHR – Simplify HR Operations for Modern Workplaces
HiHelloHR
 
Mobile CMMS Solutions Empowering the Frontline Workforce
CryotosCMMSSoftware
 
Salesforce CRM Services.VALiNTRY360
VALiNTRY360
 
Writing Better Code - Helping Developers make Decisions.pptx
Lorraine Steyn
 
An Introduction to ZAP by Checkmarx - Official Version
Simon Bennetts
 
vMix Pro 28.0.0.42 Download vMix Registration key Bundle
kulindacore
 
Feb 2021 Cohesity first pitch presentation.pptx
enginsayin1
 
Build It, Buy It, or Already Got It? Make Smarter Martech Decisions
bbedford2
 
Tally_Basic_Operations_Presentation.pptx
AditiBansal54083
 
Platform for Enterprise Solution - Java EE5
abhishekoza1981
 
Why Businesses Are Switching to Open Source Alternatives to Crystal Reports.pdf
Varsha Nayak
 
Engineering the Java Web Application (MVC)
abhishekoza1981
 
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
Unlock Efficiency with Insurance Policy Administration Systems
Insurance Tech Services
 
Alarm in Android-Scheduling Timed Tasks Using AlarmManager in Android.pdf
Nabin Dhakal
 
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
GetOnCRM Speeds Up Agentforce 3 Deployment for Enterprise AI Wins.pdf
GetOnCRM Solutions
 
Import Data Form Excel to Tally Services
Tally xperts
 
Why Businesses Are Switching to Open Source Alternatives to Crystal Reports.pptx
Varsha Nayak
 
Ad

Robust Stream Processing with Apache Flink