Apache Flink

Flink Community Update

June 2015

Marton Balassi

mbalassi@apache.org / @MartonBalassi

Apache Flink is an open source platform for

scalable batch and stream data processing.

Apache Flink is …

flink.apache.org 1

• The core of Flink is a distributed

streaming dataflow engine.

• Executing dataflows in parallel on

clusters

• Providing a reliable foundation for

various workloads

• DataSet and DataStream

programming abstractions are the

foundation for user programs and

higher layers

One engine for many use cases

flink.apache.org 2

Real time streaming

topologies

Machine Learning at scale

Graph Analysis

Long batch

pipelines

What happened?

• Hadoop Summit San Jose

• Release process for 0.9.0

• Added two new committers

• Blogpost with Bouyges Telecom on

streaming pipelines with Kafka

• Two hands-on workshops in Berlin

• Multiple conference and meetup talks

flink.apache.org 3

Hadoop Summit San Jose

flink.apache.org 4

Robert Metzger

Apache Flink deep dive

Kostas Tzoumas

Stream processing with

Apache Flink

Hadoop Summit San Jose

flink.apache.org 5

Merv Adrian, Gartner analyst telling the story

of Hadoop Summit San Jose in a nutshell

to his 20K+ followers.

Release 0.9.0

flink.apache.org 6

• Release vote passed today :)

• 4 months of work of the community

• Delivers 1200+ commits

• Highlights:

– Reworked distributed runtime in Akka

– Pipelined/Batch execution modes

– Exactly once semantics for streaming

– First full release for many libraries: FlinkML, Gelly, Table API

Welcome the new committers

flink.apache.org 7

Andra Lungu

is a master student at TU Berlin,

currently writing her thesis on Scaling up

Computations on Highly Skewed Graphs.

Andra is a prominent contributor to Gelly.

Chiwan Park

is a Korean programmer interested in

large scale data processing, stream data

processing and scalable machine

learning.

Blogpost with Bouyges Telecom

on streaming

flink.apache.org 8

http://data-artisans.com/flink-at-bouygues.html

Full-service communication operator with a revenue of 5+ billion Euros

using Flink Streaming in pre-production.

Flink workshops in Berlin

flink.apache.org 9

@ Google Berlin office @ Zalando Berlin office

With a combined attendance of ~100 people, there is need for more.

GitHub stats

flink.apache.org 10

A month of traveling and release testing for many.

Still reasonable amount of code updates. :)

Conferences

flink.apache.org 11

Berlin BuzzwordsBudapest Data Forum

• Till Rohrmann

Computing recommendations with

Apache Flink

• Stephan Ewen

Apache Flink deep dive

Meetups

flink.apache.org 12

Flink Meetup Map :)Stephan Ewen @ Bay Area

• SF Spark and Friends

• Bay Area Flink Meetup

• Chicago Flink Meetup

• Stockholm Flink Meetup

• Berlin Flink Meetup

Media Coverage

flink.apache.org 13

Datanami names Flink the #2 Big Data project to watch

http://www.datanami.com/2015/06/12/8-new-big-data-projects-to-watch/

Flink Forward registration & call for

abstracts is open now

flink.apache.org 14

• 12/13 October 2015

• Kulturbrauerei Berlin

• With Flink Workshops / Trainings!