Stream Processing & Apache Flink - News and Best Practices

Highlights from Flink Forward Seattle 2023

Written by Karin Landers | 16 November 2023

Now that Flink Forward Seattle 2023 is over, we’re excited to share that the event was a big (Flinking) success! Flink Forward is *the* conference dedicated entirely to Apache Flink®, and includes speakers, sessions, and attendees from companies like Uber, Apple, Lyft, Stripe, Netflix, and many more.

Let’s take a look at highlights from throughout the week, starting with a quick recap:

Flink Forward At-a-Glance

  • 400+ registrations and attendees from Apple, IBM, Twitter, DoorDash, and more
  • 60+ speakers from companies like AWS, Apple, Stripe, Reddit, Pinterest, INGbank, Wealthsimple, and many more
  • 4 expert keynotes 
  • 45 breakout sessions spanning 5 tracks including operations, tech deep dives, use cases, ecosystem, and an industry track focusing on finance and banking
  • 3 sponsor demos from Gold Sponsors Cloudera, Hazelcast and Decodable 
  • Expo Hall filled with sponsor booths including AWS, IBM, StreamNative, Hopsworks, Nussknacker, and DeltaStream
  • 3 sold-out Apache Flink training sessions attended by 110+ learners
  • 3 staff rescued from a stuck elevator 
  • 0 attendees (or staff!) taking unintended dips in Lake Washington, resulting in a personal $200 donation from Ververica CEO Alexander Walden to the Lake Washington Schools Foundation
  • 150 grounded drones 
  • 0 squirrels harmed 

Big Announcements 

As the proud organizers of Flink Forward, and the original creators of Apache Flink, Ververica kicked off the opening keynote with two major announcements: the new features update of our Ververica Cloud offering and the evolution of Apache Flink into Streamhouse.

Jing Ge, Head of Engineering at Ververica and Apache Flink Committer, took the stage to present Streamhouse, which unifies not only stream and batch processing, but also storage and metadata. This simplifies big data and provides better analytics by allowing one architecture the ability to solve both Batch and Streaming use cases. Ververica Cloud is also the exclusive data platform that supports both real-time streaming and Streamhouse.

Newly released, Ververica Cloud is integrated with Ververica Runtime Assembly (VERA), and boasts speeds up to 2x faster than open source Apache Flink, in a fully managed cloud-native service that easily runs your Flink applications. To explore the most recent updates and features, please visit the Ververica Documentation for detailed information.

TRY VERVERICA CLOUD

Expert Apache Flink Training Courses 

Prior to the conference, learners from around the globe gathered to participate in sold out, two-day, in-person Apache Flink Training Courses, offered by Ververica Academy

Troubleshooting and Performance Tuning for Apache Flink

Led by Ken Krugler, this advanced course was a hands-on introduction to key topics relating to putting Flink applications into production. The deep dive focused on Flink runtime, with an eye towards understanding how Flink works, and what can go wrong. Attendees learned how to avoid problems, and how to diagnose and fix common issues; including backpressure, checkpoint failures, and high latency.

Stream Processing with Flink SQL

Trainer Giannis Polyzos presented his fast-moving, hands-on training course designed for data engineers who are new to Flink SQL and want to learn what Flink offers when developing streaming applications and bringing them to life.

Stream Processing with Apache Flink for Developers

Taught  by Daniel Ciocirlan, this hands-on training course was designed for Java developers who are new to Flink and curious to learn what Flink offers when developing and creating streaming applications.

Did you miss these popular training sessions? Not to worry: Ververica Academy is planning to virtually launch each of these popular paid courses again in 2024. Sign up for the free online Apache Flink classes currently offered, and subscribe to be notified about new course offerings.

Keynotes and Breakout Sessions At-a-Glance

Speakers covered various topics on streaming at scale, handling performance and troubleshooting, leveraging the stability and efficiency available in tuning Flink clusters, and much more. To access all the available recordings, including breakout sessions and keynotes, click here. Subscribe to Flink Forward communications to ensure you are among the first to be notified of all upcoming Flink Forward updates. 

Opening Keynote: Quo Vadis Stream Processing? Dude, where is this streaming to?

Alexander Walden (Ververica CEO) discussed the evolution of Apache Flink, Flink Forward, and the Flink community, and announced the new features available in Ververica Cloud, a fully-managed cloud-native service offered by Ververica.

Technical Keynote: Flink Unveiled - Journeying from DataStream to Streamhouse

As detailed above, Jing Ge, (Ververica) announced Streamhouse and introduced the powerful benefits it offers.

Keynote 3: The Vision and Practice of Stream and Batch Unification - LinkedIn's Story

In his keynote, Becket Qin (LinkedIn, Apache Flink and Apache Kafka PMC member) shared the exploration of stream and batch unification at LinkedIn. This project included a systematic effort demanding changes in computing, storage, control plane as well as toolings, for which they ultimately picked Flink as their unified computing engine.

Keynote 4: Riding the Streams: Powering Stream Processing at Cash App with Ververica

Rashmi Shamsundar and Karel Alfonso Sague (Block) explained how Cash App implements Apache Flink, including utilizing powerful Ververica Platform to solve their stream processing use cases.

Networking & Community at Flink Forward

There’s nothing quite like finally connecting with your new Twitter pal or learning from other data streaming thought leaders in person, and Flink Forward had plenty of opportunities to network and connect!

Don’t just take our word for it, here’s just a little of what attendees had to say about the event:

It was great to meet so many like-minded folks and companies focused on the unification of batch and streaming and data processing, making it simpler and more cost-effective.” - Jove Zhong (Timeplus)

Stream and Batch unification was a hot topic during the event: it was mentioned during the opening keynote several times, and there were a few talks going deep into the unified architecture and even batch (!) improvements.” - Yaroslav Tkachenko (Goldsky)

Leveraging on top of the Spark ecosystem is a pretty clever idea! Better than coming up with something totally new that would require its own infra. Looking forward to seeing more on Streamhouse.

At Flink Forward today, the place is buzzing! Enjoying and looking forward to more good conversations about data streaming and Apache Flink!” - Hong Liang Teoh (Amazon Web Services (AWS))

This conference promises to be an incredible opportunity to connect with core Flink committers, learn from fellow stream processing enthusiasts, and exchange ideas with thought leaders in the field.” -Taruj G (VMware)

Flink Fest

Flink Forward wrapped up with only one mishap: due to technical difficulties, the much-anticipated drone show was unable to take flight… But that didn’t stop attendees from enjoying games with emcee Jeff Houghton, karaoking long into the evening, and continuing to socialize and mingle with fellow attendees, all captured by the photobooth.

Return to Our Roots: Flink Forward Heads to Berlin in 2024!

Seattle offered us (surprisingly) great weather and a beautiful view over Lake Washington, but it’s time to return to where the Apache Flink story began over a decade ago. 

Mark your calendars and save the date, as Flink Forward is leaving the Emerald City and heading back to Berlin, Germany in October 2024

Resources and Next Steps 

The event team recorded all keynotes and breakout sessions, and the videos are available to watch on-demand here

If you’d like to hear more about what people are saying about Ververica Cloud, check out these articles: 

Curious to hear from a Flink Forward 2023 attendee? Read the recent blog from Yaroslav Tkachenko, Principal Software Engineer at Goldsky.

Stay Informed: 

  • Try Ververica Cloud for Free
  • Read the Ververica Docs
  • Join Ververica Academy
  • Subscribe to Flink Forward Communications
  • Watch the Recordings from Flink Forward Seattle 2023

On behalf of the entire event team here at Ververica, we extend our thanks to all the attendees, sponsors, Program Committee members, and the Apache Flink Community for helping make this an event to remember. Until next year, keep Flinking!