Ververica is proud to host the Flink Forward conferences, uniting Apache Flink® and streaming data communities. Each year we nominate a Program Chair to select a broad range of Program Committee members to evaluate the talk submissions from the community for the Flink Forward program. Program Committee members come from various industries, all with deep knowledge of Apache Flink and streaming data technologies.
We are proud to have Erik de Nooij, Engineering lead Streaming Data Analytics at ING Bank, join us as one of our Program Committee members this year at Flink Forward Berlin 2024. Erik has worked on Apache Flink within ING since its inception in 2016, helping it grow toward the de facto standard for streaming at ING worldwide. With a wealth of experience in Apache Flink, Erik has witnessed firsthand the transformative impact of this powerful streaming data framework. I sat down with Erik to discuss his insights into Apache Flink, the future trends shaping the streaming data landscape, along with his tips for those interested in speaking at Flink Forward Berlin 2024!
Reflecting on your experience, what are some notable milestones or developments you've observed in the adoption of Apache Flink at ING Bank?
Actually, there are two significant moments. The first was in 2016 when we started with Apache Flink. Apache Flink put us on a path of building use cases we could actually put into production. Before using Apache Flink that was not the case and the focus was more on learning and experimenting without having a clear path to production.
The second pivotal moment was when we adopted a SQL-first approach, or perhaps more accurately, a Flink SQL-only approach. By then, we had already invested substantial effort in custom code. Despite doubts among some in the organization about the feasibility of transitioning to Flink SQL, we successfully managed it. This achievement was enabled, in part, by our widespread adoption of User-Defined Functions (UDFs). Despite being custom code, UDFs provided a much lighter solution compared to our previous framework, effectively minimizing our IT footprint.
What motivated you to join the program committee for Flink Forward, and what unique perspective do you bring to the table?
Being part of a community of passionate Flink enthusiasts and having the chance to evaluate abstracts promises a wealth of inspiration and intriguing discussions. With my extensive background in Flink, I believe I can contribute valuable insights and ideas to the mix.
What are some key challenges you believe the Flink community should address in the upcoming year?
One crucial challenge is ensuring compatibility, particularly as we transition to Flink 2.0. Guidance on how savepoints can be migrated would be immensely beneficial to many Flink users. Possibly, providing savepoints with a more defined structure could be part of the solution.
How do you think the conference can better support and engage with Flink users and contributors?
Maybe make it a bit easier to find people who have a common interest on certain topics. E.g. start with a survey up front and have badges with different colors corresponding to your interests.
From your perspective, what makes a talk or workshop proposal compelling for Flink Forward attendees?
Personally, I find talks and workshops most valuable when they address practical real-world problems. Then again, other perspectives on developing and operating streaming platforms are just as valuable.
What excites you the most about the potential impact of Flink Forward on the broader Apache Flink ecosystem?
The fact that it is getting more and more connected to other projects via connectors. Also, Flink is getting more attention on non-Flink conferences as well, which is a good indication of the impact it has.
Can you share any tips for individuals who are interested in presenting at Flink Forward for the first time?
Watch some videos from earlier conferences and you will see that, even though no single presentation is perfect, they don’t have to be: it’s the effort that counts, and the effort is always appreciated by the audience. Don’t worry about your presentation too much, enjoy the experience and keep it authentic.
How do you see the role of industry collaboration in shaping the content and direction of the conference?
The way I see it, the current collaboration within the community is mainly focused on Flink as a product or platform, if you will. Additionally, exploring joint implementations in specific industries could further enrich the conference’s content. For instance, developing real-time apps for scenarios like managing electricity grid fluctuations or providing city tours based on real-time locations could be interesting collaborative projects.
What do you think sets Flink Forward apart from other conferences in the streaming data space?
The long track record of a combination of both a strong product and a strong community.
How do you envision the future of Apache Flink, and what role do events like Flink Forward play in shaping it?
Personally, I believe that streams will be used more and more to make intelligent decisions by continuously feeding (generative) AI models with up-to-date features and/or prompts. These feature updates can be provided by Flink jobs running close to the operational data, potentially following a Data Mesh architecture. These ideas still need to be fleshed out further and events like Flink Forward are instrumental in shaping them.
Let's maintain the tradition of physical gatherings, as they offer invaluable opportunities for collaboration and connection. I hope to meet you all in Berlin this October!
Erik de Nooij
Engineering lead Streaming Data Analytics
ING Bank
Join the Conversation: Call for Papers (CFP) is now open!
Do you have a great streaming data story to share?
Speaking at Flink Forward Berlin 2024 is a great way to connect with hundreds of your peers, and broaden your knowledge on data streaming.
Be quick, Call For Presentation close 11:59 pm (CEST) on May 17th, 2024.
