Stream Processing & Apache Flink - News and Best Practices

Announcing Ververica Platform: Self-Managed 2.14

Written by Karin Landers and Dawn Leamon | 26 November 2024

Today, the Ververica team introduces the latest advancements in our Unified Streaming Data Platform with the release of Ververica Platform: Self-Managed, 2.14

This release focuses on empowering developers and operators with enhanced Kubernetes capabilities, streamlined operational workflows, and improved validation processes.

Ververica Platform 2.14 supports Apache Flink® 1.19, Apache Flink 1.18, and Apache Flink 1.17 under SLA.

Explore all the updates and enhancements in the 2.14 Release Notes and discover how the Unified Streaming Data Platform can elevate your streaming data solutions.

Top Highlights in this Release 

You can now:

  • Catch schema errors early in development and streamline your workflows with OpenAPIV3Schema support in Custom Resource Definitions (CRDs).
  • Define meaningful Kubernetes resource names, making it easier to manage, monitor, and troubleshoot your deployments, even in complex environments.
  • Restart deployments without configuration changes and trigger manual savepoints on demand, giving you greater control over state management and recovery.
  • Easily track and manage all resources tied to your Flink deployments with metadata.ownerReferences and custom annotations.
  • Inject sensitive configurations securely during deployments without exposing them in the UI, enhancing security and flexibility.
  • Define your own 2FA solution based on your environment and third-party services, strengthening security while maintaining user flexibility.
  • Easily understand actual resource allocations for deployments.

These updates and more put you in control, reduce errors, and enhance your ability to manage large-scale streaming workloads efficiently.

New Features

Validate CRDs with Ease

Custom Resource Definitions (CRDs) now include openAPIV3Schema details within their YAML representations. This enhancement provides:

  • Improved schema validation to enable more comprehensive schema checks for CRDs, ensuring local validation and early detection of schema errors.
  • Streamlined development process developers can use to enforce schema constraints directly within CRDs, reducing eros during deployment. 

Customize Resource Names

You can now customize the Kubernetes resource names for managed applications, making resource management easier and more intuitive.

  • Add deployment.resource-name-prefix to deployment annotations to include meaningful identifiers, such as the Flink job name or a custom string.
  • Use this feature to simplify operational workflows, improve monitoring, and accelerate troubleshooting, especially in complex environments.

Clear resource names ensure efficiency and control during deployments and incident response.

Manage State and Recovery Seamlessly

Ververica’s Kubernetes operator introduces two new features:

  • Restart Without Change: You can use the restartNonce field, to restart a deployment without changing its configuration.
  • Manual Savepoint Trigger: You can use the saveppointTriggerNonce field to create savepoints for running jobs manually, without disrupting job execution.

These features simplify state management and recovery, enhancing operational flexibility and reliability.

Track Flink Deployments Effortlessly

The Ververica Kubernetes Operator now supports setting metadata.ownerReferences allowing you to add custom metadata.labels and metadata.annotations to Kubernetes deployments created via the VvpDeployment resource. This will ultimately make it easier for tools to track and manage all resources tied to Ververica Flink deployments.

Additional Improvements

  • Secure Secrets Handling: You can safely inject sensitive configurations during deployments without exposing them in the UI. This ensures secure and streamlined deployment processes. 
  • Expanded Access to Resource Usage Tracking Endpoint: Non-admin users and tenants can now programmatically monitor resource and quota usage. This increases visibility for all stakeholders.
  • Improved UI Consistency for Resource Configurations with Pod Templates: The UI now highlights both the configured CPU settings and any overrides from pod templates. This provides greater transparency and helps users easily understand actual resource allocations for deployments.
  • Accurate Deployment  Defaults: When clearing CPU fields to use default values, the system correctly applies deployment defaults. This ensures accurate resource allocation and an enhanced user experience when setting up deployments. 
  • Support Two-Factor Authentication on Flink UI: You can define your own two-factor authentication (2FA) solution based on your environment and third party services. This strengthens security while maintaining user flexibility.

Bug Fixes

Addressing issues is an integral part of release cycles. In this release, we resolved the following issues:

  • Improved Error Message When Deleting Deployment Targets
  • Fixed NullPointerException When Deploying SQL Jobs with Custom Kafka Catalog
  • Fixed Flink UI Inaccessibility After Upgrade from Ververica Platform: Self-Managed 2.11.2
  • Corrected Error Message for Invalid Characters in Secret Names

Please read the Release Notes for additional details.

Vulnerability Fixes

Managing our platform security is a top priority for Ververica. In this release, we addressed several vulnerabilities to safeguard your deployments and ensure system integrity. For a complete list of the resolved vulnerabilities, please read the 2.14 Release Notes

Downloads

More Resources