# Ververica — core content, full text > Product, industry, customer, learning and company pages of https://www.ververica.com, each as Markdown. Blog posts: https://www.ververica.com/blog/llms-full.txt. Developer articles: https://www.ververica.com/developers/llms-full.txt. Index: https://www.ververica.com/llms.txt. --- title: "Home page" description: "The original creators of Apache Flink. Ververica delivers enterprise stream processing with 2x faster performance, 40% lower TCO, and sub-10ms latency." lastUpdated: 2026-10-14T08:38:48.000Z source_url: html: "https://www.ververica.com/" md: "https://www.ververica.com/index.md" --- # Critical won’t wait. Neither can you. Delivery Assurance for mission-critical real-time data. Built by the original creators of Apache Flink®. - Forrester Wave Leader, Q4 2025. - Sovereign by design. - Engineered in Europe. - **2X — Faster Processing**: Than open-source Apache Flink - **40% — Lower TCO**: More from the same infrastructure - **<10MS — End-to-End Latency**: From event ingestion to action - **100B+ — Events per Day**: Proven at scale ## Trusted in Production ## 3 reasons. no fluff. Why Ververica We created Apache Flink. Not adopted it. Not forked it. Created it. 10+ years of production-grade streaming. The #1 contributor to the Flink codebase. Forrester Wave Leader, Q4 2025. Every event tracked. Every transformation auditable.
SOC 2 Type II. ISO 27001. GDPR. DORA. Column-level lineage from source to action. Compliance built into the architecture, not bolted on after the audit. 2x throughput over open-source Flink. Sub-10ms latency. 100% API compatible. Deploy on BYOC, or Self-Managed. No vendor lock-in. No data egress. Your cloud, your rules. ## Banking Runs on Ververica. ### Fraud Detection Score 2B+ transactions per day. Block fraud in under 150ms. ### Real-Time Payments Instant settlement. Exactly-once processing. Peak-hour resilience. ### AML Monitoring Continuous monitoring. 60% fewer false positives. Adaptive thresholds. ### Risk Management Real-time VaR. Counterparty exposure. Continuous stress testing. ### Regulatory Reporting T+0 reporting. Automated validation. Immutable audit trail. ### Core Modernization Strangler fig migration. Zero downtime. Dual-run validation. ### Customer Personalization Act on customer behavior as it occurs. Serve the right offer in under 50ms. ### Mainframe Offloading Cut MIPS costs by 40% or more without replacing the mainframe. ### Fintech Monitoring Real-time portfolio risk. Instant wealth management signals. ## Four Pillars. One Platform. ### Stream Processing 2x faster than open-source Flink. 40% lower TCO. 100% API compatible. Enterprise compute built by the team that created Flink. ### Data Movement 100+ connectors. Flink CDC. Every source, every format. Zero custom code. ### Streamhouse Architecture Unified batch and stream. One SQL dialect. Lakehouse economics, streaming freshness. ### Real Time AI Feature engineering on live data. Sub-100ms inference. Full model lineage. ## Three Steps. Milliseconds Apart. How It Works ### Connect Ingest data from any source. Kafka, databases, APIs, IoT devices, cloud services. No custom connectors required. ### Process VERA engine processes streams in real time. Apply business logic, run analytics, execute AI models. Sub-10ms latency. ### Act Route results to any destination. Trigger alerts, update systems, feed dashboards. Decisions happen now, not later. ## Competitive comparison | Feature | Ververica | Open Source | AWS Managed Flink | | --- | --- | --- | --- | | Throughput | 6.9B records/sec | 3.4B records/sec | Not published | | Latency (p99) | 8ms | 45ms | 50-100ms | | Recovery Time | <30 seconds | 2-5 minutes | 1-3 minutes | | DORA Compliance | Built in | N/A | N/A | | Deployment Options | BYOC, Self-Managed | Self-managed only | Cloud only | _Building a platform to provide Security-as-a-Service_ ### Booking.com Booking.com, a leading travel ecosystem serving both partners and travelers, faced challenges with processing security data streams and orchestrating Flink applications with stateful upgrades and multi-tenancy requirements. Their previous approach proved cumbersome and inadequate for their needs. **Result:** To address these issues, Booking.com selected Ververica Platform to overcome their challenges. The decision to choose Ververica was driven by its ability to enable Booking.com's security teams to manage their own Flink applications independently, freeing up the Platform team to focus on maintaining the overall solution. - **10x**: More Flink Applications - **20x**: Faster Deployment ## Forrester Named a Leader in The Forrester Wave: Streaming Data Platforms, Q4 2025. Not a participation trophy. Recognition of what runs in production, at scale, across industries. --- title: "vs Open Source Flink" description: "Compare open-source Apache Flink with Ververica Platform. Download the one-page reference sheet on features, operations and support for production." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/apache-flink-vs-ververica" md: "https://www.ververica.com/apache-flink-vs-ververica.md" --- # Apache Flink® / Ververica One-Pager Download this one-page reference sheet: "Apache Flink vs. Ververica" to learn more about the differences between the open-source project and enterprise-grade solution. ## What is the difference between Apache Flink and Ververica? Apache Flink is the open-source stream processing framework, and Ververica is an enterprise platform built on it. Ververica is compatible with Apache Flink, so existing jobs run unchanged, and adds the VERA engine, auto-scaling, automatic recovery, multi-tenant governance, BYOC and self-managed deployment, and enterprise support. In today’s data-driven landscape, managing and processing large-scale streaming data is essential to staying competitive. While Apache Flink offers a powerful platform for real-time stream processing, its complexities can be a barrier, often requiring deep technical expertise and significant resources. That's where Ververica can help. While 100% compatible with and built on Apache Flink, Ververica offers additional features, benefits, and capabilities that remove the operational overhead and complexity associated with data streaming. In addition to being the original creators of Apache Flink, in this one-page reference guide, you will find: - A chart of open source vs. enterprise capabilities - 4 key takeaways to consider for any data streaming project - Notable differentiations between Flink and Ververica - ...and more! --- title: "Ververica MCP" description: "" lastUpdated: 2026-06-23T08:45:40.000Z source_url: html: "https://www.ververica.com/mcp" md: "https://www.ververica.com/mcp.md" --- # Your AI Coding Assistant Can't Touch Your Streaming Platform. Until Now. - Natural Language Deployment Creation - Log-Aware Debugging - Deployment Lifecycle Management - Import / Export Across Workspaces - Context-Aware Platform Access ## Introducing Ververica’s Model Context Protocol Server (Preview) ### Native Large Language Models (LLM) Integration for Your Unified Streaming Data Platform Ververica's Unified Streaming Data Platform Self-Managed version 3.0 isn't an upgrade; it's a complete reimagining of what a unified streaming platform can do for real-time data teams. After listening to data engineers overwhelmed by deployment complexity, operation teams battling production issues at 3 AM, and architects losing sleep over governance and compliance audits, we built Ververica Platform 3.0 to tackle these challenges head-on. ## Ververica MCP ## Ververica MCP Server Most data platforms treat AI assistants as external tools, helpful for code generation but disconnected from the platform where code runs. The result is friction, context loss, and time-consuming manual validation. With Ververica's MCP server, your AI assistant becomes an extension of your streaming data platform itself, with full visibility and control over deployments, scripts, artifacts, and logs. ## Why It Matters Accelerate development and deployment cycles. From prompt to running job in seconds. Reduce operational overhead. No manual API calls, no dashboard navigation required. Simplify debugging your workflows. AI analyzes logs and proposes fixes automatically. Minimize Configuration Drift. Get consistent deployments across every environment. Enable AI-Native Streaming Operations. Conversational platform management becomes your new standard. --- title: "Preview Program" description: "Explore Ververica's Unified Streaming Data Platform through Private and Public Preview phases, offering early access to features and opportunities." lastUpdated: 2026-10-07T11:16:53.000Z source_url: html: "https://www.ververica.com/previews" md: "https://www.ververica.com/previews.md" --- # Ververica's Unified Streaming Data Platform Preview Program New releases in the Ververica Streaming Data Platform follow a structured rollout process consisting of multiple stages. The Private Preview and Public Preview phases provide customers early access to new features and products, enabling them to explore upcoming capabilities while contributing valuable feedback. This iterative approach helps enhance the quality and reliability of our offerings before they reach full production readiness. ## Private Preview During the Private Preview phase, new products and features are made available only to a select group of users by invitation. ## Public Preview During the Public Preview phase, new products and features are accessible to all users for testing and feedback. These offerings are primarily intended for non-production environments, though customers may run production-like workloads to evaluate performance under real-world conditions. If you’re interested in accessing Public Preview offerings, please contact our [sales team](https://www.ververica.com/contact). ## General Availability Once a feature or product is fully refined and deemed stable for production environments, it reaches General Availability status. The duration of the Private Preview and Public Preview phases depends on user adoption and the resolution of issues identified during testing. ## Pricing for Preview Offerings During the Preview phase, Ververica provides free access to preview offerings. The specific access methods and requirements will be determined by Ververica’s guidelines in effect at the time. ## Preview Offering Terms By using Private Preview and Public Preview features, you agree that Ververica may contact you for feedback to help improve the product. Preview offerings are provided for testing and evaluation purposes. While we continuously refine them based on user input, they may have functionality limitations, and their progression to General Availability is not guaranteed. Based on evaluation results and customer adoption, certain features may be modified, delayed, or discontinued. Preview offerings are exclusively available to subscribed customers and are not covered by the [Ververica Service Level Agreement](https://www.ververica.com/product/deployment/cloud/service-level-agreement). For complete details on the terms governing preview offerings, availability, and usage, please refer to the [Ververica Preview Policy](https://www.ververica.com/preview-policy), [Ververica Terms of Service](https://www.ververica.com/terms-of-service), and other relevant policies (if applicable). --- title: "Platform Overview" description: "The complete enterprise stream processing platform. VERA engine, Apache Fluss, and Streamhouse architecture 2x faster, 40% lower TCO than alternatives." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product" md: "https://www.ververica.com/product.md" --- # Stream. Process. Act. Before the Moment Passes. The streaming data platform built by the creators of Apache Flink. Sovereign by design. Engineered in Europe. - Your data - Your cloud - Your rules - **2x — Faster Processing** - **40% — Lower TCO** - **DORA — Built for DORA requirements** ## What is Ververica’s Apache Flink platform? Ververica is an enterprise platform for Apache Flink. It runs streaming and batch jobs on the VERA engine and adds real-time AI, streaming data movement with Flink CDC and built-in governance, deployed as BYOC in your own AWS or Azure account or self-managed on your own infrastructure. It is compatible with Apache Flink APIs, so existing jobs run unchanged. ## Mission-Critical Results Customer Evidence - **Events Scored Daily**: Single banking deployment - **Detection Time**: From event to decision - **TCO Reduction**: vs. previous infrastructure - **Fewer False Positives**: ML on real-time data ## Ververica Platform Pillars ### Stream Processing _Compute_ 2x faster than open-source Flink. 40% lower TCO. 100% API compatible. Enterprise compute built by the team that created Flink. ### Real-Time AI _AI_ AI on governed, real-time data. Continuous feature engineering, inference pipelines, and RAG and not stale batch predictions. ### Streaming Data Movement _DATA MOVEMENT_ Flink CDC + 100+ connectors. Capture every change from every source the instant it happens — databases, queues, APIs, files. Zero-ETL. Zero batch windows. One pipeline, always current. ### Streamhouse Architecture _ARCHITECTURE_ Unified batch + stream architecture. One platform replaces separate streaming and batch systems. Lakehouse economics, streaming freshness. ## We Created Apache Flink ### Leader in Forrester Wave™ Q4 2025 Streaming Data Platforms ### 10+ Years Experience Running Flink In Prodcuction at scale across every industry ### #1 Flink Contributor More commits than any other company ### 24/7 Enterprise Support From the people who wrote the code ## Performance stats key metrics - **Faster**: Than open-source Flink - **Lower TCO**: Same workload, less cost - **Latency**: End-to-end processing - **Events/Day**: Proven at scale ## Frequently Asked Questions ### What is Ververica? Ververica is a real-time data platform built by the original creators of Apache Flink. It combines the Streamhouse unified architecture, Apache Fluss streaming storage, and enterprise-grade compute to deliver sub-millisecond analytics at scale. European-engineered. Sovereign by design. Deployed on any cloud or on-premises. ### What makes Ververica different from open-source Apache Flink? The Ververica Platform delivers 2x throughput improvement and 40% lower TCO over open-source Flink while maintaining 100% API compatibility. Beyond performance, the platform includes autopilot operations, governance, compliance frameworks (GDPR, DORA, SOC 2), and three deployment models (managed cloud, BYOC, self-managed). Built by the original creators of Apache Flink. ### Is Ververica compatible with my existing Flink jobs? Yes, with zero code changes. The Ververica Platform is 100% compatible with Apache Flink APIs, SQL, and connectors. Your existing Flink jobs run unchanged. Migration typically takes days, not months. ### Where can I deploy Ververica? Two options: BYOC (our control plane in your AWS/Azure/GCP account), and Self-Managed (any Kubernetes cluster, including on-premises and air-gapped). ### What is Streamhouse? Streamhouse is the unified architecture that combines streaming and batch processing into one platform. It uses Apache Fluss for real-time streaming storage and open table formats like Apache Paimon or Iceberg for long-term storage. --- title: "VS AWS Managed Flink" description: "Compares AWS Managed Service for Apache Flink with Ververica: where the hidden costs come from (orchestration overhead, egress fees and single-cloud lock-in), and how existing Flink applications move to Ververica on AWS, Azure or your own infrastructure without code changes." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/youre-overpaying-for-aws-flink" md: "https://www.ververica.com/youre-overpaying-for-aws-flink.md" --- # You're Overpaying for AWS Flink. See how companies cut their streaming data costs by 40% or more without rewriting a single line of code. ## What are the alternatives to Amazon Managed Service for Apache Flink? Ververica is an enterprise Apache Flink platform and an alternative to Amazon Managed Service for Apache Flink. It runs in your own AWS or Azure account through BYOC, or on your own infrastructure, so you are not tied to one cloud, and existing Flink applications move across without code changes. ## AWS Managed Flink: Convenient. Expensive. Trapped. AWS Managed Service for Apache Flink looks affordable at first glance. The listed price is $0.11 per KPU-hour in US East (N. Virginia). Simple enough. But that is not your real bill. Your actual costs include storage fees at $0.10 per GB/month, mandatory orchestration overhead, and data transfer charges that add up fast. Every application you run requires an extra KPU just for AWS to manage it. That is an invisible tax on every workload. The real surprise comes when you try to leave. Moving just one petabyte of data out of AWS costs around $90,000 in egress fees. Your data becomes hostage to the exit price. **What is really costing you:** - **Orchestration overhead:** Every application needs an extra KPU you do not use - **Egress penalties:** AWS charges you to access your own data outside their network - **Single-cloud lock-in:** No leverage to negotiate, no flexibility to optimize ## The Hidden Costs AWS Does Not Highlight AWS packages open-source Apache Flink as a managed service. Which sounds great until you realize: ### You're locked to AWS Multi-cloud? Only if you rebuild everything. ### The bill scales aggressively Per KPU. Per GB ingested. Per GB transferred. Per snapshot. Per region. Plus support. ### Features lag months behind Open-source Flink releases new features. AWS gets around to packaging them. Eventually. ### Egress fees are brutal. Want to send data to Snowflake? That'll be $92K 🤯 per PB. ### Governance is bolted on. Not built in. Hope your auditor is patient. ### The kicker? We created Apache Flink. We know exactly how AWS is limiting you. ## Why Teams Switch to Ververica The VERA engine delivers 40% lower total cost of ownership compared to other managed Apache Flink services. You get better performance while spending less. That is not a trade-off. That is an upgrade. Deploy on AWS, Azure, or your own infrastructure. Multi-cloud flexibility means you are never locked into one vendor's pricing. When your contract comes up for renewal, you have options. Ververica was founded by the original creators of Apache Flink at the Technical University of Berlin. When you have a question, you are talking to the people who wrote the code. No one knows Apache Flink better. Your existing Flink applications work on Ververica without modification. Most customers are running in production within few weeks. No lengthy rewrite projects. No compatibility surprises. ## The Math Does Not Lie Here is how AWS Managed Flink compares to Ververica for a typical enterprise workload: | Feature | Ververica | AWS Managed Flink | | --- | --- | --- | | Created by Flink founders | ✓ | ✗ | | Performance | VERA engine: 2x faster | Standard Flink | | Deployment | Multi-cloud, hybrid, on-prem | AWS only | | Cost structure | Transparent compute + storage | Per KPU + ingestion + egress + storage | | Typical enterprise TCO | $100K-$130K/year | $180K-$240K/year | | Egress fees | Optimized multi-cloud architecture | $0.09/GB (locks you in) | | Vendor lock-in | Zero (100% Flink API compatible) | By design | | Enterprise governance | Built into platform core | Basic (add-ons available) | | Support | Direct from Flink creators | AWS support (3-10% of bill) | | Best for | Enterprise production at scale | Getting started quickly in AWS | ### Do I need to rewrite my Flink applications? No. Ververica is 100% Apache Flink API compatible. Your existing code runs unchanged. If it works on standard Flink, it works on Ververica. ### Can I still run on AWS? Yes. You can deploy Ververica directly in your own AWS account using the [Bring-Your-Own-Cloud option](https://www.ververica.com/product/deployment/byoc). You can also choose Azure or on-premises deployment. The choice is yours. ### How long does migration take? Most customers move from AWS Managed Flink to Ververica and reach production within 3 weeks. Since your code does not change, migration is primarily configuration and testing. ### What about support? Ververica was founded by the team that created Apache Flink at the Technical University of Berlin. The engineers have been building and optimizing Flink since the beginning. Enterprise support includes direct access to Flink experts. --- title: "Deployment" description: "Deploy Ververica your way. Fully managed cloud, BYOC for regulated industries, or self-managed on your infrastructure. Compare all options." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/deployment" md: "https://www.ververica.com/product/deployment.md" --- # Your Data. Your Cloud. Your Rules. Two deployment models. One platform. Total sovereignty. ## How can I deploy Apache Flink with Ververica? Ververica deploys Apache Flink in two ways: BYOC, where the data plane runs in your own AWS or Azure account and Ververica operates the platform through a control plane that exchanges only metadata, and self-managed, where you run the platform on any Kubernetes cluster, in the cloud, on-premises or air-gapped. In both models, your data stays inside your environment. ## Comparison Table | Feature | BYOC | Self-Managed | | --- | --- | --- | | Time to production | Days | Days | | Infrastructure management | Shared responsibility | Customer managed | | Data residency | Your VPC / account | Your data center | | Network isolation | Full VPC isolation | Air-gapped | | Compliance scope | GDPR, DORA, NIS2, SOC 2 | Most demanding frameworks | | Scaling | Automatic within your limits | Customer controlled | --- title: "Apache Fluss" description: "Apache Fluss is the streaming storage layer purpose-built for real-time analytics. Unified stream and table storage for the Ververica Platform." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/fluss" md: "https://www.ververica.com/product/fluss.md" --- # The Storage Layer Streaming Deserved. Apache Kafka® was built for event transport, not analytics. Apache Fluss™ was built for both. Proven at 3 PB scale, processing 40 GB/s with sub-second latency. - Columnar streaming storage. - Delta Joins that cut compute by 80%. - Native Apache Iceberg tiering. ## How can I use Apache Fluss on Ververica? Apache Fluss is available on Ververica as a Private Preview. Fluss, an Apache top-level project, adds columnar streaming storage so Flink jobs read only the columns they need, Delta Joins that move join state out of Flink jobs, and automatic tiering of older data to Apache Iceberg or Apache Paimon. Union Read queries hot and lake data in one SQL statement. ## Key Statistics - **Proven Scale**: Production deployment processing 40 GB/s - **Less Compute**: CPU & memory reduction via Delta Joins - **Checkpoints**: From 90 seconds down to 1 second - **Data Freshness**: Sub-second ingestion to query ## The Problem You're using Kafka for something it wasn't built for. Your Flink jobs read 100% of every Kafka message, even when they only need a few half the columns. Stream-to-stream joins pile up 100 TB+ of fragile state in RocksDB with 90-second checkpoints. ## 4 Key Capabilities ### Column Pruning Apache Arrow columnar format, read only needed columns. Big network savings. ### Delta Joins Read only changed rows and needed columns. 80% less CPU and memory, 1-second checkpoints. ### Native Iceberg Tiering Hot data on NVMe/SSD, cold data auto-tiers to Apache Iceberg on object storage ### Union Read One SQL statement across hot and cold data transparently ## Production Proof You can't ignore - 80% Less CPU & Memory - 100 TB+ State Removed - 30% Cost Reduction - 1s Checkpoint Time ## Technical Specification | Feature | Specification | | --- | --- | | Storage Format | Apache Arrow columnar + row-based log (dual format) | | Query Interface | Flink SQL (streaming & batch), StarRocks, Spark | | Data Freshness | Sub-second (single-digit millisecond typical) | | Throughput | 40 GB/s proven in production (3 PB deployment) | | State Management | Delta Joins: externalized from compute engine | | Hot Storage | NVMe/SSD with primary key indexing (RocksDB) | | Tiered Storage | Automatic tiering to Iceberg/Paimon on object storage | | Column Pruning | Read only needed columns (Arrow columnar format) | | Checkpoint Impact | 1-second checkpoints (down from 90s in comparable systems) | | Compatibility | 100% Flink Table API / SQL compatible | --- title: "Governance" description: "Enterprise governance for stream processing. Role-based access, audit logging, data lineage, and compliance controls built into the Ververica Platform." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/governance" md: "https://www.ververica.com/product/governance.md" --- # Your Data Is Ungoverned. Regulators Have Noticed. **Built in, not bolted on.** Most streaming platforms add governance as an afterthought. Ververica was built for regulated industries from day one. Every event tracked. Every transformation auditable. - DORA - GDPR - SOC 2 - ISO 27001 - MiFID II ## What is governance and compliance on Ververica? Governance on Ververica is built into the streaming platform rather than added on. Data lineage tracks every transformation, schema management blocks breaking changes, role-based access control decides who sees what, and continuous audit trails record every change. Ververica is ISO 27001:2022 certified, holds SOC 2 Type II and SOC 3 reports, and is built for DORA and GDPR requirements. ## Banking Proof Points - **Top-10 EU Banks** - **Audit Coverage** - **Data Sovereignty Gaps** ## Problem of The Governance Gap Your batch pipelines have governance. Your data warehouse has lineage. But your streaming infrastructure? It's a black box. Regulators don't care that "streaming is different." ## Five Governance Capabilities. | Feature | Description | Regulation | | --- | --- | --- | | Complete Data Lineage | Track every transformation, column-level lineage | DORA Article 11, GDPR Article 30, SOC 2 CC6.1 | | Immutable Audit Trails | Every action logged, tamper-proof, cryptographic verification | DORA Article 12, SOC 2 CC7.2, MiFID II Article 16 | | Role-Based Access Control | Fine-grained permissions, SSO/SAML, least privilege | SOC 2 CC6.3, ISO 27001 A.9, GDPR Article 25 | | Automated Compliance Reporting | Pre-built dashboards, on-demand audit reports | DORA Article 15, SOC 2 CC4.1, GDPR Article 35 | | AI Governance | Govern AI inputs/outputs, feature lineage, model drift monitoring | EU AI Act Article 14, GDPR Article 22, DORA Article 5 | ## Regulation Coverage Digital Operational Resilience Act (financial entities) General Data Protection Regulation (personal data) Type II trust service criteria Markets in Financial Instruments Directive ## Sovereign Infrastructure ### Data Sovereignty Headquartered in Munich, Germany. Self-managed or BYOC (AWS or Azure) in the region you choose. ### Trust Center SOC 2 Type II, ISO 27001, GDPR ready, HIPAA compliant, built for DORA requirements, CSA STAR certified ### DORA Readiness Downloadable compliance checklist --- title: "How It Works" description: "See how the Ververica Platform processes 6.9B records/sec in three steps. Connect data sources, process in real time, deliver insights instantly." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/how-it-works" md: "https://www.ververica.com/product/how-it-works.md" --- # DECISIONS WHEN THEY MATTER. NOT A MILLISECOND LATER. **HOW IT WORKS** One unified, sovereign data platform. Every event ingested, processed, stored, served, and acted on before your batch job finishes loading. Built by the original creators of Apache Flink®. ## How does Ververica work? Ververica turns event streams into decisions in one pipeline. It ingests data from databases, queues and APIs through Flink CDC and pre-built connectors, processes events with Flink SQL, Java or Python on the VERA engine, stores results in the storage you choose, such as Apache Fluss, Apache Paimon or Apache Iceberg, and serves them to the applications that act on each event, with every step governed and traceable. ## Data Journey Your data has a window. Miss it, and the machine breaks. The fraud clears. Your customer disappears. Your compliance fails. Ververica is built for that window. Exactly-once processing, fault tolerant, compliant and sovereign by design. FOLLOW YOUR DATA TO DECISIONS ➔ ANY SOURCE. ANY FORMAT. ZERO CUSTOM CODE. 100+ connectors ingest the raw event data stream. Databases change. APIs fire. IoT sensors pulse. Capture it all with built-in change data capture (CDC). No custom plumbing required. - Apache Kafka, Kinesis, Pub/Sub, Event Hubs - PostgreSQL, MySQL, MongoDB, Oracle CDC - Amazon S3, HDFS, Apache Iceberg, Delta Lake, Apache Paimon - REST APIs, webhooks, custom protocols PROCESS, TRANSFORM, AND ACT AT ANY SCALE. Turn raw events into decisions at any scale and speed. Process billions of messages daily. Aggregate, join, and detect patterns across live data with exactly-once semantics. Route your data dynamically, execute event-driven applications on the runtime 2x faster than open source Apache Flink. - Flink SQL and DataStream API - Complex Event Processing (CEP) - Autopilot scaling, recovery, and optimization STORE ANYWHERE. Get the compliance, flexibility, and redundancy your business requires, with no vendor lock-in. Use your existing storage technologies or our seamless solution. The choice is yours. - Apache Fluss, HDFS, Apache Iceberg, Delta Lake, Apache Paimon, Amazon S3 - Ververica’s Streamhouse Reduce complexity and cost with unified storage, compute, and governance in one architecture EXPOSE YOUR DATA WHETHER IN-STREAM OR STORED. Processed, enriched data lands in the hands of your applications, dashboards, and APIs, always-fresh and on demand. Every consumer gets exactly the freshness they need without costly query re-runs. - Aggregated state - Materialized views - Dynamic tables EXECUTE DECISIONS THE MOMENT AN EVENT ARRIVES. Adjust the price. Follow the market. Block the fraud. Route the order. Actions fire the instant conditions are met. All of your data governed, audited, and traceable to the very end. - Automated decisions: From fraud prevention to dynamic pricing - Signaling: Trigger actions both in digital and real world environments - Monitoring and alerts: Build dashboards and autonomous systems - Application feeds: Create inventory counts or leaderboards ## SEE WHAT OTHERS ARE BUILDING WITH VERVERICA ### Mainframe offloading ### Fraud detection ### Regulatory reporting ### Digital infrastructure ### User activity ### Media and entertainment ## Autopilot Operations The Platform Runs Itself ### Auto-Scaling Resources scale with your data volume. Handle 10x traffic spikes without manual intervention. Scale back down to save costs. ### Auto-Recovery Failed jobs restart automatically from the last checkpoint. No data loss. No manual intervention. Sub-minute recovery times. ### Auto-Optimization Ververica analyzes query plans and resource usage continuously. Automatic parallelism tuning, memory allocation, and checkpoint interval optimization. ### Auto-Upgrading Platform updates with zero downtime. Rolling deployments. State preserved across upgrades. No maintenance windows. - **Uptime SLA** - **TCO reduction** - **Detection time** - **End-to-End Latency** --- title: "Integrations" description: "Connect to data sources and destinations. Kafka, databases, cloud services, ML platforms, and more — all natively integrated with Ververica." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/integrations" md: "https://www.ververica.com/product/integrations.md" --- # Connect Everything. Stream Everywhere. 100+ pre-built connectors. Every major database, message queue, data warehouse, and cloud service. Plug in and stream. No custom code required. - 100+ Connectors - 40+ CDC Sources - SQL Configuration - E2E Exactly-Once ## How does Ververica connect to my systems? Ververica connects to message queues, databases, data lakes, data warehouses, cloud services and REST APIs through pre-built Apache Flink connectors and Flink CDC, which streams every database change as it happens. Integrations are written in Flink SQL or the DataStream API, with exactly-once delivery from end to end. ## connectors categories - Message Queues - Databases (CDC) - Relational Databases - Data Lakes & Table Formats - Data Warehouses - Search & Analytics - NoSQL - Cloud Storage & Files ## Partner Ecosystem EKS, Kinesis, Redshift, S3 AKS, Event Hubs, Synapse, ADLS GKE, Pub/Sub, BigQuery, GCS Managed Kafka and PostgreSQL ## Features ### Exactly-once E2E delivery ### Automatic schema evolution ### Independent connector parallelism ### Event-time watermark strategies --- title: "Professional Services" description: "Expert stream processing consulting from the creators of Apache Flink. Architecture design, migration, optimization, and training services." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/product/professional-services" md: "https://www.ververica.com/product/professional-services.md" --- # Expert Guidance from the Creators of Apache Flink Architecture design. Migration execution. Performance optimization. Training and certification. Delivered by the engineers who built Apache Flink. ## Who offers Apache Flink consulting and professional services? Ververica Professional Services provides Apache Flink consulting for teams running Flink and the Ververica platform. It covers architecture design, migration from open-source Flink, Spark Streaming or Kafka Streams, performance tuning, Flink job code reviews, upgrades and training, and a resident engineer can join your team for longer projects. ## Jump-Start and Boost Services - Launch New Use Cases - Flink Job Code Review - Platform Installation - Healthcheck - Platform & Flink Training - Flink Version Upgrades ## Adoption and Expansion Services - Resident Engineer - Platform & Flink Training - Launch New Use Cases - Custom Connectors - CI/CD Integration - Observability Integration - Platform Upgrades - Healthcheck ## Scale and Optimize Services - Performance Tuning - Platform & Flink Training - Flink Job Code Review - Architecture Design Review ## Professional Services From the original creators of Apache Flink®. ### Architecture Consulting Stream processing architecture decisions are expensive to reverse. Topology, state management, partitioning, and connector selection determine performance, cost, and operational complexity for years. - Stream processing architecture design aligned to your workloads and SLAs - Technology selection and integration patterns - Capacity planning and resource sizing - High availability and disaster recovery design - Data governance and compliance architecture - Reference architectures and implementation roadmaps Our architects have designed stream processing systems at the largest banks, retailers, and technology companies in the world. They wrote the code that became Apache Flink. ### Migration Services Migrating to stream processing, or migrating between platforms, carries risk. Data continuity, application compatibility, and operational readiness all require planning and execution discipline. **Migration Paths We Execute:** - Batch-to-streaming migration for real-time analytics and operations - Open-source Flink to Ververica Platform for enterprise governance and VERA performance - Spark Streaming to Ververica for lower latency and true stream processing - Kafka Streams to Ververica for complex event processing and multi-source joins - Cloud-to-cloud migration for infrastructure consolidation - On-premises to BYOC for operational efficiency **Migration Process** 1. Assessment of current workloads, dependencies, and SLAs 1. Migration plan with risk analysis and rollback procedures 1. Application porting and testing in parallel environment 1. Cutover execution with zero-downtime procedures 1. Post-migration validation and performance benchmarking ### Performance Optimization Running stream processing and running it well are different problems. Throughput bottlenecks, state management overhead, garbage collection pauses, and serialization costs accumulate. Small inefficiencies compound at scale. What We Optimize - Throughput: Identify and eliminate processing bottlenecks - Latency: Reduce end-to-end latency from source to sink - Resource utilization: Lower CPU, memory, and storage consumption by up to 52% with VERA engine tuning - State management: Optimize checkpoint intervals, state backends, and compaction - Serialization: Reduce serialization overhead for high-volume workloads - Scaling: Configure auto-scaling policies for variable load patterns Optimization engagements typically deliver 30-50% resource reduction on existing workloads. Measured before and after. Not estimated. ## Engagement Models Ongoing access to Ververica architects and engineers. Architecture reviews, design guidance, and technical advisory on demand. Quarterly business reviews. Named contacts. Fixed-scope engagements for migrations, architecture design, or optimization projects. Defined deliverables, timelines, and success criteria. A Ververica engineer embedded in your team. Full-time or part-time. Hands-on development, operations, and knowledge transfer. Duration matched to your project timeline. ## Frequently Asked Questions ### Do I need professional services to deploy Ververica? No. BYOC and Self-Managed include documentation and support for self-service deployment. Professional services accelerate complex deployments, migrations, and architecture decisions. ### Can professional services work with my existing technology stack? Yes. Our team integrates with your existing infrastructure, monitoring, identity, and CI/CD systems. We do not require you to replace working components. Engagements focus on stream processing, not rearchitecting your entire stack. ### What is the typical engagement duration? Architecture consulting runs 2-4 weeks. Migrations range from 4-16 weeks depending on complexity. Optimization engagements typically complete in 1-2 weeks. Resident engineer placements run 3-12 months. ### Do you offer remote services? Yes. All service categories are available remote. On-site delivery is available for organizations that require in-person engagement. Most engagements use a hybrid model with on-site kickoff and remote execution. --- title: "Real-Time AI" description: "Run AI and ML models at stream speed with Ververica. Real-time feature engineering, model inference, and AI-powered automation at sub-10ms latency." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/real-time-ai" md: "https://www.ververica.com/product/real-time-ai.md" --- # Real-Time AI. Intelligence at the Speed of Your Data. Run ML models on every record as it arrives. Feature engineering, inference, and automated action. Sub-10ms latency. No batch scoring. No stale predictions. - Sub-100ms inference latency. - 93% fewer false positives. - Full lineage from raw event to model prediction. ## What is real-time AI on Ververica? Real-time AI on Ververica runs machine learning models directly inside Apache Flink stream processing pipelines. It covers feature engineering, model inference, A/B testing and automated actions on live data instead of stale batch predictions, and it can also call external model servers over REST or gRPC. ## Key Statistics - **Inference Latency**: End-to-end from event to prediction - **Feature Engineering**: Sub-second freshness, zero training-serving skew - **Governed Lineage**: From raw event to model prediction - **Fewer False Positives**: Fresh context vs. batch scoring ## Core Capabilities ### Real-Time Feature Engineering Compute features from streaming data as events happen. Sub-second freshness. ### Continuous Model Inference Deploy ML models directly in streaming pipelines. Hot model reloading. ### Streaming RAG Pipelines Retrieval-Augmented Generation on real-time data. No periodic batch re-indexing. ### Governed AI Full lineage from raw data to model prediction. EU AI Act compliance-ready. ## No batch windows. No scheduling. No waiting. How It Works ### Stream Features Raw events arrive from sources. The VERA engine computes features in real time: aggregations, joins, transformations, derived metrics. Feature values update with every new event. ### Score Models Computed features feed directly into embedded models or external model servers. Every record receives a prediction. Fraud score, recommendation rank, anomaly probability, price adjustment. Inference runs at stream speed. ### Act on Predictions Predictions trigger actions immediately. Block a transaction. Adjust a price. Send an alert. Update a dashboard. Route to a human reviewer. The action happens milliseconds after the prediction, not hours. ## Use Cases ### Fraud Scoring Score every transaction in real time. Compute risk features from transaction history, account behavior, and device signals. Run fraud models inline. Block suspicious transactions in under 50 milliseconds. Banks process millions of transactions per second on this pattern. ### Dynamic Pricing Adjust prices based on demand signals, inventory levels, and competitor data. Compute pricing features from streaming market data. Run pricing models on every relevant event. Push price updates to storefronts in real time. ### Predictive Maintenance Process IoT sensor streams from manufacturing equipment. Compute degradation features from vibration, temperature, and pressure data. Score predictive models on every reading. Trigger maintenance alerts before equipment fails. ### Real-Time Recommendations Personalize content, products, and offers as customers interact. Compute behavioral features from clickstream data. Score recommendation models on every page view. Deliver personalized results in the same request cycle. ### Anomaly Detection Identify unusual patterns across network traffic, financial transactions, or system metrics. Compute statistical features over sliding windows. Score anomaly models on every data point. Alert operations teams in seconds, not hours. ### Intelligent Alerting Replace static threshold alerts with ML-powered alerting. Models learn normal patterns and flag true anomalies. Reduce alert noise by 90%. Every alert that fires has a model-backed confidence score. ## Frequently Asked Questions ### What is real-time AI on the Ververica Platform? Real-time AI runs machine learning models inside streaming pipelines. The VERA engine computes features and executes model inference on every record at sub-10ms latency. Supported frameworks include TensorFlow, PyTorch, ONNX, scikit-learn, and XGBoost, plus external servers via REST and gRPC. ### How fast is real-time model inference? Embedded models run with sub-10ms inference latency per record inside the streaming pipeline. External model servers add network round-trip time, typically 5-20ms depending on model size and infrastructure. Total end-to-end latency from event to action remains under 50ms for most production workloads. ### Can I use my existing trained models? Yes. Export your model in TensorFlow SavedModel, ONNX, TorchScript, or pickle format and load it into the streaming pipeline. No retraining required. No model conversion. If your model runs in Python or Java, it runs in the Ververica streaming pipeline. ### How does real-time feature engineering work? The VERA engine computes features from streaming data as events arrive. Sliding window aggregations, cross-stream joins, running statistics, and derived calculations update with every new event. Features are always fresh, never stale batch computations. ### Does the platform support A/B testing for models? Yes. Route traffic between model versions in real time based on configurable split ratios. Track performance metrics on live data. The platform monitors prediction accuracy, latency, and business outcomes per model version to support data-driven model promotion decisions. --- title: "Streamhouse" description: "Streamhouse unifies streaming and batch in one architecture. Combine VERA engine, Apache Fluss, and Apache Paimon for end-to-end real-time analytics." lastUpdated: 2026-10-14T07:58:16.000Z source_url: html: "https://www.ververica.com/product/streamhouse" md: "https://www.ververica.com/product/streamhouse.md" --- # Streamhouse. Dual pipelines are done. **One query. Stream and table. Doesn't matter.** The streaming lakehouse, coined by the original creators of Apache Flink®. One architecture for streams and tables, real-time and historical, ingestion and analytics. One SQL engine. One governed source of truth. ## What Is Streamhouse? Streamhouse treats every stream as a queryable table and every table as a subscribable stream. One architecture runs ingestion, real-time processing, historical analytics, and AI inference on one governed set of data. Built on Apache Flink® for compute, Apache Fluss™ for low-latency streaming storage, and open table formats for the lake. Union Read crosses both tiers in a single query. Sub-second freshness when the business needs it. Lake economics when it does not. ## One query. Stream and table. Doesn't matter. STREAMHOUSE STACK | Feature | Technology | | --- | --- | | Compute | VERA / Apache Flink® | | Streaming storage | Apache Fluss™ | | Lakehouse storage | Any open table format (e.g. Iceberg) | | Ingestion | Flink CDC | | Query interface | Flink SQL | | Governance | Built in | ## One house. Four layers. Zero compromise. Streamhouse Architecture ### MATERIALIZED TABLES ON VERA APPLICATION + COMPUTE Declarative SQL. Freshness-driven refresh. VERA runs stream and batch on one engine, 2x faster than open-source Apache Flink®. ### APACHE FLUSS™ STREAMING STORAGE Low-latency columnar streaming storage. Real-time KV and log, columnar tables, lakehouse-native architecture. ### OPEN TABLE FORMATS LAKEHOUSE STORAGE Your object store. Your format. Tiering and Union Read cross streaming and lake storage in one query. ### AUTOPILOT + WORKFLOW SCHEDULER OPERATIONS Autopilot scales parallelism in real time. The Workflow Scheduler triggers batch on a freshness threshold or cron. Deterministic capacity, no surprises. ## What holds Streamhouse up. ## Two stacks. Twice the cost. Half the answers. Dual pipelines multiply everything that can break. Streaming for real-time. Lakehouse for analytics. Two pipelines. Two storage layers. Two query engines. Two governance models. Two on-call rotations. Data drifts. Costs compound. Engineers leave. Lakehouses are too slow for what comes next. Iceberg and Delta tables are minutes-to-hours fresh. Acceptable for BI. Fatal for fraud detection, real-time pricing, and agentic AI. Batch was never going to power the next decade. Pure streaming is too expensive for history. Keeping 90 days of data hot in Kafka is a budget line item nobody wants. And Kafka is not queryable. You cannot SELECT against a topic. So you build a separate lake to compensate. Now you have two systems again. ## Numbers. Not narratives. The "WHY" You should have it. - **WRITE PERFORMANCE**: Faster writes than a traditional batch lakehouse. - **QUERY PERFORMANCE**: Faster analytical queries with streaming preprocessing. - **COMPUTE THROUGHPUT**: VERA against open-source Apache Flink® - **CHEAPER THAN KAFKA**: Apache Fluss™ stateless compute and tiered storage ## Pick a stack. Pick a compromise. | Feature | STREAMING ONLY | CLASSIC LAKEHOUSE | STREAMHOUSE | | --- | --- | --- | --- | | Freshness | Sub-second | Minutes to hours | Sub-second to seconds | | Historical queries | Expensive replay | Yes | Yes, in the same query | | Storage cost | High, hot retention | Low | Low, automatic tiering | | Query interface | Custom consumers | SQL, batch only | Flink SQL, stream and batch | | Governance | Separate from lake | Lake only | One control plane | | Lock-in | Vendor runtime | Open formats | Open all the way down | --- title: "VERA Engine" description: "VERA engine delivers 2x the throughput of open-source Flink at 40% lower cost. Auto-optimized stream processing with sub-10ms latency." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/vera" md: "https://www.ververica.com/product/vera.md" --- # VERA Engine. Stream Processing, Supercharged. Binary-compatible with Apache Flink. Every existing Flink job runs faster. Zero code changes. - 2x the throughput - 40% lower cost - Sub-10ms latency ## What is the VERA engine? VERA is Ververica’s stream processing engine for Apache Flink workloads. It is binary-compatible with Apache Flink, so existing Flink applications run without code changes, and it speeds up execution through adaptive query planning, optimized memory management, operator fusion, an advanced state backend and JIT compilation. ## Performance Advantages Engine Behind the Platform VERA processes double the volume of open-source Apache Flink on identical hardware. Benchmarked on production workloads, not synthetic tests. Same data, same jobs, twice the speed. Total cost of ownership reduction across compute, operations, and infrastructure. Fewer nodes for the same workload. Lower cloud spend. Smaller operational team. End-to-end processing latency at the 99th percentile under sustained load. From record ingestion to output delivery. Measured, not estimated. Peak throughput on production-grade hardware. Processing at a scale that covers the largest enterprise workloads on the planet. Same workload. Fewer CPUs. Less memory. Smaller cluster. The efficiency gains compound at scale. ## How Does VERA Achieve 2x Performance? Five proprietary optimizations working together. ### Adaptive Query Planning VERA analyzes data characteristics at runtime and adjusts execution plans continuously. Static query plans waste resources on changing data. Adaptive plans do not. ### Optimized Memory Management Custom memory allocation reduces garbage collection pressure. Off-heap state storage eliminates JVM pauses. Predictable latency under sustained high throughput. ### Operator Fusion VERA merges multiple operators into single execution units where safe. Fewer serialization boundaries. Fewer network shuffles. Less overhead per record. ### Advanced State Backend Purpose-built state backend optimized for streaming access patterns. Faster state reads and writes. Incremental checkpointing with minimal I/O overhead. ### JIT Compilation Just-in-time code generation for SQL queries and user-defined functions. Machine-optimized execution paths generated at deployment time, not interpreted at runtime. ## VERA Key Capabilities ### Binary Compatibility Run any existing Apache Flink application on VERA. JARs, connectors, UDFs, and libraries. No recompilation. No code changes. Drop in and run faster. ### SQL and DataStream API Full Flink SQL support for declarative stream processing. Full DataStream API for programmatic control. Both run on the same optimized engine. ### State Management Maintain state across billions of keys with exactly-once guarantees. Queryable state for external access. State migration between application versions without downtime. ### Intelligent Auto-Scaling VERA monitors throughput, backpressure, and resource utilization. Scales parallelism up during traffic spikes and down during quiet periods. No over-provisioning. No manual scaling. ### Event Time and Watermarks Process events based on when they occurred, not when they arrived. Watermark tracking handles late and out-of-order data correctly across distributed processing. ### Distributed Checkpointing Incremental checkpoints with minimal performance impact. Configurable intervals. Automatic recovery from any failure. Zero data loss guaranteed. ## VERA Engine vs. Open-Source Apache Flink | Feature | VERA Engine | Open-Source Flink | | --- | --- | --- | | Throughput | 6.9B records/sec | ~3.4B records/sec | | Latency (p99) | Sub-10ms | 20-50ms | | Resource Efficiency | 52% lower | Baseline | | TCO | 40% lower | Baseline | | Auto-Scaling | Built-in, adaptive | Manual or basic reactive | | Query Optimization | Adaptive, runtime | Static plan | | State Backend | Optimized, proprietary | RocksDB | | Multi-Tenancy | Native | Not supported | | Governance | RBAC, audit, lineage | Not included | | Managed Operations | Full platform | DIY | | Binary Compatibility | Full Flink API | N/A | | Enterprise Support | 24/7, SLA-backed | Community | ## Frequently Asked Questions ### What is the VERA engine? VERA is Ververica's proprietary stream processing engine, binary-compatible with Apache Flink. It delivers 2x throughput through adaptive query planning, optimized memory management, and operator fusion. Every Flink application runs on VERA without code changes. ### Is VERA a fork of Apache Flink? No. VERA is binary-compatible with Flink, not a fork. It replaces critical execution paths with proprietary optimizations while maintaining full API compatibility. All Flink connectors, libraries, and applications run unchanged on VERA. ### Can I migrate from open-source Flink to VERA without code changes? Yes. Take your existing Flink application JAR and deploy it on the Ververica Platform. No recompilation, no code changes, no connector rewrites. VERA applies all optimizations automatically. State snapshots from open-source Flink are compatible. ### How does VERA achieve 2x throughput? Five optimizations: adaptive query planning adjusts execution at runtime, optimized memory management reduces GC pressure, operator fusion merges execution units, an advanced state backend accelerates reads and writes, and JIT compilation generates machine-optimized code. ### Does VERA support exactly-once processing? Yes. VERA provides end-to-end exactly-once semantics through distributed checkpointing, two-phase commit sinks, and transactional state management. Every record is processed exactly once, even during failures and recovery. ### What programming languages does VERA support? VERA supports Flink SQL, Java, Python, and Scala. SQL analysts and software engineers work on the same engine. All four interfaces produce applications that benefit from VERA's performance optimizations equally. --- title: "Ververica Platform" description: "Ververica Platform runs enterprise Apache Flink on your own infrastructure or cloud account, with auto-scaling, auto-recovery and zero-downtime upgrades." lastUpdated: 2026-10-14T07:58:16.000Z source_url: html: "https://www.ververica.com/product/ververica-platform" md: "https://www.ververica.com/product/ververica-platform.md" --- # Enterprise Flink. For Those Who Cannot Get It Wrong. **Ververica Platform** Auto-scaling. Auto-recovery. SOC 2 certified. Zero-downtime upgrades. Multi tenant governance. Plus 2x throughput and 40% lower TCO. 100% Flink API compatible. Drop-in replacement. - We built Flink - We know where it breaks. - We fixed it - **2x — Throughput** - **40% — Lower TCO** - **100% — Flink API compatible** - We built Flink. - We know where it breaks in production - We fixed it ## What Is Ververica Platform? The enterprise streaming and batch platform built by the original creators of Apache Flink. Unified stream and batch processing in one runtime. Materialized tables that maintain themselves. Auto-scaling, auto-recovery, SOC 2 & ISO 27001 certified, zero-downtime upgrades, multi-tenant RBAC everything production Flink needs, built in. Not bolted on. ## Key Statistics - **Faster Processing**: Throughput vs open-source Flink - **Lower TCO**: Same workload, less infrastructure - **Flink Compatible**: Zero migration effort required - **Recovery**: Automatic checkpoint restore ## The Problem Open-Source Flink Is Powerful. Production Is the Hard Part. Manual cluster sizing. No auto-scaling. 2-5 minute recovery times on failure. Upgrades require downtime. No RBAC. No compliance certifications. Auto-Scaling | Auto-Recovery | SOC 2 & ISO 27001 | Zero-Downtime Upgrades | Multi-Tenancy | Deployment Flexibility ## Technology Innovations Why Ververica Traditional: Spark batch + Flink stream, 2x cost Ververica: One SQL both modes, freshness-driven execution, 40-60% Less Cost OSS: 2-5 min recovery, full checkpoint Ververica: <30s auto-recovery, 60% smaller checkpoints Typical: Rewrite code, months to validate. Ververica: Same SQL & DataStream, deploy done. 0 Code Changes. OSS: Row-by-row, full CPU burn Ververica: Columnar SIMD batches, 40% less CPU, 2x Throughput ## Benchmarks Table | Feature | Ververica | Open-Source | Improvement | | --- | --- | --- | --- | | Throughput | 6.9B records/sec | 3.4B records/sec | 2x | | Latency (p99) | 8ms | 45ms | 5.6x | | Resource Usage | 60% | 100% | 40% less | | Recovery Time | < 30 seconds | 2-5 minutes | 10x | | Checkpoint Duration | 1.2 seconds | 3.1 seconds | 2.6x | ## Unified Processing One Platform. Streaming and Batch. Unified. - Unified Semantics - Freshness-Driven Execution - Built-In Scheduling - Resource Isolation - 10x Faster Deployment - 40-60% Cost Reduction - 97% Faster Snapshots ## Materialized Tables Define Once. Always Fresh. Query Instantly - Interactive MaterializedTableDemo component - SQL-Native - Always Fresh - Instant Lookups (<10ms) ## Compatibility (Zero Migration) Your Flink Code. Unchanged. - Flink SQL syntax - DataStream API - Table API - All connectors - State backends - Checkpointing - Savepoints - Python UDFs - Custom serializers - Metrics reporters --- title: "BYOC Deployment" description: "Stream processing in your cloud account. Data never leaves your VPC. Ververica manages the platform while you keep full control of your data." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/deployment/byoc" md: "https://www.ververica.com/product/deployment/byoc.md" --- # Your Cloud, Our Platform. Data Never Leaves Your VPC. **Available on: AWS, Azure, Google Cloud (Soon)** Ververica runs in your cloud account. You keep full data sovereignty. We manage the platform. Zero data egress. Zero shared tenancy. ## What is Ververica BYOC deployment? Ververica BYOC (Bring Your Own Cloud) runs the streaming data plane inside your own AWS or Azure account, so your data stays in your network and in the region you choose. Ververica operates the platform remotely through a secure control plane that exchanges only metadata, and handles upgrades, patching, scaling and monitoring. ## Data Sovereignty ### European Company Your contract is with Ververica GmbH, headquartered in Munich, Germany. ### Built for GDPR Requirements Data processing in your account, your region. ### Built for DORA Requirements ICT risk management with full visibility. ## Key Benefits Customer-managed KMS for all data at rest. TLS 1.3 for all data in transit. You control key rotation, access policies, and audit logging. Ververica handles upgrades, patching, scaling, and monitoring. Your team focuses on applications, not infrastructure. Platform SLA of 99.99%. Data stays in your network. No cross-cloud data transfer fees. No bandwidth charges for moving data to a third-party environment. Supports GDPR, DORA, PCI DSS, HIPAA and sector-specific requirements. Data stays in the region you choose. Audit trails for every operation. Your compliance team controls the evidence. Data never leaves your cloud account. Processing, storage, and state management execute inside your VPC. No data egress. No exceptions. ## Features - Single-tenant deployment - Network isolation - Use existing cloud credits - Custom compliance - Choose region/AZ - BYOK encryption --- title: "Self-Managed Deployment" description: "Run Ververica on your infrastructure — any cloud, on-premises, or air-gapped. Full control with enterprise platform capabilities." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/product/deployment/self-managed" md: "https://www.ververica.com/product/deployment/self-managed.md" --- # Full Control. Any Infrastructure. Deploy Ververica Platform on any Kubernetes cluster. Cloud, on-premises, or air-gapped. Your infrastructure, your rules, our enterprise support. ## What is Ververica self-managed deployment? Ververica self-managed deployment runs the Ververica platform on any Kubernetes cluster you operate, in the cloud, on-premises or air-gapped. You control the infrastructure, networking, security and compliance set-up, while Ververica provides the software, Helm charts, Kubernetes operator and enterprise support. Your data never leaves your environment. ## Statistics - Any K8s Distribution - Air-Gap Supported - On-Prem Ready - Full Control ## Supported K8s Distributions - Amazon EKS - Azure AKS - Google GKE - Red Hat - OpenShift - Rancher / RKE2 - Vanilla Kubernetes ## Platform Features - Helm Charts & Operator Pattern - Offline License & Air-Gapped Install - Custom Network Configurations - Enterprise RBAC & SSO - Observability Integration - Backup & Disaster Recovery ## Air-Gapped Use Cases Why Self-Managed On-premises for trading systems and transaction processing Classified data in SCIFs and air-gapped networks Sovereign data processing for national agencies. ITAR, FedRAMP Sensitive patient data on-premises. HIPAA compliant --- title: "Fluss Private Preview" description: "Join the private preview of Ververica Platform with Apache Fluss integration. Experience unified streaming: ingest, process, store, and query in one place." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/product/fluss/private-preview" md: "https://www.ververica.com/product/fluss/private-preview.md" --- # Complete Your Streamhouse with Apache Fluss™ Experience the industry's first unified streaming data platform with columnar, queryable storage. Transform ingestion, processing, and querying into one cohesive system. ## Infrastructure Challenge Traditional streaming platforms process data quickly but lack queryable storage. Adding databases and caches solves one problem while creating others: more infrastructure to manage, more data to move, and more delays between event and insight. ## Common Challenges Separate tools for ingestion, processing, caching, and querying multiply operational costs and complexity. Shuttling data between systems introduces latency, reduces freshness, and increases costs. External databases and batch processes prevent you from querying data as it arrives. ## How It Works Data flows into Ververica Platform and moves through three integrated stages. Apache Flink processes streaming data with exactly-once guarantees and sub-second latency. Apache Fluss stores results in columnar format while maintaining a unified log and cached table representation. You query data immediately using standard SQL, with sub-second response times on continuous updates. The architecture removes the need for external databases, caches, and data movement pipelines. Processing and storage happen in one platform, keeping data fresh and queryable without adding infrastructure. ## Why Teams Choose Ververica Platform ### Unified Streaming Infrastructure Consolidate ingestion, processing, and storage into one platform. Ververica eliminates separate databases and caches, reducing operational overhead by 40%. Your team builds streaming applications instead of managing tool chains. ### Immediate Queryability Query streaming data with sub-second latency. Apache Fluss reflects every incoming change instantly, enabling dashboards and analytics that update as events happen. No batch delays, no stale caches, no waiting. ### Simplified Architecture Replace five separate systems with one unified platform. Ververica handles everything from data ingestion through Apache Flink processing to queryable storage with Apache Fluss. Fewer tools mean lower costs, simpler operations, and faster development. ## Platform Capabilities ### Apache Fluss Integration: Columnar, Queryable Storage - Treats streams as tables with unified log and cached representation - Columnar storage optimized for analytical workloads with efficient access patterns - Native support for modern formats including Lance and LanceDB - Seamless tiering between hot data and lakehouse storage ### Apache Flink Processing - Exactly-once processing guarantees with automatic checkpointing - Sub-second event latency at enterprise scale - Unified batch and streaming APIs with SQL support - Kubernetes-native deployment with automated operations ### Zero-State Streaming Analytics - Query real-time and historical data without external state stores - Eliminate database dependencies for streaming applications - Unified view across continuous and batch data sources - Real-time feature store capabilities for AI and ML workloads ### Enterprise Operations - Automated scaling and resource management - Built-in security with RBAC and compliance controls - 24/7 support from the team that created Apache Flink - Deploy on AWS, Azure, GCP, or self-managed infrastructure --- title: "Academy" description: "Free courses on Apache Flink and stream processing from the creators of the technology. Beginner to advanced certification paths." lastUpdated: 2026-10-20T05:02:49.000Z source_url: html: "https://www.ververica.com/academy" md: "https://www.ververica.com/academy.md" --- # Ververica Academy. Learn from the Experts **No vendor pitch decks. Free. Self-paced.** The creators of Apache Flink built this curriculum. Every course draws from 10 years of production engineering. Built for engineers who ship. - Auto scaling - Auto-recovery - SOC 2 Certified - Zero-downtime Upgrades - Multi-tenant Governance - 2x Throughput - 40% Lower TCO - 100% Flink Compatible ## Where can I learn Apache Flink? Ververica Academy offers free, self-paced online courses on Apache Flink and stream processing, created by the engineers at Ververica. For engineers who already use Flink, the Academy also runs a free online Apache Flink bootcamp with hands-on exercises on state, exactly-once processing and Flink SQL. ## Why Ververica Academy? Study at your own pace and convenience, with training created by the experts at Ververica. ### Boost Your Career Enhance your career opportunities by gaining expertise in popular Apache Flink technologies, trends, and best practices. ### Flexible Learning Whether you are upgrading your skills or exploring something new, learn Flink in a convenient and engaging learning environment. ### Digital Credentials Demonstrate your newly acquired skills with badges earned upon successful course completion. --- title: "Asset Library" description: "Browse and download one-pagers, reference sheets, whitepapers, guides, and more. " lastUpdated: 2026-10-14T07:58:16.000Z source_url: html: "https://www.ververica.com/asset-library" md: "https://www.ververica.com/asset-library.md" --- # Asset Library **One-pagers, reference sheets, whitepapers, guides.** ## Data sovereignty assets ### Data Sovereignty Enterprise stream processing from a European company. Keep real-time data in infrastructure and regions you control. ### FSI Streaming Sovereignty One-Pager You outsourced operations. They took control. Vendor-managed streaming platforms hide dependencies that surface during audits, outages, and exit negotiations. This one-pager exposes the trade-offs no vendor puts in the pitch deck. ### Sovereignty Self-Assessment Checklist If your regulator asked you to prove end-to-end control over your real-time streaming data, could you answer: Where does personal data flow? Who can access it? How would you delete it on request? ### Sovereignty Playbook Comprehensive guide to data sovereignty for FSI. Learn what DORA demands, why most streaming platforms fail. ### Sovereignty Evaluation Framework Technical evaluation framework for FSI organizations to audit streaming platform sovereignty across deployment, governance, security, AI/ML, and operations ### How Ververica Delivers Sovereignty Discover how Ververica empowers financial services with architectural sovereignty, offering flexible deployment options and enhanced governance. ### Finance Service Industry Proofsheet Financial Service Institutions don't get to choose between speed and control. Download this proofsheet and see how five FSI organizations achieve streaming sovereignty with Ververica. ## Banking industry assets ### Finance Industry Reference Sheet Discover how Ververica empowers financial institutions to leverage real-time data for faster decisions and better customer experiences. ### Mainframe Offloading One-Pager Modernize your mainframe operations and reduce costs. Download our one-page guide on leveraging Ververica's solutions for real-time data processing and insights. ## Other industry and technology assets ### Ververica vs OS Apache Flink® Info Sheet While 100% compatible with and built on Apache Flink, Ververica offers additional features, benefits, and capabilities that remove the operational overhead and complexity associated with data streaming. ### Online Gambling & Gaming Industry One-Pager When timing, user engagement, fraud protection, and personalization are critical to your bottom line, every millisecond matters. ### VERA: The Path to Cloud-Native Apache Flink White Paper Get a technical introduction to Ververica's Runtime Assembly (VERA), the powerful engine that powers Ververica's Unified Streaming Data Platform. ### Apache Fluss®: The Foundation of the Unified Streaming Lakehouse This paper traces the architecture that makes that possible, from the storage engine internals to the stateless compute model to the unified feature and context store. ### Low-Code Streaming Pipelines with SQL Ebook Download the practical guide to building production-grade Flink SQL pipelines with Ververica Platform, from interactive SQL to governed production deployments. --- title: "Ecosystem Introduction" description: "Your Guide to the Ververica's Unified Streaming Data Platform Ecosystem " lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction" md: "https://www.ververica.com/ecosystem-introduction.md" --- # Ecosystem Introduction **Your Guide to the Ververica's Ecosystem** ## What is the Apache Flink ecosystem? The Apache Flink ecosystem is the set of open-source projects used with Flink to build real-time data platforms: Apache Flink for stream and batch processing, Flink CDC for capturing database changes, Apache Fluss for streaming storage and Apache Paimon for lake tables. Ververica builds its platform on these projects and runs Flink jobs on its VERA engine. ### What is Apache Flink®? Discover Apache Flink, a powerful framework for real-time stream processing and analytics, enabling event-driven applications and dynamic decision-making. ### Beginner's Guide to Real-Time Data Processing Unlock real-time, fault-tolerant data pipelines using stream processing with Apache Flink. Discover key concepts, tools, and more with Ververica ### What is Apache Paimon™? Explore Apache Paimon, the open-source, stream-native data lakehouse format. Learn how it powers real-time updates, unified analytics with Ververica. ### What is Apache Fluss™? Learn more about Apache Fluss, the open-source, unified streaming storage layer designed to optimize real-time data processing. ### What is VERA? Discover VERA, Ververica's high-performance engine that optimizes Apache Flink for seamless real-time and batch data processing ### What is Streamhouse? Streamhouse unifies batch and stream processing on data lakes, enabling real-time insights and cost-effective analytics for businesses. ### Stream Processing vs. Batch Processing Apache Flink blends batch and stream. Ververica delivers the enterprise tools to run unified data workloads reliably, efficiently, and at scale. --- title: "Academy Bootcamp" description: "An online Apache Flink Bootcamp from Ververica Academy for engineers who already use Flink and want to run it in production. Modules cover architecture, state management, exactly-once processing, Flink SQL and workflow design, with hands-on exercises and live office hours." lastUpdated: 2026-10-16T05:13:59.000Z source_url: html: "https://www.ververica.com/academy/bootcamp" md: "https://www.ververica.com/academy/bootcamp.md" --- # Apache Flink Bootcamp **Online and Free!** Nine modules. Self-paced. Taught by the team that wrote Flink. Go from Flink user to production-ready stream processing engineer. ## The Bootcamp An intensive program for engineers who already use Apache Flink and want to run it in production. Nine modules cover architecture, state management, exactly-once processing, Flink SQL, and workflow design. Every concept is paired with a practical exercise. Every exercise maps to a real production pattern. No marketing fluff. No recycled Flink documentation. The curriculum is taught by engineers who contribute to the Apache Flink project and operate it at enterprise scale. ## What You Get Architecture to workflow design. Every concept built on the previous one. Linear progression through the full stack. Production-grade coding exercises. Evaluated by Ververica engineers. Not multiple-choice theater. Start anytime. Watch recorded lectures and complete coding challenges at your own pace. Having trouble with the course code? Reach out to Ververica Academy for help. ## Curriculum ### Module 1: Apache Flink Architecture and Runtime Cluster components, job submission, deployment modes, DataStream API, resource management. ### Module 2: Basic DataStream API Transformations Filter, map, flatMap functions, rich functions, RichCo functions. ### Module 3: Event Time and Watermarks Time ordering, windowing, watermark generation, late event handling, aggregations, window types. ### Module 4: State Management and Serialization State handling, HashMap and RocksDB state backends, serialization concepts. ### Module 5: Failure Handling and Exactly-Once Processing Failover strategies, high availability, checkpoints, exactly-once semantics. ### Module 6: Enrichment and Skew Enrichment patterns, streaming joins, Streamhouse joins, performance optimization. ### Module 7: Flink SQL SQL semantics, interval and temporal joins, changelog streams, dynamic tables. ### Module 8: Table API and DataStream Integration Table API usage. Interop with DataStream pipelines. ### Module 9: Workflow Design Job and workflow separation, components, segmentation, bridging. ## Prerequisites Who Should Attend ### Programming One to two years of Java or a comparable language. Basic SQL. ### Flink Experience Hands-on API work. Job deployment and management. Event time and state concepts understood. ### System Knowledge Stream processing fundamentals. Distributed systems experience. Basic cloud platform familiarity. ETL and data pipeline exposure. --- title: "Fluss White Paper" description: "Apache Fluss® is an open-source, lakehouse-native storage layer designed to unify the fragmented infrastructure that real-time analytics, Machine Learning (ML) pipelines, and AI systems require. This paper traces the architecture that makes that possible, from the storage engine internals to the stateless compute model to the unified feature and context store. " lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/asset-library/fluss-white-paper" md: "https://www.ververica.com/asset-library/fluss-white-paper.md" --- # Apache Fluss®: The Foundation of the Unified Streaming Lakehouse A Lakehouse-native streaming storage for real-time analytics, context engineering, and AI. ## Apache Fluss Apache Fluss® is an open-source, lakehouse-native storage layer designed to unify the fragmented infrastructure that real-time analytics, Machine Learning (ML) pipelines, and AI systems require. This paper traces the architecture that makes that possible, from the storage engine internals to the stateless compute model to the unified feature and context store. ## Real-time AI systems demand infrastructure you don't have. No organization builds a real-time AI data platform from scratch. They arrive at it through a sequence of expanding ambitions: streaming ingestion, then analytics, then event processing, then ML features, then AI systems. Each stage adds a new specialized system. A message broker. A stream processor. An online store. An offline store. A synchronization layer. Every boundary between the systems is a seam where data silently diverges. ## Why does real-time analytics need a new storage layer? The Apache Fluss white paper explains why real-time analytics and AI break down when the same data sits in a message broker, a stream processor, online and offline stores and a lakehouse at once. It shows how Apache Fluss, an open-source, lakehouse-native streaming storage layer, replaces those copies with one storage layer, so the synchronization problem disappears. --- title: "Streaming Sovereignty" description: "A one-pager on the control trade-offs hidden in vendor-managed streaming platforms (hidden control planes, metadata leakage and lock-in), and how BYOC, self-managed and on-premise deployment with Ververica keep control with you." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/asset-library/fsi-streaming-sovereignty-one-pager" md: "https://www.ververica.com/asset-library/fsi-streaming-sovereignty-one-pager.md" --- # What Other Vendor-Managed Platforms Hide **The Illusion of Control** Regulators are not asking whether your vendor claims compliance. They are asking whether YOU can prove control. ## "Managed" Means Vendor-Controlled. Here's What That Costs You. You outsourced operations. They took control. Vendor-managed streaming platforms hide dependencies that surface during audits, outages, and exit negotiations. This one-pager exposes the trade-offs no vendor puts in the pitch deck. As we wrote in [Data Sovereignty Is Existential Most Platforms Treat It Like a Feature](https://www.ververica.com/blog/data-sovereignty-is-existential-most-platforms-treat-it-like-a-feature), true sovereignty requires architectural guarantees — not contractual promises. This one-pager makes that argument visual. ### The Convenience Managed streaming platforms sell simplicity. Provision in minutes. Scale on demand. No infrastructure headaches. What they don't sell: the control you surrender to get it. - **Hidden control planes.** Provisioning, scaling, monitoring, and access management flow through infrastructure you cannot audit. Your data stays in-region. The system controlling it does not. - **Metadata leakage. **Topic names, schema registries, consumer group offsets, and telemetry cross jurisdictional boundaries. Under DORA and GDPR, that is a compliance exposure. Our [Zero Trust Theater](https://www.ververica.com/blog/zero-trust-theater-and-why-most-streaming-platforms-are-pretenders) analysis explains why most platforms claiming "Zero Trust" cannot survive an actual audit. - **No real exit.** Proprietary APIs, vendor-specific tooling, platform-coupled state. Your "exit strategy" is a multi-year migration buried in a contract addendum. ## What This One-Pager Reveals A visual breakdown of the control trade-offs hidden inside vendor-managed streaming platforms. Built for leaders evaluating AWS Managed Flink, Confluent Cloud, or Azure Event Hubs. Inside the one-pager: - **"What they say" vs. "What you get"**: Side-by-side comparison of vendor promises against operational reality - **The sovereignty gap**: Where managed platforms break data residency, operational control, and technical sovereignty - **The lock-in anatomy**: How proprietary APIs, coupled state, and closed tooling eliminate your exit options - **The Ververica alternative**: How BYOC, self-managed, and on-premise deployment models deliver structural control Get the visual disqualification tool that exposes what "**managed**" actually means for your streaming infrastructure. ## ## Ready to Go Deeper? - **Assess your posture:** Use the [Streaming Sovereignty Checklist](https://www.ververica.com/data-sovereignty/streaming-sovereignty-checklist-for-financial-services-industry) to evaluate your platform against what regulators actually demand - **Score your platform:** Apply the [Sovereignty Evaluation Framework](https://www.ververica.com/data-sovereignty/streaming-sovereignty-self-assessment-technical-audit) for a structured 26-requirement technical assessment - **Read the full guide:** The [FSI Streaming Sovereignty Pillar Page](https://www.ververica.com/data-sovereignty/fsi-sovereignty-playbook) covers governance, deployment, Zero Trust, and sovereign AI in depth --- title: "Gambling and Gaming One Pager" description: "A one-page guide for gaming, betting and other high-volume digital businesses: how real-time data helps you act on fraud and engagement instantly." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/asset-library/gaming-industry-powered-by-ververica" md: "https://www.ververica.com/asset-library/gaming-industry-powered-by-ververica.md" --- # Win Big by Acting Fast Do You Operate in a Fast-Paced, High-Volume Digital Environment? Download this one-page guide to learn how Ververica can help put the odds in your favor with real-time data that supports right-now decisions. When timing, user engagement, fraud protection, and personalization are critical to your bottom line, every millisecond matters. Ververica builds scalable, low-latency data applications so you can act on your data instantly, securely, and at scale. Download this must-read if your business provides: - Gaming - Online gambling - Sports betting - Racing - Content streaming - ...or other highly-regulated or "risky" industry --- title: "Low-Code Streaming Pipelines With SQL E-book" description: "Download this practical guide to building production-grade Flink SQL Pipelines with Ververica Platform. Get a path from interactive SQL development to production Flink applications including dynamic tables, watermarks, windows, temporal joins, SQL Deployments, savepoints, and governance" lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/asset-library/low-code-streaming-pipelines-with-sql-e-book" md: "https://www.ververica.com/asset-library/low-code-streaming-pipelines-with-sql-e-book.md" --- # Low-Code Streaming Pipelines With SQL Download this free ebook to learn how to build production-grade FlinkSQL pipelines with Ververica Platform Streaming is no longer reserved for teams with deep Java expertise. Business and product teams want fresher metrics, operational triggers, and faster delivery cycles, and most transformation logic already starts life as SQL. Flink SQL makes that logic declarative, event-time aware, and production-ready. Ververica Platform takes it the rest of the way: from interactive query authoring to managed Deployments with savepoints, autoscaling, and governance built in. Developers develop business value. Infrastructure teams keep the lights on. In this ebook, learn: - How Flink SQL models streams as dynamic tables, and why that mental model matters - The building blocks of low-code pipelines: connectors, watermarks, windows, and temporal joins - Three production-proven patterns: clean and route events, real-time enrichment, and KPI aggregation pipelines - The path from SQL editor to SQL Deployment, including stateful upgrades and savepoints - How Autopilot autoscaling keeps pipelines backpressure-free while cutting costs - Governance guardrails: namespaces, templates, UDFs, and artifact management for reusable pipelines - A clear escalation path for when SQL is enough, and when to drop to Table API or DataStream - An implementation checklist to launch your first use case --- title: "Mainframe Offloading One Pager" description: "A one-page reference sheet on offloading mainframe workloads to real-time streaming with Ververica: why businesses offload legacy batch workloads, and the benefits for modern, real-time data pipelines." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/asset-library/mainframe-offloading-one-pager" md: "https://www.ververica.com/asset-library/mainframe-offloading-one-pager.md" --- # Mainframe Offloading Download this one-page reference sheet to learn how to modernize your mainframe, slash processing time, and cut MIPs. Mainframes are the backbone of many critical business operations. While reliable, these slow, batch-based systems also have high operational costs, limited agility, and struggle to support today's demand for real-time insights and actions. With Ververica, you can evolve your mainframes beyond rigid, expensive systems into modern real-time solutions. With Ververica, you can evolve your mainframes beyond rigid, expensive systems into modern real-time solutions. Unlock real-time intelligence, make decisions at lightning-fast speed, and bridge the gap between your traditional systems and AI/ML pipelines by leveraging your existing mainframe data to power your next-gen applications. Download this free one-page reference to learn: - How one global bank slashed its MIPS (Millions of Instructions per Second) cost by 90% - Why businesses choose to offload legacy workloads utilizing Ververica's Unified Streaming Data Platform - Benefits of offloading mainframe data when building modern, real-time pipelines - ...and more! --- title: "VERA Whitepaper" description: "Download the white paper \"VERA: The Path to Cloud-Native Apache Flink\", a technical introduction to Ververica Runtime Assembly (VERA): its architecture, why Ververica built it, and how it stays compatible with Apache Flink while running natively in cloud environments." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/asset-library/vera-whitepaper" md: "https://www.ververica.com/asset-library/vera-whitepaper.md" --- # VERA: The Path to Cloud-Native Apache Flink® Download the white paper "VERA: The Path to Cloud-Native Apache Flink" for a technical introduction to Ververica Runtime Assembly (VERA) In today’s data-driven landscape, managing and processing large-scale streaming data is essential to staying competitive. While Apache Flink® offers a powerful platform for real-time stream processing, its complexities can be a barrier, often requiring deep technical expertise and significant resources. That’s where Ververica’s Runtime Assembly (VERA) comes in. VERA is a cloud-native, ultra-high-performance engine that powers Ververica’s Streaming Data Platform, making stream processing accessible to all—not just the experts. By addressing key pain points like latency, scalability, and operational complexity, VERA enhances Flink’s performance and simplifies the user experience. In this white paper, you will find: - An introduction to the VERA architecture - The reasoning behind why Ververica built the VERA engine - How VERA tackles 5 key challenges of OS Apache Flink - A brief journey through the democratization of stream processing - Results of our performance comparison between VERA’s state engine (Gemini) and the open-source Apache Flink state engine (RocksDB) - Insights into how VERA remains open core and 100% Apache Flink compatible, while evolving Flink to run natively in modern cloud environments - How VERA simplifies the complexities of fault tolerance while providing better support for large stateful applications --- title: "Finance Industry Reference Sheet" description: "A downloadable reference sheet on real-time data in financial services: how continuous data streams help investment and commercial banking, customer experience and other real-time financial use cases with Ververica." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/finance-industry-reference-sheet" md: "https://www.ververica.com/ecosystem-introduction/finance-industry-reference-sheet.md" --- # Finance Industry Reference Sheet Milliseconds matter. Download this free resource to learn why financial leaders choose Ververica to solve real-time use cases. In the competitive financial services industry, fresh data fuels real-time decisions, making the difference between leading the market or falling behind. Learn how Ververica empowers financial institutions to act on data instantly, securely, and at scale. Unlock real-time intelligence, make decisions at lightning-fast speed, and stay ahead of the market. Download this free reference to learn: - The advantages investment bankers gain from using real-time data - How continuous data streams transform commercial banking - 4 ways to deliver exceptional customer experiences - 4 real-time financial use cases - What current customers achieve with Ververica - ...and more! --- title: "Unified Batch and Stream Processing with Apache Flink" description: "Explains the difference between batch and stream processing, and how Apache Flink runs both on one engine with STREAMING and BATCH execution modes and a unified Table API and SQL. Ververica provides the enterprise tools to run these unified workloads in production." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/run-batch-and-stream-with-flink-on-ververica" md: "https://www.ververica.com/ecosystem-introduction/run-batch-and-stream-with-flink-on-ververica.md" --- # Unified Batch and Stream Processing with Apache Flink Apache Flink blends batch and stream. Ververica delivers the enterprise tools to run unified data workloads reliably, efficiently, and at scale. ## Can Apache Flink run batch and stream processing in one engine? Unified batch and stream processing with Apache Flink means one engine and one API for both: Flink treats a batch as a bounded stream, so the same SQL or DataStream job can process a finite dataset and stop, or run continuously on live events. On Ververica, one job can serve both needs without a second engine or a duplicate pipeline. ## What do we mean by "Batch" vs "Stream" processing? [The difference between Batch and Stream processing](https://www.ververica.com/developers/batch-processing-vs-stream-processing) > **INFO:** Batch processing handles a finite, known dataset like a daily log file or database snapshot by running a job that processes all data, produces results, and then stops. In contrast, stream processing deals with unbounded, continuously arriving data such as sensor readings or user events, focusing on near-real-time insights and low latency. Because streaming systems never truly finish, they must manage event-time, late or out-of-order data, and maintain ongoing state for incremental computations. ### Batch processing - You process a bounded dataset, i.e., a finite collection of data that is known (or assumed) to be complete, e.g., a file, a database snapshot, or a table of logs for the day. - The job runs, processes the data, produces results, and finishes. There’s no ongoing “listening to new data forever”. - Latency (time from data arrival to result) is typically higher - we don’t expect instant results; rather we process large volumes, often for analytics, [ETL](https://www.ververica.com/product/how-it-works), and reporting. - Because the input is static/known, you can use certain optimisations (e.g., global sorts, blocking operations) because you know the whole dataset ahead of time. ### Stream processing - Here you deal with unbounded or semi‐unbounded data: data arrives continuously (e.g., sensor readings, Kafka topics, user events) and the job may never “finish”. - The goal is to process data as it arrives, often with low latency: near-real-time reactions. Examples: fraud detection, alerting, click-stream analytics. - Because data keeps coming, you must handle things like time (event-time, processing-time), windows, late data, out‐of‐order events. - You cannot assume you know the full dataset ahead of time, so you have to design for ongoing computations, incremental updates, stateful streaming. ## How Apache Flink® approaches both - unified architecture One of the big advantages of [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink) is that it supports both paradigms in a unified way - you don’t need two completely separate engines. Let’s dive into how. ### Flink’s "stream‐native" mindset - Flink was originally built as a **stream‐native** engine: meaning its core architectural model treats data as streams even when the streams are bounded. - In fact, one of the core design philosophies: _batch is just a special case of streaming_. - Because of that, even “batch” jobs in Flink often use the same fundamental runtime and APIs (for example, the DataStream API) rather than a completely separate batch engine. ### Bounded vs Unbounded & Execution Mode - In Flink you can specify an execution mode: **STREAMING** or **BATCH**. For example, the documentation: _“Apache Flink’s unified approach to stream and batch processing means that a DataStream application executed over bounded input will produce the same final results regardless of the configured execution mode.”_ - The _boundedness_ of the input matters: if all sources are bounded, then a job is bounded; if at least one is unbounded, the job is unbounded. The BATCH execution mode only makes sense for bounded jobs. - When you choose BATCH mode, Flink can apply additional optimisations (e.g., more efficient joins/aggregations, shuffle strategies) because it knows whole input is finite. ### Unified API & reuse - Because of this unification, you can often write (or reuse) code that runs either as streaming or batch with minimal changes if the logic itself is compatible. The Table API/SQL in Flink is unified for both batch and stream. - This helps in scenarios where you want to run the same logic over historical data (batch) and then continue with live incremental data (stream). For example: initial back‐fill + ongoing real-time processing. ### Optimisations when batch mode is used Some specifics of how Flink optimises when working in BATCH mode: - Because the input is finite, Flink can use blocking operators (operators that wait for all input before proceeding) and global sorting of keys, which you might not want in an unbounded stream scenario. - The shuffle and task scheduling can be more efficient: tasks can materialise intermediate results differently (even spilling less, materialising more aggressively) because termination is guaranteed. - State management can be simplified: for bounded jobs you might not need extensive checkpointing/back‐pressure in the same sense as streaming continuous jobs. ### Key implication: one engine, two modes So practically: - You use Flink’s DataStream (or Table/SQL) API. - You choose your sources (bounded vs unbounded). - You set execution mode accordingly (STREAMING vs BATCH). - Flink’s runtime adapts optimisations under the hood.Therefore you avoid the “two completely separate code‐bases” problem. ## Use-cases: when to pick batch vs streaming (in Flink) Let’s talk about when you might choose one vs the other, especially in a Flink context. ### Use-cases favouring batch mode - You have a large historical dataset you want to process (say logs for last month) where latency isn't critical; you just want a final result (report, ETL load). - You want to perform heavy aggregations, full joins, sorting across the whole dataset. Because input is bounded, optimisations can be applied. - You’re doing a “back-fill” job: fill up data until now, then you might switch to streaming. Flink supports that transition nicely. - Example: nightly job processing overnight logs, generating daily summary reports. ### Use-cases favouring streaming mode - Data arrives continuously and you need results quickly (near real-time), e.g., fraud detection, monitoring, anomaly alerts, live dashboards. - You need to maintain state over time, use event‐time semantics, handle windows, out‐of‐order events, and you cannot wait for all data to arrive. - Example: online click-stream processing, sensor data pipelines, alerting on live transactions. ### Hybrid / mixed scenarios - Many realistic architectures involve both: back‐fill large historical data (batch) then switch to live processing (stream). Flink’s unified model supports this. - You might also want incremental full + incremental updates: process backlog (batch), then process new events as stream. - It’s worth noting that having a pipeline that _mixes_ batch and streaming within the same job is possible but has constraints. For example, the input sources must support bounded/unbounded appropriately, and you must consider execution semantics accordingly. ### Rule-of-thumb in Flink - If your input is unbounded and you care about low latency → streaming mode. - If your input is bounded (finite) and latency is less critical than throughput/complete result → batch mode (or bounded streaming). - If you want “both” (initial historical + ongoing real-time) → consider unified pipeline with Flink, perhaps start bounded, then switch to streaming. ## Practical considerations & differences in Flink implementation When you’re working with Flink (or designing pipelines) you’ll want to keep in mind some of the implementation details and trade-offs. ### Time semantics, state, windows - In streaming mode, you’ll often work with **event time** (when the event occurred) vs **processing time** (when Flink sees it). Flink supports both. - For windows (e.g., tumbling, sliding windows) you need to consider late events, watermarking, state size. This is a streaming concern. - In batch mode, because the data is bounded, these concerns simplify: you might not need complex watermarking or window‐handling of late data (assuming you ingest all data). ### Fault tolerance, checkpointing, resource usage - Streaming mode needs to run indefinitely, so you care about fault tolerance (checkpoints, state backends) and resource usage over time. - In batch mode, since job terminates, you may rely less on constant checkpointing; Flink’s BATCH mode may reduce overhead for those parts. For example, one user said: “On the docs it’s said that it does not use checkpointing, back-pressure, nor even a RocksDB, and that ‘keys are sorted and processed sequentially’.” - Resource usage: streaming jobs hold resources long-term; batch jobs can release when done. ### Performance & optimisations - Because Flink uses the same engine, you get benefits of pipelining and parallel processing in both modes. - In batch mode, Flink can apply more aggressive optimisations: for example global sort, blocking operators, more efficient shuffles. - However: streaming latency demands may limit some optimisations (you can’t wait for whole dataset). - One article claims Flink can provide “up to 100× better throughput and latency for streaming workloads” compared to older batch-native systems when used appropriately. ### Code and API reuse - Because the DataStream API and Table/SQL API are unified, you can write code that works in both modes (with minimal changes) if your logic doesn’t rely on e.g., the job terminating vs being infinite. - That’s really valuable: less duplication between batch & streaming code‐bases. ### Things to watch out for - Just because you’re using bounded input doesn’t necessarily mean you should auto‐choose batch mode: if you still need near‐real-time updates you might choose streaming even on bounded input. - If your sources don’t support bounded/unbounded semantics properly (old connectors vs new), you may face issues. For example: only some sources (e.g., KafkaSource) support both bounded and unbounded mode. - Be careful about mixing bounded/unbounded sources in the same job: unbounded means you’re effectively streaming. - Ensure you pick the correct execution mode (BATCH vs STREAMING) so Flink can apply the correct optimisations. ## Summary: when to use what, and how Flink fits Let’s wrap up the main take-home points: - **Batch processing** = bounded datasets + full result + throughput prioritised over latency. - **Stream processing** = continuous data + incremental results + low latency prioritised. - Flink allows both in one engine, you don’t need entirely separate systems. - With Flink: - If you set execution mode = BATCH and have bounded sources, you can benefit from special optimisations. - If you have unbounded streams (or need low latency), use execution mode = STREAMING (or default). - Use-case oriented: choose streaming for “as events happen” processing; choose batch for “process the heavy dataset later” analytics. - For many real systems: you’ll do _both_ (e.g., back‐fill batch, then continuous streaming) and Flink supports that via unified API. - Technical considerations matter: time semantics, state management, checkpointing, resource usage, connector capabilities The code reuse advantage: with Flink you can potentially write once and run in both modes (or switch modes) rather than maintain two separate pipelines. ## Example scenario: how you might implement in Flink Here’s how you might approach a real scenario: - Suppose you are ingesting user click events for a website you want to: - A) process historical click logs (past 30 days) to compute user behaviour metrics; - B) from now on process live click events to update metrics in near real‐time. ### Step A) Batch part - Use a bounded source: e.g., read logs from S3/HDFS for the last 30 days. - Set Flink execution mode to BATCH. - Use DataStream API (or Table API) to compute aggregates, join with user profile data, etc. Because input is bounded you get full result (e.g., user metrics snapshot). - After job completes, write results to a data store (e.g., a database or data warehouse). ### Step B) Streaming part - Set up a KafkaSource (unbounded) reading live click events. - Use Flink in STREAMING mode. Use event‐time windows (e.g., tumbling 5-minute windows) to compute sliding metrics. Maintain state for each user. - Update results in near‐real‐time (e.g., a live dashboard, alerting, incremental enrichment). - Possibly you join with the historical snapshot from the batch job or keep that in a store. ### Optionally You might combine the logic so that you have _one_ Flink job: ingest the historical (bounded) data + stream data, and set the job to STREAMING mode but the bounded source is processed as “bounded”. Flink will handle it. But you’ll want to ensure you design appropriately (windows, eventual termination or graceful switch). ## How Ververica Platform supports both batch and stream processing Here are the key ways [Ververica Platform](https://www.ververica.com/product) (and its ecosystem) makes life easier for unified batch + streaming with Flink: ### Unified Processing and Batch Support That’s why our Ververica Platform fully supports bounded streaming applications — batch-type jobs that run through the same runtime. When a batch job finishes successfully, the Deployment simply transitions to the **FINISHED** state. We’ve built our entire ecosystem to bring together ingestion, processing, and storage into a cohesive, cloud-native platform for both batch and streaming. ### Closing the Gap Between Batch and Streaming: The Streamhouse We introduced the concept of the [**Streamhouse**](https://www.ververica.com/ecosystem-introduction/what-is-streamhouse) to unify streaming and batch analytics. Traditionally, organizations ran separate systems for batch (data lakehouses) and for real-time streaming. With the Streamhouse, we’re eliminating that divide. You can now achieve incremental updates, low-latency analytics, and unified storage and compute for both historical and real-time data — all in one place. ### Engine Enhancements and Operations Tooling Under the hood, our [**VERA (Ververica Runtime Assembly)**](https://www.ververica.com/product/vera) engine combines the best of open-source Flink with our own advanced innovations. Operationally, we provide a rich set of capabilities for managing both long-running streaming applications and bounded (batch) jobs. With our platform, you get cluster management, lifecycle UI, bounded-job awareness, deployment states, and more — all designed to simplify operations at scale. Our ecosystem also includes built-in support for ingestion (e.g., CDC), storage (via Flink CDC + [Paimon](https://www.ververica.com/ecosystem-introduction/what-is-apache-paimon)), state backends, and more — making end-to-end batch and streaming pipelines truly turnkey. ### How We Can Help You Bringing it all together: - You can run batch (bounded) and streaming (unbounded) workloads on the same platform and tooling, minimizing fragmentation. - You get production-grade infrastructure, tooling, and operational support — no need to build everything yourself. - Our unified Streamhouse architecture reduces duplication — no separate data lake and streaming stack required. - Our optimized runtime (VERA) delivers exceptional performance, resiliency, and scalability. - You gain [enterprise-grade features](https://www.ververica.com/product) like governance, lifecycle management, monitoring, and cluster operations — all in one integrated platform. ## Advantages of using Ververica Platform compared to open-source Apache Flink Now let’s compare: if you just use open-source Apache Flink vs. adopt Ververica Platform (which builds on Flink). Here are the advantages (and trade-offs). ### Key advantages 1. **Operational & deployment maturity** - With open-source Flink you get a world-class engine, but you often need to build out the operational layer (cluster management, scalability, multi-tenant, monitoring, UI, lifecycle). Ververica builds much of that out of the box. - Ververica provides enterprise support, structured best practices, and integrations suited for production-grade, large‐scale systems. - Example: Job lifecycle management for bounded jobs (batch) is explicitly supported. Open source might require you to build additional orchestration. 1. **Unified architecture for both batch & streaming with less friction** - While Flink supports both batch & streaming, Ververica emphasises this unification with features like Streamhouse, tightly integrated storage + compute + ingestion. That means less “glue code” or custom architecture. - This is especially helpful if you’re doing hybrid workloads (back-fill + real-time) which many organisations are. Ververica is designed for that. 1. **Performance and scalability enhancements** - The VERA runtime claims improved performance, optimised for cloud-native, large state, etc. For example: “Flink-powered lakehouse with 5-10× faster processing…” - These enhancements might reduce need for tuning and deep internals knowledge (you still need good design). 1. **Integrated ecosystem and tooling** - With Ververica you don’t just get the engine you get ingestion (CDC), storage (Paimon), unified platform, UI, dashboards, deployment tools. That reduces engineering overhead. - For example: if you’re building a combined real-time + historical analytics pipeline, having Lakehouse + Streamhouse in one place is a big plus. 1. **Support and risk mitigation** - Enterprises often prefer vendor support, SLAs, proven track record for critical use cases (finance, IoT, etc). Ververica’s heritage (the original creators of Flink) and its enterprise offering give that. - If you rely purely on open-source Flink, you may have to invest more in in-house expertise, custom tooling, and risk mitigation. ### What to consider / trade-offs - Cost: The enterprise platform (Ververica) may involve licensing, support costs vs “free” open-source Flink. - Flexibility: With open source you’re free to customise deeply; a managed platform may impose some constraints (though likely minor). - Vendor lock-in: If you adopt platform-specific enhancements, you might tie in to that vendor’s ecosystem (though Ververica is built on open source). - Overhead vs light use-case: If you have a small project, simple pipeline, maybe open source Flink is sufficient without full platform overhead. ## Conclusion When choosing between **stream processing** and **batch processing**, the decision always comes down to your data and your goals: - If your data is _finite_ and you can afford to wait for complete results, **batch processing** is usually more efficient. - If your data arrives _continuously_ and you need insights as it happens, **stream processing** is the way to go. With [**Apache Flink**](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink), you don’t have to pick one world or the other; it's built around the idea that _batch is just a special case of streaming_. That means you can process both bounded and unbounded data within the same engine, APIs, and architecture. It gives you the flexibility to run large-scale historical analyses one moment and handle real-time event streams the next, all with consistent semantics and performance. Now, where [**Ververica Platform**](https://www.ververica.com/product) really shines is in making all of this easier to manage and operate. It builds directly on Apache Flink but adds a full layer of **enterprise-grade features**: deployment automation, monitoring dashboards, lifecycle management for both long-running streaming apps and finite batch jobs, and built-in optimizations via the [**VERA runtime**](https://www.ververica.com/product/vera). It’s designed to take care of the “plumbing” so teams can focus on building data logic, not managing infrastructure. On top of that, Ververica’s [**Streamhouse**](https://www.ververica.com/ecosystem-introduction/what-is-streamhouse) architecture unifies streaming and batch data under one consistent ecosystem, so historical and real-time analytics no longer live in separate systems. Whether you’re replaying data to rebuild models, running continuous ETL, or feeding real-time dashboards, Ververica Platform gives you a single, cloud-native environment for it all. In short: - **Flink** gives you the flexibility and unified model for stream and batch. - **Ververica Platform** gives you the tools, automation, and reliability to run it at scale, in production, with much less operational friction. If you’re experimenting, open-source Flink is an excellent place to start. But if you need to move from prototypes to dependable, large-scale, always-on workloads or want an environment where both batch and stream pipelines can coexist seamlessly, Ververica Platform is built exactly for that. ## FAQ ### What’s the difference between batch and stream processing? **Batch processing** handles a finite, known dataset like a daily log file or database snapshot by running a job that processes all data, produces results, and then stops. In contrast, **stream processing** deals with unbounded, continuously arriving data such as sensor readings or user events, focusing on near-real-time insights and low latency. Because streaming systems never truly finish, they must manage event-time, late or out-of-order data, and maintain ongoing state for incremental computations. ### How does Flink support batch and stream processing ? **Apache Flink** supports both batch and stream processing through a unified architecture, meaning you don’t need separate engines for each paradigm. It was built as a stream-native engine, treating all data as streams—batch processing is simply a special case of streaming with bounded input. Users can choose between **STREAMING** or **BATCH** execution modes, allowing Flink to apply optimizations like efficient joins or global sorting for finite data. This unified model lets developers reuse the same APIs and logic for both historical (batch) and real-time (streaming) data processing seamlessly. ### When should I use Apache Flink batch mode? Use **Batch mode** when your data is **bounded** (finite) and you want to process the entire dataset to produce a **complete result**. It’s ideal for offline analytics, historical data backfills, or workloads where **throughput** is more important than latency. In this mode, Flink applies **optimizations** like global sorting, blocking operators, and efficient joins since it knows all data is available upfront. ### When should I use Apache Flink streaming mode? Use **Streaming mode** when your data is **unbounded** (continuous) or when you need **real-time, low-latency processing**. This mode is best for event-driven applications, live monitoring, or any “as events happen” scenario. Flink’s **stream-native runtime** ensures consistent, incremental updates with strong guarantees on time semantics and state management. --- title: "What is Stream Processing" description: "Unlock real-time, fault-tolerant data pipelines using stream processing with Apache Flink. Discover key concepts, tools, and more with Ververica" lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/stream-processing-with-apache-flink-beginners-guide" md: "https://www.ververica.com/ecosystem-introduction/stream-processing-with-apache-flink-beginners-guide.md" --- # How to "Do" Stream Processing with Apache Flink® Unlock real-time, fault-tolerant data pipelines using stream processing with Apache Flink. Discover key concepts, tools, and more with Ververica ## How do you do stream processing with Apache Flink? Stream processing with Apache Flink means writing a job that reads events from sources such as Apache Kafka, transforms each one with the DataStream API or Flink SQL, and writes results to sinks such as databases. Flink keeps state between events, recovers from failures through checkpoints and uses event time, so results stay correct when data arrives late. ## A Beginner's Guide to Real-Time Data Processing > **INFO:** To “do” stream processing with Apache Flink, you build applications that continuously process unbounded data streams. You define Sources to ingest data, apply Transformations to process it (e.g., filter, aggregate, join using Flink's DataStream API or Flink SQL), and direct the results to Sinks (e.g., a database or another message queue). Flink ensures fault tolerance via checkpointing and handles event time for accurate results. Using [Ververica's Unified Streaming Data Platform](https://www.ververica.com/product) enhances this by providing enterprise-grade capabilities for simplified deployment, monitoring, and scaling. Ververica is powered by VERA, the engine that revolutionizes Flink by boosting Flink's performance. In addition, the Ververica ecosystem includes Apache Paimon, which enables real-time updates on data lakes, and Apache Fluss, which provides unified, low-latency storage for analytical queries on streaming data.  Whether from website clicks or sensor readings, financial transactions or social media updates, data is constantly being generated. To meet modern business demands, the ability to process this continuous stream of information in real time and extract immediate insights has become a necessity. This is where stream processing comes in and [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink) stands as a leading stream processing framework at its forefront. This guide will introduce you to Apache Flink, explain how to do stream processing with Apache Flink, and highlight the key differences and advantages of using Ververica's Unified Streaming Data Platform enterprise-grade capabilities, including the VERA engine, Apache Paimon, and Apache Fluss, to build a robust real-time data and stream processing solution. ## Apache Flink in One Minute At its core, [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink) is an open-source, distributed stream processing framework designed for high-performing, fault-tolerant, and stateful computations over unbounded (continuous) and bounded (batch) data streams. Unlike traditional systems that process data in fixed chunks or "batches," Flink processes data as it arrives, enabling real-time responses to dynamic events. Here's what makes Apache Flink a unique and powerful choice for data stream processing: - **Unified Batch and Stream Processing:** Flink famously treats batch data as a finite stream. This "stream-first" approach means you can use the same APIs and programming model for both real-time streams and historical datasets, simplifying development and maintenance. - **Stateful Computations:** Many real-world applications need to remember information about past events to process current ones (e.g., counting unique users over time, tracking session activity). Flink offers robust, fault-tolerant state management, ensuring that your application's memory of past events is reliable, even if something goes wrong. - **Fault Tolerance:** Flink guarantees that your data processing will be accurate even in the face of machine failures. It achieves this through a mechanism called "checkpointing," which periodically saves the state of your application. If a failure occurs, the job can restart from the last successful checkpoint, ensuring exactly-once processing semantics, meaning each event is processed precisely once, avoiding duplicates or data loss. - **Event Time Processing:** Data events often don't arrive in the exact order they occurred. Flink's "event time" processing and "watermarks" allow it to correctly process events based on their actual occurrence time, even if they arrive late or out of order, which is critical for accurate real-time analytics. ## Introduction to Apache Flink Stream Processing (for Beginners) To understand how to "do" stream processing with Apache Flink, let's look at its basic building blocks. At the most simplistic level, a Flink stream processing application consists of three main parts: **1) Sources:** These are where data streams originate. Flink can connect to a wide variety of data sources that produce continuous streams, such as: - **Message Queues:** Apache Kafka, Apache Pulsar, AWS Kinesis. - **Databases**: Using Change Data Capture (CDC) connectors to read real-time changes from databases like MySQL, PostgreSQL, or Oracle. - **File Systems:** Reading new files as they appear in directories. **2) Transformations:** This is the core logic where you process your data. Flink provides a rich set of operators to transform, filter, aggregate, and enrich your data streams. Some common transformations include: - **Map**: Apply a function to each individual element in the stream (e.g., convert temperature from Kilograms to Pounds). - **Filter**: Selectively keep or discard elements based on a condition (e.g., only process transactions over $10). - **KeyBy**: Group elements by a specific key. This is essential for stateful operations or aggregations on specific groups (e.g., count of sales per product ID). - **Windowing**: Process data within specific time boundaries. - **Tumbling Windows:** Fixed-size, non-overlapping windows (e.g., sum sales every 5 minutes). - **Sliding Windows:** Fixed-size, overlapping windows (e.g., sum sales over the last 5 minutes, updated every 1 minute). - **Joins**: Combine data from two or more streams (e.g., join user click data with user profile data). - **Aggregations**: Compute sums, counts, averages, min/max over windows or groups. **3) Sinks:** These are where the processed data streams are sent. Flink can write data to various destinations, including: - **Message Queues:** Sending processed events back to Kafka or other queues. - **Databases**: Updating real-time dashboards or operational databases. - **File Systems/Object Storage:** Storing processed data for later analysis (e.g., on S3, HDFS). - **Dashboards/Monitoring Tools:** Directly feeding data to visualization tools. **Programming Flink:** You can write Apache Flink applications using its DataStream API (in Java or Scala for fine-grained control) or Flink SQL, which allows you to process streams using standard SQL queries, making stream processing more accessible to a wider audience familiar with relational databases. ## Apache Flink Use Cases and Best Practices Apache Flink excels in scenarios where real-time responsiveness, high throughput, and data accuracy are paramount. ### Real-World Apache Flink Use Cases: - **Real-Time Fraud Detection:** Flink can analyze financial transactions as they happen, using [Complex Event Processing (CEP)](https://www.ververica.com/blog/real-time-fraud-detection-using-complex-event-processing) to detect suspicious patterns (e.g., multiple transactions from different locations within seconds) and trigger immediate alerts or blocks. Ververica provides detailed insights into real-time fraud detection using Flink and CEP. - **Customer 360 & Personalization:** By processing customer interactions like clicks, purchases, and browsing history in real-time, Flink helps build a unified, continuously updated view of each customer, enabling personalized recommendations and dynamic pricing. An example of this is seen in real-time feature engineering for e-commerce, like [KartShoppe's use case](https://www.ververica.com/blog/kartshoppe-real-time-feature-engineering-with-ververica). - **IoT Analytics & Predictive Maintenance:** Flink can ingest massive streams of sensor data from industrial machinery or smart devices, detect anomalies, and predict equipment failures before they occur, optimizing maintenance schedules. - **Real-Time ETL (Extract, Transform, Load):** Modern ETL pipelines are moving from batch to stream. Flink performs [continuous transformations on data as it moves](https://www.ververica.com/product/how-it-works) from source to destination, eliminating batch delays and providing fresh data for analytics. - **Security Information and Event Management (SIEM):** Flink is used to process security logs in real-time [detecting threats, policy violations, and unusual activities](https://www.ververica.com/case-study/xm-cyber) as they unfold, enabling immediate incident response. - **Airlines & Logistics:** Optimizing operations by correlating real-time events like flight delays, gate changes, and baggage handling. Flink enables [real-time insights for airlines](https://www.ververica.com/case-study/airbus) with Complex Event Processing. ### Best Practices for Apache Flink Applications: - **Understand Event Time:** Design your application around event time and watermarks for accurate results, especially with out-of-order data. - **Manage State Efficiently:** For stateful jobs, understand how Flink manages state, choose appropriate state backends, and optimize state serialization to ensure performance and fault tolerance. - **Parallelism Tuning:** Configure the parallelism of your Flink job to match your available resources and the volume of your data. Too little means bottlenecks, too much can waste resources. - **Monitor Backpressure:** Regularly monitor for backpressure indicators, as this signals that parts of your pipeline are slowing down, which impacts latency and throughput. - **Leverage Flink SQL:** For simpler transformations and aggregations, Flink SQL often provides a faster development cycle and is more accessible to data analysts. ## Open Source Apache Flink vs. Ververica’s Unified Streaming Data Platform Enterprise-Grade Capabilities While open-source Apache Flink is incredibly powerful and “free” to use, running it at scale in a production enterprise environment comes with its own set of challenges. ### Challenges with Open Source Apache Flink: - **Operational Complexity:** Deploying, upgrading, scaling, and managing Flink clusters, especially in cloud environments, requires significant DevOps expertise. - **Monitoring & Debugging:** Setting up comprehensive monitoring, alerting, and debugging tools for distributed Flink applications can be complex and time-consuming. - **State Management at Scale:** While Flink handles state, optimizing its performance for very large, highly concurrent stateful applications can be challenging without deep expertise. - **High Availability & Disaster Recovery:** Ensuring seamless failover and disaster recovery in self-managed Flink setups requires careful planning and robust infrastructure. - **Security & Governance:** Implementing enterprise-grade security features, access controls, and data governance policies across Flink deployments can be intricate. - **Total Cost of Ownership (TCO):** The operational overhead and the need for specialized engineering teams can lead to a higher TCO than initially perceived. ### Ververica Platform Capabilities for Stream Processing Ververica's Unified Streaming Data Platform addresses these challenges head-on, providing an integrated, enterprise-ready solution for stream processing with Apache Flink. In addition to being 100% compatible with open-source Flink, it also transforms the power of Flink into a managed, [production-ready offering](https://www.ververica.com/apache-flink-vs-ververica) with features designed to simplify operations and accelerate value, including: - **Application Lifecycle Management:** Simplifies the deployment, versioning, and scaling of Flink applications. You can quickly deploy jobs, manage updates, and scale resources up or down with ease. - **Monitoring and Observability:** Provides a user-friendly interface for tracking job metrics, resource utilization, and debugging, offering deep insights into your stream processing pipelines. - **High Availability and Fault Tolerance:** Streamlines the setup and management of resilient Flink clusters, ensuring your critical data stream processing applications are always available. - **Enterprise Security:** Offers end-to-end encryption, flexible access control, and integrates with cloud-native security tools, aligning with [zero-trust principles](https://www.ververica.com/blog/your-cloud-your-rules-ververicas-bring-your-own-cloud-deployment). - **Cost Efficiency & Performance at Scale:** Complements Flink's high-performance runtime with autoscaling and capacity planning capabilities, ensuring optimal resource utilization and reducing TCO, sometimes reporting **40% lower TCO** than alternative Flink solutions [as mentioned in the VERA whitepaper](https://www.ververica.com/asset-library/vera-whitepaper). - **Turn-Key Solution:** Comes with all required dependencies for cloud and on-premise deployments, providing a highly available stream processing runtime and an integrated development environment for Flink SQL. ## Enhancing Performance and Capabilities: VERA, Apache Paimon, and Apache Fluss Ververica's Unified Streaming Data Platform extends Apache Flink's capabilities even further with innovative components like the VERA engine, Apache Paimon, and Apache Fluss, creating a comprehensive ecosystem for advanced data stream processing. ### VERA Engine: Revolutionizing Apache Flink's Performance The VERA engine (Ververica Runtime Assembly) is Ververica's cloud-native, ultra-high-performance runtime that sits at the core of the Unified Streaming Data Platform. VERA optimizes open-source Apache Flink to deliver unprecedented speed and efficiency: - **Ultra-High Performance:** VERA is designed for extreme throughput and low latency, capable of processing billions of events per second with millisecond-level delays. This makes stream processing flink significantly faster than open source Flink and any alternatives. - **Optimized Stateful Computations:** VERA includes an advanced state engine ("Gemini") that replaces standard Flink state backends like RocksDB, delivering superior performance in state-intensive, large-scale stream processing environments. This is crucial for applications that manage vast amounts of continuous state. - **Cloud-Native & Elastic Scale:** VERA separates compute and storage, allowing for effortless scaling of large stateful applications, ensuring consistent performance even as data volumes fluctuate.  ### Apache Paimon: The Stream Native Data Lakehouse For persisting and querying streaming data efficiently, Apache Paimon is a game-changer. As a stream-native data lakehouse format, Paimon enables Apache Flink to continuously write and update data directly on a cost-effective data lake (like S3), forming the foundation of [Ververica's Streamhouse concept](https://www.ververica.com/product/streamhouse). - **Real-Time Updates:** Paimon allows Flink jobs to perform real-time upserts and updates on data lake tables, ensuring that analytical queries always reflect the freshest data with low latency. - **Unified Batch and Stream Access:** Paimon tables can be accessed by both stream processing jobs (for continuous updates) and batch queries (for historical analysis), creating a single source of truth for all data. - **Cost-Efficient Storage:** It leverages affordable object storage while offering data warehouse-like properties (ACID transactions, schema evolution), making large-scale data stream processing more economical.  ### Apache Fluss: Unified Streaming Storage for Next-Gen Analytics Apache Fluss is a groundbreaking unified streaming storage layer designed specifically for high-performance, low-latency analytical access to streaming data. It integrates deeply with Apache Flink to simplify analytical pipelines: - **Sub-Second Latency Analytics:** Fluss offers sub-second latency for both writing streaming data and querying it, making it ideal for real-time dashboards and interactive analytics. - **Eliminates Intermediaries:** Fluss can replace the need for separate message queues (like Kafka) and traditional OLAP systems for analytical workloads, simplifying your architecture and reducing costs. - **Columnar Reads & Projection Pushdown:** Its columnar format and intelligent "projection pushdown" feature optimize read performance, fetching only necessary data and significantly boosting analytical query throughput. ## Conclusion Stream processing with [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink) represents the modern approach to handling data, moving beyond the limitations of traditional batch processing to embrace real-time insights and event-driven architecture. For beginners looking to dive into this exciting field, Apache Flink offers a powerful and flexible stream processing framework. While open-source Flink provides immense capabilities [Ververica's Unified Streaming Data Platform](https://www.ververica.com/product) transforms it into an enterprise-ready solution, simplifying operations, ensuring reliability, and providing advanced tooling. When combined with the performance acceleration of the [VERA engine](https://www.ververica.com/product/vera), the unified streaming storage capabilities of Apache Paimon (for your [Streamhouse](https://www.ververica.com/product/streamhouse)), and the low-latency analytical power of Apache Fluss, Ververica's Unified Streaming Data Platform offers a comprehensive ecosystem to build, deploy, and manage the most demanding data stream processing applications, truly unlocking the value of real-time data for any enterprise. ## FAQ ### What is stream processing with Apache Flink? Stream processing with Apache Flink is a method of analyzing and acting on data in real time as it arrives, using Flink's distributed engine to process continuous data streams. ### How does Apache Flink differ from batch processing? Apache Flink processes data as it flows in (real time), while batch processing handles fixed groups of data at scheduled intervals, making Flink ideal for instant analytics and decision-making. ### What are the main components of an Apache Flink streaming application? Key components include Sources (data inputs), Transformations (operations like filtering and aggregating), and Sinks (outputs such as databases or dashboards). ### How does Apache Flink achieve fault tolerance and exactly-once processing? Flink uses checkpointing and advanced state management to recover from failures, ensuring each event in the stream is processed exactly once. ### What are common use cases for Apache Flink stream processing? Typical applications include real-time fraud detection, IoT analytics, personalized recommendations, security monitoring, and real-time ETL for data pipelines. --- title: "What is Apache Flink" description: "Apache Flink is the leading open-source stream processing framework for real-time analytics, event-driven applications, and data pipelines. " lastUpdated: 2026-10-14T09:41:38.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/what-is-apache-flink" md: "https://www.ververica.com/ecosystem-introduction/what-is-apache-flink.md" --- # What is Apache Flink®? Learn more about the leading technology for processing both unbounded (streaming) and bounded (batch) data. ## What is Apache Flink and what is it used for? Apache Flink is an open-source framework for stateful stream processing: it processes data continuously as it arrives, with exactly-once state consistency and event-time handling, and runs batch jobs on the same engine. Companies use it for fraud detection, real-time analytics, streaming ETL and event-driven applications. ## Apache Flink, the leading stream processing framework. > **INFO:** Apache Flink is a powerful Open Source stream processing framework for data stream processing and distributed stateful computations on large-scale data streams. Unlike traditional batch processing, Flink operates on data continuously as it arrives, enabling real-time processing with low latency. This makes Flink a cornerstone of modern event-driven architecture, streaming analytics, and real-time data pipelines in many organizations. In this comprehensive guide, explore what Apache Flink is, how it works, and its key applications in **event-driven applications**, **real-time analytics**, and **data pipelines**. In addition, learn about common use cases like fraud detection, anomaly detection, rule-based alerts, business process monitoring, dynamic pricing, Extract, Transform, and Load (ETL), and streaming Artificial Intelligence and Machine Learning (AI/ML), including explanation of each and how Flink empowers these scenarios. Flink’s design provides **in-memory speed and scalability** for handling high-throughput event streams. It manages events with **exactly-once state consistency** and offers powerful primitives for processing time, (event-time windows, and watermarks) and for maintaining application state across events. In practice, this means Flink applications can perform complex operations like aggregations, joins, pattern detection, machine learning inference, etc. on the fly, producing instantaneous results as new data events come in. The ability to handle unbounded streams in real time while also managing bounded (batch) data sets makes Flink a powerful solution suited for a range of scenarios. Below, we explain how Flink is used in event-driven systems, streaming analytics, and data pipelines, with real-world examples and use cases. ## Apache Flink in Event-Driven Applications In an **event-driven application**, systems react to events (such as user actions, sensor readings, [financial transactions](https://www.ververica.com/banking), etc.) as they occur. Instead of processing data in periodic batches, event-driven architectures continuously stream events through processing logic to trigger immediate actions or downstream processes. Apache Flink is often the “brain” in these architectures, ingesting event streams, evaluating conditions or patterns, and responding in **milliseconds**. Flink’s **low-latency, stateful stream processing capabilities** allow developers to implement **complex event logic** that remembers past events, correlates multiple event streams, and maintains counters or profiles, all in real time. **Why Flink for Event-Driven Architecture?** Event-driven systems require processing incoming events reliably and quickly. Flink provides exactly-once processing guarantees (meaning an event will never be lost or processed twice, even in failures) and can scale to millions of events per second with horizontal clustering. Its event-time processing ensures out-of-order events can be handled correctly (which is important in distributed event sources), and its state management lets you create sophisticated **event pattern detectors** and alerting rules. In short, Flink can detect **anomalies**, trigger **alerts**, or update **aggregates** the moment relevant events happen. This makes it ideal for mission-critical applications where immediate reaction can prevent loss or capitalize on an opportunity. ### Key Event-Driven Use Cases Powered by Flink: - **Real-Time Fraud Detection:** Fraud detection is a significant challenge across finance, e-commerce, insurance and more, requiring identification of fraudulent transactions hidden in vast data streams. Apache Flink enables organizations to analyze events like credit card swipes, logins, or money transfers as they happen, comparing against patterns of legitimate behavior to catch anomalies. By correlating event streams and applying machine-learning models or rules, Flink can flag suspicious activities in milliseconds, helping businesses **prevent fraud losses before they occur.** For example, Flink might detect a sequence of transactions or account events indicative of fraud and immediately block the transaction or alert security teams. Real-world users leverage Flink for fraud detection to adapt to evolving fraudster tactics with millisecond anomaly detection. _(See_ [_Fraud Detection use case_](https://www.ververica.com/banking/fraud-detection) _for more details.)_ - **Anomaly Detection & Security Alerts:** Beyond financial fraud, many systems need to detect **anomalies** in streaming data – from IT security breaches to equipment sensor faults. Flink’s stream processing can compute statistics or apply ML models on sliding windows of events to recognize unusual patterns instantly. For instance, in cybersecurity (SIEM systems), Flink can ingest logs from servers, network devices, etc., and perform **real-time complex event processing** to identify signs of an attack or policy violation. It processes security events at ingestion time to enable immediate detection of anomalies or intrusions. This proactive approach means potential threats trigger alerts or automated responses within seconds, rather than hours or days after analysis. Flink’s **event-driven architecture** excels here by correlating events across disparate sources in real time, something traditional batch-based monitoring can’t easily do. _(See_ [_Security Information and Event Management (SIEM) use case_](https://www.ververica.com/case-study/xm-cyber) _for more details.)_ - **Rule-Based Alerting:** Many business applications rely on **rule-based alerting**, where certain combinations of events or threshold conditions should trigger a notification or action. Apache Flink includes a library for **Complex Event Processing (CEP)** that lets developers define event pattern rules (for example, "if Event A is followed by Event B within 2 minutes, and Event C has not occurred, raise an alert"). Flink will continuously look for these patterns in the event stream and fire alerts the moment a rule is satisfied. This is used in scenarios like monitoring transactions for specific sequences (e.g., multiple failed logins followed by a high-value transaction) or IoT sensor networks (e.g., if temperature > X and pressure > Y within 5 seconds, signal an alarm). Because Flink’s CEP rules can be updated and evaluated on live data, businesses can respond to operational conditions instantly with automated workflows. For example, a Flink application can send a real-time notification to operators when certain business process steps are delayed or abnormal. Flink’s ability to trigger notifications and alerts on defined patterns in real time has been used to implement dynamic **rule-based alerting** in fraud prevention systems and others. _(See_ [_Real-Time Fraud Detection Using Complex Event Processing_](https://www.ververica.com/blog/real-time-fraud-detection-using-complex-event-processing) _for more details.)_ - **Real-Time Business Process Monitoring:** Companies often use Flink to monitor end-to-end **business processes** by tracking events from multiple systems that make up a workflow. Consider an e-commerce order fulfillment process: an order placed event, a payment processed event, a warehouse shipment event, delivery confirmation, etc. These might each originate from different services or databases. Flink can ingest all these event streams and join or correlate them by Order ID (maintaining state for each active order), thereby reconstructing the live status of each order through its lifecycle. If a step is missing or delayed (e.g., payment received but no shipment event after X minutes), Flink can detect the discrepancy and raise an alert or initiate a compensating action. This kind of **business process monitoring** ensures visibility into complex workflows in real time. It also provides live metrics (throughput, latency per step, etc.) for operational dashboards. By leveraging Flink for such monitoring, organizations gain a live pulse of their business operations and can react quickly to bottlenecks or failures. In essence, Flink enables a **digital twin of your business** processes via event streams, facilitating agility and responsiveness. Apache Flink’s strength in event-driven scenarios lies in its ability to **process incoming events immediately and contextually.** It keeps relevant state (such as user session data, running counts, or machine learning features) in memory, so each new event can be analyzed in light of prior events. This allows applications like fraud detection to not just inspect one transaction in isolation, but compare it to historical patterns or combine it with related events (e.g. multiple accounts logging in from the same device) before deciding to flag it. As a result, organizations can move from reactive post-hoc analysis to **proactive event-driven action**, catching problems in-flight. Real-time streaming with Flink lets you “stop suspicious activity in its tracks”, whether that’s halting a fraudulent transaction or preventing damage to equipment after anomaly signals. In summary, Flink provides the real-time nervous system for event-driven applications, powering use cases from fraud prevention and IoT monitoring to user engagement triggers, by responding to events as they happen. ## Apache Flink for Real-Time Data Analytics Applications Another major use of Apache Flink is in **real-time data analytics**, which is deriving insights and making data-driven decisions continuously from streaming data. Traditional analytics often meant pulling data into a data warehouse and running batch queries or reports (which could take hours or days). With Flink’s stream processing, analytics can happen **on the fly**, allowing dashboards, algorithms, or users to see up-to-the-moment information and trends. Flink essentially turns queries into long-running applications that constantly update their results as new data arrives, enabling what’s known as **streaming analytics** or real-time business intelligence. **Streaming Analytics vs. Batch Analytics:** In batch analytics, you might compute yesterday’s sales total or last hour’s sensor averages after collecting all the data. In contrast, Flink lets you compute metrics continuously – e.g., a running count of sales in the last 5 minutes that updates every second, or an alert if sensor readings deviate from the norm right now. This not only reduces the latency of insight (from hours to seconds) but also allows automated actions in response to analytics (closing the loop from analytics to operation). Flink achieves this via features like event-time windows (which aggregate over sliding or tumbling time periods), incremental aggregation algorithms, and [Flink SQL](https://www.ververica.com/developers/apache-flink-sql-past-present-and-future)/Table API for writing continuous queries. Common patterns where Flink is applied in analytics include **real-time dashboards**, **alerts on KPI thresholds**, **trend detection**, and **personalization**. Because Flink can join multiple data streams, it’s possible to enrich live data with reference data (e.g., joining user events with customer profiles) to get deeper insights in real time. ### Key Analytics Use Cases Leveraging Apache Flink - **Dynamic Pricing Optimization:** In dynamic pricing, businesses adjust prices or rates in real time based on current demand, supply, and other factors. This is common in industries like ride-hailing, food delivery, airline tickets, and e-commerce flash sales. Apache Flink enables dynamic pricing by continuously analyzing streams of events such as user requests, driver availability, inventory levels, competitor pricing, and even external signals like weather or traffic. With Flink, raw data from these streams is **transformed instantly into actionable insights that guide pricing in real time**. For example, a ride-sharing platform can use Flink to calculate surge pricing multipliers per city in real time by analyzing trip request rates and driver supply on a minute-by-minute basis. As conditions change (a sudden spike in demand or a big drop in available drivers for example), the Flink job updates the pricing recommendations immediately. The outcome is optimal resource allocation, more balanced supply-demand matching, and improved revenue – [studies show](https://www.mckinsey.com/industries/retail/our-insights/how-retailers-can-drive-profitable-growth-through-dynamic-pricing) companies using dynamic pricing see an average 5% increase in profit margin per product or service sold. Flink’s ability to integrate historical data (e.g., typical demand patterns) with live feeds ensures pricing decisions are data-driven and responsive rather than static or intuition-based. _(See_ [_Dynamic Pricing use case_](https://www.ververica.com/retail) _for more details.)_ - **Customer 360 and Personalization:** Modern enterprises strive for a **360-degree view of the customer**, aggregating all customer interactions and data points to better understand and serve each person. Apache Flink is a key technology in building these real-time customer analytics systems. By streaming in data from various touchpoints (website clicks, mobile app events, purchases, support tickets, social media, etc.) and joining it with historical customer data, Flink helps create a live unified profile for each customer. This unified stream can drive personalized recommendations, targeted marketing, or tailored user experiences in real time. For instance, an online retail platform might use Flink to continuously update a customer’s profile with what they're browsing and buying, then use that to recommend products or content immediately (rather than in the next day’s email). With every interaction, Flink **turns customer data into valuable, actionable insights** for the business. The result is a **data-rich, holistic view** of each customer that enhances decision-making and personalization strategies on the fly. Companies using Flink to gain a Customer 360 can engage customers in real time with context-aware offers, detect churn signals as they happen, or route high-value customers to special handling, all based on streaming analytics. Additionally, Flink’s stateful processing allows applying business rules or machine learning models per customer in real time (for example, calculating a live customer lifetime value or propensity score as events stream in). _(See_ [_Customer 360 use case_](https://www.ververica.com/banking/customer-personalization) _for more details.)_ - **Real-Time Dashboards & Operational Analytics:** Many organizations use Flink to power live dashboards and monitor key performance indicators (KPIs) continuously. Instead of waiting for end-of-day reports, operations teams can watch metrics updated second-by-second. For example, Flink might drive a dashboard of website activity (like current active users and clicks per second), application performance (real-time error rates, latency percentiles), or business metrics (orders per minute, revenue today vs. yesterday in real time). Flink jobs can aggregate event streams and output results to a dashboarding system or database that backs a visualization. Because Flink windows and aggregations can maintain summaries with each new event, the dashboard shows trends and anomalies instantly. This real-time visibility enables faster response to issues and data-driven business decisions. If a metric goes out of bounds, teams can be alerted immediately (tying into the rule-based alerting mentioned earlier). **Business process monitoring** (discussed above) is a special case of this, where the dashboard might reflect the state of each process instance. Another example is Internet of Things (IoT) analytics: Flink can analyze data from thousands of IoT sensors in real time to track metrics like machine utilization, environmental readings, or fleet vehicle locations, giving companies up-to-the-moment insight into their operations. (See [Using Real-Time Data to Optimize the Electric Vehicle Industry](https://www.ververica.com/blog/driving-efficiency-using-real-time-data-to-optimize-the-ev-industry) blog post) - **AI-Driven Analytics and Recommendations:** Apache Flink is often used in conjunction with **machine learning (ML)** to enhance analytics. In streaming recommendation engines (like content or product recommendations on platforms), Flink can continuously update recommendations based on recent user behavior. For instance, Flink might maintain a running model of trending topics or popular items and immediately suggest them to users while trends are hot. In financial trading analytics, Flink can incorporate ML models to detect market anomalies or predict price movements on streaming tick data. What sets Flink apart is its ability to handle **both historical and real-time data** together – you might feed models with historical features but update those features with live data as time progresses. Moreover, Flink’s integration with ML libraries (e.g., Flink ML or external APIs) allows scoring of events in real time (such as classifying an event or forecasting a metric) and then acting on that prediction within the stream. A concrete example is **real-time anomaly detection using ML**: Flink can calculate anomaly scores for each incoming reading (like credit card transaction or sensor value) via an ML model, and if the score indicates an outlier, immediately flag it. This blends analytics with automated action. _(See_ _AI/ML use case_ _for how Flink supports real-time machine learning pipelines.)_ Apache Flink’s rich analytic capabilities (event-time windows, aggregations, joins, SQL, machine learning algorithms, etc.) make it a versatile engine for extracting insight from streams. It essentially brings the power of SQL and dataframes to unbounded live data. Importantly, Flink’s **stateful streaming** means you can do things like join a stream with a reference dataset or keep running counts indefinitely – enabling analytics that are not possible with stateless systems. Many companies have built **real-time analytics platforms** on Flink that feed both internal users (through live metrics and alerts) and external users (through personalized content or adaptive experiences). By processing data continuously, organizations can move from after-the-fact analysis to continuously updated intelligence and gain a competitive edge by responding to information the moment it’s available. One example of this is Chinese e-commerce giant Alibaba, who uses Apache Flink to power search and recommendation use cases, processing **over one trillion events per day** and over **470 million transactions per second** during peak events – a [testament to Flink’s ability to handle extreme scale](https://www.alibabacloud.com/blog/why-did-alibaba-choose-apache-flink-anyway_595190) in streaming analytics. ## Apache Flink in Data Pipelines and Streaming ETL The third major arena for Apache Flink is building **data pipelines**, in particular, streaming **ETL (Extract, Transform, Load)** processes that move and transform data in real time. ETL traditionally uses batch processing tools. During the process, data is extracted from sources, stored, then transformed and loaded into targets (like data warehouses or data lakes) in periodic batches. This results in high latency (data is hours or days old by the time it’s available for use) and often complex architectures with staging tables or multiple processing layers. Flink offers a better approach with continuous ETL. With Flink, data flows from sources to targets with transformations applied on the fly, so that **data is available in seconds instead of days**. As a result, organizations can simplify architectures by eliminating unnecessary interim storage and get up-to-the-moment data in their analytical systems. **Streaming ETL and Data Integration:** Apache Flink excels at integrating data from disparate systems in real time. Via **Change Data Capture (CDC)**, Flink can consume streams from databases, including every insert/update/delete as they occur, message queues like Apache Kafka, sensors, log files, or APIs, and then transform and harmonize these streams before delivering to a sink, such as a data warehouse, data lake, search index, or another service. Because Flink pipelines run continuously, data is fed through as soon as it’s generated. This means no more waiting for nightly batch jobs, instead, reports and data-driven applications can use data that’s only seconds old. In scenarios where data is siloed across different systems, Flink can serve as the **unifying layer**, merging streams into one common data model. For example, a company can stream customer data from a Customer Relationship Manager (CRM), clickstream data from web logs, and transaction data from a billing system, all into Flink, which can join/enrich these in real time and load into a unified database or feature store. This process effectively creates a live **single source of truth**. One immediate benefit is that decisions are based on the latest information, not stale snapshots of older data. **Real-time ETL isn't just an upgrade, it's imperative for staying competitive and taking timely, informed actions in today’s instant-feedback world. **_(See_ [_ETL use case_](https://www.ververica.com/product/how-it-works)_to learn more.)_ Flink’s approach to streaming ETL addresses several limitations of traditional ETL: - **Ultra-Low Latency:** Traditional ETL processes data in batches, introducing delays since data must be collected and stored before processing. In contrast, **Apache Flink enables real-time ETL**, allowing applications to **ingest, transform, and act on data instantly without having to store it first.** This means as soon as an event occurs in a source system, it can be in the target system moments later. For businesses, this could mean having up-to-the-second inventory levels, or immediately propagating a new customer signup to all relevant systems. - **Simplified Architecture:** Legacy ETL pipelines often involve multiple stages (bronze, silver, gold data layers, etc.) and intermediate storage, which add complexity and cost. Flink’s **event-driven pipeline** can perform on-the-fly transformations and filtering, reducing the need for staging areas or multiple passes over the data. In other words, you can do **continuous transformation** as data streams through the pipeline, using one system (Flink) rather than a chain of tools. This simplifies data architecture and reduces maintenance. [Ververica’s Unified Streaming Data Platform](https://www.ververica.com/product) (which is 100% compatible with Flink) highlights that by removing batch layers and moving to streaming, you eliminate redundant storage and achieve more efficient, lean data pipelines. - **Scalability and Efficiency:** Flink is built to handle high volume streams and can scale out horizontally. Traditional ETL jobs often hit scalability issues or require expensive scaling of database engines to handle large batch jobs. Flink pipelines can run on commodity clusters and scale linearly for throughput. Moreover, continuous processing spreads out the workload over time (instead of peaking during batch windows), often leading to better resource utilization. Flink’s incremental processing also means it can handle late-arriving data or out-of-order events gracefully (via event time handling), ensuring correctness without needing complex reprocessing logic. - **Data Freshness and Quality:** By delivering data continuously, Flink ensures that downstream systems are always operating on **fresh data**. This significantly improves the quality of analytics and machine learning, which benefit from fresh, up-to-date information. For example, an analytics dashboard fed by a Flink pipeline will reflect the latest business conditions, instead of batch processing which will result in delayed insights. Similarly, a machine learning model retrained using a Flink pipeline can incorporate today’s data and adapt faster. (See [_Why Streaming ELT Fuels Next-Gen Machine Learning_](https://www.ververica.com/blog/why-streaming-etl-fuels-next-gen-machine-learning) blog post) ### Key Data Pipeline Use Cases with Flink: - **Real-Time ETL and Change Data Capture:** A classic use case is replicating database changes in real time to other systems. Apache Flink can connect to databases via Change Data Capture (CDC) connectors that emit every change event (like insert/update/delete). Flink jobs then stream these changes to a destination. For instance, you might capture changes from a production MySQL database and use Flink to stream them into a cloud data warehouse for search analytics. Along the way, Flink can transform the data (including filter columns, mask sensitive data, join with reference data, etc.). This pipeline ensures the target is continuously synced with the source with minimal lag (seconds instead of hours or days). Compared to hourly dumps or nightly ETL, this is a game-changer for data freshness. Many companies use Flink in this way to maintain real-time replicas of transactional data for reporting or to offload analytical queries from primary databases. Change Data Capture with Flink allows maintaining an up-to-date **data lake or data warehouse** without lengthy batch import jobs. - **Unified Data Streams for Analytics:** Apache Flink often sits between data producers and consumers as a real-time transformation layer. For example, imagine a data source receiving raw events like clicks or page views from a website. Downstream, there are multiple consumers, including one system that wants to count page views per minute, and another that needs to store events, while yet another wants to detect certain sequences for monitoring. Instead of each consumer doing redundant processing, a Flink job can consume the raw stream from data source, perform several transformations (for example: parse logs, enrich with geo-information, or filter irrelevant events), and then publish refined streams to new sinks for each use. This **streaming data pipeline** offloads common transformation logic to Flink and provides each consumer with a clean, processed stream. It’s essentially the streaming equivalent of an ETL workflow, but continuously running. The benefit is consistency, as all consumers see the same clean data, and low latency. Such pipelines are used in **customer analytics**, where events from multiple channels are unified and cleaned in real time, then fed to various teams/apps. [Ververica’s Unified Streaming Data Platform](https://www.ververica.com/product) emphasizes real-time **data movement and processing at millisecond latencies**, so you can make decisions and take action instantly on your data. - **Machine Learning Model Pipelines (Streaming AI/ML):** As organizations adopt AI and ML, data pipelines feeding those models become critical. Apache Flink is increasingly used to build **real-time ML** pipelines for both model training and model serving. For training, Flink can stream data into feature stores or directly into an online training process (for example, updating a model incrementally with new data, a concept known as online learning). This ensures models are always learning from the latest data rather than being retrained only on static batches. More commonly, Flink is used for **feature engineering in real time**. Flink can compute features from raw data streams (such as aggregating user behavior over a window, or joining a stream with another to get context) and feed those features to an ML model service. This enables **real-time predictions**, where as soon as an event arrives, the model can score it with up-to-date features. An example of this is real-time fraud scoring. When a transaction event comes in, a Flink job might aggregate the user’s past 5 minute spending, count of transactions, location info, etc. (features), then call a model to get a fraud probability, and finally act on that (block or allow), all within a second. Flink’s ability to handle the data plumbing for ML in real time keeps models effective and responsive. In fact, Ververica’s AI/ML use case highlights that Flink **processes data in real time, enabling you to ingest, process, and deliver insights to models as events occur**, ensuring your AI/ML systems always operate on up-to-date data. Moreover, Flink’s integration with **AI pipelines** means it can also take the outputs of models and trigger downstream actions (like automated alerts or decisions), enabling autonomous decisioning. For example, Flink can work with an “[Agentic AI](https://www.ververica.com/blog/outrun-fraudsters-with-agentic-ai-and-ververica)” approach where the system not only detects an anomaly but also automatically investigates and creates an alert or recommendation, using both real-time and historical data._(See_ _AI/ML use case_ _to learn how Flink supports live data for machine learning.)_ - **Event-Driven Microservices & Data Sharing:** In modern microservice architectures, there’s often a need to share data changes between services in real time (to avoid each service working in isolation on stale data). Flink can serve as the backbone for an **event-driven data mesh,** where each important data change is broadcast as an event and Flink pipelines route and transform these events to other services that need them. For instance, when a user updates their profile in a User service, a Flink pipeline can capture that event and immediately propagate the change to a Recommendation service’s database and to a Marketing analytics system. This pattern ensures eventual consistency across services in seconds, and Flink’s processing can handle retries, transformations, and ordering concerns centrally. Similarly, for **IoT data pipelines**, Flink can collect streams from devices, perform streaming analytics (including filtering and aggregating), and deliver processed streams to different consumers. For example, it can trigger alerts for anomalies (as discussed), update real-time digital twins dashboards, and store raw data in a cluster for later analysis, all simultaneously. The pipeline concept here is “write once, deliver to many” via transformations that Flink is uniquely positioned to create. In summary, Apache Flink enables building **real-time data pipelines** that are more efficient, simpler, and faster than traditional ETL pipelines. With Flink, **data is continuously in motion**, flowing from sources to destinations with minimal delay, and with all necessary processing done en route. This means businesses don’t have to choose between having up-to-date data and having complex processing, instead, Flink delivers both. ROI is expressed as more timely analytics, more responsive applications, and the ability to do things that were previously impractical, like constantly updating machine learning models or maintaining live replicas of data. Companies adopting Flink-based pipelines report improved agility as they can integrate new data sources quickly and react to new information immediately. In a world where data value decays over time, moving to real-time pipelines ensures you get the **maximum value from your data when it’s most relevant**, rather than after the fact. ## Conclusion Apache Flink has emerged as a leading technology for **stream processing** in the era of big data and real-time demands. By providing a platform for **event-driven, real-time computations** at scale, Flink enables organizations to transition from batch-oriented thinking to streaming-first architectures. Whether it’s powering complex event-driven applications (like catching fraud or triggering alerts within milliseconds), driving **real-time analytics** (like live dashboards, dynamic pricing, and personalized experiences), or streamlining **data pipelines** (real-time ETL and continuous machine learning data feeds), Flink delivers the speed, scalability, and reliability required. It abstracts away much of the difficulty of handling streams (such as fault tolerance, ordering, state management) and lets developers focus on business logic, whether that is defining event patterns, completing analytic aggregations, or creating transformation workflows. The impact of adopting Apache Flink is transformative. Businesses become more **responsive** to events (turning streams of data into instant action), more **insightful** (gaining visibility into live trends and behaviors), and more **unified** in their data infrastructure (with a single engine handling batch and stream processing). Case studies from industry leaders underscore Flink’s versatility, for example, financial institutions using Flink to detect fraud in real time, tech companies using it to deliver real-time customer 360 analytics, ride-sharing apps optimizing prices dynamically, and many enterprises overhauling legacy ETL with streaming pipelines for fresher data. In the broader landscape of stream processing frameworks, Flink is known for its **state-of-the-art architecture** and robust features (such as exactly-once semantics and event-time processing) that set it apart. It has a thriving open-source community and is the backbone behind commercial, enterprise-grade offerings including [Ververica's Unified Streaming Data Platform](https://docs.ververica.com/introduction/about-ecosystem/apache-flink/) that further simplify building streaming applications. For any organization looking to implement **event-driven architecture**, real-time analytics, or modern data pipelines, Apache Flink is a proven choice that can handle the demands of **stream processing with Apache Flink** at scale. Embracing Flink means your business can process, analyze, and act on NOW data, gaining a critical edge in today’s fast-paced, data-driven world. Ververica enhances Apache Flink with proprietary innovations like the [VERA](https://www.ververica.com/ecosystem-introduction/what-is-vera) engine (delivering up to 2x performance improvement) and [Streamhouse](https://www.ververica.com/product/streamhouse) (enabling real-time data warehouse capabilities directly on streaming data). These technologies, combined with our team's deep expertise as Flink's original creators, make Ververica the premier choice for organizations deploying Apache Flink at enterprise scale. ## Open Source Apache Flink vs Ververica Platform | Feature | Ververica Platform | Open Source Apache Flink | | --- | --- | --- | | Core Engine | Apache Flink + VERA engine (2x performance) | Standard Apache Flink | | Deployment | Fully managed cloud or self-managed with automation | Self-managed, manual setup | | Performance | Up to 2x faster with VERA optimization | Standard throughput | | Operations | Auto-scaling, monitoring, alerts included | Manual cluster management | | Support | 24/7 expert support from Flink creators | Community forums | | Enterprise Features | SSO, RBAC, compliance (SOC2, ISO 27001), SLAs | Basic features | | Best For | Production workloads, enterprise scale | POCs, small deployments, learning | ## FAQ ### What makes Apache Flink unique among stream processing frameworks? Apache Flink stands out because of its powerful real-time processing capabilities, especially its ability to handle stateful computations accurately with **exactly-once guarantees.** Flink efficiently processes both **real-time streaming data and batch data** in a unified system, making it highly versatile. Its advanced event-time handling ensures accurate results **even with out-of-order data** streams. ### When should you use Apache Flink vs. other streaming technologies? Apache Flink is ideal when your business needs **real-time insights** with guaranteed accuracy and consistency. It's particularly suited for **complex, stateful applications** such as fraud detection, real-time analytics, and real-time machine learning.  ### How do you optimize Flink jobs for low latency and high throughput? To optimize Flink jobs, consider tuning **task parallelism,** optimizing **state backends**, configuring **efficient checkpoint intervals**, and leveraging Flink’s **built-in windowing mechanisms.** Ensuring your data streams are partitioned evenly and using the right state storage options (like RocksDB) can also significantly enhance performance. ### What tools are available for monitoring Flink jobs and clusters? Several effective tools exist for monitoring Flink jobs and clusters, including **Flink's built-in web dashboard**, which provides comprehensive job and cluster metrics. Additionally, external monitoring systems such as **Prometheus**, **Grafana**, and commercial platforms like [**Ververica's Unified Streaming Data Platform**](https://www.ververica.com/product) offer advanced monitoring capabilities, real-time alerts, and detailed performance analytics. ### How does real-time data streaming with Flink drive business decision-making? Real-time data streaming with Flink enables immediate data analysis and actionable insights, allowing businesses to make faster, more informed decisions. This capability **improves responsiveness**, helps identify **opportunities** and threats as they occur, and **enhances customer experiences** through real-time personalization and timely interventions. --- title: "What is Apache Fluss" description: "Apache Fluss is a streaming-native lakehouse storage engine for Apache Flink. Achieve sub-second latency and stream-table duality at scale." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/what-is-apache-fluss" md: "https://www.ververica.com/ecosystem-introduction/what-is-apache-fluss.md" --- # What is Apache Fluss™? Learn more about Apache Fluss, the open-source, unified streaming storage layer designed to optimize real-time data processing. ## What is Apache Fluss? Apache Fluss is an open-source streaming storage layer built for real-time analytics with Apache Flink. It stores streams in a columnar format, so jobs read only the columns they need, supports primary-key tables with changelogs, and tiers older data to lake formats such as Apache Paimon or Apache Iceberg. Fluss is a top-level project of the Apache Software Foundation. > **INFO:** Apache Fluss is an open-source, unified streaming storage layer designed to optimize real-time data processing with Apache Flink®. It bridges the gap between streaming and analytical storage by providing a single, high-performance platform for both real-time updates and historical queries on data. In order to get immediate insights from ever-growing data volume, modern enterprises face a constant challenge: **how to efficiently store and query vast streams of information in real-time, while also retaining the ability to perform deep historical analysis**. Traditional architectures often consist of disparate systems for messaging, stream processing, and long-term storage. As a result, these separate systems introduce complexity, latency, and significant operational overhead. This is precisely the critical gap that Apache Fluss aims to close. Fluss is an open-source project that acts as a unified streaming storage layer, and is built for next-generation data analytics and to revolutionize real-time data processing with Apache Flink. Fluss is built in column format instead of log format, and fills a missing gap in streaming technologies. It is a streaming storage layer that resembles a data warehouse while also being native to Flink. It allows decisions based on both historical and current data at the millisecond level because you don’t have to re-read the entire data set to get the relevant data out each time. ## Why Fluss and Columnar Streaming Reads Are Essential for Analytics Fluss is built as a columnar streaming reads, retrieving data by **columns** instead of rows. This approach significantly boosts performance in analytics scenarios, including: - **Targeted Data Access:** Only the necessary columns are read, reducing data transfer and processing time. For example, in a dataset with 50 columns, if analytics only require 3, columnar reads avoid loading irrelevant data. - **Optimized Compression:** Columnar formats store similar data types together, enabling better compression ratios and faster decompression during reads. - **Accelerated Query Speed:** Processes analytics queries like aggregations and filters directly on columns, leading to significant speedups compared to row-based processing. ## The Evolution of Data Storage: Why Fluss is Essential For years, the backbone of event-driven architecture and data [stream processing](https://www.ververica.com/ecosystem-introduction/stream-processing-with-apache-flink-beginners-guide) has largely relied on message queues like Apache Kafka®. However, these systems are not designed to serve as a primary analytical storage layer. This limitation creates a significant bottleneck for applications demanding both high-throughput writes and low-latency analytical reads directly on streaming data. Challenges include: - **High Network Costs and Bottlenecks:** Kafka's architecture often necessitates extensive data movement across networks, **leading to considerable infrastructure costs and performance issues**, particularly when scaling for comprehensive real-time analytics. - **Missing Columnar Streaming Storage:** There is a gap in the ecosystem for a streaming storage solution that natively supports a columnar format, which is essential for fast analytical queries and efficient data compression. Existing solutions struggle to transform to meet these rigorous needs. - **Complex and Disconnected Pipelines:** Integrating message queues, stream processing frameworks like Apache Flink, and separate OLAP (Online Analytical Processing) systems for analysis often results in complex, multi-layered pipelines that are difficult to manage, debug, and optimize. Recognizing these longstanding challenges, a team of experts with deep knowledge of Apache Flink began work on a new project to create a dedicated streaming storage layer that is tuned for today's stream processing demands. This effort created Fluss, which is now a top-level project of the Apache Software Foundation. Fluss is named from the German word for 'river,' symbolizing the continuous data flow it's built to handle. Fluss marks a significant advance toward a fully unified platform for both batch and streaming data, because it allows for seamless data management from the point of ingestion through to when that data is analyzed. To learn more, read the blog: "[Fluss: Unified Streaming Storage For Next-Generation Data Analytics.](https://www.ververica.com/blog/introducing-fluss)" ## A Unified Streaming Storage Solution As a high-performance, scalable, and fully integrated storage system, Fluss is designed to power real-time analytics. It combines the best attributes of streaming and analytical storage, providing a unified layer that eliminates the need for separate message queues and additional OLAP systems in many analytical workflows. At its core, Fluss: - **Offers Unified Batch and Stream Processing:** Fluss provides a singular platform that handles both batch and streaming data seamlessly. This integration is crucial for optimizing infrastructure for sophisticated AI, ML, and analytical workloads, allowing businesses to perform efficient historical processing alongside live data streams. - **Bridges the Gap for Real-time Analytics:** Fluss directly addresses the shortcomings of traditional architectures by offering a storage layer optimized for continuous updates and lightning-fast analytical queries on streaming data. This is particularly beneficial for applications where low latency is paramount. - **Streamlines Data Pipelines:** By integrating natively with Apache Flink, Fluss simplifies complex data pipelines. **It removes the necessity for intermediate Kafka topics, reducing infrastructure costs,** enhancing scalability, and improving overall performance for high-throughput, low-latency analytics. To meet use cases that rely on fast and efficient analytics (like powering dashboards, detecting anomalies, or training machine learning models), **columnar data formats** are essential. - Kafka’s **log-based design** is suited for transporting events but falls short when deep analysis or rapid insights are required. - Systems like **Fluss** fill this gap by offering **columnar streaming reads**, enabling immediate, high-performance access to data for analytical workloads while maintaining compatibility with streaming use cases. In short, while Kafka excels as a message broker for streaming events, its **row-based log format** limits its utility for analytics. For analytics at scale and speed, a columnar storage and processing layer like Fluss becomes crucial. The key differentiators Fluss offers are: - Supports large scale data in motion and allow that data to flow faster (just like a river) - Bridges compute and data lakes - Delivers data faster, with less delay and millisecond level latency **With Fluss, the line between storing data and making decisions with that data becomes indistinguishable.** Apache Fluss is production-ready, running internally with the team that developed it prior to it becoming an open-source project, demonstrating its readiness and robust capabilities. Its status as an Apache project underscores its commitment to open-source collaboration and community-driven development, further solidifying its role as a key stream processing framework component. This important step was announced in the recent blog: [Fluss Is Now Open Source.](https://www.ververica.com/blog/fluss-is-now-open-source) ## Core Features and Advantages of Fluss Fluss is packed with innovative features that set it apart as a premier streaming storage solution, making stream processing with Apache Flink even more powerful. Some of these features include: ### Sub-Second Latency - **What it Does:** Delivers sub-second latency for both streaming reads and writes. - **Why it Matters:** Critical for time-sensitive applications like monitoring systems and financial platforms, ensuring instant data availability for actionable insights. ### Stream-Table Duality with Updates and Changelogs - **What it Does:** Supports stream-table duality, providing changelogs for efficient updates and consistent data flow. - **Why it Matters:** Enables accurate real-time and historical insights within a unified system. ### Ad-hoc, Interactive Queries - **What it Does:** Provides a fully queryable storage layer for direct inspection of data. - **Why it Matters:** Simplifies debugging, reduces development complexity, and enables immediate access to live insights without additional processing layers. ### Unified Batch and Stream - **What it Does:** Seamlessly combines batch and streaming data processing. - **Why it Matters:** Optimizes infrastructure for AI, ML, and analytics workloads, allowing smooth transitions between historical and real-time processing. ### Projection Pushdown - **What it Does:** Optimizes streaming reads by fetching only the required fields. - **Why it Matters:** Minimizes data transfer, improves query performance up to 10x, and reduces network costs. ### Columnar Streaming Reads - **What it Does:** Stores and processes data in a columnar format for streaming reads. - **Why it Matters:** Improves compression efficiency and accelerates analytics, making it suitable for high-volume, real-time applications. ### Integration with Lakehouses - **What it Does:** Supports bi-directional communication with lakehouse tiered storage systems like [Apache Paimon](https://www.ververica.com/ecosystem-introduction/what-is-apache-paimon) and Apache Iceberg. - **Why it Matters:** Enables efficient initialization of streaming jobs from batch sources, and ensures seamless synchronization between batch and streaming data. ### Seamless State Initialization and Synchronization - **What it Does:** Fluss allows a streaming job to load state directly from batch sources. - **Why it Matters:** This capability enables seamless state initialization and synchronization between batch and streaming data, providing a truly unified data experience. ### Simplified Pipeline Architecture - **What it does:** With its native integration with Apache Flink, **Fluss eliminates the need for intermediate Kafka topics** and additional OLAP systems. - **Why it matters:** This dramatically simplifies pipeline architecture and **reduces infrastructure costs while enhancing scalability.** ## Fluss in the Ververica Unified Streaming Data Platform Fluss is not just a standalone technology; it's a critical component that enhances the capabilities of [Ververica's Unified Streaming Data Platform.](https://www.ververica.com/product) By providing a scalable, unified batch, and streaming data solution, Fluss addresses key challenges in real-time data processing and storage within the Ververica ecosystem. - **Enhancing Flink's Capabilities:** Fluss complements Apache Flink's powerful stream processing capabilities by providing the missing piece: **an optimized storage layer for real-time analytics.** This allows users to leverage Apache Flink for both computation and a direct, high-performance storage solution. - **Driving Real-Time Intelligence:** Fluss is designed to power complex real-time intelligence pipelines, enabling organizations to build highly responsive systems that react instantly to data changes. This is fundamental for modern event-driven architecture and applications that demand immediate insights. - **Cost Efficiency and Performance:** By streamlining the architecture and optimizing for real-time analytics, **Fluss significantly reduces operational overhead and infrastructure costs** compared to traditional multi-system approaches. Its sub-second latency and columnar reads contribute directly to superior performance, making data [stream processing](https://www.ververica.com/ecosystem-introduction/stream-processing-with-apache-flink-beginners-guide) more efficient than ever before. - **Foundation for AI/ML Workloads:** The unified batch and streaming capabilities, combined with real-time updates, make Fluss an ideal foundation for feeding fresh data to AI and ML models, accelerating model training and inference. In addition, Fluss equips businesses to stay competitive, adapt quickly, and future-proof investments through scalable, efficient, and modern data solutions. Key results include: ### Stay Competitive - **How it Helps:** Fluss gives you a technological edge by enabling faster, smarter decisions. - **What is the Impact:** Fluss processes and delivers data instantly, enabling your teams to respond to events like market changes, system alerts, or customer actions in real time. ### Adapt Quickly - **How it Helps:** Fluss provides seamless integration and fast data flow. - **What is the Impact:** You can adapt to changing market conditions and customer needs quickly. ### Future-Proof Your Investments - **How it Helps:** Fluss provides a scalable, adaptable, and forward-looking data infrastructure designed to support evolving technology trends and business needs. - **What is the Impact:** Fluss is part of an essential part of a forward-looking data strategy. By providing a platform that evolves with technological advancements and business demands, Fluss empowers organizations to make investments that deliver long-term value and adaptability. ### Simplify Your Data Architecture - **How it Helps:** By combining real-time and historical data handling into one platform, Fluss eliminates the need for separate systems - **What is the Impact:** reducing operational complexity and ensuring compatibility with future data demands. ### Seamless Integration With Emerging Technologies - **How it Helps:** By supporting advanced AI, machine learning, and analytics workloads, your business can leverage cutting-edge tools and methodologies - **What is the Impact:** No need to overhaul your data architecture. ### Built For Scalability - **How it Helps:** Fluss is designed to scale with growing data volumes. - **What is the Impact:** Fluss ensures that your business can handle increasing workloads as operations expand without sacrificing performance. ### Compatibility With Modern Data Ecosystems - **How it Helps:** Fluss integrates seamlessly with popular data lakehouse systems (like [Apache Paimon](https://www.ververica.com/ecosystem-introduction/what-is-apache-paimon) and Apache Iceberg). - **What is the Impact:** Fluss ensures that your business can adapt to the latest data management paradigms without disrupting existing workflows. ### Open-Source Flexibility With Commercial Support - **How it Helps:** Fluss is operated on open-source principles and flexibility in combination with Ververica’s expert help. - **What is the Impact:** Fluss ensures your business retains control and avoids vendor lock-in while benefiting from Ververica’s commercial enhancements and support. ### Optimize Resources - **How it Helps:** Delivers efficient data processing and reduced network costs - **What is the Impact:** Fluss provides a cost-effective solution, especially for data-intensive industries like finance, retail, and technology. ## Use Cases Transformed by Fluss The introduction of Fluss opens up new possibilities and transforms existing use cases across various industries: - **Real-time Dashboards and Monitoring Systems:** Powering dashboards with sub-second latency, providing up-to-the-minute views of business operations, system health, and key performance indicators. - **Streaming ETL and ELT:** Streamlining the [Extract, Transform, Load](https://www.ververica.com/product/how-it-works) processes by enabling continuous data movement and transformation directly within the streaming storage layer. - **Real-time Intelligence Pipelines:** Building sophisticated pipelines for [fraud detection](https://www.ververica.com/banking/fraud-detection), [personalized recommendations](https://www.ververica.com/banking/customer-personalization), anomaly detection, and [other](https://www.ververica.com/case-study) applications that require immediate decision-making based on live data. - **Streaming Data Warehouses:** Serving as the real-time data layer on the Lakehouse, allowing for continuously updated data warehouses that support both streaming-first architectures and traditional batch queries. - **Operational Analytics:** Enabling direct data inspection for debugging and troubleshooting live applications, reducing time-to-insight for operational teams. Fluss's ability to provide real-time updates makes it a natural fit for scenarios where data freshness is critical, ensuring that decisions are always based on the most current information. ## The Road Ahead The decision to open-source Fluss and donate it to the Apache Software Foundation marks a significant milestone. This commitment ensures that Fluss will benefit from community-driven development, fostering innovation and wider adoption. As the project continues to evolve, capabilities will expand and further solidify Fluss’s position as a leading solution for unified streaming storage. Ververica remains committed to supporting and contributing to Fluss, ensuring its continued integration and optimization within [Ververica’s Unified Streaming Data Platform.](https://www.ververica.com/product/deployment) Apache Fluss represents a pivotal innovation in the world of stream processing and data analytics. By addressing the critical need for a unified, high-performance streaming storage layer, it simplifies complex data architectures, **reduces costs**, and accelerates the delivery of real-time insights. Its deep integration with Apache Flink makes stream processing with Apache Flink more powerful and efficient than ever before. For organizations striving to build agile, event-driven architecture and unlock the full potential of their data stream processing capabilities, Fluss offers a clear path forward, streamlining the journey **from raw data to actionable intelligence**. ## FAQ ### How is Fluss different from Apache Flink? Flink is a stream processing engine for building and running pipelines, while Fluss is a storage/serving layer that keeps materialized views over streams for fast queries and lookups. ### What problems does Fluss solve? Fluss reduces stack complexity by replacing a patchwork of stream processor + message bus + cache + OLAP store with a single streaming data layer that provides consistent, sub‑second reads. ### Can Fluss handle late or out‑of‑order events? Yes, Fluss is designed to work with time‑aware processing (e.g., event‑time and watermarks) so derived tables stay consistent as late data arrives. ### What are typical use cases for Fluss? Common uses include real‑time dashboards, fraud/risk scoring, personalization, operational analytics, leaderboards, and fast dimension lookups for streaming joins. --- title: "What is Apache Paimon™?" description: "Explore Apache Paimon, the open-source, stream-native data lakehouse format. Learn how it powers real-time updates, unified analytics with Ververica." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/what-is-apache-paimon" md: "https://www.ververica.com/ecosystem-introduction/what-is-apache-paimon.md" --- # What is Apache Paimon™? Explore Apache Paimon, the open-source, stream-native data lakehouse format. Learn how it powers real-time updates, unified analytics with Ververica. ## What is Apache Paimon? Apache Paimon is an open-source, stream-native lake table format. Its log-structured merge tree lets Apache Flink write continuous updates, upserts and changelogs straight into tables on object storage, which Flink, Spark, Trino and other engines can then query. Flink CDC can stream database changes into Paimon tables as they happen. ## What is Apache Paimon? > **INFO:** Apache Paimon is an open-source, stream-native data lakehouse format. It's designed to bring real-time data stream processing capabilities directly to your data lake, enabling efficient updates, consistent changelogs, and powerful analytical queries on continuously changing data. It integrates tightly with Apache Flink®, allowing you to build unified pipelines for both streaming and batch workloads. In the world of big data, processing and analyzing vast amounts of information in real time is a key challenge. Traditional data systems struggle to handle both historical data and continuous data streams effectively. This is where Apache Paimon comes in. Paimon acts as a stream-native data lakehouse that allows for real-time updates directly within your data lake. For companies using Apache Flink to process data streams, Apache Paimon is a foundational tool that is included as part of Ververica's [Streamhouse](https://www.ververica.com/product/streamhouse) architecture. ## The Genesis of Apache Paimon Apache Paimon was created due to the recognition of a critical need in the stream processing framework landscape. While Apache Flink has evolved into a powerful unified engine for both batch and streaming data, a direct, queryable storage layer for intermediate and final tables in a streaming context was lacking. While powerful, Flink's "Dynamic Tables" aren't directly queryable, which limits the immediate accessibility of continuously updated data. As a result, FLIP-188 ("Introduce Built-in Dynamic Table Storage,") was proposed as an initiative that eventually evolved into Apache Paimon. The fundamental idea is both simple and profound: _provide Apache Flink with a robust storage layer that leverages a table format, allowing intermediate data in dynamic tables to be directly accessible and queryable._ This concept aligns perfectly with the expanding "Lakehouse" paradigm, which combines the flexibility and scalability of data lakes on affordable storage (like S3) with the structured querying and optimization typically found in data warehouses. While other technologies like Apache Iceberg™, Delta Lake, and Apache Hudi also embrace the Lakehouse approach, Paimon's unique strength lies in its **stream-first design.** This makes it exceptionally suitable for continuous updates and real-time analytics, as detailed in the recent blog: [Apache Paimon: The Streaming Lakehouse](https://www.ververica.com/developers/apache-paimon-the-streaming-lakehouse). Ververica integrated Paimon as a key ecosystem component, particularly in our Streamhouse architecture, because Paimon provides a robust, stream-native storage layer that significantly enhances Flink's capabilities for real-time data processing and advanced analytics. This strategic integration allows Ververica to offer comprehensive solutions for enterprises seeking modern, zero-trust-compliant data infrastructures. ## A Stream-Native Data Lakehouse Apache Paimon is an open-source table format designed to enable the construction of a real-time Lakehouse architecture with both streaming and batch operations. It combines the benefits of a data lake format with a Log-Structured Merge-tree (LSM) structure, bringing real-time streaming updates directly into the data lake. Key characteristics that define Apache Paimon include: - **Unified Data Lakehouse Storage:** Paimon serves as a single storage layer that bridges the gap between batch and streaming data. It offers the scalability and flexibility of data lakes alongside the structured querying and schema enforcement of data warehouses. This unification simplifies data architectures, reducing complexity and operational overhead. - **Real-Time Ingestion with CDC Support:** Paimon excels at real-time ingestion, with strong support for Flink Change Data Capture (CDC). This enables incremental updates, allowing data changes from operational databases to be continuously written to Paimon tables, ensuring data freshness. - **Unified Workloads (Batch and OLAP):** Paimon is optimized for both analytical queries (OLAP) and batch processing, making it versatile for diverse workloads within a single system. This means you can run long-running historical queries alongside real-time analytical dashboards. - **Tight Integration with Apache Flink:** Apache Paimon integrates tightly with Apache Flink. Flink can seamlessly use Paimon as both a source (to read data) and a sink (to write data). This deep integration empowers Flink jobs to process data in real time, incrementally update Paimon tables, and query those tables for immediate analytical insights. This makes stream processing with [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink) significantly more powerful and flexible. ## Core Features and Capabilities of Apache Paimon Apache Paimon’s design incorporates several powerful features that make it an ideal choice for data [stream processing](https://www.ververica.com/ecosystem-introduction/stream-processing-with-apache-flink-beginners-guide) and building a modern data lakehouse: - **Real-Time Updates with Primary Key Tables:** Paimon supports primary-key tables that enable real-time streaming updates of large amounts of data. This is crucial for maintaining data freshness, with queryable updates often available within minutes or even at sub-minute latency, depending on checkpoint intervals. Paimon offers various "merge engines" (like deduplicate, partial-update, aggregate, or first-row) to handle multiple records with the same primary key, providing flexibility in how updates are applied. - **Flexible Updates with Merge Engines:** A key strength of Paimon lies in its support for rich merge engines. This allows users to define how records are updated, whether by keeping the last row, performing partial updates (updating only specific columns), or aggregating records. This flexibility is vital for complex data stream processing scenarios. - **Change-Tracking Updates with Changelog Producers:** Paimon supports "changelog producers" that generate correct and complete changelogs from any data source. This is essential for downstream consumers to always see correct results and for building an accurate event-driven architecture. - **Append-Only Tables:** For use cases that only require data insertion (e.g., log data synchronization) and do not need update or delete operations, Paimon offers append-only tables. These tables provide large-scale batch and streaming processing capabilities efficiently. - **Data Lake Capabilities:** Inheriting the advantages of a data lake, Paimon provides low-cost storage, high reliability, and scalable metadata management. It supports features like Time Travel, allowing users to query previous versions of data by leveraging snapshots, and Full Schema Evolution, adapting to changes in data structure without disrupting pipelines. - **Query Data Skipping:** Paimon enhances query performance through indexes (like min/max) that filter irrelevant files, leading to faster data retrieval. - **Resource-Efficient Automatic Data Compaction:** To optimize read and write speeds, Paimon automatically and asynchronously combines smaller files into larger ones through a process called compaction. This prevents "small file" problems common in data lakes, resulting in faster queries and improved performance. - **Robust Upsert Support:** Paimon's use of LSM trees and various merge engines, including the powerful partial update merge engine, allows for efficient upserts (updates or inserts) directly on the Lakehouse. This can eliminate the need for costly streaming joins on primary keys, making streaming ETL more cost-effective. - **True Streaming Reads:** Paimon provides safeguard mechanisms and a consumer-id mechanism to ensure true streaming reads. This means downstream consumers can reliably track changes, even when data files expire or are deleted due to snapshot management. ## Apache Paimon in the Ververica Ecosystem Within the Ververica ecosystem, Apache Paimon plays a pivotal role in enabling a truly [Unified Streaming Data Platform](https://www.ververica.com/product). It seamlessly integrates with other key components like Apache Flink (the core processing engine) and Flink CDC (for real-time data ingestion). Foundation for Streamhouse: As described in the "[The Streamhouse Evolution](https://www.ververica.com/blog/streamhouse-evolution)", Apache Paimon is the streaming storage layer for Ververica's Streamhouse architecture. This combination allows for a single, cohesive platform for both real-time stream processing and historical analysis on the data lake. **Enhanced Flink Capabilities:** Paimon directly extends Apache Flink's capabilities by providing a persistent, transactional, and queryable storage layer for its dynamic tables. This allows Flink jobs to not only process data in motion but also to maintain and query the state of that data efficiently in a cost-effective data lake. **Building Real-Time Data Views:** Paimon is instrumental in building real-time data views. Data from various sources is ingested (often via Flink CDC) into Paimon tables, aggregated by [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink), and then made available for real-time visualization through BI tools or custom applications. This end-to-end streaming pipeline eliminates recurrent batch jobs, as elaborated in the Building Real-Time Data Views with Streamhouse blog. **Data Sovereignty and Governance:** Paimon aligns with Ververica's focus on data sovereignty and compliance by keeping data storage within the customer's cloud environment. It also enforces granular access policies and integrates with cloud-native security tools, ensuring that organizations retain full control over their data. **Scalability and Cost Efficiency:** Paimon is designed to handle massive-scale data workloads and supports hybrid or multi-cloud environments. Its optimization for streaming significantly reduces costs associated with traditional batch-processing architectures by leveraging cheap object storage while maintaining performance. **Developer-Friendly Design:** Paimon supports tools and APIs familiar to developers working with Apache Flink, making it easier to adopt within the Ververica Unified Streaming Data Platform and simplifying the operational complexity of managing data stream processing pipelines. ## Use Cases and Applications The capabilities of Apache Paimon open up a wide array of use cases, particularly in scenarios demanding both real-time data freshness and cost-effective storage: - **Real-Time ETL:** Transforming and loading data continuously into a data lake for immediate consumption. - **Real-Time Data Warehousing:** Building a data warehouse that is continuously updated with fresh data, enabling real-time analytics and reporting. - **Streaming Data Marts**: Creating specialized, continuously updated data marts for specific business functions. - **Feature Stores for AI/ML:** Providing fresh data features for machine learning models by continuously updating tables that serve as feature stores. - **Backfilling and Historical Analysis:** Combining real-time updates with the ability to query historical snapshots for auditing, debugging, or retrospective analysis. - **Change Data Capture (CDC) into Data Lake:** Leveraging Flink CDC to ingest changes from transactional databases into the data lake in real time for unified analytics. Apache Paimon is particularly beneficial for businesses looking to upgrade high-latency batch jobs to near real-time, assess the ROI of stream processing, or seamless migration of existing Lakehouse workloads to a streaming-first paradigm. ## Conclusion Apache Paimon represents a significant leap forward in data stream processing and data lake architectures. By providing a stream-native, unified storage layer for Apache Flink, it effectively bridges the long-standing gap between real-time and historical data. Its robust features, tight integration with Flink, and role within [Ververica's Streamhouse](https://www.ververica.com/product/streamhouse) architecture empower organizations to build highly scalable, cost-efficient, and responsive data pipelines. For businesses seeking to harness the full power of their streaming data, achieve true event-driven architecture, and unlock immediate insights from their data lakes, Apache Paimon stands as a pivotal component, leading the way towards a future where stream processing with [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink) on a data lake is not just a possibility, but an accessible and efficient reality. ## FAQ ### What makes Paimon stream-first vs Iceberg or Delta? Apache Paimon is designed for continuous, low-latency ingestion and updates with a streaming-first architecture, prioritizing real-time reads/writes, primary-key upserts, LSM-based compaction, and CDC-friendly changelogs, whereas Iceberg and Delta are optimized primarily for batch-heavy analytics and snapshot-driven workflows. Paimon’s focus on real-time stream compaction and freshness makes it better suited to high-frequency updates and millisecond-to-seconds decisioning, while Iceberg and Delta generally excel at large-scale batch analytics and broader snapshot/time-travel capabilities. ### How does Paimon work with Apache Flink (source/sink, CDC, streaming reads)? Paimon integrates deeply with Apache Flink as both a source and sink, supporting real-time ingestion (including Flink CDC) into primary-key or append-only tables and enabling true streaming reads via changelog producers and snapshot-aware mechanisms. This integration allows Flink jobs to write CDC streams (e.g., from databases or Kafka) into Paimon, perform incremental processing, and read back latest state or changelogs continuously with low latency for operational and analytical use cases. ### Does Paimon support primary keys, upserts, partial updates? Yes. Paimon supports primary-key tables with multiple merge engines (deduplicate, partial-update, aggregate, first-row), enabling efficient upserts and flexible column-level updates; it also supports upsert mode for non-PK tables via upsert keys and optional sequence fields for deterministic merges. The partial-update merge engine lets multiple messages incrementally complete a record under the same primary key, and streaming queries can combine partial-update with specific changelog producers to ensure correctness. ### What latency can Paimon achieve for streaming reads/writes? Paimon is engineered for low-latency streaming pipelines, achieving near-real-time ingestion and consumption when paired with Flink, with latency driven by checkpoint intervals, compaction settings, and changelog configuration; its architecture targets lower streaming latency than batch-first formats due to stream-optimized write/read paths and LSM-based compaction. In practice, deployments use Paimon’s Flink connector metrics to monitor source and sink latency and tune for sub-minute to seconds-level end-to-end freshness in continuous workloads. ### What engines query Paimon tables (Flink, Spark, Trino, StarRocks, Doris)? Beyond Apache Flink, Paimon tables can be queried by Apache Spark, Trino, StarRocks, Apache Doris, and Apache Hive, enabling a multi-engine lakehouse where streaming updates are immediately consumable across SQL and OLAP engines. This broad interoperability lets teams use Flink for ingestion and incremental processing while serving analytics via Spark, Trino, StarRocks, or Doris without data duplication. --- title: "What is Streamhouse?" description: "Streamhouse unifies batch and stream processing on data lakes, enabling real-time insights and cost-effective analytics for businesses." lastUpdated: 2026-10-14T09:40:12.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/what-is-streamhouse" md: "https://www.ververica.com/ecosystem-introduction/what-is-streamhouse.md" --- # What is Streamhouse? Streamhouse unifies batch and stream processing on data lakes, enabling real-time insights and cost-effective analytics for businesses. ### Bridging the Gap Between Real-Time and Historical Data > **INFO:** Streamhouse, a streaming lakehouse concept introduced by Ververica, unifies batch and stream processing on a data lake. It combines Apache Flink® for compute with Apache Paimon™ for stream-native storage to deliver near real-time results, cost-efficiency, and simplified data architectures. Essentially, it allows businesses to leverage the benefits of a data lake (cost, scalability) with the speed and real-time capabilities of stream processing, making current and historical data immediately accessible for analytics and event-driven architecture. Businesses are constantly seeking more efficient and cost-effective ways to harness their vast amounts of data. The traditional divide between batch processing for historical data and stream processing for real-time insights often leads to complex, siloed, and expensive data architectures. This is where Streamhouse, also known as Streaming Lakehouse, emerges as a transformative solution. At its core, Streamhouse is Ververica's innovative approach that unifies batch and stream processing, allowing organizations to achieve near real-time results on their data lake while maintaining cost-efficiency and simplifying their data infrastructure.With Streamhouse, businesses can get the best of both worlds: accessing the power and cost-effectiveness of their traditional Data Lakehouse at the speed of real-time streaming. With Streamhouse, you can fill the gap between traditional streaming and batch architectures, getting stream processing capabilities while maintaining (near) real-time results on the Data Lake. ## Why Streamhouse? The Evolution of Data Architectures. For years, organizations have grappled with the challenges of managing increasingly large and diverse datasets. The rise of machine learning (ML), artificial intelligence (AI), and the escalating demand for real-time decision-making intensify the need for immediate data insights. Traditionally, this led to two distinct data paradigms: - **Data Warehouses:** Optimized for structured data and complex analytical queries, but often expensive and not well-suited for unstructured data or real-time ingestion. - **Data Lakes:** Excellent for storing vast amounts of raw, diverse data at low cost, but typically required separate processing engines for different data types and lacked strong ACID (Atomicity, Consistency, Isolation, Durability) guarantees. - **Real-Time Streaming Architectures:** Built for low-latency processing and immediate insights, essential for event-driven architecture, but can be expensive and resource-intensive to maintain at scale. While Data Lakehouses bridge the gap between data warehouses and data lakes, offering cost-effective storage with analytical capabilities, they still primarily focus on batch processing and struggle to deliver real-time insights effectively. This creates a significant void: businesses need a solution that seamlessly unifies real-time and batch processing without compromising on performance, cost, or complexity. This unaddressed need is the key driver behind the Streamhouse evolution. Ververica recognizes that to truly empower businesses, a solution is required that provides fast, fresh data while leveraging the cost benefits of data lakes. Streamhouse capabilities address these limitations, simplifying architecture, reducing operational complexity and infrastructure costs, and empowering businesses to harness the full potential of their data. As explored in the [Streamhouse Evolution blog](https://www.ververica.com/blog/streamhouse-evolution), it is about moving beyond the "deadlock" of separate systems to a cohesive, unified platform. ## What is Streamhouse? The Core Concept The Streamhouse concept is straightforward yet revolutionary: it combines the best aspects of traditional Data Lakehouses with the agility and speed of real-time streaming. It's designed to process and analyze vast amounts of data in real-time while simultaneously allowing for deep analytical queries on historical data, all within a single, integrated platform. At its technical heart, Ververica’s Streamhouse capabilities leverage a powerful combination of open-source technologies: - **Apache Flink for Unified Compute**: Apache Flink is a powerful Open Source stream processing framework for data stream processing and distributed stateful computations on large-scale data streams. In Streamhouse, Flink serves as the unified compute engine, capable of processing both continuous streams and historical batch data. This allows for flexible and powerful stream processing with Apache Flink. - **Flink Change Data Capture (CDC) for Unified Ingestion**: Flink CDC enables efficient and real-time capture of data changes from various operational databases (like MySQL or Postgres) directly into the Streamhouse. This ensures that data is ingested and made available for processing almost instantaneously. - **Apache Paimon for Unified Lakehouse Storage**: Apache Paimon provides a streaming storage layer that allows Flink to perform stream processing directly on the data lake. Paimon offers ACID transactions, schema evolution, and efficient data versioning, bringing data warehouse capabilities to the low-cost, scalable data lake. Its role is crucial in enabling the unified approach to data management within Streamhouse. This powerful synergy allows Streamhouse to bridge the cost/latency gap, providing near real-time results on the data lake by merging the cost-efficiency of data lakes with the speed of real-time data. ## Key Benefits Ververica’s Streamhouse is part of the Unified Streaming Data Platform, and features several key value propositions that make it a compelling technology to address modern data challenges: - **Significant Latency Improvement**: Streamhouse offers a substantial reduction in latency, transforming high-latency batch jobs (which might take days) to near real-time results (within minutes). This dramatic improvement is crucial for businesses that rely on fresh data for timely decisions. - **Flexibility with Flink SQL**: By leveraging Flink SQL, users gain the flexibility to seamlessly transition between Streamhouse capabilities and pure real-time streaming based on their evolving business requirements. This adaptability ensures that the solution can grow and change with an organization's needs. - **Cost-Effectiveness and ROI Assessment**: Streamhouse provides a more efficient, cost-effective, and low-risk method for system upgrades. It allows businesses to assess suitability and ROI by running operations and gauging results before committing to full real-time streaming, offering a balanced approach between cost and latency. - **Seamless Migration**: The unified Flink engine and Flink SQL facilitate seamless migration, minimizing the effort involved in upgrading from traditional Lakehouse architectures to Streamhouse and potentially to full real-time streaming pipelines. - **Optimized Performance and Resource Efficiency**: Streamhouse can significantly cut costs compared to always-on real-time streaming for use cases where minute-level data latency is acceptable. Furthermore, employing streaming preprocessing within Streamhouse yields **five times better write** performance and **eight times better query performance** compared to traditional batch Lakehouses.. - **Enhanced Data Freshness:** By continuously processing and updating data on the data lake, Streamhouse ensures that analytical queries always reflect the freshest available information, making it an ideal choice for data stream processing needs. - **Simplified Architecture**: Consolidating batch and stream processing into a single platform reduces architectural complexity, making systems easier to build, maintain, and scale. ## Common Data Processing Patterns Enabled by Streamhouse Because Streamhouse combines the capabilities of Apache Flink, Flink CDC, and Apache Paimon, it enables a variety of crucial data stream processing patterns directly on the data lake. These patterns are designed to provide near real-time results and significantly reduce engineering maintenance efforts: - **Event Deduplication:** Eliminating duplicate events is critical for data quality. Streamhouse achieves this using Apache Paimon's deduplicate and first-row merge engines, ensuring data integrity. - **Table Widening**: The partial-update merge engine in Paimon allows users to update specific columns of a record through multiple updates without resorting to expensive streaming joins, streamlining data enrichment processes. - **Out-of-Order Event Handling:** Real-world data often arrives out of sequence. Streamhouse addresses this by allowing users to specify sequence fields and sequence groups, guaranteeing data correctness even when events arrive late. - **Automatic Aggregations:** For business analysis, Streamhouse supports automatic aggregations, making it easier to derive summary insights from large datasets. - **Time Travel Capabilities:** Leveraging snapshots and tags, Streamhouse allows users to query previous versions of data, which is invaluable for auditing, debugging, and historical analysis. - **CDC Data Lake Ingestion with Schema Evolution:** Streamhouse provides robust integration with CDC connectors for various databases, automatically handling schema changes as data evolves, ensuring seamless data ingestion into the data lake. - **Data Enrichment with Lookup Joins:** Streamhouse supports data enrichment by building RocksDB indexes within Paimon for improved performance and allowing asynchronous lookups with retries for late-arriving records. ## Building Real-Time Data Views with Streamhouse One of the most compelling applications of Ververica’s Unified Streaming Data Platform Streamhouse capabilities is its ability to facilitate the creation of real-time data views. This is achieved through a streamlined, three-stage pipeline: ingestion, aggregation, and visualization. - **Ingestion:** Data from diverse sources, including message queues like Kinesis and Kafka, or relational databases via Flink CDC, is efficiently ingested into the Streamhouse environment. This data is then stored in Paimon tables. Notably, "Append Only" Paimon tables can even replace message queues for intermediate data ingestion, offering a more persistent and queryable storage layer. - **Aggregation:** Once ingested, data is joined and processed to build aggregated data tables. For instance, you can calculate revenue per country by joining order and customer data. Paimon's features, such as merge-engine=aggregation and changelog-producer=full-compaction, enable automatic aggregation and efficient change log management, ensuring that your aggregated views are continuously updated. - **Visualization:** The processed and aggregated data is then presented to users in real-time. This can be achieved by integrating with popular business intelligence (BI) tools or by utilizing the Paimon Java API to build custom web applications that stream updates continuously, providing truly live dashboards and reports. The benefits of using Streamhouse for real-time data views are extensive, including continuous real-time processing, efficient storage on cheap services like S3 with data warehouse properties, simplified data ingestion via Flink CDC, enhanced performance thanks to Ververica's VERA engine (which offers 2x better performance for real-time data processing), and remarkable flexibility in visualization. This process is detailed in the [Building Real-Time Data Views with Streamhouse blog](https://www.ververica.com/blog/building-real-time-data-views-with-streamhouse). ## Streamhouse and the Broader Data Landscape The Streamhouse concept represents a significant step towards a truly unified data architecture. By bringing stream processing capabilities directly to the data lake, it eliminates the need for separate systems for real-time and historical analytics. This convergence is vital for organizations moving towards an event-driven architecture, where immediate reactions to changes in data are paramount. The integration of Apache Flink as the compute engine and Apache Paimon as the storage layer positions Streamhouse as a robust and scalable stream processing technology and addition to the Unified Streaming Data Platform. It allows businesses to gain instant access to insights, power real-time applications, and conduct deep historical analysis without the complexities and costs associated with maintaining disparate systems. ## Conclusion Streamhouse is more than just a new buzzword; it's a practical and powerful technological concept for the modern data challenges. By combining the strengths of Apache Flink for stream processing with the cost-efficiency and analytical capabilities of data lakes via Apache Paimon, Ververica has crafted a [platform that truly unifies data stream processing](https://www.ververica.com/product). It enables businesses to move from batch-oriented insights to a world of real-time decision-making, all while simplifying their data infrastructure and optimizing costs. For any organization looking to achieve true data agility and harness the full potential of its streaming and historical data, exploring the components of the Unified Streaming Data Platform, including Streamhouse is an essential next step. --- title: "What is VERA?" description: "Explore VERA, a high-performance engine optimizing Apache Flink for seamless real-time and batch processing." lastUpdated: 2026-10-14T09:40:33.000Z source_url: html: "https://www.ververica.com/ecosystem-introduction/what-is-vera" md: "https://www.ververica.com/ecosystem-introduction/what-is-vera.md" --- # What is VERA? Discover VERA, Ververica's high-performance engine that optimizes Apache Flink for seamless real-time and batch data processing ## What is VERA (Ververica Runtime Assembly)? VERA (Ververica Runtime Assembly) is the stream processing engine at the core of Ververica’s Unified Streaming Data Platform. It is compatible with Apache Flink, so existing Flink jobs run without code changes, and it improves Flink’s performance, state management and day-to-day operations for both streaming and batch workloads. > **INFO:** VERA (Ververica Runtime Assembly) is Ververica's cloud-native, ultra-high-performance engine that powers Ververica's Unified Streaming Data Platform. The VERA engine optimizes Apache Flink® and is designed to simplify complex stream processing, making real-time data accessible and actionable for a wider audience. VERA enhances Flink's performance, improves user experience, and addresses key pain points like latency and operational complexity in data stream processing, especially within event-driven architecture contexts. With VERA, you can connect, process, analyze, and govern your data in one ultra-high performance streaming data solution. Created to solve both batch and real-time streaming use cases, VERA makes it easy for you to harness insights from your data at any volume and scale. Businesses constantly seek ways to derive immediate insights from their ever-growing streams of data. While traditional batch processing is able to solve some use cases, the demand for real-time decision-making is paving the way for advanced stream processing technologies. At the forefront of this evolution is VERA (Ververica Runtime Assembly), a cloud-native engine that serves as the core of [Ververica's Unified Streaming Data Platform](https://www.ververica.com/product). VERA revolutionizes Apache Flink, making both powerful real-time stream and batch processing accessible to a broader audience, extending beyond specialized technical experts. ## The Genesis of VERA: Addressing Stream Processing Challenges For over a decade, Apache Flink has been a leading solution for stream processing. However, as data scales exponentially and systems become increasingly distributed, the complexities associated with managing and operationalizing open-source Flink have grown. Many organizations lack the internal resources and specialized expertise required to build and maintain the intricate supporting architectures for complex real-time solutions. Ververica recognizes these critical pain points. While Flink is the de-facto gold standard streaming project, all projects have limitations. VERA was created to address challenges including Flink's historical single-tenant design, high operational complexity and costs, the "black box" behavior described by developers, and to address an architecture that has not fully evolved to support modern workloads and cloud environments. VERA democratizes stream processing, maintaining 100% compatibility with Flink while solving these known open source limitations. ## VERA's Core Purpose and Functionality VERA’s primary purpose is to simplify the complexities of data stream processing, enabling businesses to operationalize streaming data with ease. It allows users to seamlessly connect, process, analyze, and govern their data within a single, high-performance streaming data solution. VERA optimizes Apache Flink, enhancing its performance and significantly improving the overall user experience. The capabilities of VERA are built upon three fundamental pillars, designed to provide a unified and efficient approach to real-time data management: - **Streaming Data Movement:** This pillar tackles the challenge of disparate data sources (like relational databases such as Postgres or MySQL) that hold critical business information. VERA utilizes Flink Change Data Capture (CDC) to absorb diverse streaming data and events, transforming them into a uniform, actionable format. This process involves loading and transforming data, real-time processing to make it actionable, and delivering the processed data to various destination systems while maintaining data lineage and security. This ensures easy access, formatting, processing, and storage of data, supporting future decision-making, a unified real-time data view, and application development with support for Java, Python, or SQL. - **Real-Time Stream Processing:** At its heart, VERA focuses on extracting immediate meaning and insights by processing data in real time, eliminating the historical lag inherent in batch processing. VERA’s architecture decouples storage and compute layers, enabling it to deliver both stream processing and batch processing for stateful computations over data streams. It efficiently manages Flink application execution and integrates seamlessly with Flink APIs, empowering developers to process and analyze vast amounts of data and extract real-time insights. This unified approach facilitates fast, reliable decisions based on the freshest available information, as data can be acted upon immediately upon arrival, regardless of its format. - [**Streamhouse**](https://www.ververica.com/ecosystem-introduction/what-is-streamhouse) **(Streaming Lakehouse)**: This innovative pillar combines Apache Flink for stream processing with [Apache Paimon™](https://www.ververica.com/ecosystem-introduction/what-is-apache-paimon) on the streaming storage layer. Streamhouse provides a storage solution that merges the benefits of real-time streaming with the cost-effectiveness and query capabilities of traditional Lakehouse batch processing. It allows users to leverage nearly unlimited storage within their stream processing framework, query petabytes (or even exabytes) of data cost-effectively, and perform both real-time and near real-time stream processing from a single engine. This empowers informed decision-making by leveraging both current and historical data. ## Key Features and Advantages of VERA VERA offers a multitude of features and advantages that position it as a leading stream processing framework: - **Ultra-High Performance:** VERA is engineered for exceptional speed, capable of processing billions of events per second with sub-second latency. It demonstrates performance **up to 2x faster than self-managed open-source Flink**. - **Infinite Scalability and Elasticity:** With its architecture that separates compute and storage layers, VERA reduces costs and significantly enhances performance, enabling it to scale infinitely to meet demand. - **High Availability and Robust Fault Tolerance**: VERA boasts a 99.99% uptime SLA. Its advanced fault-tolerant features, including tiered state and faster checkpoints, ensure uninterrupted operation even in the face of failures, simplifying complexities often found in large stateful applications. - **Cloud-Native Design:** VERA is built to run natively in modern cloud environments, leveraging the benefits of cloud infrastructure for resilience and scalability. - **100% Apache Flink Compatibility:** VERA maintains full compatibility with Apache Flink, preventing vendor lock-in and allowing existing Flink users to easily transition and benefit from VERA's optimizations. This commitment ensures a seamless evolution of Flink for modern workloads. - **Simplified Operations and Reduced Cost:** By abstracting away much of the operational complexity associated with open-source Flink, VERA reduces the need for extensive internal resources and expertise, lowering operational costs and increasing ROI. - **Developer-Friendly Experience:** VERA aims to provide a more intuitive and streamlined experience for developers, allowing them to focus more on business outcomes rather than infrastructure management. - **Unified Batch and Stream Processing:** VERA supports continuous processing for both batch and real-time pipelines, allowing for instant access to data and extraction of insights as soon as data becomes available. This addresses the increasing need for converged architectures that handle both historical and real-time data within a single system. - **Security and Data Governance:** VERA prioritizes security, streamlines data access management, and supports multi-tenancy and robust data governance policies. ### VERA and Event-Driven Architecture VERA is inherently aligned with an event-driven architecture. In an event-driven system, real-time events drive business processes and decisions. VERA's ability to ingest, process, and analyze data streams in real-time makes it an ideal engine for building responsive and scalable event-driven applications. It transforms raw events into uniform datasets, enabling immediate action and informed decisions as soon as data arrives. This is crucial for modern applications that demand immediate reactions to changes in data, such as fraud detection, personalized recommendations, and real-time analytics dashboards. ## The Future of Stream Processing with VERA VERA democratizes stream processing by simplifying its inherent complexities. The goal is to enable users to effortlessly access both fresh and historical data, facilitating well-informed business decisions. VERA is designed to seamlessly handle both batch and real-time streaming data use cases. In the near future, users will experience a streamlined process: simply select their deployment method (on-premise or cloud), choose their data sources, fine-tune the solution, and immediately begin to see results, regardless of the specific use case. This innovative approach is also poised to significantly reduce operational costs and enhance ROI, as VERA provides a managed solution that handles updates, feature innovations, and continuous monitoring, effectively eliminating the need for burdensome self-management. ## Conclusion VERA represents a significant leap forward in the realm of data stream processing. By optimizing Apache Flink and delivering a cloud-native, high-performance engine, Ververica has created a solution that addresses the critical needs of modern enterprises. Whether it’s for stream processing with Apache Flink, building an event-driven architecture, or achieving a unified view of real-time and historical data through Streamhouse, VERA empowers organizations to unlock the full potential of their streaming data, enabling faster insights and more agile business operations. ## FAQ ### What is VERA in the context of stream processing? VERA (Ververica Runtime Assembly) is a cloud-native, ultra-high-performance engine that optimizes Apache Flink for unified streaming and batch data processing on the Ververica platform. ### How does VERA improve performance compared to open source Apache Flink? VERA enhances Flink’s speed, provides sub-second latency, and offers up to 2x better performance and scalability by decoupling compute and storage and introducing advanced state management features. ### Is VERA fully compatible with Apache Flink APIs and jobs? Yes, VERA guarantees 100% compatibility with Apache Flink, enabling seamless migration without vendor lock-in or code rewrites. ### What are the main advantages of using VERA? Key benefits include ultra-fast processing, infinite scalability, high availability (99.99% SLA), simplified operations, strong security and governance, and unified support for both real-time and batch pipelines. ### How does VERA support event-driven architectures? VERA enables real-time ingestion, processing, and analysis of data streams, making it ideal for event-driven applications like real-time analytics, fraud detection, and personalized recommendations. --- title: "Case Studies" description: "Discover how organizations leverage Ververica Platform for stream processing needs. From booking.com to FinTech Studios, see real-world use cases." lastUpdated: 2026-10-14T07:58:16.000Z source_url: html: "https://www.ververica.com/case-study" md: "https://www.ververica.com/case-study.md" --- # PROOF, NOT PROMISES. **case studies** Real teams. Real production workloads. Real results at stream scale. These are their stories. ## One Mount > Ververica Platform Has Been Key To Deploying And Managing Our Flink Jobs. I Can’t Imagine Managing Our Multi-Tenant Stream Processing Architecture Without It. > — Elad Hayun, Head of Core Group at XM Cyber ## Case Studies ### OKX Migrated from AWS Flink to Ververica on AWS. ### FairMoney The Seamless Integration With Our Existing Ecosystem ### Booking.com From 20 Flink apps to 250 with zero added headcount. ### Airbus Read how Airbus reclaimed engineering time with Ververica's managed platform. ### Fintech Studios Zero to production-ready Flink in 30 days. ### One Mount Group Millions of payments daily. Fraud caught in real time. ### Humn.ai Real-time risk. Dynamic pricing. ### XM Cyber 100+ customer environments. 1TB per customer per day. ### VIPKid 100,000 lessons a day. 60% of help requests now resolved automatically. ### Weibo 100 billion model parameters. ML cycles cut from weekly to 10 minutes. _Building a platform to provide Security-as-a-Service_ ### Booking.com Booking.com, a leading travel ecosystem serving both partners and travelers, faced challenges with processing security data streams and orchestrating Flink applications with stateful upgrades and multi-tenancy requirements. Their previous approach proved cumbersome and inadequate for their needs. **Result:** To address these issues, Booking.com selected Ververica Platform to overcome their challenges. The decision to choose Ververica was driven by its ability to enable Booking.com's security teams to manage their own Flink applications independently, freeing up the Platform team to focus on maintaining the overall solution. - **10x**: More Flink Applications - **20x**: Faster Deployment ## Trusted in Production --- title: "Airbus Case Study" description: "Transforming real-time intelligence at Airbus with Ververica. Streamlined operations, 24/7 service desk support, and enhanced visibility." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/airbus" md: "https://www.ververica.com/case-study/airbus.md" --- # Airbus How Ververica Transformed Real-Time Intelligence at Airbus Ververica was tasked with providing a solution to support Airbus, a leading aircraft manufacturer, in deploying their Apache Flink® jobs at scale. Their existing custom-built Java solution required manual handling of jobs and unexpected interruptions, making it difficult to maintain efficiency and keep costs low. To overcome these obstacles, Ververica Platform was selected by Airbus for its ability to streamline real-time data processing without adding complexity to its existing technology stack. Ververica offered a cost-effective, scalable solution, which supported Airbus in building robust real-time intelligence for the aviation market. Download the case study to learn how Ververica helped Airbus achieve: - Streamlined operations without maintenance. - Quick, concise responses to queries via Ververica’s 24-7 service desk. - Ease of visibility to all jobs and their current state & much more. Overall, this contributed to greater stability of the real-time streaming pipeline, a necessary requirement to deliver a service with high-quality standards. --- title: "Booking.com Case Study" description: "Discover how Booking.com utilized Ververica Platform to enhance security data processing, streamline Flink applications, and boost operational efficiency. " lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/booking" md: "https://www.ververica.com/case-study/booking.md" --- # Booking.com Building a platform to provide Security-as-a-Service. Delivering streaming-in security related data for processing Booking.com, a leading travel ecosystem serving both partners and travelers, faced challenges with processing security data streams and orchestrating Flink applications with stateful upgrades and multi-tenancy requirements. Their previous approach proved cumbersome and inadequate for their needs. To address these issues, Booking.com selected Ververica Platform to overcome their challenges. The decision to choose Ververica was driven by its ability to enable Booking.com's security teams to manage their own Flink applications independently, freeing up the Platform team to focus on maintaining the overall solution. Download the case study to discover how Ververica supported Booking.com in achieving the following: - Instant short-term wins that reduced downtime and quicker issue resolution. - The ability to scale jobs, leading to operational efficiency - Reduced deployment time and stateful upgrade time; from hours to a matter of minutes - Invaluable support for Booking.com’s internal team, offering fast response times and access to Flink experts for guidance and assistance. Overall, Ververica Platform proved to be an effective and transformative solution for Booking.com, providing seamless Flink application lifecycle management, scalability, and improved operational efficiency, ultimately enabling a more secure and reliable experience for their users and partners. --- title: "FairMoney Case Study" description: "How FairMoney, a leading African digital bank, reshaped risk model development with Ververica Platform and Apache Flink, self-operated on Amazon EKS. It kept its existing Flink job logic and gained direct configuration control at a lower cost than a per-job managed service." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/fairmoney" md: "https://www.ververica.com/case-study/fairmoney.md" --- # FairMoney How FairMoney Reshaped Risk Model Development at Africa’s Leading Digital Bank With Ververica Platform. FairMoney evaluated several stream-processing platforms before selecting Ververica Platform (VVP) running Apache Flink®, on a self-operated deploy- ment on Amazon EKS. The deciding factor was fit: unlike alternatives that would have required rebuilding the processing architecture from scratch, Ververica Platform let FairMoney keep its existing Flink-based job logic largely unchanged while gaining direct control over configuration, at a ma- terially better cost structure than a per-job managed service. --- title: "FinTech Studios Case Study" description: "Unlock the power of real-time data and stream processing with FinTech Studios using Ververica Platform. Empower the finance industry of tomorrow." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/fintech-studios" md: "https://www.ververica.com/case-study/fintech-studios.md" --- # FinTech Studios Powering the finance industry of tomorrow with real-time data. Leveraging the power of stream processing and real-time data to empower a data-driven finance industry of tomorrow. As a technology-first organization, FinTech Studios leverages Ververica Platform to ingest millions of research, news and regulatory sources in real-time and build an AI-based intelligent search and analytics platform for financial services professionals. This case study explains: - How FinTech Studios developed continuous delivery pipelines for Flink applications in production. - How FinTech Studios deployed Ververica Platform and Apache Flink to build real-time streaming pipelines and ingest them to the company’s Machine Learning (ML) and Natural Language Processing (NLP) platform. - How the company performs enrichment processes to the real time data in order to provide data-driven workflows for financial services professionals. > _"With the help of Ververica Platform, we were able to move our Kubernetes-native, continuous delivery pipeline to production and push new features to our technology faster."_ _Senior Engineer, Fintech Studios_ --- title: "Humn.ai Case Study" description: "Discover how Humn.ai leveraged Ververica Platform and Apache Flink to enhance real-time risk assessment and dynamic pricing for commercial fleet insurance." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/humn-ai" md: "https://www.ververica.com/case-study/humn-ai.md" --- # Humn.ai Machine Learning-based models to quantify commercial fleet exposure at individual insured asset risk in real-time. Humn.ai uses Ververica Platform and Apache Flink to build a Machine Learning-based platform producing dynamic risk assessment models and real-time pricing for tomorrow’s insurance industry. Humn.ai deployed Ververica Platform with Apache Flink to harden production Flink jobs seamlessly and quickly. They developed a Machine Learning-based platform that is Kubernetes-native and implements ML-based algorithms for calculating the risk of vehicles in real-time. The risk scores for each insured asset is calculated dynamically and then updates a premium variable part of the insurance policy in real-time. This case study includes: The challenges Humn.ai faced with their previous technology stack and how Apache Flink helped in resolving them The results achieved by deploying Ververica Platform and Apache Flink to production A detailed overview of Humn.ai's experience using Ververica Platform "When we migrated to Kubernetes with Ververica Platform it was much easier for our developers to define and configure jobs, as well as manage and monitor them in an integrated and efficient manner." Alberto Romero, Co-founder and CTO, Humn.ai --- title: "OKX Case Study" description: "OKX Did Not Want To Leave AWS. It Simply Needed A Better Flink On AWS" lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/okx" md: "https://www.ververica.com/case-study/okx.md" --- # OKX Why OKX chose to migrate their data pipelines to Ververica from Amazon’s Managed Service for Apache Flink®. OKX handles the two challenges by migrating their streaming workloads to Ververica Platform in the Bring Your Own Cloud (BYOC) deployment on AWS. This maintains the workload on their AWS cloud instance, while offering total data sovereignty and access to an improved Flink backend engine and overall experience. Ververica are the original creators of Apache Flink, and built the BYOC deployment to address exactly the challenges OKX faced. --- title: "One Mount Group Case Study" description: "Discover how One Mount Group utilizes Ververica Platform and Apache Flink for real-time fraud detection in OneU, Vietnam's consumer app. " lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/one-mount-group" md: "https://www.ververica.com/case-study/one-mount-group.md" --- # One Mount Group Detecting fraudulent transactions in real-time for OneU, Vietnam’s consumer application trusted by millions of users. One Mount Group harnesses Ververica’s Unified Streaming Data Platform to power real-time fraud detection for its consumer application, OneU. By leveraging Ververica Platform: Self-Managed as a Platform-as-a-Service (PaaS), One Mount Group has streamlined the deployment and management of its real-time applications, allowing its tech team to operate with greater efficiency. By partnering with Ververica, One Mount Group effectively addresses real-time scalability and high availability challenges while ensuring compliance with strict Service Level Agreement (SLA) requirements. Additionally, One Mount Group’s internal teams achieved operational efficiency and standardized the deployment and operation of Flink jobs across the organization. --- title: "VIPKid Case Study" description: "Discover how VIPKid uses Ververica Platform to enhance online education with real-time feedback and optimizations facilitating over 100,000 lessons per day." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/vipkid" md: "https://www.ververica.com/case-study/vipkid.md" --- # VIPKid Over 100,000 lessons per day! How does VIPKID solve the problem of online education real-time live broadcast interaction? VIPkid uses Ververica Platform to provide real-time feedback on learning experiences and efficiently optimise the quality of their online courses. Real-time computing applications have many scenarios for companies in the online education space. From performing data computations providing real time feedback on the learning experiences, to efficiently optimizing the quality of the courses over the internet, continuous exploration with real-time data is necessary in our field. This case study will share, in three sections, how to improve human efficiency and system processing capabilities through real-time analytics and computations during live broadcast classes. --- title: "Weibo Case Study" description: "Learn how Weibo leverages Apache Flink and Ververica Platform for real-time data processing and Machine Learning at scale. Dive into their success story." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/weibo" md: "https://www.ververica.com/case-study/weibo.md" --- # Weibo Learn how Weibo uses Apache Flink and Ververica Platform to unify offline and online data processing and run Machine Learning pipelines at scale. This case study presents how social networking platform Weibo uses Apache Flink and Ververica Platform to run real time data processing and Machine Learning on their platform. In the different sections of the case study, Weibo's Machine Learning platform is introduced. Its architecture is revealed, focusing on how Machine Learning pipelines are developed in real-time using Apache Flink and Ververica Platform on Alibaba Cloud. There is further insight into the company’s plans for Flink’s usage at Weibo and how open source technology has been utilized and adopted in the organization overall. What you will learn from the case study: - A deep dive into Weibo's Machine Learning platform, its components, and technology stack - How Weibo uses Apache Flink for online Machine Learning - How Weibo plans to further integrate and expand the usage of Apache Flink and Ververica Platform in the future --- title: "XM Cyber Case Study" description: "Discover how XM Cyber partnered with Ververica Platform to build a robust multi-tenant Flink architecture for real-time, scalable cyber security solutions." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/case-study/xm-cyber" md: "https://www.ververica.com/case-study/xm-cyber.md" --- # XM Cyber Building a robust multi-tenant platform for the future of tomorrow. XM Cyber, a leading hybrid cloud security company founded by top executives from the Israeli cyber intelligence community, is a technology leader in its approach to cyber risk. XM Cyber utilizes its Apache Flink® (Flink) applications to ensure unlimited scaling for customers, in real-time, regardless of data volumes (up to 1 terabyte every 24 hours for each of its customers). When deciding how to deploy a multi-tenant Flink architecture with the capability to scale, XM Cyber turned to Ververica, the Original Creators of Apache Flink®, and its solution, Ververica Platform, to help manage the Flink jobs that form part of its industry-leading security solutions. Real-time stream processing is critical for XM Cyber to maintain high performance and security within its Cyber Security solutions. Utilizing stream processing enables XM Cyber to respond to data changes and potential threats before they happen. Integrated with Ververica Platform, XM Cyber’s streaming architecture has evolved to be a true multi-tenant model. Download the case study to discover how Ververica Platform supported XM Cyber in achieving: - Unlimited flexibility and scaling - handling terabytes of data from it’s customers - The low latency requirement without resource over allocation - Reduced running costs and engineering overheads and more --- title: "About" description: "Ververica is the company founded by the original creators of Apache Flink. European-engineered in Munich, powering real-time data for global enterprises." lastUpdated: 2026-04-16T07:14:13.000Z source_url: html: "https://www.ververica.com/about" md: "https://www.ververica.com/about.md" --- # We Created Apache Flink. Now We Are Perfecting It. The company behind the most widely deployed stream processing framework in the world. Founded by researchers. Built by engineers. Trusted by enterprises processing trillions of records daily. --- title: "Careers" description: "Join the team that created Apache Flink. Engineering, product, sales, and marketing roles at Ververica. Munich HQ with remote-friendly culture." lastUpdated: 2026-10-13T07:44:52.000Z source_url: html: "https://www.ververica.com/careers" md: "https://www.ververica.com/careers.md" --- # Join a World-Class Team We are looking for the brightest people to work on fundamental problems that matter in practice. Take a look at our open positions and meet the team! ## Why work with us? We enjoy working with people who share our values and love challenges. Take a look at some of the reason why joining Ververica should be your next decision. ### Growth Opportunities Your ambition and contributions are valued here. Together, we grow and succeed. ### Flexibility Let’s face it life happens. We get it. That’s why we offer flexible working hours to help you balance your personal needs, along with health insurance and other essential benefits. ### Customize Your Tech Choose the gear that helps you work best. Let us know your preferences, and we’ll provide the tech that keeps you comfortable and productive. ### Relocation Assistance Don’t worry we support your relocation, if needed, and take care of any necessary paperwork to make the process smooth. ### Great Team Success is most rewarding when we achieve it together. We collaborate, share ideas, and celebrate every win as one team. ## Hiring Process 1. **Submit your CV** 2. **Skills Assessment** 3. **HR/ Technical interview** 4. **Final Interview** 5. **Job offer** ## Current Openings --- title: "Contact" description: "Talk to Ververica about Ververica Platform, Apache Flink, deployment options, support or a demo. We reply within one business day." lastUpdated: 2026-10-20T08:03:04.000Z source_url: html: "https://www.ververica.com/contact" md: "https://www.ververica.com/contact.md" --- # Let’s Talk Stream Processing **Contact us** Questions about Apache Flink, deployment options, support or a demo? Tell us what you are working on. We reply within one business day. ### Get in touch Complete the form below, and our team will get in touch with you shortly. ## Trusted by Engineering Teams Worldwide ## Frequently Asked Questions ### How quickly will I get a response? We respond to all inquiries within one business day. Technical questions are routed directly to engineering. Sales inquiries receive a response within 4 hours during European business hours. ### Can I get a live demo of the platform? Yes. Book a demo through our [demo request page](https://www.ververica.com/demo). A Ververica engineer walks you through the platform in a live environment. No slides. Running systems. ### Is there a free trial available? Yes. The Ververica Cloud free trial gives you full platform access with $400 in credits. No credit card required. Start at [app.ververica.com.](https://app.ververica.cloud/authenticate/sign-up) ### Where is Ververica located? Headquarters in Munich, Germany. Additional offices in Frankfurt, Berlin, Barcelona, Spain and London, England. Engineering is distributed across Europe. We serve customers globally. --- title: "Demo" description: "Request a technical demo of Ververica Platform. A solutions engineer first scopes your workload, stack and goals, then walks you through a live scenario that matches your use case." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/demo" md: "https://www.ververica.com/demo.md" --- # See Ververica Platform Run Your Streaming Workloads **Request a Demo** A technical walkthrough with the creators of Apache Flink. Bring a use case. Leave with an architecture. - **>10ms — Sub-10ms latency** - **40 — Proven at ING Bank** - **DORA — Built for DORA requirements** ## A Process Built for Technical Evaluation ### Schedule Pick a 30-minute slot. We confirm within one business day. ### Scope A 15-minute pre-call to understand your workload, stack, and goals. We tailor the demo. ### Demo Live walkthrough of the platform against a scenario that matches your use case. ### Next steps Architecture notes, a proof-of-concept plan, or pricing, depending on your stage. ## Frequently Asked Questions ### How long does the demo take? Thirty minutes for the core walkthrough, plus a 15-minute scoping call beforehand. Longer sessions available on request. ### Do I need Flink expertise to join the demo? No. The demo is calibrated to your team. We run sessions for platform teams new to Flink as well as teams already in production. ### Can we use our own data or workload in the demo? Yes. Share a workload description during scoping. We prepare a relevant scenario and, where appropriate, can run against sample data that matches your shape. ### What deployment options are covered? Fully managed Ververica Cloud, self-managed Ververica Platform, and bring-your-own-cloud. We walk through the operational and commercial differences. ### Is this a sales call? A solutions engineer runs the session. Pricing only comes up if you ask. --- title: "Events" description: "Join Ververica at X-Stream Labs, meetups, webinars, and conferences. Learn stream processing from the creators of Apache Flink. Register for upcoming events." lastUpdated: 2026-10-14T09:40:12.000Z source_url: html: "https://www.ververica.com/events" md: "https://www.ververica.com/events.md" --- # Events. In Person. Online. On Demand. X-Stream Labs, meetups, webinars, and conferences. Technical content from the team that created Apache Flink. No sales pitches. Engineering depth only. --- title: "Conferences" description: "Find Ververica at industry conferences worldwide. Meet the Apache Flink creators, see live demos, and attend our sessions." lastUpdated: 2026-10-14T07:58:16.000Z source_url: html: "https://www.ververica.com/events/conferences" md: "https://www.ververica.com/events/conferences.md" --- # Meet Ververica at Industry Conferences The team that created Apache Flink. At the booth, on the stage, in the hallway track. Live demos. Architecture discussions. Direct access. ## What We Bring Why Ververica VERA engine processing 6.9 billion records per second. Streamhouse architecture in action. Flink SQL on live streams. Not slides. Running systems. Bring your architecture diagram. Sit down with a Ververica engineer. Get direct feedback on your streaming data design. 30-minute sessions at the booth. Talk with with Apache Flink Committers and Original creators. Ask about roadmap, internals, or your specific production challenge. --- title: "Meetups" description: "Join Apache Flink and stream processing meetups hosted by Ververica. Connect with the community, hear technical talks, and share experiences." lastUpdated: 2026-05-27T07:00:02.000Z source_url: html: "https://www.ververica.com/events/meetups" md: "https://www.ververica.com/events/meetups.md" --- # Apache Flink and Stream Processing Meetups Technical talks. Live demos. The people who build and operate stream processing in production. Free admission. Every time. ## What to Expect ### Technical Deep-Dives A practitioner presents a real production use case, architecture decision, or open-source contribution. Code on screen. Numbers on the board. ### Demo or Lightning Talk A shorter session. Product demo, new feature walkthrough, or community project showcase. ### Q&A Open floor. Questions directed at speakers, Ververica engineers, or the room. ### Networking Food and drinks provided. Conversations that start at the meetup often continue on Slack and in pull requests. --- title: "Webinars" description: "Live and on-demand webinars on Apache Flink and stream processing. Technical deep-dives, product demos, and industry use cases from Ververica experts." lastUpdated: 2026-05-27T06:59:20.000Z source_url: html: "https://www.ververica.com/events/webinars" md: "https://www.ververica.com/events/webinars.md" --- # Stream Processing Webinars Technical depth. No filler. Live sessions and on-demand recordings from Ververica engineers and production practitioners. ## Watch. Learn. Build. Past webinars Dual Pipelines Are Done | Ververica's Streamhouse & Materialized Tables Streaming at Scale: Unity’s Flink & Paimon Data Platform Whole Lotta Stream: The Streamhouse Architecture at Fresha Evolution of Yelp's Data Infrastructure to Streamhouse Data Lineage in Data Streaming Driving Energy Transformation with Real Time Data Webinar: Fluss 0.7 - A New Stream of Possibilities --- title: "X-Stream Lab" description: "Hands-on workshops with the creators of Apache Flink. Build real-time applications in guided lab sessions. View upcoming X-Stream Labs and register." lastUpdated: 2026-10-14T09:40:12.000Z source_url: html: "https://www.ververica.com/events/x-stream-lab" md: "https://www.ververica.com/events/x-stream-lab.md" --- # X-Stream Lab. Hands-On Stream Processing Workshops Write code. Deploy applications. Break things and fix them. Guided by the engineers who run Apache Flink daily. ## What Are X-Stream Labs Full-day workshops. Small groups. Ververica engineers at the whiteboard and in the code alongside you. Every X-Stream Lab is structured around a production-grade project. You build a working application from ingestion to output. Your hands on the keyboard, deploying to a live Ververica Platform environment. Groups are capped at 30 participants. Questions get answered. Problems get debugged. Nobody falls behind. ## X-Stream Lab Topics ### Hands-On Expertise Build a streaming application from the ground up following a real use case. ### Real-Time Concepts Get a practical introduction to key concepts in stream processing and real-time data. ### Network with Peers Connect with other data professionals, share insights, and build valuable relationships. ### Real Life Adoption Work through a complete scenario that reflects a real-world stream processing use case from ingestion to output. --- title: "Banking Hub" description: "Real-time fraud detection, payments, AML monitoring, and regulatory reporting for banks. Powered by Apache Flink with sub-10ms latency." lastUpdated: 2026-10-14T09:41:38.000Z source_url: html: "https://www.ververica.com/banking" md: "https://www.ververica.com/banking.md" --- # Real-Time Stream Processing Built for Banking. Fraud detection in under 10ms. Instant payments at 6.9B records/sec. Continuous AML monitoring, risk management, and regulatory reporting. All on one platform, built by the creators of Apache Flink. - **<10ms — Sub-10ms latency** - **6.9B — records/sec** - **DORA — Built for DORA requirements** ## How do banks use stream processing? Stream processing for banking means processing every transaction, payment and market event the moment it arrives, instead of in overnight batches. Banks use Apache Flink on Ververica for fraud detection, instant payments, AML monitoring, risk management and regulatory reporting, with exactly-once processing so no transaction is lost or counted twice. ## Trusted in Production ## Batch Processing Is a Liability In Modern Banking The Problem ### Fraud settles before jobs finish. AML alerts arrive 24 hours late. Regulatory reports miss the deadline. Customers receive yesterday's offers. ### The financial cost. Billions in undetected fraud, millions in compliance fines, irreversible customer attrition. ### Built for a slower world. Batch was built for a slower world. Banking moved on. Your infrastructure did not. ## Banking Solutions ### Fraud Detection Score 2B+ transactions per day. Block fraud in under 150ms. ### Real-Time Payments Instant settlement. Exactly-once processing. Peak-hour resilience. ### AML Monitoring Continuous monitoring. 60% fewer false positives. Adaptive thresholds. ### Risk Management Real-time VaR. Counterparty exposure. Continuous stress testing. ### Regulatory Reporting T+0 reporting. Automated validation. Immutable audit trail. ### Core Modernization Strangler fig migration. Zero downtime. Dual-run validation. ### Customer Personalization Act on customer behavior as it occurs. Serve the right offer in under 50ms. ### Mainframe Offloading Cut MIPS costs by 40% or more without replacing the mainframe. ### Fintech Monitoring Real-time portfolio risk. Instant wealth management signals. ## Why Banks Run On Ververica ### Sub-10ms Latency Fraud, payments, and risk decisions execute in under 10 milliseconds. Not aspirational. Measured in production across tier-1 banks. ### 6.9B Records/Sec Throughput The VERA engine processes 6.9 billion records per second. Peak volumes during market events and payment surges do not degrade performance. ### 2x Faster Than Open-Source Flink VERA engine delivers double the throughput of standard Apache Flink at 52% lower resource consumption. Built by the team that created Flink. ### 40% Lower TCO Measured total cost of ownership reduction versus legacy stream processing stacks. Fewer clusters. Less operational overhead. Same workloads. ### Exactly-Once Processing Financial transactions demand zero data loss and zero duplicates. Ververica guarantees exactly-once semantics at scale. No exceptions. ### Multi-Cloud Deployment Deploy on AWS, Azure, GCP, or on-premises. BYOC mode keeps all data in your environment. Engineered in Europe. Sovereign by design. _CUSTOMER STORY_ ### ING BANK ING Bank deployed Ververica to process financial events in real time across its global operations. The result: fraud detection that acts before transactions settle, not after. **Result:** Fraud detection that acts before transactions settle, not after. 2B+ events per day. Fraud detection across 40 countries. Sub-second decision latency. - **2B+**: Events scored per day - **>1MS**: Sub-second decision latency ## Frequently Asked Questions ### What is real-time stream processing for banking? [Real-time stream processing](https://www.ververica.com/ecosystem-introduction/stream-processing-with-apache-flink-beginners-guide) analyzes financial data the instant it arrives, without waiting for batch windows. Banks use it for fraud detection, payment processing, AML monitoring, and regulatory reporting. Ververica processes 6.9B records/sec with sub-10ms latency on the VERA engine. ### How does Ververica differ from open-source Apache Flink? Ververica was founded by the creators of [Apache Flink](https://www.ververica.com/ecosystem-introduction/what-is-apache-flink). The VERA engine delivers 2x the throughput at 52% lower resource consumption. Ververica adds enterprise security, multi-tenancy, automated operations, and 24/7 support. It is Flink, built for production banking workloads. ### How does Ververica support DORA requirements? Under DORA, the bank stays responsible for its compliance. Ververica is built for DORA requirements: it supports ICT risk management, continuous monitoring, incident detection and operational resilience testing. Ververica holds SOC 2 Type II and ISO 27001 certifications and is built for GDPR requirements. BYOC and self-managed deployment keep data in the bank’s own environment. ### What deployment options are available for banks? Banks can deploy Ververica Cloud (fully managed), BYOC (your cloud, our platform), or self-managed (your infrastructure). BYOC is the most common choice for regulated institutions. All data remains in your environment. No vendor access to production data. ### How long does implementation take for banking use cases? Typical banking deployments reach production in 8 to 12 weeks. Fraud detection and real-time payments are the most common first use cases. Ververica provides pre-built connectors for Kafka, mainframe CDC, and core banking systems. Professional services teams specialize in financial services. ### Does Ververica support exactly-once processing for financial transactions? Ververica guarantees [exactly-once processing semantics](https://www.ververica.com/developers/flink-exactly-once-kafka). Every financial transaction is processed once and only once, even during failures or restarts. This is non-negotiable for payments, settlements, and regulatory reporting. The guarantee holds at full throughput. --- title: "Data sovereignty" description: "Why European banks need sovereign real-time data. Run Apache Flink with Ververica on your own infrastructure or in your own AWS or Azure account." lastUpdated: 2026-10-19T08:43:28.000Z source_url: html: "https://www.ververica.com/data-sovereignty" md: "https://www.ververica.com/data-sovereignty.md" --- # Keep Your Real-Time Data Under Your Control Ververica is a European company, headquartered in Munich, Germany. Apache Flink® born in Berlin. Run Ververica self-managed in your own data center, or as BYOC in your own AWS or Azure account, so your data stays in the infrastructure and region you choose. ## What does data sovereignty mean for a real-time streaming platform? Data sovereignty for real-time streaming means you, not your vendors, control where streaming data is stored and processed and who can reach it. Ververica GmbH is a European company based in Munich, Germany. Its Apache Flink platform runs where you decide: self-managed on your own infrastructure, which needs no connection to Ververica, or as BYOC in your AWS or Azure account and region. ## They Told You Cloud Was Borderless. They Lied. ### Data has nationality. ### And yours is probably American 🇺🇸 by default. When your streaming platform runs on AWS, Azure, or GCP: - Your data is subject to US surveillance laws ([CLOUD Act](https://www.justice.gov/criminal/cloud-act-resources), [FISA 702](https://www.congress.gov/crs-product/R48592)), even in EU regions - American intelligence agencies can access it without notifying you - Your GDPR compliance is theater when the platform provider has US headquarters - EU data centers don't protect you when the company answers to US jurisdiction **Confluent? Databricks? Kinesis?** American companies. US jurisdiction. **Great tech. Wrong passport.** It's not paranoia. It's the law. The CLOUD Act gives US authorities jurisdiction over data controlled by US companies, _regardless of where it's stored._ You didn't build your European business to hand your most sensitive data to a foreign government. But that's exactly what happened. ## Here's what it requires What Real Data Sovereignty Looks Like ### European Corporate Entity Ververica GmbH is registered in Munich, Germany (Amtsgericht München, HRB 309285). Your contract and your support are with a European company. ### Your Data Stays With You With self-managed deployment, Ververica does not run your clusters and has no access to your data. With BYOC, your data stays in your own cloud account and region. Your contract is with Ververica GmbH in Munich. Which laws apply to your cloud provider is a question for your own legal team. ### Full Deployment Control - Self-managed: Your EU infrastructure, your servers, your control - BYOC: your own AWS or Azure account, in the region you choose - Managed Service: EU data centers, EU contracts, EU operations teams ### Apache Flink®: Born in Berlin Created at TU Berlin by our founders. European innovation from day one. The team that invented Flink still leads it. ### Auditable Stack Self-managed deployments need no connection to Ververica. Complete data lineage supports GDPR Article 30 records and DORA audit trails. When regulators ask where the data went, you can show the **lineage.** ## Your Data Sovereignty Checklist | Feature | Ververica | US Platforms | | --- | --- | --- | | EU corporate entity | ✓ | ✗ | | Contract with an EU company | ✓ | ✗ | | EU-only deployment options | ✓ | ✗ | | European technology origins | ✓ | ✗ | | Auditable supply chain | ✓ | ✗ | | Self-managed option with no connection to the vendor | ✓ | ✗ | ## If You're in These Industries, This Isn't Optional If your regulator would flinch at "Our real-time platform is American," keep reading. DORA mandates third-party risk management. If your regulator asks why transaction data touches US infrastructure, what's your answer? Score your platform with the self-assessment checklist for financial institutions. Patient data sovereignty under GDPR Article 9. Clinical trials and genomic research can't be exposed to foreign governments. NIS2 essential entities. Sovereign data requirements for national security. Procurement favors EU providers. Network telemetry = national security. Real-time subscriber analytics for hundreds of millions of EU citizens. ## FAQ ### Can't we just use AWS EU regions with data residency controls? Data residency (where data is stored) is not the same as data sovereignty (which laws can reach it). Under the US CLOUD Act, providers subject to US jurisdiction can be required to disclose data they control, even when it is stored outside the US. With Ververica BYOC, the data plane runs in your own AWS or Azure account; with self-managed, it runs wherever you choose, including your own data center. ### What about managed streaming services in EU regions? Check where the provider is headquartered and which laws apply to it. A service in an EU region can still be run by a company that is subject to non-EU law. ### Does this cost more than US platforms? Ververica delivers 40% lower TCO. But the real question: What's the cost of regulatory non-compliance? Of being the next Schrems case study? The cheapest option is the one that doesn't shut down your business. ### What if we need global deployment? Data sovereignty doesn't mean isolation. Run Ververica globally with EU data in EU regions. The difference: You control where each workload runs. --- title: "Manufacturing" description: "Real-time quality monitoring, predictive maintenance, and supply chain optimization for manufacturing. Process IoT data at 6.9B records/sec." lastUpdated: 2026-10-14T07:55:04.000Z source_url: html: "https://www.ververica.com/manufacturing" md: "https://www.ververica.com/manufacturing.md" --- # Real-Time Intelligence for the Factory Floor Thousands of sensors. Millions of events. Every production line, every machine, every checkpoint generating data. Defects caught. Downtime prevented. Throughput maximized. ## Key Reasons To choose Ververica Why Ververica A single production line generates millions of data points per hour. Temperature, vibration, pressure, torque, speed. Batch systems cannot ingest it all. Data gets sampled. Signals get lost. When a defect is detected at final inspection, every unit produced since the defect originated is suspect. Minutes of detection delay translate to hours of rework, scrap, and warranty exposure. A stopped production line costs thousands per minute. Reactive maintenance waits for failure. Predictive maintenance acts on data. The difference is real-time processing. Parts shortages, logistics delays, and demand shifts propagate through the production schedule. By the time batch reports surface the problem, the line is already idle. ## Factories generate data at machine speed. Processing it on batch schedules is waste. Use Cases ### Predictive Maintenance Analyze vibration, temperature, and operational data from equipment sensors in real time. Detect degradation patterns before failure occurs. Schedule maintenance during planned windows. Eliminate unplanned downtime. ### Quality Monitoring Process inspection data, sensor readings, and visual inspection results as they are generated. Detect quality deviations within seconds of occurrence. Stop defective production before it propagates downstream. ### Supply Chain Optimization Ingest data from suppliers, logistics providers, and warehouse systems in a single streaming pipeline. Detect delays and shortages in real time. Adjust production schedules before the line runs out of parts. ### Energy Management Monitor power consumption across every machine, line, and facility in real time. Detect energy anomalies. Optimize consumption patterns. Reduce energy costs by correlating usage with production output continuously. ### Production Line Optimization Process OEE data, cycle times, and throughput metrics from every station in real time. Identify bottlenecks the moment they form. Adjust line parameters to maximize output without compromising quality. ### Digital Twin Feed real-time sensor data into digital twin models. Run simulations against live production conditions. Test parameter changes on the digital model before deploying them to the physical line. ## Performance at Retail Scale ### 6.9B Records/Sec VERA engine throughput. Benchmarked at enterprise scale. Every click, transaction, and inventory update processed. ### Sub-10ms Latency From event ingestion to action. Pricing updates, fraud scores, and personalization signals delivered before the page loads. ### 2x Faster Double the throughput of standard Apache Flink. Same workloads. Half the compute. ### 40% Lower TCO Measured total cost of ownership reduction across compute, operations, and infrastructure. Fewer servers. Same results. ## Frequently Asked Questions ### Can Ververica process data from industrial IoT sensors and SCADA systems? Yes. The platform ingests data from MQTT brokers, OPC-UA servers, Kafka topics, and industrial databases. Pre-built connectors and CDC capture handle standard manufacturing data sources without custom integration code. ### How fast can the platform detect quality defects on a production line? VERA engine processes sensor and inspection data with sub-10ms latency. Quality rules execute against every data point in real time. Defect detection occurs within seconds of the deviation, not hours later at final inspection. ### What deployment options work for manufacturing environments with air-gapped networks? Ververica offers self-managed deployment that runs entirely within your infrastructure. No external network access required. The platform operates behind factory firewalls with full functionality. SOC 2 Type II and ISO 27001 certified. ### How does predictive maintenance work with Ververica? The platform processes equipment sensor data, including vibration, temperature, and operational parameters, in real time. ML models score degradation patterns continuously. Maintenance alerts fire when thresholds are crossed, before failure occurs. ### Can Ververica integrate with existing MES and ERP systems like SAP? Yes. The platform connects to SAP, major MES platforms, and industrial databases through pre-built connectors, CDC, and REST APIs. Data flows bidirectionally. Production insights feed back into existing systems without replacing them. --- title: "Retail" description: "Power real-time retail experiences with Ververica. Dynamic pricing, inventory optimization, and personalization at 6.9B records/sec throughput." lastUpdated: 2026-10-14T07:55:04.000Z source_url: html: "https://www.ververica.com/retail" md: "https://www.ververica.com/retail.md" --- # Real-Time Data for Retail That Moves at Customer Speed Your customers act in milliseconds. Your data infrastructure should too. Ververica processes 6.9B records/sec so pricing, and personalization run at the speed of demand. ## Key Reasons To choose Ververica Why Ververica Stores, apps, marketplaces, fulfillment centers. Each channel generates data in isolation. Decisions arrive late. Customers leave. Batch pipelines deliver recommendations hours after intent signals fire. By then, the cart is abandoned and the competitor has closed the sale. Demand shifts in minutes. Legacy systems report in hours. The gap between those two numbers is lost revenue, dead stock, and empty shelves. ## Retail runs on velocity. Batch processing does not. Use Cases ### Real-Time Inventory Track stock across every warehouse, store, and fulfillment center in real time. Eliminate overselling. Eliminate stockouts. Every SKU accounted for, every second, across every channel. ### Dynamic Pricing Adjust prices based on demand signals, competitor data, and inventory levels as they change. Not hourly. Not daily. Now. Protect margins and move product at the right price point. ### Customer Personalization Act on clickstream, purchase history, and behavioral data the moment it arrives. Deliver product recommendations, offers, and content that match current intent, not yesterday's activity. ### Supply Chain Optimization Process signals from suppliers, logistics partners, and weather systems in a single stream. Detect delays before they cascade. Reroute shipments before shelves go empty. ### Fraud Prevention Score every transaction in real time against behavioral patterns and known fraud signals. Flag anomalies in sub-10ms. Block fraudulent orders before fulfillment begins. ### Demand Forecasting Combine real-time sales data with external signals to generate demand predictions that update continuously. Feed purchasing, staffing, and logistics systems with forecasts that reflect current conditions. ## Performance at Retail Scale ### 6.9B Records/Sec VERA engine throughput. Benchmarked at enterprise scale. Every click, transaction, and inventory update processed. ### Sub-10ms Latency From event ingestion to action. Pricing updates, fraud scores, and personalization signals delivered before the page loads. ### 2x Faster Double the throughput of standard Apache Flink. Same workloads. Half the compute. ### 40% Lower TCO Measured total cost of ownership reduction across compute, operations, and infrastructure. Fewer servers. Same results. ## Frequently Asked Questions ### How does stream processing differ from batch processing for retail? Batch processing collects data and runs it on a schedule, typically hourly or daily. Stream processing acts on each event as it arrives. For retail, this means pricing, inventory, and personalization update continuously instead of operating on stale snapshots. ### Can Ververica process data from all retail channels in one pipeline? Yes. Ververica ingests data from POS systems, e-commerce platforms, mobile apps, marketplaces, and IoT sensors simultaneously. All channels feed into a unified streaming pipeline with sub-10ms processing latency. ### What scale of retail operations can the platform support? VERA engine processes 6.9B records per second. That covers transaction volumes, clickstream data, and inventory events for retailers operating thousands of stores and millions of daily online sessions without performance degradation. ### How does Ververica integrate with existing retail technology stacks? The platform connects to Kafka, databases via CDC, REST APIs, cloud storage, and data lakes out of the box. Pre-built connectors cover major retail platforms. No custom integration code required for standard sources and sinks. ### Is Ververica compliant with data privacy regulations for retail? Ververica is SOC 2 Type II certified, ISO 27001 certified, and GDPR compliant. BYOC deployment keeps all customer data within your own cloud environment. No data leaves your infrastructure unless you route it there. --- title: "Software" description: "Build real-time features faster with Ververica. Event-driven architecture, real-time analytics, and stream processing for software companies." lastUpdated: 2026-10-14T07:55:04.000Z source_url: html: "https://www.ververica.com/software" md: "https://www.ververica.com/software.md" --- # Ship Real-Time Features Without Building Infrastructure Your users expect real-time. Building the infrastructure to deliver it burns months and millions. You build features. We run the platform. ## Key Reasons To choose Ververica Why Ververica Activity feeds that update in minutes. Dashboards that refresh on reload. Alerts that arrive after the damage is done. Users notice. Users leave. A production-grade Flink cluster requires months of engineering time, dedicated operations staff, and ongoing maintenance. That is time not spent on product. Your team builds a working prototype. Then it hits production load. Then it breaks at 10x. Then the on-call rotation burns out your best engineers on infrastructure instead of features. ## The math is clear. Build on managed infrastructure. Ship product. Use Cases ### Real-Time Analytics Process event streams and deliver live metrics to your users. Dashboards, reports, and KPIs that update continuously. No batch lag. No stale numbers. Analytics that reflect what is happening now. ### Event-Driven Microservices Build services that react to events the moment they occur. Decouple systems with streaming pipelines instead of synchronous API chains. Eliminate cascading failures. Scale each service independently. ### ML Feature Stores Compute features from streaming data in real time and serve them to ML models at inference time. Fresh features produce better predictions. Stale features produce stale results. ### Activity Feeds Power real-time feeds for social, collaboration, and marketplace platforms. Process millions of events per second. Deliver ordered, deduplicated, personalized feeds with sub-10ms latency. ### Usage-Based Billing Meter every API call, compute second, and storage byte in real time. Generate accurate usage records as consumption happens. Bill precisely. Eliminate month-end reconciliation surprises. ### Anomaly Detection Monitor application metrics, user behavior, and system health in real time. Detect anomalies the moment patterns deviate. Alert operations teams before users file support tickets. ## Performance at Retail Scale ### 6.9B Records/Sec VERA engine throughput. Your SaaS events, API calls, and user actions processed at scale no internal team can match. ### Sub-10ms Latency From event to action. Real-time features that feel real-time. ### 2x Faster Double the throughput of standard Apache Flink. Ship features on infrastructure that outperforms what your team would build. ### 40% Lower TCO Compared to self-managed Flink. Redirect that budget to product engineering. ## Frequently Asked Questions ### How does Ververica compare to building our own Flink infrastructure? Self-managed Flink requires dedicated infrastructure engineers, ongoing cluster management, and custom tooling. Ververica delivers a fully managed platform with 2x better performance and 40% lower TCO. Your team builds features instead of operating infrastructure. ### What programming languages does the platform support? Ververica supports SQL, Java, and Python for writing streaming applications. All three use standard Apache Flink APIs. Your engineers work in familiar languages with full access to the Flink ecosystem and VERA engine optimizations. ### Can we integrate Ververica into our existing CI/CD pipelines? Yes. The platform provides REST APIs and an SDK for programmatic deployment. Streaming jobs deploy through the same Git-based workflows your team uses for application code. GitHub Actions, GitLab CI, and Jenkins integrations are documented. ### How does pricing work for software companies? Ververica offers consumption-based pricing that scales with your workload. No upfront capacity planning. No overprovisioning. Pay for the compute your streaming applications consume. Contact sales for volume pricing. ### Does Ververica support multi-tenant deployments? Yes. The platform provides namespace isolation, resource quotas, and role-based access control. Run multiple teams and applications on a single deployment with strict isolation between tenants. --- title: "Telecom" description: "Real-time network monitoring, customer experience management, and fraud detection for telecom. Process 6.9B records/sec with sub-10ms latency." lastUpdated: 2026-10-14T07:55:04.000Z source_url: html: "https://www.ververica.com/telecom" md: "https://www.ververica.com/telecom.md" --- # Stream Processing for Telecom Networks at Scale Billions of network events per second. Millions of subscribers generating data simultaneously. Ververica processes it all at 6.9B records/sec with sub-10ms latency. ## Key Reasons To choose Ververica Why Ververica Every cell tower, every base station, every subscriber device generates continuous telemetry. Legacy systems collapse under the load. Sampling data means missing problems. Edge computing, IoT devices, and network slicing multiply event volumes by orders of magnitude. Infrastructure built for 4G workloads cannot keep pace with 5G reality. By the time batch analytics identifies a dissatisfied subscriber, they have already ported their number. Retention campaigns that arrive 48 hours late are campaigns that fail. SIM swaps, subscription fraud, and international revenue share fraud execute in minutes. Daily fraud reports are post-mortems, not prevention. ## Telecom networks generate data at machine speed. Processing it on human timelines is not an option. Use Cases ### Network Monitoring Process CDRs, performance counters, and alarm data from every network element in real time. Detect degradation the moment it begins. Correlate events across radio, transport, and core layers in a single stream. ### Predictive Maintenance Analyze equipment telemetry patterns to identify hardware failures before they cause outages. Schedule maintenance during low-traffic windows. Reduce mean time to repair by acting on predictions, not incidents. ### Churn Prevention Score subscriber behavior in real time. Detect dissatisfaction signals from call drops, billing disputes, and usage pattern changes as they occur. Trigger retention actions within minutes, not days. ### Revenue Assurance Reconcile CDRs, billing records, and interconnect data in real time. Detect revenue leakage the moment records diverge. Close the gap between service delivery and accurate billing. ### Fraud Detection Score every SIM activation, international call, and data session against fraud patterns in sub-10ms. Block fraudulent activity before it generates cost. Eliminate the delay between fraud execution and detection. ### 5G Edge Processing Process data at the network edge with low-latency streaming pipelines. Support network slicing, MEC applications, and IoT device management with real-time event processing where the data originates. ## Performance at Telecom Scale ### 6.9B Records/Sec VERA engine throughput. CDRs, network events, and subscriber data processed without sampling or dropping events. ### Sub-10ms Latency From network event to action. Fraud detection, alarm correlation, and subscriber scoring at wire speed. ### 2x Faster Double the throughput of standard Apache Flink. Process 5G data volumes on infrastructure sized for 4G budgets. ### 40% Lower TCO Measured total cost of ownership reduction. Fewer servers processing more data. Operations costs cut proportionally. ## Frequently Asked Questions ### Can Ververica process the data volumes generated by 5G networks? Yes. VERA engine processes 6.9B records per second. That capacity covers CDRs, IoT telemetry, network performance counters, and subscriber events from 5G networks simultaneously without sampling or data loss. ### How does real-time fraud detection work for telecom? Every event, including SIM activations, international calls, and data sessions, is scored against fraud models in sub-10ms. The system applies rules and ML models to detect patterns like SIM swap fraud and Wangiri attacks before they generate financial loss. ### Does Ververica integrate with existing BSS/OSS systems? The platform connects to mediation platforms, billing systems, CRM, and network management tools through pre-built connectors and standard APIs. CDC connectors capture changes from existing databases without modifying source systems. ### What deployment options exist for telecom operators with data sovereignty requirements? Ververica offers BYOC and self-managed deployment. Both keep all data within your own infrastructure. No subscriber data, CDRs, or network telemetry leaves your environment. SOC 2 Type II and ISO 27001 certified. ### How does the platform support network alarm correlation? Ververica processes alarm streams from radio, transport, and core network layers in a single pipeline. It correlates related alarms, suppresses noise, and identifies root causes in real time, reducing alert volume by up to 90%. ## More documents Left out to keep this file small; each link returns the full Markdown. - [AML Monitoring](https://www.ververica.com/banking/aml-monitoring.md) - [Core Modernization](https://www.ververica.com/banking/core-modernization.md) - [Customer Personalization](https://www.ververica.com/banking/customer-personalization.md) - [Fintech Monitoring](https://www.ververica.com/banking/fintech-monitoring.md) - [Fraud Detection](https://www.ververica.com/banking/fraud-detection.md) - [Mainframe Offloading](https://www.ververica.com/banking/mainframe-offloading.md) - [Real-Time Payments](https://www.ververica.com/banking/real-time-payments.md) - [Regulatory Reporting](https://www.ververica.com/banking/regulatory-reporting.md) - [Risk Management](https://www.ververica.com/banking/risk-management.md) - [Finance Service Industry Proofsheet](https://www.ververica.com/data-sovereignty/finance-service-industry-proofsheet.md) - [Sovereignty playbook for FSI](https://www.ververica.com/data-sovereignty/fsi-sovereignty-playbook.md) - [Sovereignty Evaluation Framework](https://www.ververica.com/data-sovereignty/fsi-streaming-platform-evaluation-framework.md) - [How Ververica Delivers Sovereignty](https://www.ververica.com/data-sovereignty/how-ververica-delivers-sovereignty-for-financial-services-industry.md) - [Sovereignty Checklist for Financial Industry](https://www.ververica.com/data-sovereignty/streaming-sovereignty-checklist-for-financial-services-industry.md) - [Streaming Sovereignty Self Assessment Checklist](https://www.ververica.com/data-sovereignty/streaming-sovereignty-self-assessment-checklist-for-financial-industry.md) - [Sovereignty Evaluation Framework Checklist](https://www.ververica.com/data-sovereignty/streaming-sovereignty-self-assessment-technical-audit.md)