Decodable Extends Data Platform with PyFlink Support to Enhance Real-time Data Processing for AI Applications

San Francisco, August 14, 2024 — Decodable, a real-time data platform powered by Apache Flink and Debezium, announces the integration of PyFlink support into its platform, aimed to significantly boost real-time data processing capabilities for AI applications.

PyFlink, the Python-based API for Apache Flink, empowers development teams to build scalable batch and streaming workloads, including real-time data processing pipelines, large-scale exploratory data analysis, Machine Learning (ML) pipelines, and ETL processes. This capability is particularly advantageous for teams more familiar with Python than Java, who seek to leverage Flink’s powerful stream processing capabilities seamlessly.

"Incorporating PyFlink into Decodable’s platform opens up new opportunities for AI-driven applications," said Sharon Xie, Head of Product at Decodable. "Python's extensive third-party ecosystem, encompassing libraries for data engineering, scientific computing, and AI such as Pandas, NumPy, PyTorch, and TensorFlow, makes PyFlink an ideal bridge between these fields and real-time stream processing. This integration enables developers to train ML models on real-time event data sourced from production RDBMS—performing data cleaning, filtering, and aggregation efficiently all in one unified platform."

"Databases designed to support high-dimensional vector data for Retrieval-Augmented Generation (RAG) systems are pivotal for AI applications," added Xie. "With PyFlink complementing Decodable’s existing Flink SQL and Java capabilities, the platform now offers developers unmatched flexibility to optimize vector data stores with real-time updates. This enhancement ensures that AI models are continuously fed with the latest data, enhancing the precision and reliability of AI-driven insights."

By deploying your PyFlink jobs as custom pipelines on Decodable, you can focus solely on implementing your job, while leaving all the aspects of running the job—like provisioning Flink clusters, securing and updating the underlying hardware, scaling, monitoring, and observing—to the fully-managed Decodable platform. This approach streamlines development efforts and ensures robust, scalable, and secure operations for AI applications—distinguishing Decodable from complex alternatives.

"We've seen firsthand how Decodable accelerates the development of AI applications," said Lior Solomon, VP of Engineering at Drata. "Our engineers swiftly created a prototype in just 12 days, allowing us to expedite the launch of our AI product within two months. Decodable was pivotal in our mission to build a reliable knowledge base for our customers, streamline compliance processes, and significantly reduce response times. It shows how crucial real-time, accurate data is for operational efficiency. With the integration of PyFlink, we're excited to further enhance our ability to leverage Flink’s powerful stream processing capabilities.”

Decodable’s commitment to innovation and customer-centric solutions remains steadfast as it continues to empower data teams in e-commerce, finance, healthcare, and beyond with cutting-edge real-time data processing capabilities.

For more information on how Decodable enhances real-time data processing and AI capabilities with PyFlink support, visit Decodable Summer Updates: PyFlink, Declarative Magic, and more.