.svg)
Delta Lake Overview
Delta Lake is an open source project that enables building a lakehouse architecture on top of data lakes such as Databricks. Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing on top of existing data lakes, such as S3, ADLS, GCS, and HDFS.
Decodable + Delta Lake
Delta Lake connector is available to any Decodable user who wants to use Databricks with data in other systems. It enables ingestion of data into Databricks at the Bronze and Silver stages of the Databricks medallion data layer architecture. Using the Decodable service with the new connector is the simplest way to get data into Databricks so that application developers and data engineers can leverage the AI-driven power of Databricks, but do so via a data ingestion pipeline that is less expensive, less complex and simpler than present batch ingestion. The following video demonstrates the use of Decodable to filter and transform records of Tweets from Twitter before ingesting into Databricks' Delta Lake. Filtering and processing in Decodable saves both storage and processing costs inside Databricks.
