Fast, Cheap and Easy Data Ingestion with AWS Lambda and Delta Lake

Explore how to leverage AWS Lambda and Delta Lake for efficient data ingestion and processing in this 31-minute conference talk. Dive into practical examples of working with Delta tables using AWS Lambdas written in Python and Rust. Learn how Lambda's fast, easy, and cost-effective execution environment can be triggered by various services like Kinesis, SQS, Kafka, and S3 Event Notifications to transform your data platform from batch processing to event-driven architecture. Discover techniques for exploring the transaction log, writing updates, performing table maintenance, and querying Delta tables in milliseconds using native Python or Rust libraries. Gain insights to incorporate Lambda into existing AWS-based data platforms, enabling a more responsive and efficient data processing workflow.