Learn about data pipeline management in the age of Large Language Models through this 30-minute conference talk from OSA Con 2023. Explore why traditional ETL (Extract, Transform, Load) approaches remain relevant despite new LLM application development tools. Discover how Singer and the Meltano community offer robust solutions for production-grade data pipelines, comparing their capabilities with LangChain and LlamaIndex's ETL features. Gain insights into emerging data pipeline requirements within AI applications and understand how established data engineering principles can be effectively applied to meet these new challenges.
Overview
Syllabus
ETL with Meltano + Singer in the LLM era
Taught by
OSACon