Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Applying DevOps Practices in Data and ML Engineering

MLOps.community via YouTube

Overview

Explore how to apply DevOps practices in data and machine learning engineering in this comprehensive talk from the MLOps Community Meetup. Learn about the challenges of adopting DevOps principles in data engineering and discover how the open-source project Versatile Data Kit (VDK) addresses these issues. Follow along as Antoni Ivanov, Software Engineer at VMware, demonstrates creating and productionizing an end-to-end data pipeline efficiently. Gain insights into automating and abstracting the development process, managing the data journey, and extending functionality through practical examples and a step-by-step demo. Understand how VDK can transform data engineering towards a code-first, fully automated, and decentralized approach, potentially revolutionizing your data and ML workflows.

Syllabus

[] Introduction to Antoni Ivanov
[] Applying DevOps Practices in Data and ML Engineering
[] Agenda
[] DevOps Challenges
[] Versatile Data Kit
[] DevOps for Data as a Service
[] Components
[] VDK, SDK, and VDK Runtime Control Service
[] Automate and Abstract the Development Process
[] Quick Example: DevOps Plugin
[] Automate and Abstract the Data Journey
[] Quick Example: vdk-impala, vdk-trino
[] Data Journey: Ingestion
[] Ingestion Job
[] Demo
[] Step 1: Explore VDK's Functionalities
[] Step 2: Create a Data Job
[] Features in real-time
[] Working with streaming tools
[] VK as a training model
[] Step 3: Ingestion Job
[] Step 4: Processing Job
[] Step 5: Deploy
[] Step 6: Extend Anonymize
[] Step 6: Extend SQL Validation

Taught by

MLOps.community

Reviews

Start your review of Applying DevOps Practices in Data and ML Engineering

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.