Scaling ML Pipelines on Kubernetes Using Tekton - Overcoming Complexity
CNCF [Cloud Native Computing Foundation] via YouTube
Overview
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the challenges and solutions for scaling machine learning pipelines on Kubernetes using Tekton in this 31-minute conference talk by Tommy Li from IBM. Delve into the limitations of Tekton Pipelines' native functionality for complex ML tasks and discover how custom tasks were developed in collaboration with the Tekton community to overcome single pod environment constraints. Learn about advanced ML scenarios, including data passing, network communication, and resource distribution, and how they can be handled more efficiently in a Kubernetes-native way. Gain insights into the integration of Tekton with KubeFlow Pipelines to eliminate storage bottlenecks and achieve truly scalable ML pipelines in Kubernetes environments.
Syllabus
The Complexity on Scaling ML Pipelines on Kubernetes Using Tekton - Tommy Li, IBM
Taught by
CNCF [Cloud Native Computing Foundation]