Completed
Spark-submit example - SparkPi
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Migrating Airflow-Based Apache Spark Jobs to Kubernetes - The Native Way
Automatically move to the next video in the Classroom when playback concludes
- 1 Introduction
- 2 What will you learn?
- 3 Nielsen Identity in numbers
- 4 Common data pipeline pattern - Airflow DAG
- 5 Spark clusters
- 6 What is EMR?
- 7 EMR pricing - example
- 8 Running Airflow-based Spark jobs on EMR
- 9 Basic Kubernetes terminology
- 10 Kubernetes auto-scale
- 11 Spark-On-Kubernetes overview
- 12 Spark-submit example - SparkPi
- 13 Spark-On-Kubernetes operator example - SparkPi
- 14 Airflow Spark Kubernetes integration
- 15 Common data pipeline pattern - revised
- 16 Connecting the dots... making it production-ready
- 17 Visibility
- 18 Robustness
- 19 Airflow integration current status
