Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Data Microservices in Apache Spark Using Apache Arrow Flight

Databricks via YouTube

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the world of machine learning pipelines and learn how to enhance them using Apache Arrow Flight in this 24-minute talk from Databricks. Dive into the fundamentals of Apache Arrow and Arrow Flight, understanding their architecture and key features through examples of Flight servers and clients. Discover the Arrow Flight Spark data source and its potential for building microservices with and for Spark. Compare the performance of Flight against other common transport protocols through benchmarks. Watch a demonstration of a machine learning pipeline running in Spark, powered by Arrow Flight data microservices, to see firsthand how this technology simplifies and accelerates the process. Gain valuable insights into improving data transport efficiency in machine learning pipelines using Apache Arrow Flight and Spark.

Syllabus

Introduction
What Arrow isnt
Building Blocks
What is Arrow Flight
GRPC
Microservices
Parallel streams
Python clients
Spark Source
Spark Source for Flame
Benchmarks

Taught by

Databricks

Reviews

Start your review of Data Microservices in Apache Spark Using Apache Arrow Flight

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.