Coursera Cuts Jobs Despite $100M Revenue Milestone

Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Inside TensorFlow - tf.data + tf.distribute

TensorFlow via YouTube

Start learning Write review

Overview

Explore best practices for tf.data and tf.distribute in this 46-minute TensorFlow presentation by Software Engineer Jiri Simsa. Dive into building efficient TensorFlow input pipelines, improving performance with the tf.data API, and implementing distributed training strategies. Learn about software pipelining, parallel transformation, and parallel extraction techniques. Discover the benefits of TensorFlow Datasets (TFDS) and various distributed training approaches, including multi-GPU all-reduce synchronous training and multi-worker setups. Gain insights into performance optimization for ResNet models, parameter servers, and central storage concepts. Understand the programming model and features supported in TensorFlow 2.0 Beta for distributed training.

Syllabus

Intro
ML Building Blocks
TensorFlow APIs
Why input pipeline?
tf.data: TensorFlow Input Pipeline
Input Pipeline Performance
Software Pipelining
Parallel Transformation
Parallel Extraction
tf.data Options
TFDS: TensorFlow Datasets
Why distributed training?
tf.distribute. Strategy API
How to use tf.distribute.Strategy?
Multi-GPU all-reduce sync training
All-Reduce Algorithm
Synchronous Training
Multi-GPU Performance ResNetso v1.5 Performance with
Multi-worker all-reduce sync training
All-reduce sync training for TPUs
Parameter Servers and Workers
Central Storage
Programming Model
What's supported in TF 2.0 Beta

Taught by

TensorFlow

Reviews

Start your review of Inside TensorFlow - tf.data + tf.distribute

Start learning

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.