Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Inference with Torch-TensorRT Deep Learning Prediction for Beginners - CPU vs CUDA vs TensorRT

Python Simplified via YouTube

Overview

Explore deep learning prediction using Torch-TensorRT in this comprehensive tutorial video. Learn to accelerate inference speed by comparing CPU, CUDA, and TensorRT implementations. Set up the development environment with Docker and Nvidia tools, then dive into using PyTorch to load and utilize the ResNet50 neural network for image classification. Discover techniques for image preprocessing, batch processing, and interpreting model predictions. Implement and analyze benchmarks to compare performance across different hardware configurations. Follow along to trace models, convert to TensorRT, and optimize inference speed. Gain practical insights into deep learning deployment and performance optimization for beginners and intermediate practitioners alike.

Syllabus

- intro
- clone Torch-TensorRT
- install and setup Docker
- install Nvidia Container Toolkit & Nvidia Docker 2
- Torch-TensorRT container option #1
- Torch-TensorRT Nvidia NGC container option #2
- import Pytorch
- load ResNet50
- load sample image
- sample image transforms
- batch size
- prediction with ResNet50
- softmax function
- ImageNet class number to name mapping
- predict top 5 classes of sample image topk
- speed test benchmark function
- CPU benchmarks
- CUDA benchmarks
- trace model
- convert traced model into a Torch-TensorRT model
- TensorRT benchmarks
- download Jupyter Notebook
- HOW DID I MISS THIS???
- thanks for watching!

Taught by

Python Simplified

Reviews

Start your review of Inference with Torch-TensorRT Deep Learning Prediction for Beginners - CPU vs CUDA vs TensorRT

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.