Explore the process of converting a model to Tensor-RT format in this 34-minute video tutorial. Learn how to compare throughput and inference time by varying batch size and data precision, using both native PyTorch inference and TensorRT runtime. Gain practical insights into optimizing GPU inference for machine learning models, with a focus on LLMOps techniques. Access the accompanying notebook on GitHub to follow along and implement the demonstrated concepts in your own projects.
Overview
Syllabus
LLMOps: How to use Nvidia TensorRT SDK for GPU Inference #datascience #machinelearning
Taught by
The Machine Learning Engineer
Related Courses
-
Optimize TensorFlow Models For Deployment with TensorRT
-
LLMOps: Como usar Nvidia TensorRT SDK para Inferencia en GPU
-
LLMOps: Comparison of OpenVino, ONNX, TensorRT, and PyTorch Inference
-
LLMOps: Comparación de Openvino, ONNX, TensorRT y Pytorch para Inferencia
-
LLMOps: Quantization Models and Inference with ONNX Generative Runtime
-
Inference with Torch-TensorRT Deep Learning Prediction for Beginners - CPU vs CUDA vs TensorRT
