Learn how to convert a LLAMA3.1 8B Instruct model to IR OpenVino format and quantize it to 4-bit integer precision in this 24-minute video tutorial. Discover the process of performing inference on CPU using the quantized model. Follow along with the provided GitHub notebook to gain hands-on experience in LLMOps, focusing on the OpenVino Toolkit for model optimization and deployment. Explore practical applications in data science and machine learning as you master techniques for efficient model compression and inference on CPU hardware.
LLMOps: OpenVino Toolkit Quantize to 4int LLama 3.1 8B Inference on CPU
The Machine Learning Engineer via YouTube
Overview
Syllabus
LLMOps: OpenVino Toolkit Quantize to 4int LLama3.1 8B Inference CPU #datascience #machinelearning
Taught by
The Machine Learning Engineer
Related Courses
-
LLMOps: Comparison of OpenVino, ONNX, TensorRT, and PyTorch Inference
-
LLMOPs - Inference in CPU with Phi3 4k Instruct ONNX 4-bit Model Using C#
-
LLMOPs: Multimodal Prompting and Inference with Phi-3 Vision 128K Instruct on CPU - ONNX 4-Bit Quantization in C#
-
LLMOps: Converting Video Classifier (ViViT) to ONNX and Inference on CPU
-
MLOps: OpenVino Quantized Pipeline for Grammatical Error Correction
-
LLMOps: Comparación de Openvino, ONNX, TensorRT y Pytorch para Inferencia
