Accelerating Generative AI on Arm CPUs in the Cloud and in Your Pocket

Overview

Discover in this 35-minute conference talk from All Things Open 2024 how to efficiently run AI workloads on modern Arm CPUs without relying on power-hungry GPUs. Learn about the latest advancements in hardware instructions like Neon and SVE/2, ML libraries including Arm Compute Library and Kleidi AI, and explore the evolution of Small Language Models and quantization methods. Gain practical insights into implementing generative AI workloads on personal servers or embedding them into smartphone applications, with a focus on making AI solutions effective, affordable, and widely accessible.

Syllabus

Accelerating Generative AI on Arm CPUs, in the Cloud and in your Pocket - Michael Hall

Taught by

All Things Open

Reviews

Start your review of Accelerating Generative AI on Arm CPUs in the Cloud and in Your Pocket

Taught by

Generative AI with Large Language Models

Intel® Solutions Pro – Principles of AI Everywhere

AI Infrastructure and Operations Fundamentals

Optimizing oneDAL and oneDNN for ARM CPUs to Accelerate AI Workloads

Optimizing Large Language Model Inference for Arm CPUs

Using Intel AMX CPUs for AI and LLMs on VMware vSphere

Never Stop Learning.