Efficient and Portable AI/LLM Inference on the Edge Cloud - Workshop

Overview

Explore efficient and portable AI/LLM inference on the edge cloud in this 48-minute workshop presented by Xiaowei Hu from Second State. Learn about the challenges of running AI workloads on heterogeneous hardware and discover how WebAssembly (Wasm) offers a lightweight, fast, and portable solution. Gain hands-on experience creating and running Wasm-based AI applications on edge servers or local hosts. Examine practical examples using AI models and libraries for media processing (Mediapipe), computer vision (YOLO, Llava), and natural language processing (Llama2 series). Follow along with live demonstrations and run all examples on your own laptop during the session, gaining valuable insights into efficient AI deployment strategies for edge computing environments.

Syllabus

Workshop: Efficient and Portable AI / LLM Inference on the Edge Cloud - Xiaowei Hu, Second State

Taught by

Linux Foundation

Reviews

Start your review of Efficient and Portable AI/LLM Inference on the Edge Cloud - Workshop

Taught by

Tags

Efficient and Cross-Platform LLM Inference in the Heterogeneous Cloud

Cloud-Native AI: Wasm in Portable, Secure AI/ML Workloads

AI Inference on the Edge Cloud Using WebAssembly

Leveraging Wasm for Portable AI Inference Across GPUs, CPUs, OS and Cloud-Native Environments

Leveraging Wasm for Portable AI Inference Across GPUs, CPUs, OS and Cloud-Native Environments

Creating Cloud Native Agents and Extensions for Large Language Models

Never Stop Learning.