Global Intelligence Pipeline - Crafting Inference at the Edge

Overview

Explore the cutting-edge world of edge computing and AI inference in this conference talk from Conf42 ML 2024. Delve into the concept of a Global Intelligence Pipeline, learning how to craft inference solutions at the edge for low-latency, real-time processing. Discover Gcore's edge AI solutions, including the use of NVIDIA H100 and A100 GPUs with InfiniBand. Examine market trends in AI adoption, real-time LLM inference examples, and specific use cases for edge AI. Gain insights into solving challenges in LLM inference, understanding network latency impacts, and implementing real-time end-to-end processing. Witness a demonstration of AIoT architecture and learn about achieving optimal network latency goals for edge inference applications.

Syllabus

intro
preamble
gcore at a glance
gcore edge ai solutions
global intelligence pipeline
nvidia h100 and a100 + infiniband gpu
where can i serve my trained model with low latency?
market overview: increasing rever adopting ai
real-time llm inference example
ai use case at the edge
edge ai inference requirements
solving challenges in llm inference
network latency
real-time end-to-end processing
aiot architecture
demo
inference at the edge
network latency goal
thank you!