Deploying AI Models Using Intel AMX CPUs on VMware vSphere with Tanzu Kubernetes

Learn how to deploy AI models using Intel AMX CPUs in a VMware environment through this technical presentation from AI Field Day 4. Discover VMware Private AI with Intel, a collaboration enabling enterprises to build and deploy secure AI models on VMware Cloud Foundation while leveraging Intel's AI software suite and 4th Generation Xeon Scalable Processors. Master the setup process for Tanzu Kubernetes to run AI/ML workloads using AMX CPUs, including specific requirements like Sapphire Rapids or Emerald Rapids CPUs, Linux kernel 5.16+, and hardware version 20 for AMX instruction virtualization. Explore real-world demonstrations of video processing with OpenVINO on vSphere 8, showcasing high-performance AI workloads without dedicated GPUs. Follow detailed guidance on configuring Tanzu with AMX support, from content library setup to cluster definition file creation. Examine performance metrics of the Llama 2 7B LLM inference running on a single fourth-gen Xeon CPU, achieving sub-100ms latency suitable for chatbot applications.