Leveraging ODSA's BoW Die-to-Die Link Technology for Generative AI Inference Transformation

Learn about cutting-edge developments in Generative AI acceleration through this 21-minute technical talk from D-Matrix's Vice President of Product, Sree Ganesan. Explore how D-Matrix is revolutionizing GenAI inference with their Corsair product, featuring innovative technologies like an 8TB/s die-to-die interconnect compatible with ODSA Bunch of Wires (BoW) PHY specification. Discover the unique challenges of GenAI workloads, including compute-bound prompt processing and memory-bandwidth bound token generation, and understand how Corsair's chiplet-based architecture addresses these challenges. Examine the industry's first digital-in-memory compute (DIMC) engine, offering 10X higher memory bandwidth compared to GPUs using high-bandwidth memory, along with advanced numerical formats for memory compression. Gain insights into how these technologies contribute to faster generation speed, improved power efficiency, and lower total cost of ownership compared to traditional GPU solutions.