Imbue - Training a 70B Model from Scratch - Infrastructure and Challenges

Dive into a comprehensive 59-minute video featuring Bowei from Imbue discussing their ambitious project of training a 70B model from scratch. Explore the intricate details of building infrastructure to support such a massive undertaking, as outlined in Imbue's detailed blog post. Learn about Bowei's background, Imbue's research focus, and the challenges of training a 70B model. Gain insights into the process of building a cluster from scratch, and enjoy anecdotes and a Q&A session. The video covers topics ranging from Hyperstack GPUs to the intricacies of large-scale model training, offering valuable knowledge for those interested in cutting-edge AI infrastructure and development.