Explore the evolution of Domain Specific Architectures for Deep Neural Networks through three generations of Tensor Processing Units (TPUs) in this distinguished lecture by David Patterson. Delve into the resurgence of domain-specific architectures driven by the success of deep neural networks and the slowing of Moore's Law. Learn about Google's TPUv1 and its significant performance improvements for inference tasks. Discover how Google developed the first production DSA supercomputer for the more challenging task of training, deployed in 2017. Examine the impressive capabilities of TPUv2 and TPUv3 supercomputers in training production DNNs with near-perfect linear speedup and superior energy efficiency compared to general-purpose supercomputers. Gain insights from Patterson's extensive experience in computer architecture, including his work on RISC, RAID, and Network of Workstation projects, which have had a profound impact on the industry.
Domain Specific Architectures for Deep Neural Networks: Three Generations of Tensor Processing Units
Paul G. Allen School via YouTube
Overview
Syllabus
Allen School Distinguished Lecture: David Patterson (UC Berkeley/Google)
Taught by
Paul G. Allen School
Related Courses
-
A Decade of Machine Learning Accelerators: Lessons Learned and Carbon Footprint
-
Fast and Accurate Deep Neural Networks Training
-
Effectively Scheduling Computational Graphs of Deep Neural Networks toward Their Domain-Specific Accelerators
-
Mathematics for Deep Neural Networks: Advantages of Additional Layers - Lecture 3
-
Stanford Seminar - Neural Networks on Chip Design from the User Perspective
