8-bit Methods for Efficient Deep Learning

Explore 8-bit methods for efficient deep learning in this comprehensive talk by Tim Dettmers from the University of Washington. Delve into the challenges of training and inferencing large language models, and discover how 8-bit techniques can improve efficiency without compromising performance. Learn about quantization precision, its impact on model performance and training stability, and how these factors change with scale. Examine 8-bit optimizers for training and Int8 inference for large language models up to 175B parameters. Gain insights into various topics, including quantization basics, floating point data types, dynamic exponent quantization, block-wise quantization, and mixed precision decomposition. Understand the challenges of quantizing outliers with large values and explore emerging features in model training. Investigate bit-level scaling laws, instruction tuning with 4-bit adapters, and the concept of 4-bit Normal Float (NF4). This talk offers valuable knowledge for researchers and practitioners looking to make large models more accessible and efficient.