Numerical Stability of Algorithms at Extreme Scale and Low Precisions
International Mathematical Union via YouTube
Overview
Syllabus
Intro
TOP500: June 2022 Frontier at Oak Ridge AMD EPYC 64C 2GHz, AMD Radeon Instinct GPU. 8,730,112 cores
Growth of Problem Size in TOP500
Today's Floating-Point Arithmetics
Backward Error Analysis for LU Factorizatia
Low Precision in Deep Learning
The (Partial) Explanation
Blocked Inner Products: 2 Pieces
Block Summation
FABsum Error Bound
Random Uniform (0, 1), b = 128, fp32
Extended Precision Registers
Mixed Precision Block FMA
Block FMA Hardware
Error Analysis of Block FMAS Blanchard, H, Lopez, Mary, & Pranesh (2020). Analysis of algs for matrix mult c - AB based on block
NVIDIA V100
Probabilistic Error Analysis Rounding error bounds above are worst-case.
Statistical Effects
Standard Tool for Rounding Error Analysis Theorem If | sufori - 1 n and nu 1 then
Assumptions for Probabilistic Analysis
Probabilistic Analysis Theorem (Connolly, H & Mary, 2021)
Linear Systems
Real-Life Matrices
Probabilistic QR Error Bound Theorem (Connolly & H, 2022) Under Model Mand a technical assumption, for the
Stochastic Rounding Forsythe (1950). .... Croci et al. (2022).
Stagnation Harmonic sum 1/k in fp16.
Random Data
Putting It All Together
Conclusions
Taught by
International Mathematical Union