Overview
Syllabus
Intro
(Non)-convex learning
Differential Privacy
Cross-device federated learning
Differentially private stochastic gradient descent DP
DP-SGD: Key insights
DP-Federated Averaging (DP-FedAvg)
Challenges for Amplification by Sampling in FL
Deconstructing the SGD model update
Noise Accumulation in Prefix Sums
Towards Tree Aggregation
Interlude: Follow-the-regularized-leader (FTRL)
DP-Follow-the-regularized leader (DP-FTRL)
DP-FTRL: Online learning properties
Privacy-Utility Trade-offs for Stackoverflow
Production model with formal DP
Matrix factorization view of prefix sum estimation
Matrix factorization view of DP prefix sum
Future directions
Acknowledgements
Taught by
TheIACR