Explore the intersection of private machine learning and public data in this 29-minute conference talk by Gautam Kamath from the University of Waterloo. Delve into the concept of differentially private stochastic gradient descent (DPSGD) and its application to datasets like MNIST and CIFAR-10. Examine the impact of public pre-training on private machine learning models, including zero-shot learning techniques for ImageNet. Consider the ethical implications of using public data in private ML contexts, addressing concerns about personal information and privacy. Gain insights into the future of private machine learning and the importance of developing meaningful benchmarks for privacy in the field.
Overview
Syllabus
Intro
Private Machine Learning
Stochastic Gradient Descent (SGD)
Differentially Private Stochastic Gradient Descent (DPSGD)
MNIST
The story so far
Enter "pre-training"
Pre-training for privacy
Why is (public) pre-training useful?
Does public pre-training help? (CIFAR-10)
Does public pre-training help? ImageNet
Zero-Shot Learning for ImageNet
JFT what?
Zero-Shot Learning is taking over
Is this really "private"?
Your secrets
Personal Information
Secrets about you
How can private ML stay relevant?
Focus on more meaningful benchmarks fe privacy
Closing
Taught by
Fields Institute
Related Courses
-
Improving the Privacy-Utility Tradeoff in Differentially Private Machine Learning with Public Data
-
Differentially Private Diffusion Models
-
Federated Learning with Formal User-Level Differential Privacy Guarantees
-
Adversary Instantiation - Lower Bounds for Differentially Private Machine Learning
-
Differentially Private Model Publishing for Deep Learning
-
Differentially Private Online-to-Batch Conversion for Stochastic Optimization
