Watch a research presentation from the 2022 Symposium on Foundations of Responsible Computing where Jiayuan Ye from the National University of Singapore explores how differential privacy analysis of randomized learning algorithms can be improved through hidden state analysis. Learn about extending hidden-state analysis to various stochastic minibatch gradient descent schemes, including "shuffle and partition" and "sample without replacement" approaches. Discover how this analysis yields significantly smaller privacy bounds compared to traditional composition bounds when training with many iterations on high-dimensional data. Examine experimental results from training classification models on MNIST, FMNIST and CIFAR-10 datasets that demonstrate improved accuracy under fixed privacy budgets using hidden-state analysis.
Differentially Private Learning Needs Hidden State or Much Faster Convergence
Harvard CMSA via YouTube
Overview
Syllabus
Jiayuan Ye | Differentially Private Learning Needs Hidden State (or Much Faster Convergence)
Taught by
Harvard CMSA
Related Courses
-
Differentially Private Diffusion Models
-
Improving the Privacy-Utility Tradeoff in Differentially Private Machine Learning with Public Data
-
Premonitions of Public Data for Private ML
-
Differentially Private Model Publishing for Deep Learning
-
The Value of Collaboration in Convex Machine Learning with Differential Privacy
-
New Local Differentially Private Protocols for Frequency and Mean Estimation
