Federated Hyperparameter Tuning - Challenges, Baselines, and Connections

Explore the challenges, baselines, and connections to weight-sharing in federated hyperparameter tuning through this comprehensive conference talk. Delve into the complexities of tuning hyperparameters in federated learning environments, where models are trained across distributed networks of heterogeneous devices. Learn about key challenges in federated hyperparameter optimization and discover how standard approaches can be adapted to form baselines. Gain insights into a novel method called FedEx, which accelerates federated hyperparameter tuning by connecting to neural architecture search techniques. Examine theoretical foundations and empirical results demonstrating FedEx's superior performance on benchmarks like Shakespeare, FEMNIST, and CIFAR-10. Understand the importance of efficient hyperparameter tuning in federated learning and its impact on model accuracy and training budgets.