The Elusive Role of High Dimensions in Modern Optimization and Generalization

Explore a thought-provoking lecture on the theoretical challenges of deep learning, focusing on high-dimensionality's complex role in modern optimization and generalization. Delve into recent advancements that address large-scale optimization problems in neural networks, examining how high-dimensionality interacts with large gradients, parallel updates, and post-overfitting training. Learn why high-dimensional problems, contrary to conventional wisdom, may actually be more manageable in modern machine learning contexts. Delivered by Guy Kornowski, an Azrieli graduate fellow and PhD student at the Weizmann Institute of Science, who brings insights from his research in optimization and machine learning theory, including experience from Apple ML Research. The 57-minute presentation challenges traditional perspectives on neural network complexity and offers fresh theoretical frameworks for understanding deep learning success.