Memory as a Lens to Understand Efficient Learning and Optimization

Explore the role of memory in learning and optimization through this 48-minute lecture presented by Vatsal Sharan from the University of Southern California at IPAM's EnCORE Workshop. Delve into the optimal convergence rates for various optimization problems and examine whether simpler, faster, and memory-limited algorithms like gradient descent can achieve these rates. Discover a potential dichotomy between memory-efficient techniques and those requiring substantially more memory. Investigate how exploring memory-limited optimization reveals new problem structures and suggests novel variants of gradient descent. Gain insights into the relationship between computational efficiency and memory usage in optimization algorithms.