Discovering Black-Box Optimizers via Evolutionary Meta-Learning

Watch a 50-minute AutoML seminar exploring the discovery of black-box optimizers through evolutionary meta-learning. Learn how to develop effective update rules for evolution strategies and genetic algorithms using meta-learning approaches. Dive into the implementation of black-box optimizers with self-attention-based architectures that ensure update rule invariance to candidate solution ordering. Understand how meta-evolving on low-dimensional analytic optimization problems leads to discovering new evolutionary optimizers that generalize across unseen optimization challenges, population sizes, and horizons. Explore the performance comparison against neuroevolution baselines in supervised and continuous control tasks. Examine the neural network components, reverse engineering of learned optimizers into explicit heuristic forms, and the transfer of neural network-based operators to white-box optimizers. Discover how a self-referential training approach can be used to train an evolution strategy from scratch using learned update rules for meta-learning loops. The talk includes practical demonstrations using the JAX-based Evolution Strategies library evosax and covers both Learned Evolution Strategy (LES) and Learned Genetic Algorithms (LGA) implementations.