Multiscale Generalized Hamiltonian Monte Carlo with Delayed Rejection

Explore advanced sampling techniques for multiscale distributions in this comprehensive lecture on Multiscale Generalized Hamiltonian Monte Carlo with Delayed Rejection. Discover how combining generalized Hamiltonian Monte Carlo and delayed rejection creates a sampler as efficient as Hamiltonian Monte Carlo, but capable of adapting step sizes for multiscale distributions. Learn about the challenges posed by multiscale distributions, such as Radford Neal's funnel example, and how this new approach overcomes limitations of fixed step sizes. Delve into the mechanics of generalized HMC, including its equivalence to Metropolis-adjusted Langevin dynamics and the use of partial momentum refreshment. Understand the importance of delayed rejection in maintaining directed exploration and how it allows for step size adjustments with Hastings-style corrections. Compare the performance of this method to standard Hamiltonian Monte Carlo techniques, including dynamic forms like the no-U-turn sampler. Conclude with insights into ongoing research on automatic tuning methods using complementary parallel chains, as developed by Matt Hoffman and Pavel Sountsov for their MEADS sampler.