Subtractive Mixture Models: Representation and Learning

Explore a 52-minute lecture on subtractive mixture models presented by Antonio Vergari from the University of Edinburgh as part of the Simons Institute's series on Probabilistic Circuits and Logic. Delve into the concept of subtractive mixtures, which can reduce the number of components needed to model complex distributions by allowing the subtraction of probability mass or density. Examine the challenges of learning these models while maintaining non-negative functions, and discover how deep subtractive mixtures can be learned and inferred by squaring them within the framework of probabilistic circuits. Understand how this approach enables the representation of tensorized mixtures and generalizes other subtractive models like positive semi-definite kernel models and Born machines. Learn about the theoretical proof demonstrating that squared circuits with subtractions can be exponentially more expressive than traditional additive mixtures. Analyze empirical evidence showcasing this increased expressiveness in real-world distribution estimation tasks, and discuss the tractable inference scenarios for this new class of circuits.