Modeling Hierarchies of Sound for Audio Source Separation and Generation - 2024 JSALT

Explore the intricacies of hierarchical sound modeling in this comprehensive lecture on audio source separation and generation. Delve into the multi-level granularity of sound signals, from complex music compositions to individual frequency components. Discover how incorporating prior knowledge of hierarchical relationships enhances explainability and controllability in data-driven audio signal processing models. Learn about innovative approaches to audio source separation, enabling novel control mechanisms for interacting with these models. Examine the importance of multi-granular features in detecting and quantifying training data memorization in large text-to-audio diffusion models. Gain insights into the use of classifier probes to understand a large music transformer's knowledge of music and develop fine-grained, interpretable controls. This in-depth talk provides a thorough exploration of cutting-edge techniques in audio processing and generation, offering valuable knowledge for researchers and practitioners in the field.