Transformers for Generative Music AI - Part 2: Decoder and Music Generation
Valerio Velardo - The Sound of AI via YouTube
Overview
Syllabus
Intro
Decoder intuition
Decoder input
Decoder block
Training / inference discrepancy
Masked multi-head attention
Add & norm
Multi-head attention
Feedforward
Decoder block
Linear & softmax
Decoder step-by-step
Training a transformer
Music generation with transformers
Valerio's music generation transformer routine
Music data is key
Pros and cons
Most promising research
Key takeaways
What's up next?
Taught by
Valerio Velardo - The Sound of AI