Anyone who has worked on a movie or video game production can attest to the importance of sound in those mediums. While the use of pre-recorded samples has dominated the last few decades, there is an alternate approach that involves synthesizing sounds from first principles using DSP algorithms. These algorithms take up a much smaller memory footprint, while also giving the designer a more interactive way to shape their sounds’ behavior.

But while acoustic models are well developed and understood, their brute numerical implementations are far too computationally demanding. Real-time interactive systems need more specialized DSP techniques. Unfortunately, most resources out there don’t focus on modeling the physical aspects of sound, but on emulating analog electronic circuits for music applications. Those that do are often too mathematical for the sound designer or programmer who lacks the relevant background.

This is why the Physics-Based Sound Synthesis for Games and Interactive Systems course, by Professor Perry R. Cook (with contributions from Julius O. Smith III), is so valuable. Not only is he highly regarded as a researcher, having advanced many new techniques in the field, but he is also known to be an excellent teacher. This makes him ideal for presenting this advanced and specialized subject in an accessible way to the beginner. Furthermore, Cook helped author both the Synthesis ToolKit (STK) C++ library and the ChucK programming language, both of which feature heavily in this course.

The course

The course is available on the Kadenze platform. You can try the course for free, but if you want to submit coursework and get a certificate you will need a premium membership (at $20 per month). You also get a discount offer for Cook’s excellent book on the topic, (20% for those enrolled for free and 50% for premium members). The course should really be treated as a companion to the book, making it essential to read.

I’ve taken a few other Kadenze courses, and in my experience, this course covers the broadest range of topics in the shortest amount of time (each session is only 20-30 minutes long)! It is astonishing how much material has been packed, and the videos are so concise that every minute seems to cover an important concept. It never gets boring, but if you are not familiar with the topics you will need to explore them further on your own.

The coursework assignments consist of multiple-choice tests and practical projects using the ChucK language. You are not required to be proficient in ChucK, as you are often tasked with modifying code that has been prepared for you. Grading criteria for the practicals aren’t strict, and their purpose seems to be to encourage exploration and experimentation with the presented concepts.

While it may not be as well known as other audio languages (such as CSound or PureData), ChucK is easy to learn and has some unique and interesting features. It is also the easiest way to start playing with the STK library if you don’t know C++. If you are not comfortable with any programming language, then I recommend you at least take the course available on Kadenze on the ChucK language, to help with the practicals.

After introducing the fundamental concepts of sound, Cook explains how digital filters work and how biquads can synthesize the modes of rigid objects. These modal representations are much more versatile than traditional samples, as they offer a deeper level of control than recorded samples.  For the practical coursework, you are tasked with recording some rigid object being struck and then extracting and resynthesizing its modes using a provided algorithm. I chose to record my adjustable swing-arm lamp which has some springs inside making an interesting sound. I was pleased with the results and how close they sounded. I had to do this kind of modal analysis for vibrating systems in my previous job, but I never had the opportunity to listen to the results in this way.

Cook then examines the Karplus Strong algorithm, and how it can model a range of closed-loop systems such as strings, tubes, bars, membranes, and even rooms. These sessions introduce one of the most important principles in acoustics – how boundary conditions determine the spectral character of a system. It is vital to take the time to develop an intuitive understanding of this principle.

The rest of the course covers a wide range of topics, including techniques for vocal synthesis, a particle system for ensemble sounds, and how to shape noise textures to synthesize natural ambiances. The next session discusses how to combine these techniques to model complex instruments such as the guitar.  This includes an interesting interview with Julius O. Smith, a pioneer in this field, where he talks about the many challenges involved in modeling these instruments and some clever tricks to overcome them. The final session demonstrates how to interactively control these models.

Takeaways

The course covers fairly advanced topics, what you would expect in the first two years of an undergraduate engineering curriculum. But Cook does a great job making the material accessible, emphasizing intuition rather than mathematical rigor. This doesn’t mean the course is completely devoid of math, but it’s mostly limited to pre-calculus.

This makes the course suitable for a wide audience – from computer scientists and electrical engineers, with a good grasp of DSP concepts but who wish to learn more about the physical aspects of sound, to sound designers wishing to start developing their own algorithms. Even artists from other fields could benefit from understanding the nature of sound in the world around them.

The biggest takeaway I took from the course was that despite being derived from elementary oscillators and noise generators, these synthesized sounds can still come across as natural. It shows the importance of the temporal characteristics of a sound, such as its envelopes and modulation. Spectral characteristics still matter, but we also have a high acuity of how sounds evolve over time and modeling that correctly is critical.

Further Resources

It’s not easy to find resources covering such advanced topics in an accessible manner, but they do exist. Andy Farnell’s Designing Sound is considered a bible on the topic, and written in the same spirit. There’s also a series of articles by Gordon Reid written for Sound on Sound magazine, although they focus on using classic analog synths instead of computers.

There are also other audio prototyping languages worth learning such as Faust (which Kadenze also has a course). Faust has several advanced libraries available including some of the models described in Session 8.

Overall, I highly recommend this course to anyone looking to establish a solid foundation in physical modeling techniques.