apfeatured

Heading into the world of audio programming may seem a little daunting at first, even with prior programming experience.  There are quite a number of tools and languages available in this area, so I decided to put together this little primer containing various resources to learn more about this fascinating corner of the programming world.  It has been divided into sections by the most common programming languages/tools used, though they are by no means intended to be mutually exclusive.  Though the most common areas that involve audio programming would be video games and audio software/plug-in development, it also has great application in theatre and as a compositional tool.

C/C++, C#

The C-based languages (especially C and C++) are pretty standard requirements in many professional audio programming positions.  A fairly solid understanding of these languages is required due to the high demands of performance-critical DSP.  With their complexity, however, comes almost limitless options and a high degree of control over what you can do.  Here are some good resources and tools to help get started:

musicdsp.org – This site contains a great back-catalogue of DSP algorithms from filtering and EQ, to interpolation and other mathematical functions common to signal processing. Amazing resource!

kvraudio.com – The forums from musicdsp.org migrated over here, and there are many very knowledgeable people that regularly offer up advice and discussion on all things audio programming-related, from theoretical to practical, at the DSP and Plug-In forum.

libsndfile – This is a C library API that allows you to read and write most of the common audio file formats (except MP3, which you can read about on the site why it isn’t included).  This is very useful because writing your own audio file handling routines from scratch is time-consuming and error-prone.

portaudio – This is another C-based audio API, but it differs from libsndfile by allowing real-time recording and playback through your computer’s audio interface.

VST 2.4 – Programming your own standalone audio apps is one way to go, but making plug-ins is also a great way to build your skills up.  Steinberg’s VST is probably the best one to start with since there is a fair amount of resources available on it.  It’s also free (but it does require creating a developer ID with Steinberg), and is supported by a wide variety of host applications on both Windows and Mac.

Max/MSP, Pure Data

Here we have a collection of graphical programming APIs that, nevertheless, offer a great deal of power and flexibility.  These programs can be used to build anything from simple oscillators to entire sythesizers, plug-ins, and even audio engines.  Both are extensible by allowing additional controls and audio routines written in C, Python, Lua, and more.  They also use OSC (Open Sound Control) that allows them to be integrated with other languages and applications.  Max carries with it a price tag of $400 (though a trial is available), but Pure Data is free, making it a great place to start.  A great benefit of these graphical programming environments is that they include very helpful documentation, including examples, of all the objects available for use.  Furthermore, since they are graphical in nature, they lend themselves a little better to the YouTube treatment than traditional programming, so search for tutorials and how-to’s on YouTube as an additional resource.

Max/MSP – This is the official page for Max that also includes many great tutorials and ways to get started.

GamesSound Tutorials – Here are a few tutorials on using Max from the author of the book Game Sound: An Introduction to the History, Theory and Practice of Video Game Music and Sound Design.

Pure Data – This is the official site, offering extensive lessons and documentation on learning how to program with Pure Data.

SuperCollider, Csound

I’m not too familiar with either of these, nor have I worked with them, but Csound in particular is one of the pioneering languages in audio programming (having been around since the mid-1980s) that offers almost limitless possibilities in sound/music creation and manipulation.  It is extensible through programming opcodes.  SuperCollider, like Max and Pure Data, utilizes OSC so it can also be integrated with other environments to great effect.  In fact, there is a wonderful blog site (link below) that showcases SuperCollider integrated with UDK (Unreal Development Kit) in creating procedurally generated sound effects that results in vastly more variations than can be managed by using traditional raw audio assets.

Csound – The official site that includes downloads, samples, and other information on the language.

Csound @ Flossmanual – This site offers a very comprehensive manual for most everything related to Csound, including how to integrate it with Max, Pure Data, Abelton, and more.

SuperCollider – The official SC site where you can find many of the resources you’ll need in getting started, or learning more about this language.

Graham Gatheral – This blog site, as described above, contains tutorials and videos of his work in integrating SuperCollider with UDK through OSC and all the wonderful things you can do with it.

DSP Theory

The ability to code in any of the above programming languages is an important step, but knowledge in actual signal processing theory can’t be overlooked.  Not only for the sake of implementation, but understanding the strengths/weaknesses of particular algorithms, what filters are good for a specific purpose, etc. will be a valuable tool in any audio programmer’s arsenal.

The Scientist and Engineer’s Guide to Digital Signal Processing – This free, on-line book by Steve Smith of over 30 chapters is a great resource for anyone, whether beginner or professional.  It covers a wide range of topics, including filters, FFT, and convolution.

Center for Computer Research in Music and Acoustics – Julius O. Smith provides us with a huge database of information in just about every major area of signal processing.

Bores Signal Processing – Another great web resource that offers a free on-line DSP book that covers the basics of sampling, aliasing, and some other foundational topics.  Additional on-line books that go further into detail come at a price.  If you’re in the UK, there are even on-site classes being offered.

Designing Audio Effect Plug-Ins in C++ – This book was just released at the end of 2012, and contains some very good explanations of DSP theory, including the unit circle and filter design.  The author, Will Pirkle, also uses his own abstraction layer called RackFX to make the VST plug-ins, greatly simplifying the GUI part of the process.

The Audio Programming Book – What is most impressive about this book is the sheer number of supplemental material it comes with.  The DVD includes well over 20 additional chapters on topics ranging from CSound opcodes, to Mac App/iPhone development, to continued discussions of topics in the book.  In addition, source code is included for just about everything covered in the book.

One thing to keep in mind in regards to signal processing is the math involved.  Surely as you browse the resources on theory given above, you will encounter some very heavy math; notably calculus, matrices, and vector algebra.  Don’t let this overwhelm you.  Good math skills is absolutely a requirement in this area, but much of the theory-based stuff you’ll come across includes proofs and derivations that won’t be a necessary requirement at the early stages when you just want to get started implementing, testing, and having fun with it. And believe me, it is fun!