In last week’s post, I discussed how we decided to integrate Pure Data though a native Unity plugin. This week I’ll discuss what the plugin actually does, and what sort of role it plays in bridging Unity and Pure Data.
Pure Data works with metaphorical “patches,” sets of signal-processing or message-handling objects which are connected to each other with patch cords. There are three main ways we need to interact with a patch: we need to be able to send messages to our patch, to receive messages back from our patch, and to actually process and retrieve audio samples for playback. All this has to happen fairly efficiently and not block from either side.
In order to understand how this all works, we need to know a bit about how message processing interacts with audio processing in Pure Data. Pure Data works with 64-sample “ticks,” in that whenever audio is processed it is done in multiples of 64 samples. Once per tick, all currently incoming messages are handled and outgoing messages are sent out.
This poses a bit of a problem for us. In Unity, as with virtually all other game engines, audio needs to be processed in a different thread from the game logic. This is because the audio always needs to be output at a fixed rate of 44100 (or sometimes 48000) samples per second, regardless of game logic or framerate. If so much as a single sample is missed, it’s clearly audible as an unpleasant “click” in the audio. The problem comes from the fact that we want the Pure Data audio processing to happen in our game’s audio thread, but we also want to send and receive messages from the main game update loop.
To deal with this, we need a couple of queues between the API and Pure Data, one to buffer incoming messages and one to buffer outgoing messages. (I used a lockless queue implementation from PortAudio.)
Whenever FRACT OSC sends messages to Pure Data, they get put in the incoming queue. As soon as the audio thread requests samples (in this case from Unity’s MonoBehaviour.OnAudioFilterRead callback,) audio is processed and the incoming messages are handled, often resulting in new outgoing messages which are added to the outgoing queue. When the game loop runs its next update, it pulls the messages out and handles them however it wants.
This is the complete high-level flow! Next time I’ll take a look at exactly what a message is, and how I went about dispatching messages from/to the right Unity scripts and Pure Data sub-patches.