Snowy Monkey

Building a software synth

My Audio Setup Code February 6, 2010

Filed under: iphone — snowy monkey @ 11:22 am
Tags: , ,

I previously stated that I had a fix for the AudioOutputUnitStart -66681 error, then updated the post stating I’d failed.

Today, Sijo asked if I had any idea about how to fix the problem. Unfortunately I don’t…. But I haven’t had that error for a long time, so I figured I might as well post my audio setup code here (along with the callbacks). Hopefully it’s a useful comparison. It works, and is reasonably documented (IMO). It’s not my exact code, it’s a couple of classes copied & pasted, so it’ll have to be adapted somewhat. Let me know if there are any problems.

First, the AudioSourceInterface. Implement this to provide the audio samples.

#pragma once

#include <Coreaudio/CoreAudioTypes.h>       // All this for SInt16....

class AudioSourceInterface
{
public:
    virtual void audioRequestedFloat(float* output, int numSamples, int numChannels) = 0;
    virtual void audioRequestedInt(SInt16* output, int numSamples, int numChannels) = 0;
};

Now the AudioEngine base class. First the header, which is later subclassed for the iphone implementation.

#pragma once

#include <AudioUnit/AudioUnit.h>

class AudioSourceInterface;

class AudioEngine
{
public:
    AudioEngine();
    virtual ~AudioEngine() {}

    void setup(AudioSourceInterface* audioSource);
    void exit();

protected:
    AudioUnit               mAudioUnit;
    AudioSourceInterface*   mAudioSource;
    float                   mSampleRate;

    virtual void createAudioUnit() = 0;
    virtual void enableAudioUnit() {}
    void setupAudioUnit();

    void setupIntAudioStream(AudioStreamBasicDescription* audioFormat) const;
    void setupFloatAudioStream(AudioStreamBasicDescription* audioFormat) const;

    static OSStatus floatRenderer(void*                       inRefCon,
                                  AudioUnitRenderActionFlags* ioActionFlags,
                                  const AudioTimeStamp*       inTimeStamp,
                                  UInt32                      inBusNumber,
                                  UInt32                      inNumberFrames,
                                  AudioBufferList*            ioData);

    static OSStatus intRenderer(void*                          inRefCon,
                                AudioUnitRenderActionFlags*    ioActionFlags,
                                const AudioTimeStamp*          inTimeStamp,
                                UInt32                         inBusNumber,
                                UInt32                         inNumberFrames,
                                AudioBufferList*               ioData);
};

And the AudioEngine cpp body…

#include "AudioEngine.h"

#include <AudioUnit/AudioUnit.h>
#include <AudioUnit/AudioComponent.h>

#include "AudioSourceInterface.h"

#include "Core.h"

AudioEngine::AudioEngine():
    mSampleRate(kSampleRate)
{
}

void AudioEngine::setup(AudioSourceInterface* audioSource)
{
    mAudioSource = audioSource;

    createAudioUnit();
    enableAudioUnit();
    setupAudioUnit();
}

void AudioEngine::setupAudioUnit()
{
	// We tell the Output Unit what format we're going to supply data to it
	// this is necessary if you're providing data through an input callback
	// AND you want the DefaultOutputUnit to do any format conversions
	// necessary from your format to the device's format.
	AudioStreamBasicDescription audioFormat;
    setupIntAudioStream(&audioFormat);

    OSStatus err = AudioUnitSetProperty(mAudioUnit, kAudioUnitProperty_StreamFormat,
                                        kAudioUnitScope_Input, 0, &audioFormat, sizeof(audioFormat));
    assert(err == noErr);

    // Initialize unit
	err = AudioUnitInitialize(mAudioUnit);
    assert(err == noErr);

    // Set render callback
    AURenderCallbackStruct input;
	input.inputProc = intRenderer;
	input.inputProcRefCon = this;
    err = AudioUnitSetProperty(mAudioUnit, kAudioUnitProperty_SetRenderCallback,
                               kAudioUnitScope_Input, 0, &input, sizeof(input));
    assert(err == noErr);

	// Start the rendering
	// The DefaultOutputUnit will do any format conversions to the format of the default device
	err = AudioOutputUnitStart(mAudioUnit);
    assert(err == noErr);

    // we call the CFRunLoopRunInMode to service any notifications that the audio
    // system has to deal with
	CFRunLoopRunInMode(kCFRunLoopDefaultMode, 2, false);
}

void AudioEngine::exit()
{
	OSStatus err = AudioOutputUnitStop(mAudioUnit);
    assert(err == noErr);

    err = AudioUnitUninitialize(mAudioUnit);
    assert(err == noErr);

    err = AudioComponentInstanceDispose(mAudioUnit);
    assert(err == noErr);
}

void AudioEngine::setupIntAudioStream(AudioStreamBasicDescription* audioFormat) const
{
    memset(audioFormat, 0, sizeof(audioFormat));
    audioFormat->mSampleRate		= mSampleRate;
    audioFormat->mFormatID			= kAudioFormatLinearPCM;
    audioFormat->mFormatFlags		= kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
    audioFormat->mFramesPerPacket	= 1;
    audioFormat->mChannelsPerFrame	= 1;
    audioFormat->mBitsPerChannel	= 16;
    audioFormat->mBytesPerPacket	= 2;
    audioFormat->mBytesPerFrame		= 2;
}

void AudioEngine::setupFloatAudioStream(AudioStreamBasicDescription* audioFormat) const
{
    memset(audioFormat, 0, sizeof(audioFormat));
    audioFormat->mSampleRate		= mSampleRate;
    audioFormat->mFormatID			= kAudioFormatLinearPCM;
    audioFormat->mFormatFlags		= kAudioFormatFlagIsFloat | kLinearPCMFormatFlagIsNonInterleaved;
    audioFormat->mFramesPerPacket	= 1;
    audioFormat->mChannelsPerFrame	= 2;
    audioFormat->mBitsPerChannel	= 32;
    audioFormat->mBytesPerPacket	= 4;
    audioFormat->mBytesPerFrame		= 4;
}

OSStatus AudioEngine::floatRenderer(void*                       inRefCon,
                                    AudioUnitRenderActionFlags* ioActionFlags,
                                    const AudioTimeStamp*       inTimeStamp,
                                    UInt32                      inBusNumber,
                                    UInt32                      inNumberFrames,
                                    AudioBufferList*            ioData)
{
    assert(inNumberFrames <= kAudioBufferSizeFloats);

    AudioEngine* self = (AudioEngine*)inRefCon;

    float* buffer = static_cast<float*>(ioData->mBuffers[0].mData);
    uint bufferSize = ioData->mBuffers[0].mDataByteSize;

    // Get synth sound data for one channel
    self->mAudioSource->audioRequestedFloat(buffer, inNumberFrames, 1);

    // Duplicate single channel across all channels
    for (UInt32 channel = 1; channel < ioData->mNumberBuffers; channel++)
		memcpy (ioData->mBuffers[channel].mData, buffer, bufferSize);

    return noErr;
}

OSStatus AudioEngine::intRenderer(void*                          inRefCon,
                                  AudioUnitRenderActionFlags*    ioActionFlags,
                                  const AudioTimeStamp*          inTimeStamp,
                                  UInt32                         inBusNumber,
                                  UInt32                         inNumberFrames,
                                  AudioBufferList*               ioData)
{
    assert(inNumberFrames <= kAudioBufferSizeFloats);

    AudioEngine* self = (AudioEngine*)inRefCon;

    SInt16* buffer = static_cast<sint16*>(ioData->mBuffers[0].mData);

    // Get synth sound data for one channel
    self->mAudioSource->audioRequestedInt(buffer, inNumberFrames, 1);

    // Duplicate single channel across all channels
    uint bufferSize = ioData->mBuffers[0].mDataByteSize;
    for (UInt32 channel = 1; channel < ioData->mNumberBuffers; channel++)
		memcpy (ioData->mBuffers[channel].mData, buffer, bufferSize);

    return noErr;
}

The AudioEngineIPhoneImpl header.

#pragma once

#include "AudioEngine.h"

class AudioEngineIPhoneImpl: public AudioEngine
{
protected:
    void enableAudioUnit();
    void createAudioUnit();
};

And the AudioEngineIPhoneImpl body.

#include "AudioEngineIPhoneImpl.h"

#include <AudioUnit/AudioComponent.h>

// http://michael.tyson.id.au/2008/11/04/using-remoteio-audio-unit/

const int kOutputBus = 0;
const int kInputBus = 1;

void AudioEngineIPhoneImpl::createAudioUnit()
{
    // Create the audio component instance
    AudioComponentDescription desc;
	desc.componentType = kAudioUnitType_Output;
	desc.componentSubType = kAudioUnitSubType_RemoteIO;
	desc.componentManufacturer = kAudioUnitManufacturer_Apple;
	desc.componentFlags = 0;
	desc.componentFlagsMask = 0;

    AudioComponent comp = AudioComponentFindNext(NULL, &desc);
    assert(comp);

	OSStatus err = AudioComponentInstanceNew(comp, &mAudioUnit);
    assert(err == noErr);
}

void AudioEngineIPhoneImpl::enableAudioUnit()
{
	// Disable input - flag must be 0 to enable larger speakers,
    // otherwise small ear speaker is used.
    UInt32 flag = 0;
	OSStatus err = AudioUnitSetProperty(mAudioUnit, kAudioOutputUnitProperty_EnableIO,
                                        kAudioUnitScope_Input, kInputBus, &flag, sizeof(flag));
    assert(err == noErr);

	// Enable IO for output
    flag = 1;
	err = AudioUnitSetProperty(mAudioUnit, kAudioOutputUnitProperty_EnableIO,
                               kAudioUnitScope_Output, kOutputBus, &flag, sizeof(flag));
    assert(err == noErr);
}

Hopefully all that is a good comparison!

 

How to Access iPhone File Paths Properly January 10, 2010

Filed under: iphone — snowy monkey @ 6:38 pm
Tags: , ,

Paraphrased from Erica Sadun on TUAW over 2 years ago, but very useful for the newbie.

You want to access a data file in your iphone app.  Let’s call it ‘MyAppSettings.xml’.  First thing to do is create it and add it into your Xcode Resources folder (although it can physically reside anywhere.  This should automatically be added to the ‘Copy Bundle Resources‘ list in your target.  When you next build your app, your file will be in the root of your bundle.  You can check by going into the app bundle in your build directory. Right-click build/Release-iphoneos/MyApp (for example) and select ‘Show Package Contents‘.  You should see your file there.

You find the proper path to this file using the following statement:

[[NSBundle mainBundle] pathForResource:@"MyAppSettings" ofType:@"xml" inDirectory:""]]

This will return a NSString with the full path you need.

 

Xcode Developer Documentation is Crashy January 8, 2010

Filed under: Uncategorized — snowy monkey @ 7:58 am
Tags:

I really like most of the developer documentation app that is part of Xcode.  It’s nice and fast, better than accessing it from the browser.  The part I don’t like is that it crashes far too frequently for a piece of non-beta professional software.  And the part I really hate is that when it crashes, it takes the rest of Xcode with it.  I’d love to be able to run it externally to Xcode.  Even better would be if it didn’t crash!  Finally, it’s a bit annoying not being able to cmd-tab between the main IDE window and the docs – if I’ve got more then one other IDE window, cycling to and from the documentation is a pain.

Tags:

 

Frozen Britain from Above January 8, 2010

Filed under: Uncategorized — snowy monkey @ 7:41 am
Tags: , ,

Britain looks like it’s had a fine dusting of icing – incredible photo (from BBC article here).  This photo was taken by Nasa’s Terra satellite on January the 7th.

Tags:

 

Alchemy 008 Released January 8, 2010

Filed under: alchemy — snowy monkey @ 7:31 am
Tags:

Alchemy 008 has just been released.  You can download it here.

I haven’t had a chance to play with it yet, but the main new feature seems to be the addition of modules.  You can add or remove modules as you like, including those you’ve created yourself.  Some examples mentioned on the website are:

  • Shout at the computer. Use your voice to control the width of a line or the form of a shape.
  • Draw ‘blind’. Turn off the canvas display and explore what shapes emerge from the ‘darkness’.
  • Create random shapes. Generate shapes that can be used as a starting point for characters, spaceships, or whatever shape you see in the ‘clouds’.
  • Mirror draw. Draw mirrored symmetrical forms in realtime.
  • Randomise. Mess up and distort shapes.

It’s nearly the weekend now, maybe I can find some free time…

Technorati Tags:

 

iPhone Audio Unit Output Formats January 1, 2010

Filed under: Uncategorized — snowy monkey @ 1:56 am
Tags: ,

I’ve spent a lot of time trying to use the same audio renderer for both mac & iphone versions of the synth.  I developed the code using float buffers, which I can then feed directly to the audio unit of the mac without issues.  However, I’d always had problems doing the same on the iphone – it would come out a bit crackly.  Turns out the iphone doesn’t support using float buffers with the audio units.

Now I have to decide whether to continue to use floating point buffers (keeping the bulk of the audio code identical) and converting to ints, to write a separate, int-based note generator, or to move entirely to ints for both platforms.

On the one hand, I find floats more natural to use, but 16-bit ints would use half as much memory, be faster on the iphone (which is after all the intended platform) and perhaps be more suitable in general.  Check out this forum discussion on the topic of ints vs floats – I’ve skimmed it, but not fully absorbed it all yet.

For now I’ll probably leave the code the same, but convert the buffer before rendering it.  In the future, I’d like to write some solely int-based synth code.  Another for the todo list.

 

The Glitch is Dead! December 29, 2009

Filed under: iphone, synth — snowy monkey @ 12:54 pm
Tags:

Merry Christmas – I’m happy because I’ve fixed the envelope glitch.  In doing so I seem to have ditched the component model I spent ages setting up and put everything into a single note class, but it’s probably no bad thing as I want this to work on the iphone.

Speaking of which, I need to fix that version now – it was left in a slight state of disrepair as a result of my previous misguided efforts to fix the glitch.

So now that the glitch is dead, I’m onto the next glitch, and another redesign of my basic architecture.

I’d initially set it up so that there’s a trigger for each note.  Each trigger had an object attached which contained the state information for the note.  This works fine for short notes, but if the note is longer than the step interval, it’ll get cut off.

The current plan is to have a list of all active notes (better a pool eventually, but for now a list will do) and add to the list when a note is triggered.

Funny how my initial design is pretty much out the window…

 

Ping December 23, 2009

Filed under: iphone, synth — snowy monkey @ 8:24 pm
Tags: , ,

Dead?  Not really.  Just temporarily quiet.

I’ve been busy with other things, mostly non-programming, but I’ve recently picked up the synth again.  After 3 or 4 months on inactivity, I’ve finally figured out what the little audio glitch was….  and it wasn’t what I thought.

I’d spent time thinking maybe the audio buffer size, audio settings or performance of the synth components would be to blame.  Turns out that wasn’t the case at all, and I’m kicking myself for not realising that.

The problem (which I haven’t actually fixed yet…) is that the volume isn’t being updated properly while filling the buffer.  This would only be a problem with the ADSR envelope, as it modified the volume of the input gradually.  I am currently updating the volume of components in one thread, while filling buffers in another.  This leads to a slight ’stepping’ in the audio at the start and end of a note – the glitch I was hearing.

The solution is probably to update the volume of components in the same thread as the audio data is processed.  I just need to figure out how to handle the timing now – I haven’t had a chance to sit down with it and concentrate yet, what with Christmas and all.

Hopefully Santa can deliver a fix on Christmas day.

 

HTML 5/Canvas Demo August 6, 2009

Filed under: Uncategorized — snowy monkey @ 6:25 am
Tags: ,

I’ve been interested in the possibilities for HTML 5 for a while now, quite liking the (relative) ubiquity of flash, but disliking the integration with the browser.  It’s only now that HTML is becoming a viable alternative, with seemingly pretty decent support in Firefox 3.5 and the Chrome.

I’d been thinking it would be good to try and write a few vizualisation demos/toys in HTML 5 & Javascript, but haven’t got round to it yet.  These guys have.  Very impressive!

 

Game of Synth-Life July 11, 2009

Filed under: iphone, synth — snowy monkey @ 4:48 pm
Tags: ,

I thought if would be fun to get a game of life simulation running in the synth.  Sound isn’t the best – I really need to sort that out, but I did get to tidy things up a bit.  Anyway, here it is, enjoy.