Audio Engine Programming

Lecture 4

Introduction to FMOD programmers API

3D Sound

Sounds have position and velocity There is a listener component Relationship between the two

Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities)

Lots of “psycho-acoustic” options

Fmod audio development

There are three primary components: The Sound Designer The Sandbox The API

We’ll use all three Start with Sound Designer

Data Driven vs. Low Level API Low-Level API

Everything is done using programming The programmer has complete control

The benefits are lower memory overhead

Continued…

Data-Driven API Takes assets from FMOD Designer

Sound Designer has control over the audio Integrates them into the code using the

EventSystem object

Key Features

The data-driven approach of the EventSystem: Support’s event logic

Allows the Sound Designer to fine-tune or ‘tweak’ the final mix without the need for Programmer intervention

Creates a logical separate of concerns for the Programmer and Sound Designer

Overview of event model

FMOD Low Level

FMOD EventSystem

FMOD FMOD EXEX

FEVevent data

FSBaudio data

Assets from the sound designer

The build process creates the following assets: .FEV file - contains the event meta data .FSB file(s) - contains the audio data The ‘Programmer report ‘- a human readable list of

event names, sound definitions [Optional] A ‘header’ file - A source code header file [Optional] Cache files - Files used to speed up subsequent

builds (These files are not required by the Programmer)

Linking the library

Include the EventSystem header file #include "fmod_event.hpp“

Declare a pointer to an EventSystem FMOD::EventSystem *eventsystem;

Create the object (allocate memory) FMOD::EventSystem_Create(&eventsystem);

Initialize the object eventsystem->init(64, FMOD_INIT_NORMAL, 0,

FMOD_EVENT_INIT_NORMAL);

Loading files

Set the media path (location of files) eventsystem->setMediaPath((char *)MEDIA_PATH);

Load the .FEV file eventsystem->load("examples.fev", 0, 0);

Load an event group eventsystem->getGroup("examples/examples", FMOD_EVENT_DEFAULT,

&eventgroup);

Load an individual event eventgroup->getEvent(“car", FMOD_EVENT_DEFAULT, &car);

Loading Audio

The required .FSB file(s) are listed in .FEV file Both the .FEV and .FSB files should be located in the

directory specified with setMediaPath() The location of audio within the .FSB files is

stored in the .FEV FMOD will load the audio data from the .FSB

files as required for an event, automatically

Loading Options

FMOD_EVENT_DEFAULT Specifies default loading behaviour Event data for the whole group is NOT cached Loading will block any other task until complete.

FMOD_EVENT_NONBLOCKING FMOD will use a thread to load the data Use Event::getState to find out when loading is

complete.

Continued...

FMOD_EVENT_INFOONLY Does not allocate instances or load data Creates a ‘handle’ that allows the programmer to

get information from the event

EVENT Playback

Each events have a number of methods and attributes

To start playback of an event: example->start();

To stop an event: Example->stop();

Updating

Updating the event system should be called once per 'game' tick (or once per frame) eventsystem->update();

This allows the FMOD Engine to refresh data such as: the position of 3D sounds changes to volume, pitch, effects as well as internal mechanisms

Demonstration

Simple Event Demonstrates the necessary code to start

playback of an event

PROVIDING Parameter DATA

Get the parameter called ‘test’ example->getParameter(“test", &test); test will be a EventParameter object

To get the range of the parameter test->getRange(&rangemin, &rangemax);

To set the value of the parameter test->setValue(value);

Demonstration Parameter

Demonstrates the necessary code to provide parameter data to an event

PROVIDING a KEY-OFF If an event parameter is currently sustaining

on a ‘sustain point’,  triggering a keyoff will release it and allow the parameter to continue test->keyoff()

Demonstration

Keyoff Demonstrating the necessary code to leave a

sustain point

Selecting sounds at run-time

The sound to be played can be selected at runtime This is required for situations such as ‘colour

commentary’ in sport games or other situations that require dynamic dialogue

The are two methods: Programmer Sounds Programmer Selected

Demonstration

Programmer Sound When a sound definition containing a ‘ProgrammerSound’ is to

be played, a callback is made The code must then provide a sound object at runtime just

before it's played

Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSound’ method

Demonstration

Programmer Selected When a sound definition with the playmode

‘ProgrammerSelected’ is to be played, a callback is made The code must then provide the index number at runtime just

before it's played The index number identifies which file within the sound definition

is to be played

Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSelected’ method

Closing the Event System

Unload event data eventsystem->unload();

Release audio data in memory fsb->release();

Release memory and close eventsystem memory eventsystem->release();

Setting up a Project

Take the advice of the video tutorials

Have a separate folder Copy sounds into a “sounds” directory

Keeps the safe Can have sub-directories

Create an “output” directory

Designer Interface

Can delete this if you want, but need to have atleast one group

Events

Used to define sound properties for an event Can be comprised of

One sound Several sounds randomly chosen Monophonic vs Polyphonic Can randomize pitch and attenuation

Events can be Simple Multi-track

A Simple Event

Granular Sounds

These sounds aren’t looping

Allows for a sounds to occur between time ranges

Allows for polyphony

Event Options

Can vary attenuation (dropoff with distance)

Can vary pitch Creates a sound

considerably smaller than a “soundtrack”

Plays forever and is always random!

Sound Definitions

Required for multi-track events

Multi-track Events

Comprised of sound defs in layers

Multi-track Events

Careful when adding sounds Choose sound def Ambient noise is looping Other sounds are granular,

so choose “Oneshot” Have parameters Not based on time!

Multi-track Events

Can cross-fade and set fade out time

Effects

Parameter Properties

Can define range and velocity (to simulate time)

Engine Designer

fmod can specifically work with engine sounds Need for Speed 2

Based on the “load” of the engine Right-click on a

sound instance->properties Auto-pitch

Window->fmod Engine Designer

The Build Process

Know which platform you’re targeting Changes are applied only to that platform

Project->Clean, Project->Build

Interactive Music

Comprised of several short segments of music Intro music Darkening or discovery Fighting/intense fighting Release

Before you begin, you must know Tempo of music (beats per minute – bpm) Beats per measure (time signature)

Cues

Themes and Auditioning

Transitioning via Parameters

Other Things

Supports deployment of multiple languages Can deploy different builds based on

Platform Language

A Look at the API

Basic API Designer API

Built on basic API Can read .fev files

Written for C/C++ #include <fmod.h>// C #include <fmod.hpp> // C++

Read the documentation fmodex.chm

API Parts A System is the fmod engine A Sound is the raw resource

2D uses FMOD_2D 3D uses FMOD_3D

result = system->createSound(“music.wav", FMOD_2D, 0, &sound);

API Parts

A Channel is an instance of a Sound Each time you play a sound, you get a new

channel Channels can start out paused You can set the

Volume (0.0f – 1.0f) Pan (-1.0f – 1.0f) Frequency (in Hz as a float)

Always use FMOD_CHANNEL_FREE to pick for you

#include <iostream>#include <fmod.hpp>#include <fmod_errors.h>

using namespace std;using namespace FMOD;

void main () {FMOD_RESULT result;System* system;// Create the fmod system. We only need one of theseresult = System_Create(&system);// Specify a max number of simultaneous channelsresult = system->init(100, FMOD_INIT_NORMAL, 0);

Sound* sound;// Decompress the entire mp3 into 16-bit PCM in memoryresult = system->createSound("winning.mp3", FMOD_DEFAULT, 0, &sound);

if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;

}

Channel* channel; // Used for setting volume, pan, pausing...// You have the option of passing that channel as the last parameterresult = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency (44100.0f);

if (result != FMOD_OK) {

cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}while (true) {

cout << "1"; system->update(); // Now required in fmod}

}

Virtual Voices Supports “virtual voices” when hardware is

limited Set from System::init() System::getCPUUsage() Try to keep < 1000

May need to set the priority of a channel if it’s important

channel->isVirtual()

3D Sound in fmod

Sound attenuation Logarithmic Set the mindistance of a sound channel to start

attenuation Bee = 0.1f Jet = 50.0f

Leave max distance alone (default is 10,000) Set 3D settings with System::set3DSettings()

Doppler Distance factor (in cm, m, feet) Rolloff scale (attenuation models)

3D Sound

fmod uses a left-handed coordinate systemresult=system->init(100,

(FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D),0);

With velocity, you must pass it as metres per second

velx = (posx-lastposx) * 1000 / timedelta;

vel = 0.1 * 1000 / 16.67 = 6 meters per second

vel = 0.2 * 1000 / 33.33 = 6 meters per second

void main () {FMOD_RESULT result;FMOD_VECTOR soundPos, soundVel; FMOD_VECTOR listenerPos, listenerVel, listenerForward, listenerUp;

System* system;result = System_Create(&system);result = system->init(100, (FMOD_MODE)(FMOD_3D), 0);

int numHC = 0;result = system->getHardwareChannels(&numHC);cout << "Hardware channels: " << numHC << endl;

Sound* sound;result = system->createSound(“train.mp3", FMOD_3D, 0, &sound);

if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;

}

Channel* channel;result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency(44100.0f); if (result != FMOD_OK) {

cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}float dsp, stream, geometry, update, total;soundPos.y = soundVel.x = soundVel.y = 0.0f;soundPos.z = -100.0f; soundPos.x = 5.0f;soundVel.z = 6.0f;channel->set3DMinMaxDistance(10, 10000);listenerPos.x = listenerPos.y = listenerPos.z = 0.0f;listenerVel.x = listenerVel.y = listenerVel.z = 0.0f;listenerForward.x = listenerForward.y = listenerUp.x = listenerUp.z = 0.0f;listenerForward.z = listenerUp.y = 1.0f;system->set3DListenerAttributes(0, &listenerPos, &listenerVel, &listenerForward, &listenerUp);

while (true) {system->update(); system->getCPUUsage(&dsp, &stream, &geometry, &update, &total);cout << total << endl; soundPos.z+=0.01f;channel->set3DAttributes(&soundPos, &soundVel);

}}

Interfacing with Sound Designer

You should have created .fev file (Designer file – event data) .fsb file (raw audio data) .txt file (describes the events and parameters)

May want to start by creating a helper function

void checkErrors(FMOD_RESULT result) { if (result != FMOD_OK) { cout << "fmod error: " << FMOD_ErrorString(result) << endl; exit(1); }}

Imagine That This is our Project

#include <iostream>#include <fmod.hpp>#include <fmod_event.hpp>#include <fmod_errors.h>using namespace std;using namespace FMOD;

void main () { EventSystem* eventSystem = NULL; EventGroup* eventGroup = NULL; Event* myEvent = NULL; EventParameter* eventPar;

FMOD_RESULT result = EventSystem_Create(&eventSystem); checkErrors(result); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); eventSystem->setMediaPath("..//"); result = eventSystem->load("fmodTest.fev", 0, 0); result = eventSystem->getGroup("fmodTest/beeps", false, &eventGroup); result = eventGroup->getEvent("PossessedComputer", FMOD_EVENT_DEFAULT, &myEvent); result = myEvent->getParameter("proximityToComputer", &eventPar); myEvent->start();

float dir = 0.0001f; float currentParVal = -1.0f; eventPar->setValue(0.1f); while (true){

cout << currentParVal << endl;eventPar->getValue(&currentParVal);currentParVal+=dir;eventPar->setValue(currentParVal);if ((currentParVal >= 1.0)||(currentParVal <= 0.0)) {

dir = -dir;}

}}

Interactive Music

void main () {EventSystem* eventSystem = NULL;MusicSystem* musicSystem = NULL;MusicPrompt* introPrompt, *fightPrompt, *fight2Prompt, *releasePrompt;FMOD_MUSIC_ITERATOR cueIter;FMOD_MUSIC_ITERATOR paramIter;FMOD_MUSIC_ID intensityParID = -1.0f;

FMOD_RESULT result = EventSystem_Create(&eventSystem);result = eventSystem->init(64, FMOD_INIT_NORMAL, 0,

FMOD_EVENT_INIT_NORMAL);result = eventSystem->load("acid_test.fev", 0, 0);result = eventSystem->getMusicSystem(&musicSystem);musicSystem->setVolume(1.0f);result = musicSystem->getCues(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fightPrompt);result = musicSystem->getNextCue(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fight2Prompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &introPrompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &releasePrompt);

musicSystem->getParameters(&paramIter);intensityParID = paramIter.value->id;cout << "paramIter.value->name is " << paramIter.value->name << endl;musicSystem->setParameterValue(intensityParID, 7.5f);result = introPrompt->begin();

while (true){eventSystem->update();

}}

Other Things You Should Look Into…

The 3D Reverb API Asynchronously loading data (by default) Memory management for non-PC platforms

FMOD::Memory_Initialize() // fix size of fmod Using compressed samples

Built-in DSP

Configuration Notes

You’ll be working with Header files

C/C++ -> General->Additional Include Directories Lib files

Linker->General->Additional Library Directories Linker->General->Additional Dependencies

DLLs (which you should put into Windows/System32)

There’s also a FMOD.NET site…

Configuration Notes

You have the option of 64-bit and 32-bit libraries You might have to create a new platform