COMPUTER AUDIO/FMOD

CGDD 4603

Why fmod?

Sound you might have in a game: Background music Sound effects Dialog

How do you currently use these in your games?

fmod is very popular Powerful – both 2D and 3D sound,

DSP/effects… Cross platform (PS3, Wii, PC…) Has a designer, sandbox and API

Partial List of Games that use fmod

Allods Online Audition Online Battlestations: Pacific BioShock BioShock2 Brutal Legend Clive Barker's Jericho Cortex Command Crysis Darkfall DJ Hero Batman: Arkham

Asylum De Blob Dragon Age: Origins Dogfighter Far Cry Forza Motorsport 2 Forza Motorsport 3 Guild Wars

Guitar Hero III Guitar Hero: Aerosmith Guitar Hero: World Tour Heavenly Sword Heroes of Newerth Hellgate: London Jurassic Park: Operation

Genesis Just Cause 2 League of Legends Lego Universe LittleBigPlanet/LBP2 Metroid Prime 3 Natural Selection 2 Need for Speed: Shift Nicktoons Unite! Nicktoons: Across the

Second Dimension

• Plants vs Zombies• Pure• Rise of Flight: The First Great Air War• Second Life• Shatter• Shattered Horizon[4]

• Silent Hill: Shattered Memories• StarCraft II: Wings of Liberty• Stargate Worlds• Stranglehold• TimeShift• TNA iMPACT!• Tom Clancy's Ghost Recon• Tom Clancy's Rainbow Six Vegas 2• Tomb Raider: Underworld• Torchlight• Tropico 3• vSide• World of Tanks• World of WarCraft• You Don't Know Jack• Zuma

Things You Should Know…

ADC/DAC – Audio to Digital Conversion Sampling rate

# of times per second the computer “listens”

44.1 KHz is CD 22 KHz is good

Bits per sample Computers have to approximate 4 bits = 16 levels of approximation 8 bits = 256 levels of approximation

Original Sound

Low Sampling Rate

TIME

Low Sampling RateWhat the computer hears

TIME

High Sampling Rate

TIME

High Sampling Rate

TIME

2 bits per sample4 Approximations

TIME

StairStep Effect

TIME

3 bits per sample8 Approximations

TIME

Less StairStep

TIME

Capturing Sounds

Usually done with: a microphone (such as voice) Line in CD Hollywood Edge®

Computer has sound card Input types (RCA, MIDI, mini, ¼”, XLR) Card has quality (plays 16-bit sound)

Need some kind of software SoundForge/Audacity Windows SoundRecorder (gag)

                                                                                                                                                     

                                                      

                    

3D Sound

Sounds have position and velocity There is a listener component Relationship between the two

Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities)

Lots of “psycho-acoustic” options

Back to fmod

There are three primary components: The Sound Designer The Sandbox The API

We’ll use all three Start with Sound Designer

Setting up a Project

Take the advice of the video tutorials Have a separate folder Copy sounds into a “sounds” directory

Keeps the safe Can have sub-directories

Create an “output” directory

Designer Interface

Can delete this if you want, but need to have atleast one group

Events

Used to define sound properties for an event

Can be comprised of One sound Several sounds randomly chosen Monophonic vs Polyphonic Can randomize pitch and attenuation

Events can be Simple Multi-track

A Simple Event

Granular Sounds

These sounds aren’t looping

Allows for a sounds to occur between time ranges

Allows for polyphony

Event Options

Can vary attenuation (dropoff with distance)

Can vary pitch Creates a sound

considerably smaller than a “soundtrack”

Plays forever and is always random!

Sound Definitions

Required for multi-track events

Multi-track Events

Comprised of sound defs in layers

Multi-track Events

Careful when adding sounds Choose sound def Ambient noise is looping Other sounds are granular,

so choose “Oneshot” Have parameters Not based on time!

Multi-track Events

Can cross-fade and set fade out time

Effects

Parameter Properties

Can define range and velocity (to simulate time)

Engine Designer

fmod can specifically work with engine sounds Need for Speed 2

Based on the “load” of the engine Right-click on a

sound instance->properties Auto-pitch

Window->fmod Engine Designer

The Build Process

Know which platform you’re targeting Changes are applied only to that platform

Project->Clean, Project->Build

Interactive Music

Comprised of several short segments of music Intro music Darkening or discovery Fighting/intense fighting Release

Before you begin, you must know Tempo of music (beats per minute – bpm) Beats per measure (time signature)

Cues

Themes and Auditioning

Transitioning via Parameters

Other Things

Supports deployment of multiple languages

Can deploy different builds based on Platform Language

A Look at the API

Basic API Designer API

Built on basic API Can read .fev files

Written for C/C++ #include <fmod.h> // C #include <fmod.hpp> // C++

Read the documentation fmodex.chm

API Parts

A System is the fmod engine A Sound is the raw resource

2D uses FMOD_2D 3D uses FMOD_3D

result = system->createSound(“music.wav", FMOD_2D, 0, &sound);

API Parts

A Channel is an instance of a Sound Each time you play a sound, you get a new

channel Channels can start out paused You can set the

Volume (0.0f – 1.0f) Pan (-1.0f – 1.0f) Frequency (in Hz as a float)

Always use FMOD_CHANNEL_FREE to pick for you

#include <iostream>#include <fmod.hpp>#include <fmod_errors.h>

using namespace std;using namespace FMOD;

void main () {FMOD_RESULT result;System* system;// Create the fmod system. We only need one of theseresult = System_Create(&system);// Specify a max number of simultaneous channelsresult = system->init(100, FMOD_INIT_NORMAL, 0);

Sound* sound;// Decompress the entire mp3 into 16-bit PCM in memoryresult = system->createSound("winning.mp3", FMOD_DEFAULT, 0, &sound);

if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;

}

Channel* channel; // Used for setting volume, pan, pausing...// You have the option of passing that channel as the last parameterresult = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency (44100.0f);

if (result != FMOD_OK) {

cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}while (true) {

cout << "1"; system->update(); // Now required in fmod}

}

Virtual Voices

Supports “virtual voices” when hardware is limited Set from System::init() System::getCPUUsage() Try to keep < 1000

May need to set the priority of a channel if it’s important

channel->isVirtual()

3D Sound in fmod

Sound attenuation Logarithmic Set the mindistance of a sound channel to start

attenuation Bee = 0.1f Jet = 50.0f

Leave max distance alone (default is 10,000) Set 3D settings with System::set3DSettings()

Doppler Distance factor (in cm, m, feet) Rolloff scale (attenuation models)

3D Sound

fmod uses a left-handed coordinate system

result=system->init(100,

(FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D), 0);

With velocity, you must pass it as meters per second

velx = (posx-lastposx) * 1000 / timedelta;

vel = 0.1 * 1000 / 16.67 = 6 meters per second

vel = 0.2 * 1000 / 33.33 = 6 meters per second

void main () {FMOD_RESULT result;FMOD_VECTOR soundPos, soundVel; FMOD_VECTOR listenerPos, listenerVel, listenerForward, listenerUp;

System* system;result = System_Create(&system);result = system->init(100, (FMOD_MODE)(FMOD_3D), 0);

int numHC = 0;result = system->getHardwareChannels(&numHC);cout << "Hardware channels: " << numHC << endl;

Sound* sound;result = system->createSound(“train.mp3", FMOD_3D, 0, &sound);

if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;

}

Channel* channel;result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency(44100.0f); if (result != FMOD_OK) {

cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}float dsp, stream, geometry, update, total;soundPos.y = soundVel.x = soundVel.y = 0.0f;soundPos.z = -100.0f; soundPos.x = 5.0f;soundVel.z = 6.0f;channel->set3DMinMaxDistance(10, 10000);listenerPos.x = listenerPos.y = listenerPos.z = 0.0f;listenerVel.x = listenerVel.y = listenerVel.z = 0.0f;listenerForward.x = listenerForward.y = listenerUp.x = listenerUp.z = 0.0f;listenerForward.z = listenerUp.y = 1.0f;system->set3DListenerAttributes(0, &listenerPos, &listenerVel, &listenerForward, &listenerUp);

while (true) {system->update(); system->getCPUUsage(&dsp, &stream, &geometry, &update, &total);cout << total << endl; soundPos.z+=0.01f;channel->set3DAttributes(&soundPos, &soundVel);

}}

Interfacing with Sound Designer You should have received

.fev file (Designer file – event data) .fsb file (raw audio data) .txt file (describes the events and parameters)

May want to start by creating a helper function

void checkErrors(FMOD_RESULT result) { if (result != FMOD_OK) { cout << "fmod error: " << FMOD_ErrorString(result) << endl; exit(1); }}

Imagine That This is our Project

#include <iostream>#include <fmod.hpp>#include <fmod_event.hpp>#include <fmod_errors.h>using namespace std;using namespace FMOD;

void main () { EventSystem* eventSystem = NULL; EventGroup* eventGroup = NULL; Event* myEvent = NULL; EventParameter* eventPar;

FMOD_RESULT result = EventSystem_Create(&eventSystem); checkErrors(result); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); eventSystem->setMediaPath("..//"); result = eventSystem->load("fmodTest.fev", 0, 0); result = eventSystem->getGroup("fmodTest/beeps", false, &eventGroup); result = eventGroup->getEvent("PossessedComputer", FMOD_EVENT_DEFAULT, &myEvent); result = myEvent->getParameter("proximityToComputer", &eventPar); myEvent->start();

float dir = 0.0001f; float currentParVal = -1.0f; eventPar->setValue(0.1f); while (true){

cout << currentParVal << endl;eventPar->getValue(&currentParVal);currentParVal+=dir;eventPar->setValue(currentParVal);if ((currentParVal >= 1.0)||(currentParVal <= 0.0)) {

dir = -dir;}

}}

Interactive Music

void main () {EventSystem* eventSystem = NULL;MusicSystem* musicSystem = NULL;MusicPrompt* introPrompt, *fightPrompt, *fight2Prompt, *releasePrompt;FMOD_MUSIC_ITERATOR cueIter;FMOD_MUSIC_ITERATOR paramIter;FMOD_MUSIC_ID intensityParID = -1.0f;

FMOD_RESULT result = EventSystem_Create(&eventSystem);result = eventSystem->init(64, FMOD_INIT_NORMAL, 0,

FMOD_EVENT_INIT_NORMAL);result = eventSystem->load("acid_test.fev", 0, 0);result = eventSystem->getMusicSystem(&musicSystem);musicSystem->setVolume(1.0f);result = musicSystem->getCues(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fightPrompt);result = musicSystem->getNextCue(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fight2Prompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &introPrompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &releasePrompt);

musicSystem->getParameters(&paramIter);intensityParID = paramIter.value->id;cout << "paramIter.value->name is " << paramIter.value->name << endl;musicSystem->setParameterValue(intensityParID, 7.5f);result = introPrompt->begin();

while (true){eventSystem->update();

}}

Other Things You Should Look Into…

The 3D Reverb API Asynchronously loading data (by default) Memory management for non-PC

platforms FMOD::Memory_Initialize() // fix size of fmod Using compressed samples

Built-in DSP

Configuration Notes

You’ll be working with Header files

C/C++ -> General->Additional Include Directories

Lib files Linker->General->Additional Library Directories Linker->General->Additional Dependencies

DLLs (which you should put into Windows/System32)

There’s also a FMOD.NET site…

Configuration Notes

You have the option of 64-bit and 32-bit libraries You might have to create a new platform