R Programming & Digital Audio

Donald Byrd9 April 2007

Copyright © 2006-07, Donald Byrd

rev. 7 Feb. 2007 2

Elements of Digital Audio (1)

• Requirements of discrete time sampling– Pohlmann’s “video of ride over bumpy road” analogy

• Sampling rate determines maximum frequency– Human hearing goes up to ca. 15-20 KHz– Sampling Theorem (Nyquist, Shannon, etc.): need 2

samples per cycle• Less than 2 samples/cycle => aliasing• Visual equivalent: wheels “going backwards”

– Video & movies have low sampling (frame) rates

• Practically, need more than 2• CD sampling rate = 44,100 = 20 KHz * 2.205

rev. 6 Feb. 2007 3

Elements of Digital Audio (2)

• Sample depth (bits per sample) determines freedom from quantization noise– Also called bit depth, sample width, or bit width– SQNR (Signal-to-Quantization Noise Ratio) = about 6

dB per bit– For digital audio, almost always 8, 16, or 24 bits– Usually (CDs, etc.) 16 bits => ca. 96 dB SQNR

• Exceed maximum sample => clipping– A nasty type of distortion– Very different from overdriving analog media

rev. 3 Feb. 2007 4

Elements of Digital Audio (3)

• A simple example– Input: sound waves => microphone => Analog to

Digital Converter (ADC) => computer, etc.– ADC needs low-pass filter to avoid aliasing– Output: computer, etc. => Digital to Analog

Converter (DAC) => loudspeaker => sound waves– DAC needs low-pass filter to avoid imaging (related

to & often confused w/ aliasing)• NB: theoretically, should apply “sinc” function

instead of low-pass filter, but that’s impractical (equiv. to ideal low-pass filter?)

• Process introduces noise & distortion

15 Sep. 2006 5

Audio in R: the tuneR Package

• tuneR is an “add-on” library for R• Adds functions to create, work with, &

analyze Wave (.wav audio) files• Installation: type “install.packages()”, or

(with the R GUI) use menu command Packages>Install Packages

• For more information, see “tuneR” under Packages at http://www.r-project.org/

rev. 20 Jan. 2007 6

Structure of the tuneR Wave Object

• left: vector containing samples for left channel

• right: vector containing samples for right channel (NULL if mono)

• stereo: a boolean to say if stereo or mono

• samp.rate: sampling rate(e.g., 44,100 = 44,100 samples per sec. for CD quality)

• bit: sample depth, in bits: controls quantization (usually 16, e.g., for CDs; can be 8 for low quality)

left

samp.rate

right

stereo

bit

An object in R has slots. The Wave object has 5 slots.

rev. 3 Apr. 07 7

Creating a Wave Object from Scratch• install.packages() # do only once after installing R on a computer• library(tuneR) # do every time you run R & need tuneR• setWavPlayer("/Library/Audio/playRWave")'") # do every time you run

R & need to play sounds (OS X only)• #• # Create & play 2.5-sec. sine wave with pitch about middle C (262 Hz)• wavs <- sine(262, duration=2.5, samp.rate=16000, bit=16,

xunit="time")• play(wavs)

rev. 3 April 07 8

Creating a Wave Object from a File

• install.packages() # do only once after installing R on a computer• library(tuneR) # do every time you need tuneR• setWavPlayer("/Library/Audio/playRWave")'") # do every time you run

R & need to play sounds (OS X only)• #• # Set the working directory to the correct path for your computer.• setwd("work")• # wavStereo is an object of type Wave. Read its waveform from the

file.• wavStereo <- readWave("Piano.mf1st5sec.A4.wav")• # Convert it from stereo to mono by averaging samples (into the left #

channel; the right channel is empty in a mono Wave).• wav <- mono(wavStereo, "both")• play(wav)• # For later use, make a copy of the samples and a few other things.• sampdata <- wav@left• sampleRate <- wav@samp.rate• sampleDepth <- wav@bit• len <- length(sampdata)

rev. 27 Jan. 2007 9

What Do We Have?• play(wav)

– Uses whole Wave object• plot(wav, nr=1000)

– Uses whole Wave object• plot(wav@left) --------->• plot(wav@left[1:500])

– Uses just vector of samples• wav

– Shows the Wave’s 5 slots:

Wave Object Number of Samples: 198562 Duration (seconds): 4.5 Sampling rate (Hertz): 44100 Channels (Mono/Stereo): Mono Bit (8/16): 16

rev. 3 Apr. 07 10

Wave Manipulation Example #1

• # Assumes “Creating a Wave Object” already done

• sampdata1 <- sampdata*3

• plot(sampdata1)• wav1 <-

Wave(left=sampdata1, samp.rate=sampleRate, bit=sampleDepth)

• play(wav1)

rev. 3 Apr. 07 11

Wave Manipulation Example #2

• R code– # Assumes “Creating a Wave Object”

already done– newSampleRate <- sampleRate/2^(6/12)– wav2 <- Wave(left=sampdata1,

samp.rate=newSampleRate, bit= sampleDepth)

– play(wav2)

• Effect: pitch is 6 semitones = tritone lower

rev. 3 Apr. 07 12

Wave Manipulation: More Techniques in R

• Not Wave-specific, just standard R– See “An Introduction to R” (R-intro.pdf)

• Under Manuals, at http://www.r-project.org/

• 1. Extract every nth element– sampdata3 <- sampdata[seq(3, len, by=3)]

• 2. Make two sounds overlap– # Append 0’s to sampdata3, or there would

be NA which causes error later– sampdata3[len:round(0.5*sampleRate)] <- 0– sampdata3 <- sampdata3+sampdata

31 Jan. 2007 13

Programming in General (1)

• Details are often vital (& errors are costly)– A great many details really are. Commonly:

• Quote marks, including single/double• Capitalization

– “tuneR” isn’t “TuneR”; also “Wav”/“wav”– TIP: “steal” as much as possible!

• Via Copy & Paste is ideal: avoids typos

• Programs tend to be very hard to understand– TIP: include useful, readable comments– TIP: choose variable names for clarity

• “wavdata” isn’t good; how about “samples”?– TIP: consistency helps clarity and correctness

• Don’t mix “v = expr”, “v <- expr”, and “expr -> v”

• Program defensively

rev. 3 Apr. 07 14

Programming in General (2)

• Comments– Classic example of a bad comment

• x <- x+1 # add 1 to x– Doesn’t explain anything!

• Good commenting style (thanks to Ed Wolf)# Using the Add Sines Demo, create and play a wave at G3,# then do the same for a wave at 5/4 this frequency. Finally, # normalize the sum of the two waves and listen to result.…

# create and play first sound waveswave1 <- sine(f, duration=secs, samp.rate=sr, bit=16,

xunit="time")play(swave1)…

3 April 2007 15

Programming in General (3)• Block comments (w/ overall description) more

important than comments on single stmts• Ideal: say just the right things: Not too much

or too little– Basic principle of all human communication– …including this slide show & music notations (CMN,

tablature, etc.)– …and comments in a program

• Other aspects of formatting & style– Variable names

• Choose variable names for clarity• camelCase is helpful

– Space around operators– “v <- f(expr)”, not “v<-f(expr)”

rev. 20 Jan. 2007 16

Programming in R (1)

• R offers to save workspace when you quit– Are you sure it’s what you want?– TIP: Just say no.

• Can restore original with ‘load(".Rdata")’ (?)

– TIP: Use a text editor & files to save work• If real text editor (not word processor) file, can run with R

“source” command• Regardless, can Copy & Paste, even just part of file

• setwd() to correct path for your computer– Depends on where you have files– Can be tricky, esp. in Windows

• Typical Windows ex.: setwd("C:/Documents and Settings/donbyrd.ADS/Teaching/N560")

• Easier: use R GUI “Change Working Directory” menu command!

rev. 30 Jan. 2007 17

Programming in R (2)• Many useful built-in functions

– Many handle vectors (no loop needed)• diff(v) => vector of consecutive differences• sum(v) => sum of vector elements

– read.table– table (and related functions)– sine, square, noise, bind, mono, etc. (tuneR)

• R (and tuneR) have excellent on-line help– Type either ‘help(sine)’ (e.g.) or ‘?sine’

• …but NB: sometimes need ‘help(“sine”)– TIP: Copy & Paste from help window!– Caveat: terminology is statistics oriented

rev. 3 Apr. 07 18

Programming in R (3)

• Introducing loops– Loops are hard for many beginners– A very simple (though pointless) example

• mnnV <- 1:6 # make mnV a 6-place vector• mnnV # see what mnnV is before loop

• for (n in 1:6) {• mnnV[n] <- n+59• }• mnnV # ...and after

– Instead of “in 1:6”, can use any vector!– C, Perl, etc. users can put the vector in the “for”

• for (n in seq(1, 6)) { – Loop is a type of control statement

9 April 07 19

Debugging in R (1)• Experience: all complex programs have bugs

– Judge in Florida e-voting case: claim voting machine software was buggy is speculation

– True, but… !• Disclaimer: I don’t know any hard evidence

• A good debugger can be very helpful– W/ complex programs– Learning a new language– R has one; user reports on it are welcome!

• W/o a debugger, use print (or cat), & plot– E.g., before & after doing something questionable

• print(c("max before scaling=", max(wNotes@left)))• wNotes <- wNotes*2.5• cat("max after scaling=", max(wNotes@left), “\n”)

– cat merges its arguments, gets rid of the extra parens

9 April 07 20

Debugging in R (2)• Debug on short/simple cases, not

long/complex ones– Makes it practical to look at results of several print

statements– “short/simple” often means not too much data

• It’s usually easy to turn lots of data into a little data– Real situation: themeMaxNum <- 3509– For testing: themeMaxNum <- 4 (or some such)

30 Jan. 2007 21

Dangers of R (1)

• R is more dangerous (w/r/t bugs) than many programming languages & environments– No explicit types => doesn’t warn of

questionable usage– No variable declarations => doesn’t catch typos– Both above like Perl (e.g.), but Java (e.g.) is

great on both => Java programmers likely to be careless

• Defensive programming– always important: bugs can waste tremendous

amounts of time– …but especially in dangerous environments like

R

31 Jan. 2007 22

Dangers of R & tuneR (2)

• “Gotchas” in the loop demo– Operator precedence in “for” statement– Undocumented requirements: integer frequency

• Other real-life examples from I545– Undeclared variable: “allNotes” vs. “allnotes”– Call function that returns a value but ignore value

• Danger is much worse because R often gives lousy feedback for errors– Exception: play w/ unnormalized values– But that’s tuneR, not R

rev. 27 Jan. 2007 23

Programming in R with tuneR

• On OS X (and LINUX): play() problem– Must say what program to use to play Waves

• Either setWavPlayer once, or add 2nd param. to each play()

– OS X can use QuickTime Player• It’s on every OS X machine, & it works, but…

– Usually gives scary error messages; must hit the escape key to get R to continue; leaves open more & more QuickTime Players

– OS X alternative: playRWave• Works perfectly so far, but…• Not pre-installed; you must get & install it.• Instructions at:

– http://www.informatics.indiana.edu/donbyrd/tuneRPlaybackOnMacs.txt