Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | adam-curtis |
View: | 236 times |
Download: | 2 times |
Introduction to Interactive Media
10: Audio in Interactive Digital Media
Introduction to Interactive Media
What is Sound?
• All sounds are produced by the conversion of energy into vibrations.
• Vibration becomes a wave. When it reaches the ear, it causes the eardrum to vibrate at the same frequency
Introduction to Interactive Media
Audible Spectrum
• Humans can hear sounds in range of 20 Hz – 20 kHz.
• Highest note on piano is approx. 4kHz.
Introduction to Interactive Media
Sound Over Time
• Most sounds change frequency over time.
• Represented in waveform.
Introduction to Interactive Media
Music vs. Speech
Music waveform
Speech waveform
Waveform conveys gross character and dynamics of sound. Can provide cues in syncing media.
Introduction to Interactive Media
The Nature of Sound
• Wave captures three features of sound:– Amplitude
• Perceived as volume.
– Frequency• Perceived as pitch.
– Duration• Length of time sound lasts.
Introduction to Interactive Media
Getting Sound in and Out of the Computer
• Analog to Digital Converter captures separate measures of sound amplitude.– Samples are recorded as
digital numbers.
• Digital values are used to recreate the analog form using a Digital to Analog Converter.
Introduction to Interactive Media
Recording Sound
• Best to not do directly into computer as noise from within the computer is picked up.
• Sample rate can be reduced when recording speaking.
Introduction to Interactive Media
Recording Sound
• When capturing sound, adjust levels.– If amplitude is too low, sound quality is reduced; – If amplitude is too high, clipping occurs and
produces distortion
Introduction to Interactive Media
Sampling Sound
• Quality of the sampling depends on:– Sample resolution– Sample rate.
Introduction to Interactive Media
Sample Resolution
• Number of bits to encode amplitude. • Like images, more bits used to describe
information – more accurately it will be represented.
• Two common sample resolutions are 8-bit and 16-bit.– 8-bit resolution captures 256 different amplitude
levels.– 16-bit sound has 65,000 different levels.
• CD quality sound.
• Inadequate sample resolution can distort the sound.
Introduction to Interactive Media
Sample Rate• Number of samples taken in a fixed interval of time.
– Stated in thousands of Hertz, or kilohertz.– CD-quality sound captures 44.1kHz
to record frequencies as high as 22.05kHz. (The highest frequency the human ear can detect is 20kHz.)
• Two measurements capture each cycle of the sound wave:
• High value or peak• Low value or trough.
Introduction to Interactive Media
Sound File Formats
• Common sampled sound file formats: – WAV– AIFF– AU
• MP3 is most popular compressed format (lossy)– 10:1 file size ratio – WAV:MP3
Introduction to Interactive Media
Manipulating Sound
• Common applications that allow for recording and generate effects.– Apple’s GarageBand– Adobe’s Soundbooth– Sony’s Sound Forge
Introduction to Interactive Media
Common Audio Edits
• Hiss Removal– sample the hiss and remove hiss based on
what you sampled.
• Normalization – brings the average or peak amplitude to a
target level
• Time stretching
• Pitch alteration
Introduction to Interactive Media
Use of Audio in Interactive Media
• Ambience
• Sound Effects
• Auditory Feedback to aid Usability
• Music
• Speech
Introduction to Interactive Media
Sound on the Web
• Not always needed (can be annoying) – Became popular on the Web when it
became possible (late 1990s – early 2000s)
– Fell out fashion on the Web– Web has become more cinematic
Introduction to Interactive Media
How to Integrate Sound
• Authoring applications allow for precise syncing of audio within interactive experience.
Introduction to Interactive Media
How to Integrate Sound
• HTML5– Prior to HTML 5, there was not a standard
for playing audio files on a web.– HTML 5 defines a new element which
specifies a standard way to embed an audio file on a web page: the <audio> element.
• For more information re: format and browser support: http://www.w3schools.com/html/html5_audio.asp
Introduction to Interactive Media
Beyond Sampled Sound
• Sampled sound– All audio discussed so far: WAV, MP3,
AIFF, AU, etc…– Sound is stored as a description of the
sound.
Introduction to Interactive Media
Synthesized Sound
• Sound is stored as a series of commands for the computer to reproduce the sounds.
• Analogous to vector-based graphics.• MIDI (Musical Instrument Digital Interface).
– Codes provided for:• Specific instruments• Notes• Force and duration of note• Routing commands to different instrument channels• Specialized control functions.
Introduction to Interactive Media
MIDI
• Simplest system contains:– Digital musical instrument to create messages– Sound synthesizer to interpret the messages – Amplifier/speaker output system.
Introduction to Interactive Media
MIDI Videos
• Composing – http://www.youtube.com/watch?v=pjPf24XQo1s
• Building Your MIDI Studio– http://www.youtube.com/watch?v=JJLgwaIFePk&feature=fvw