COLLINS KAREN Mobile Phone Music in Video Games Corrected_karen

8/10/2019 COLLINS KAREN Mobile Phone Music in Video Games Corrected_karen

1/21

P A R V I

GAM ING AE STHE TICS

OUP UNCORRECTED PROOF FIRSTPROOFS, Mon Jul 09 2007, NEWGEN

oxfordhb 9780199913657 part 6.indd 381oxordhb-9780199913657-part-6.indd 381 7/9/2007 8:52:42 AM7/9/2007 8:52:42 AM


2/21




3/21

CHAPTER

A HIS TORY OF HANDH ELD

AND MOBILE VIDEO

GAM E SOU ND

KAREN COLL INS

I

history o handheld gaming is an interesting one or many reasons: it represents

a constant battle o technological constraint against aesthetic choice; it has echoed

the development o home game consoles and arcades in its progress and approach tothese constraints; and it illustrates an ongoing desire or consumers to be able to play

games anywhere, anytime. O particular importance is that there are several distinc-

tions between console and handheld gaming that have impacted the way that handheld

gamesand their soundhave developed. Te portability o handhelds has meant that

these games are ofen played in a public space, and thus designers are conscious o the

act that sound may become intrusive, rather than an aid to gameplay. However, this

does not mean that sound does not play an important role. In act, the development

o headphones designed or handheld gaming, as well as rhythm-action and sound- or

music-based games or handhelds indicates that these devices are by no means meant to

be played silently. Rather, the consideration has to be made at the design stage what rolesound will play, and how to give an alternative to sound in the event that it is switched

off. For instance, speech balloons can be used to indicate events that have been typi-

cally used to represent sound effects, such as in Legend o Zelda: Phantom Compass

(Nintendo ), in which a balloon is used in conjunction with a sound change to

indicate approaching enemy. Tis Zeldagame, or Nintendo DS (a portable device), is

highly effective in integrating speech balloons so that the user can play the game with

the sound off and still be cued to important changes.

A second major distinction is, o course, the processing and memory capabili-

ties o portable devices. Tis directly inuences the type o games that get built, since



Phantom

Hourglass, 2007


4/21

KAREN COLLINS

large games (particularly in the downloadable space) can create processing problems.

Likewise, the act that these games are ofen played in public spaces means that there

is an expectation that the player may be interrupted, or just play or a short time (e.g.,on the bus on the way to school or work, waiting in line at an event, and so on). As

such, casual games have risen to the oreront o the handheld market. Te International

Game Developers Association (IGDA) denes casual games as games that generally

involve less complicated game controls and overall complexity in terms o gameplay or

investment required to get through [the] game (IGDA ). Tis idea o investment

is particularly important to the development o games or the devices: players generally

invest less time, invest less emotion, as well as invest less money in these types o games.

A market report on causal games notes that the games are

un, quick to access, easy to learn, and require no previous special video game skills,

expertise, or regular time commitment to play. Many o these games are basedaround amiliar game concepts that consumers played in arcades or the amily Atarirom childhood. In addition, casual games are usually easy to pause, stop and restartwith little consequence to the players enjoyment. (Casual Games Association )

A consequence o the casual nature o the games and lower price-point o handheld

games has meant that audio has in many cases been developed by non-proessional

sound designers and composers, as the games are ofen developed by an individual or

very small team. Many o the largest mobile phone developers, or instance, still have

no in-house sound designers or composers. Rather, sound is either contracted out

at best, or assigned to a team member whose primary unction is not sound-related.

ypically, contracting out would take place afer the game is already developed, and soaudio can play only a limited role, whereas with console games, it is increasingly the case

that the sound team gets involved early on in the design process in order to play a more

signicant role.

Te history and development o sound in handheld devices is interesting, there-

ore, because it highlights some o these distinctions between the handheld and home

console markets. Tis chapter explores this history o sound in handheld and mobile

games, ocusing on how the sound development has been shaped by both the techno-

logical constraints, and the constraints o the portability and unctions o gaming on

the devices. I rst introduce the market or handhelds, beore delving into this history.

I divide the history into several sections: the pre-digital era, early digital games, the riseo systems with dedicated sound chips, more recent handheld systems, and then nally

games on mobile phones and convergent media devices.

A I M M

Te astest growing area o the video game industry today is undoubtedly that o mobile

and handheld games. Having a consumer base now at ease with carrying a mobile




5/21

A HISTORY OF HANDHELD AND MOBILE VIDEO GAME SOUND

phone, MP player, and other electronic portable devices like digital cameras, portable

DVD players, and GPS units means that game developers can expect that we are now

accustomed to constant uninterrupted access to our media, whether it be music, theinternet, movies, or games. Although handheld portable electronic game devices have

been around or three decades, the convergence o these different portable hardware

units into smartphones and multimedia players has pushed the portable game market

into the mainstream, particularly as adults are no longer embarrassed to be carrying

around a catalogue o games in the guise o a work-related device. Some experts predict

that the industry, currently worth billion a year in the United States, will soar to .

billion by (Duryee ).1

Mobile smart phones are now cutting into the dedicated handheld gaming market.

Nintendo currently has a base o about million users or its DS/DSLite/DSi, and Sony

has about hal that or their Playstation Portable (PSP). However, Apples iPod ouch

is growing at a aster rate than each o these devices and has a base o about thirty mil-

lion users, and the iPhone an estimated twenty million (Wagner ).2Some orecasts

suggest that in the next ew years, the iPhone and iPod ouch will account or o

total handheld games sold (Graf ). In act, the iPhone has shown mobile phone

developers that it is the applications, not the hardware, that will drive mobile sales in the

uture (El Akkad ).

Te audience or mobile gaming is incredibly diverse. Currently, the Asia Pacic

region dominates the market or mobile games at about o the market-share, with

Europe ollowing at about and North America (the rest o the world making up

the remainder).3Statistics on gender rom show a near even distribution o women

and men playing mobile games, although males tend to play or longer lengths o time,and men tended to download more games ( o downloads) (Te Online Reporter

). However, more recent reports suggest that this is changing, and that women now

account or more than hal o mobile game purchases (Hetherington ). Likewise,

Nintendo DS reported a near even split on sales to women and men in , and has

begun targeting women with pink hardware, as well as advertising aimed at emale gam-

ers, such as the I Play or Me campaign that used Liv yler and Carrie Underwood

(OConnor and Breckon ).

T B D H G

Beore delving into a history o digital handheld games, it is worth mentioning a ew

pre-digital precursors. Some o the concepts that were and still are a part o the hand-

held game console marketparticularly the way that users interact with the devices

can be ound in much older non-electronic toys and games. Small mechanical wind-up

clockwork tableau, other clock-work toys in the late nineteenth century, and later

electric toys introduced elements o player interactivity with some orm o built-in

animated, automatic (mechanical or electro-mechanical) response. More recently,




6/21

KAREN COLLINS

Etch-A-Sketchintroduced an interace resembling a handheld television with a screen

and turning knobs (Ohio Art Company ) that brought the idea o a screen-based

toy to children.4

Teickle Bee(Endless Games ) preschool game used magnets anda magnetic wand to guide a bee around through a park without getting stung (touch-

ing the edges), a game play element that has been mimicked ofen in the Nintendo

DS touch-screen stylus, or instance in mini-games incorporated into the latest Zelda

game, Phantom Hourglass(Nintendo ). At times, some spring-driven or mechani-

cal handheld toys like handheld pinball machines and automata would have had small

bells or, later, piezo-electric buzzers and beepers, but the real contribution made was

the tools or devices o player interaction in a handheld device, including winding keys,

wands, turning knobs and buttons. Te simple pleasure and tactility o these interac-

tions undoubtedly helped to open up the market or the later digital games.

In addition to handhelds, o course, it is also necessary to point out that home video-

game consoles and arcades were an important precursor to their mobile cousins. Te

Magnavox Odyssey, the Atari VCS, various Pong games, and popular arcade games

helped to drive the desire or handhelds. Many o these early games were and still are

reproduced in a handheldNamco Bandai has re-released their classic games on the

Nintendo DS (as Namco Museum DS ) and or various mobile phone platorms.

Teir list o top selling games currently includes Pac-Man(originally ), Dig-Dug

(originally ) and Galaga(originally ).5

Finally, it is worth mentioning musical toys as precursors to the music-based and

rhythm-action handheld games in particular. Although musical toys have existed since

at least the eighteenth century with tiny bird-shaped whistles or children, the more

likely precursors to music-based video games are s toys like the Fisher-Price GiantScreen Music box V o , which played melodies while pictures scrolled across a

small screen, and Stylophone () and other pocket synthesizers and experimen-

tal synthesizer-based toys like General Electrics ote-A-une () and Ohio Arts

Sketch-a-une(). It is also worth mentioning Speak & Spelland other electronic

puzzle/learning games, some o whose ideas are ound in games like Nintendos Brain

Age. Lastly, electronic virtual toys and pets like amagotchi, cross over into the video

game realm and have been adapted or new simulation animal games on the Nintendo

DS. O course, toys that make sound are always popular with children, and its likely

that the developers o the very rst handheld games were well aware that the addition o

sound would be an important element in the adoption and popularity o these devices,and help to distinguish them rom existing portable electronics, which were largely con-

ned at the time to business devices.

E D H G

Mattels LED-based Auto Race,6 which came out in , was the rst all-electronic

handheld videogame. Using a simple variation o a calculator chip, it had just bytes




7/21


o memory. Programmed by Mark Lesser, he noted in an interview, Te sound was

implemented using a single output and a piezo-ceramic speakerone output line to

generate sounds and music, such as it was, without any sound driver hardware. Te tog-gling o the speaker had to occur within the program loop, with variable timing to pro-

duce tones (Stilphen n.d.). What this means was that the speaker itsel was a simple

tone generator. Te requency o a tone, measured in Hertz, is the cycles per second o

a waveorm. o generate a Hz tone through a speaker, a waveorm can be created by

toggling a voltage wave ten times per second. For a Hz tone, the wave would need

to be toggled at times per second. In other words, the speaker, sometimes reerred

to as a beeper or bipper, could only make one tone at a time and in only a primitive

ashion. Games, thereore, would have simple beeping sounds, or basic monophonic

melodies made out o these beeps or just a ew notes at a time. AlthoughAuto Racemay

have been the rst, it was Mattels Footballgame o that was probably Mattels most

popular seller. Right rom these early days, sound was nearly always used as a selling

point. A description in the Mattel oy Fair Catalogue advertised, Cut back, avoid

tacklers, run or daylight! Score and hear the Victory Sound! (). Football II, released

the ollowing year, boasted Simulated game sounds include whistle and the victory

tune or those hard-ought touchdowns and eld goals (Mattel oy Fair Catalogue

:). A simple monophonic melody o six notes was about all the games processor

could handle or its theme song. Although these early Mattel LED games primarily had

just simple, quite abstract square wave and noise sound effects, a ew did have such short

melodies at the start or end o the game.Missile Attack(; later released as Battlestar

Galactica Space Alert) or instance played aps when the player lost.

Another toy company, Milton Bradley, was also quick to get in on the handheld mar-ket in the late s with their LCD Microvision, which had been designed by Jay Smith,

previously the creator o the Vectrex home console game system. Te most signicant

difference between the Microvision and Mattels predecessors was that the handheld

could use interchangeable cartridges o games such as Star rek: Phaser Strike, Bowling,

and Pinball. Like the Mattel systems, sound was created with a piezoelectric speaker

beeper, and was limited to simple sound effects o hisses and buzzes, with the occasional

very brie monophonic melody at best.

aking advantage o the basic sound capabilities o early handhelds, Atari was one o

the rst to release an electronic musical-memory game with their ouch Me, released

as a coin-operated arcade machine in and later as a handheld in . ouchMes original yer discussed the use o sound: Beep or Bleep? ouch Mechal-

lenges the player to remember the sequence o sight and sound, and correctly repeat

the pattern. . . . When the sound occurs, the corresponding button lights to give a visual

clue. . . . Did you correctly repeat those sounds when the button lit?7Tere were our

colored buttons that lit up, and our tones on the machine whose patterns would get pro-

gressively more diffi cult. Ralph Baer recalled:

I was in Chicago attending an MOA (Music Operators o America) show o coin-opdevices. I went to these shows routinely on Sanders and Magnavoxs behal to check




8/21

KAREN COLLINS

on the presence o games that might inringe our patents, or which Magnavoxwas our primary licensee. Atari had several coin-op units at the show. One o thesewas ouch-Me. . . . Howard Morrison also saw ouch Me and played it. . . . Some

time later, we discussed the game. We both came to the same conclusion: Nice gameidea, terrible execution . . . visually lousy, miserable sounds! (n.d.)

ouch Me did not catch on, however, until it was redesigned by Baer or Milton

Bradley and released as Simonin . Similar to ouch Me, each button on Simonhad

a corresponding sound and color, and increasingly diffi cult melodies had to be memo-

rized. Tis time, Baer improved on the sounds, as he describes here:

I took on the job o selecting the our tones, which was a non-trivial matter becausethe tones actually dene much o Simons character. Looking through my kidsCompton Encyclopedia or an instrument that can play a variety o tunes with only

our notes, I ound what I was looking or: Te bugle! Henceorth, Simonwas pro-grammed to beep G, C, E and G . . . the bugle sounds that can be played in anysequence and still sound pleasant! (n.d.)

Simon,o course, perhaps because o its improvement overouch Mestones, became

a huge success. In the wake o Simonssales, a wave o copy-cat games ollowed, includ-

ing Super Simon (a two-player version by Milton Bradley in ), Parker Brothers

Merlin(), the iger Copy Cat(), and Castles Einstein(). More complex

was the Logix .E.A.M.M.A..E(Parker Bros ), a tabletop console, which had a

speaker and a primitive sequencer that allowed the user to create their own melo-

dies, described in a November Consumer Reportsmagazine: By way o intro-

duction to this musical entertainment, the toy provides a preprogrammed versiono Oh, Susanna. It didnt really encourage us to proceed, since it was rhythmically

aulty. Persevering, though, we ound music-writing rather a challengeat rst. But

we became rustrated when our laborious compositions were erased when the toy was

turned off ().8Simonhad perhaps paved the way or the success o handheld elec-

tronic games, proving to toy manuacturers that handhelds could enjoy tremendous

prots. It is interesting that Simon, one o the most popular early handheld games,

was sound-based. While other games rarely yet had an off switch or the sound, sound

was nevertheless ofen treated like an aferthought, and certainly played second ddle

to graphics. Simonmanaged to propel sound to the oreront o the handheld gam-

ing industry or a time. Finally, handheld manuacturers realized the important rolethat sound could play in games, and the technology was soon to enable the advance o

sound beyond monophonic beeping.

As the video game industrys rst wave o real successes propelled it beyond mere nov-

elty and into homes with the rise o home consoles like the Atari VCS, the Intellivision

and the Colecovision, handhelds also witnessed increasing sales and popularity. It was

a company which was not yet a household name in the Western world that would intro-

duce one o the most important early handheld systems: Nintendo. Nintendos Game

and Watch series, an LCD-based system eaturing a clock timer and alarm (perhaps the

rst handheld game device to include non-game unctions), went through a series o




9/21


different incarnations. Te rst was the Silver series o , with the game Ball. Other

games could be had, including Flagman, Vermin, and Fire. Te Gold version ollowed

a year later, along with a widescreen version, and then a multi-screen in . Tesegames had a simple piezoelectric speaker, like the Microvision and Mattel handhelds,

and so undemanding sound effects were the norm; sine-wave beeps o varying pitch

and white noises used or crashing sounds, gunshots, and explosions. Te rst really

signicant change, however, was the abletop editions in , with VFD color screens,

and improved use o the simple beeper to include some minimal music. Tese were

mini arcade consoles o single games and were soon copied or licensed by competi-

tors. Coleco, or instance, licensed the popular Donkey Konggame rom Nintendo. Te

manuals or the tabletop games ofen advertised that they played and sounded like the

original arcade versions o games like Frogger, Donkey Kong, Pac-Man,and Galaxian,

indicating that the goal was clearly to emulate the success o the arcade by mimicking

(albeit in a more limited ashion) its game play, graphics, and sound. Te games could

manage monophonic square wave or sine wave music as introduction and game over

music, but during gameplay it was still rare to have music, due to the demands o the

music on the processor.

It is interesting to note that the Consumer ReportsNovember report on these

games mentions sound in a ew cases, indicating that the sound played an important

role even this early on, although its effectiveness was questionable. Te description o

Vanity Fairs Computer Matician tabletop model or instance, which asked a series

o math questions, included sound: Te correct answer elicits a high-pitched, repeat-

ing beep and the word right ashing at the top o the screen. An incorrect answer is

punished by a penetrating low buzz and the word wrong ashing at the bottom othe screen. . . most o the children were unnerved and annoyed by the strident beeping

and buzzing (). Indeed, it was the lack o an understanding o the use o sound as a

eedback device that ofen led to the acoustic rustration o the early days. Since play-

ers could trigger the sounds, yet memory was limited, the sounds would repeat end-

lessly with ofen no control over volume. Moreover, the technology o the time meant

that harsh, abstract square wave sounds and white noise were the norm when it came

to sound, and though children may have enjoyed button-mashing to make noise, those

around them rarely did.

Te rst to introduce cartridge-based handheld tabletops was the Entex Adventure

Vision (Entex has previously made a variety o handheld tabletops like Pac-ManandGalaxian). It also improved sound quality over its predecessors, even including a head-

phone jack. Instructions or the hardware advertised, Your Adventure Vision game

unit has a special internal sound effects generator and a ull speaker to urnish the user

with ull delity sound effects (Entex Industries Inc. ). More critical to advancing

sound capabilities was a new hardware advance: a sound co-processor chip.9Te chip

in the Adventure Vision was capable o creating pure tones o about an octave and a

hal (feen tones), as well as thirteen different sound effects created rom noise. Te

chip, however, like that o the early Atari VCS home console was not precise, and the

requency could vary by about (Boris ; Collins ). With these advances, its




10/21

KAREN COLLINS

perhaps not surprising, then, that along with hardware innovations came some impor-

tant changes in the use o sound. Not merely sound effects, sound had a new unction in

requiring the listener to actively listen. Electronic Gamesmagazine rom July tellsthe player o Adventure Visions Super Cobra, a game that resembled Venture: Super

Cobra makes use o a range o sound effects to enhance the play. Listen to these spe-

cial tones since they requently warn you o what type o enemy the helicopter will ace

next (Electronic Games :). O Adventure Visions Space Force, the same guide

notes Pay close attention to the sound effects. Te approach o alien ships are signaled

by alien cannon re. Tis alerts you to be prepared to shoot. Since the small spacecraf

is worth points, you dont want to let it slip past you (). Space Force, however,

was suspiciously similar in sound to the arcade gameAsteroids, with a simple two-note

descending A to B-at repeated moti, with very similar laser-like space ship sounds. On

the other hand, perhaps or the rst time in handheld sound, the sound effectsbecame

an integral part o the game experience, and turning sound off (as was capable on the

Adventure Vision though not most o its predecessors) could be detrimental to game

play. As sound played more important alerting unctions in games, sound was less likely

to be turned off by the player, and on-going advances ollowed to improve quality o

sound through headphones and speaker, and the rise o dedicated sound chips.

T G B D S

C H

Te Nintendo Game Boy was a signicant advance in handheld game sound, having

a dedicated polyphonic sound chip.10Te original system was released in , and it

went through several changes (Pocket in , Light in , Color in ), eventually

selling over million units. Tough each o the models improved upon the graph-

ics and increasingly streamlined the hardware, the sound remained the same until the

Game Boy Advance. Also, although the sound chip was similar to that o the Nintendo

Entertainment System home console, the Game Boy had a very distinct sound, which

is perhaps one o the reasons why it has been so popular with chiptune musicians. Te

chip had three -bit semi-analog tone channels and one noise channel pseudo-stereoprogrammable sound generator. Te rst channels were square waves, which were com-

monly used or chords or melody. Tere were our duty cycle options or the square

waves, which set the timbre o the wave sound (., , , and ). Te rst

channel had volume and requency sweep, while the second was limited to volume con-

trol, which meant the rst channel was more commonly used or melodies (Gevaryahu

). Channel three was a variable wave channel with -bit programmable samples,

and was commonly used or bass-lines. Channel our was a white noise generator, most

ofen used or percussion sounds. A simple -bit digital to analogue converter with

high pass and low pass lter options output the sound to the amplier. With Game Boy,




11/21


thereore, polyphonic music could be created, and or the rst time complete songs were

heard during game play, such as in etris. Sound effects were still synthesized sounds,

mostly rom the white noise generator, however, and so Game Boy helped to create itsown aesthetic, one which is still popular with chip-tune composers today.

Te rst major challenger to the Game Boy was the Atari Lynx, released the same year

(). It was more expensive than the Game Boy, with an original US sales tag o .

versus or the Game Boy (Console Passion), and so despite the advanced sound

and graphics, did not are as well in terms o consumer sales. Te Lynx sound was con-

trolled by the -bit CPU, although it had a separate sound engine and an -bit digital to

analogue converter (DAC). Tere were our channels o -bit sound available (mono in

its original, and stereo in a redesign). Each channel had a programmable base re-

quency, programmable sequence generator, volume control and a wave shape selector

(including square, triangle, pseudo-random noise and sine waveorms).Te Lynx could,

like the Game Boy, handle polyphonic music and play background music during game

play, but like the Game Boy, the sound effects were most ofen white noise. Te docu-

mentation or the Lynx explains the choice o sound hardware:

What in the world were we thinking when we designed the sound hardware? Te

original design goals were that the circuits

. Be cheap

. Require relatively little CPU to make useul game noises

. Have suffi cient range and accuracy or tolerable music

. Have our channels

. Have direct access to the DACs. Be cheap (Domin and Schick )

Clearly, then, cost was a primary driving actor or the sound technology. Music was at

best an aferthought (note only the reerence to the goal o tolerable music). In these

early days o games, sound effects were viewed as more important than music, and the

ways that sound could be used in games were not ully realized.

Even less successul commercially (likely due to its cost) though equally important

in terms o advances in audio capabilities was the NEC urbo Express, basically a por-

table version o the home console urbograx. In act, the urbo Express could play

the home console cartridges (called Hucards), making it convenient or those who pur-chased the home version. It was, however, released at a prohibitively expensive original

price-point (), and so despite advanced capabilities was not widely adopted. Te

urbo Express eatured a separate sound CPU (a Z, popular in the arcades), and a

six-channel programmable sound generator. Compared to other handhelds, thereore,

the possibilities or polyphonic music were ar superior. Te rst two channels were

capable o simulating a low requency oscillator to create FM-synthesis-like sounds.

Channels ve and six could generate noise and so were used requently or sound effects.

Te channels could even use direct memory access to stream up to -bit sampled sound

into any channel.




12/21

KAREN COLLINS

H G A

Segas Game Gear was released in at a price o US, going backward in sound

quality rom the relatively progressive six-channel PSG o the urbo Express. Te Game

Gear was airly successul despite the lesser sound capabilities, partly due to the large

repertoire o games at Segas disposal. Te console eatured a simple exas Instruments

SN programmable sound generator.11Tis sound chip had been popular in the

arcades in the early s, and was the same chip ound in the ColecoVision, and BBC

Micro.12It had three channels o square wave sound generators, one noise generator,

and output to a mono speaker (stereo sound could be had via headphones). Although

Game Gears sonic capabilities did not represent an advance, its reduced price, along

with Segas large catalogue o games, ensured the success o the handheld.

Sega improved on the Game Gears sound with the release o their Nomad, a

handheld version o the popular Genesis/MegaDrive. Tis included the same SN

chip, but it was peripheral to the main sound chip, a Yamaha YM FM synthesis

chip that had six channels o digitized stereo sound, and one PCM -bit sample chan-

nel (the same chip used in the popular Yamaha DX and DX music keyboards).

Just as on the Genesis, instrument sounds and sound effects had to be coded by hand

in a cumbersome ormat, and thus sounds and instruments were used many times in

different games to save coding new ones. Each FM channel had our operators (wave-

orms) assigned, each with a requency and an envelope with which to modiy the

input. Operator One (the oscillator, or carrier wave) could eed back into itsel, creatinga more complex waveorm, and thus, a different instrument sound. Te other operators

served as modulating waveorms (known as slots). Te oscillator would produce a

basic sine-wave tone, and the modulators could then affect the output waveorm, alter-

ing the complexity o the waveorm, and thereore, its timbre. Tere were eight different

algorithmic congurations o these our operators, presented with suggested instru-

ment uses in Segas technical manual. Although they could be used or other instru-

ment sounds, these suggested sounds became the most common. Te result o the

assembly programming and the eight algorithms meant that many o the same types o

sounds were used over and over in games. ypically, one channel handled percussion,

one bass, one melody, and others were used or ller chords, or arpeggios. Te multiplechannels were also used to create various effects, including phasing and double track-

ing (Collins :).

Although Sega and Nintendo were the clear leaders in the handheld market in the

s, there were a ew other systems that had some moderate success. Te SNK Neo

Geo Pocket and Pocket Color o , a -bit handheld version o the Neo Geo

home console, with essentially two o the SN chips seen on the Game Gear, but

with just kB o RAM available or the sound processor (a Z) to access, and as a result

was similar to the Game Gear in its use o music and sound. Te Bandai Wonderswan

series, designed by Nintendos Game Boy creator Gunpei Yokoi, released in




13/21


with a our-channel digital sound chip also attempted to nudge in on the handheld mar-

ket without much success, although it managed to exclusively license the Final Fantasy

games or a time, leading to some moderate sales in Japan.It was to be the successor to Yokois popular Game Boy that would dominate the

handheld market or a time. Te Game Boy Advance, released in , was the rst o

the Game Boy series to upgrade the sound, by adding two -bit DACs to the original

conguration. Not only the hardware, but the sofware was also improved: the Advance

BIOS contained many sound-related unctions or converting MIDI to Game Boy data,

although it is worth noting that Nintendo did not even bother to release inormation

about sound in their press release specications, suggesting that sound was not consid-

ered as relevant on handheld games as it was on home consoles.

C P H

G C

Te Nintendo DS (and later DS Lite and DSi) was originally released in with a

ew very innovative ideas integrated, including built-in WiFi, microphone, and the

DS stylus. Te DS has virtual surround sound and a sixteen-channel Direct Memory

Access sound chip. Te DS has also become a popular sound toy and has a wide series o

home-brewed music sofware available. Tese include sequencers such as Blipracker,

Nitrotracker, DSstep, glitchDS, DStar; samplers such as Protein[DS], repeaterDS, andSampling Keyboard; and synthesizers, including Midipad, Soundcells, Monome DS,

KaosDS(which converts X/Y touch-screen inormation into synthesized sound), and

Pulse DS(which can use the DSs PSG chip to produce sounds similar to Game Boy).13

What is unique about the DS is the use o the stylus, and thus the Kaos pad can be treated

much like a Stylophone.

Tere are also several notable uses o sound in DS games, including in the Legend o

Zelda: Phantom Hourglass(music by Kenta Nagata and oru Minegishi; sound effects

by oru Asakawa and Sanae Susaki). Te game eatures several places where the user

is required to make noise (via the microphone) in order to advance, even at one place

having to shout at the machine. Sound has, thereore, become an input device or somerecent games.

Another innovative game in which sound plays an important role is Electroplankton,

a kind o art-piece by media artist oshio Iwai, who is the creator o a new handheld

instrument by Yamaha called enori-on. When players draw on the Electroplankton

game screen, plankton react and create unusual sounds. A rather Brian Eno-esque

ambient soundscape is created as a result. One mini-game inside Electroplanktoncalled

Beatnesuses tracks rom Nintendos early NES games, such as Super Mario Bros, and

allows the user to remix these tracks in real time by adjusting the plankton to make the

sound effects rom the game over the music.




14/21

KAREN COLLINS

Te Sony PlayStation Portable (PSP) was originally released in and remains

the primary competitor to Nintendos DS. More a ull multimedia device than merely

a game machine, the PSP is designed or music and movie playback, and so supportsDolby . multi-channel sound and integrated stereo speakers. Because the PSP sup-

ports MP and MP playback, there are ewer limitations in terms o audio capabili-

ties or games beyond the memory/CPU ootprint. It even has a built-in equalizer or

music playback. Although not quite as popular or music-makers as the DS, the PSP

does have a ew music-making options, including a homebrew wireless X/Y MIDI and

mouse control with sofware to control music sequencing sofware Ableton Live rom

the PSP calledplayLive. Tere is also PSPSeq, another Ableton Live controller, andPSP

Rhythm. Te rise o popularity in music-based applications in particular has meant

that handheld devices are increasingly improving sound technology or the devices.

Tis is particularly the case with the recent explosion o rhythm-action games, which

are helping to drive improvements in sound technologies and interace devices or

handhelds.

R-A G H

Rhythm-action games, in which the player must directly interact with the music and/

or sound as the main game play element, to some extent have been incorporated into

some other types o video gamesin several o the Legend o Zeldagames or the Game

Boy Advance, or instance, the player is required to memorize short melodies and playthem back (Ocarina o ime), or have their character dance to various rhythms (Oracle

o Ages, Nintendo ). Full rhythm-action games have become ar more popular in

the wake o the success o Guitar Heroand Rock Band. Guitar Hero, o course, has been

released or the DS as Guitar Hero: On our(Activision ), which used a special

add-on hardware unit called the Guitar Grip. Elite Beat Agents(or the Nintendo DS

), a sequel to the Japanese-only Ouendon,had a similar concept, asking players to

tap the screen with the stylus according to the beat o a song. Cross-over music training

games have also been released, including Rhythm N Notes(Agetec, Inc. ), offering

tonal and rhythmic music training.

Te PSP also has a series o rhythm-action games, including Gitaroo Man Lives!(Koei ), a game o bizarre brilliance that uses the PSP stick to mimic pitch shif-

ing on a guitar. Lumines(Q Entertainment ) is a popular puzzle game based on

patterns o sound and light, created by etsuya Mizuguchi, who was behind the inno-

vative music-based Rezgame or the Playstation. A block-like game vaguely similar to

etris, Lumines(pronounced luminous) integrated the music right into the game play.

Blocks move in time to the beat, and elements o game play change the music (which

includes Japanese acid jazz stars like Mondo Grosso) as you play. Moving or rotating

blocks or creating combinations add layers o sound to the soundtrack in real time.

Another highly notable rhythm-action game or the PSP is Patapon(Sony ), a D




15/21


platorm action-adventure in which players must use musical beats to lead tribal armies.

Different drumbeats are used to signal marching, deending, retreating, and attacking.14

A very recent interesting development in the handheld rhythm-action genre is theincorporation o user-generated playlists into rhythm-action games by pulling a users

MP les off the device, turning their playlist into an interactive music game. Harmonix

(who created Guitar Hero) developed a game called Phaseor the iPod ouch to do pre-

cisely this. Te complexity o the music maps in the game, thereore, can depend on the

type o music the user listens to, and there is a never-ending possibility o new game play.

C O M P

T

Beore delving into mobile phone/personal digital assistant (smart phone) gaming, it is

worth taking a minute to examine a ew systems that attempted to bring together phone

or smart phone capabilities to gaming systems. Tese game systems all ailed to achieve

any degree o success and with the arrival o smartphones are likely to never be attempted

again. iger Electronics game.com system released in was perhaps the rst game

console aimed at an older audience. It eatured a touch screen and stylus like the smart

phone systems o the time and could be connected to a modem (although it was limited

to text-only access to the Web). Only a ew games were ever released or the system,

although some o its innovations were incorporated into the Nintendo DS. Nokia simi-larly achieved little success with their N-Gage, released in . It was essentially a game

system with cell phone unctionality, MP playback, and Bluetooth connectivity. What

is interesting about the N-Gage is that Nokia (previously a phone company) attempted

to get in on the gaming market by launching their own gaming division, producing a ew

games and licensing some other successul games. iger elematics (not to be conused

with iger Electronics) also tried to bring smart phone unctionality to a game system

with the Gizmondo in . Te Gizmondo had GPS, Bluetooth, and WiFi, but unor-

tunately the system was so unsuccessul the company was orced to le or bankruptcy.

M P G

Te history o mobile phone games (distinguished rom handheld gaming in that games

are not the primary intended use o the machines) in many ways echoes the history

o consoles and handhelds. Te earliest mobile phone game, Snake, written by aneli

Armanto on the Nokia in , was black and white and lacked sound. Te phone

also eatured basicMemoryand Logicgames. Tese games were originally installed on

phones by the phone companies, but are now commonly purchased through mobile




16/21

KAREN COLLINS

networks or through internet download. Furthermore, mobile phones are increasingly

becoming small computers, known as smartphones, allowing or more complex games,

and opening up a whole new market in the mobile game territory.One o the most important elements o mobile gaming has been network capabilities.

Even many early games had multiplayer modes, so simple games like checkers could

be played with live competitors. Real-time network play on mobile phones has become

increasingly popular, particularly with games like Bejeweled Multiplayer(Jamdat ).

Some games can also interace with their PC counterparts, such as Ragnarok Mobile

Mage(Skyzone Mobile ), so players can continue their game afer they have lef

their house.

Mobile gaming is rapidly becoming an important part o the games industry as the

technology converges, with publishers like Jamdat joining orces with Electronic Arts.

Some companies that have been in the business a long time are now re-releasing their

classic arcade games or the mobile community, such as Namco, Capcom, Konami, and

Vivendi Games.

Mobile games today are increasing at a tremendous rate. However, considering that

one o the purposes o such devices is to communicate through sound, it is ironic that

audio capabilities lag behind graphics. Many o the same diffi culties and constraints that

plagued early computer games have been seen in the mobile game audio world. Audio

resources on some phones can be as small as kB, and as such, many o the same

techniques o the -bit era have been reproduced to save space. As ringtone composer

Peter Drescher elaborated a ew years ago, Te most important trick is to be as ruth-

lessly effi cient as possible. You want to squeeze every last drop o variation out o each

and every byte o audio data at your disposal. Repetition is the enemy, compression isyour ally, and creative use o limited resources is your battle cry (). A developer

at Glu Mobile elaborates: Te challenge with mobile games is that we have sound size

limitations . . . While we dont use voice actors, we do use music a lot. We have to think

about what will take up the majority o space. We have to think about the visuals and the

soundwhat will create the richer experience (Hetherington ). A constant balance

is struck, thereore, between what is needed and what can be technically accomplished.

Tis problem with technological constraints is rapidly changing as phones move

into the smartphone territory and MP sampling or the games. Te Apple iPhone

or instance can record and play back audio in a variety o ormats. Many popular

games or the iPhone are old games revisited, including etris (Electronic Arts),Sega Columns, Ms Pac-Man, and Frogger. O course, there are also many other games

available. Rhythm-action games like Guitar Rock ourand ap ap Revengehave also

been released. But along with traditional games have also come a series o very inno-

vative changes to the nature o games that grew out o the technology. Soul rapper

Episode , or example, is a game rom Realtime Audio Adventures, a kind o cross

between a radio drama and old-ashioned Choose Your Own Adventure game. Te

company promises Realtime Audio Adventures take your mind deeply into the

spectrums o sound and imagination or a truly unique gaming experience.15David

Warhol, who runs the company, noted Everyone seems to appreciate that we pulled




17/21


out all the stops to make the audio as compelling as possible (iPhone Footprint

). Despite the desire or good audio, though, hardware limitations in terms o

processing power and memory remain. Soul rapper, or instance, is an enormous MB, a massive download size or a phone. Te result means heavily compressed

samples that signicantly reduce sound delity. In other cases, some platorms dont

support simultaneous layering o audio tracks, meaning there can be sound effects or

music, but not both at the same time.

Te iPhone and iPod ouch have more recently become musical instruments with the

development o sequencers and sound tools/sound toys.Looptasticand Sound Warp(both

by Sound rends) or instance, allow users to mix loops, manipulate sound effects, record

music, and play back songs. Perhaps even more interesting is, due to the accelerometer

available on the device, the development o gestural music applications, likeBravo Gustavo,

a conductor game in which the player swings the phone, released by the LA Philharmonic.

Despite a seeming desire, then, at least amongst some companies, to create quality

audio or mobile games, even as the technology catches up, there are still distinct di-

erences between mobile games and home console counterparts. Mobile audio has di-

erent requirements than other games due to the casual nature o the gamespeople

playing mobile games are, at present, primarily using their phone to play games while

they kill time, and games are secondary to the other unctions o these machines.

Nokias guide to sound or phone games, or instance, warns, Te game should be

playable without the sounds. Allow silent starting o games. I intro music is imple-

mented, there must be a way to switch it off. Prompt or sound settings at the start o the

game. . . . Do not include loud or high-pitched sounds, have sounds on as the deault,

[or] use sounds that are similar to ring tones and alert tones (Nokia Corporation ).Some mobile games do not even include sound or have minimal sound effects at

best. Fieldrunners,an iPod/iPhone game listed by imemagazine as one o the top

mobile games o , did not initially have any sound, although the . upgrade

included sound effects and a theme song (Grossman ). One article on game

sound or the iPhone asks, Can you hear that? Probably not, i like most gamers

you ick the silent switch on your iPhone beore playing a game. O the hundreds o

games populating the App Store, ew earn the prestigious honour o having the vol-

ume turned up (Erickson ). Magmics Joshua Ostrowalker described to me: Te

rst thing you want is a screen that gives the options to turn on or off the sound.16It is

interesting, then, that there is a growing number o audio-based and rhythm-actiongames or the mobile phone market. It appears that as gaming on phones increases,

people are more inclined to carry around headphones to play games and use sound.

T F H G A

Now that smartphone mobile games are at about the same technical level as gaming

handhelds, the desire or better mobile audio is increasing. Where previously games on




18/21

KAREN COLLINS

mobile phones could get away with little or no audio, the demand or better gamesand

better game audioon mobiles is increasing as these platorms converge. Nevertheless,

there remain some diffi culties and constraints that are holding back audio advance-ments on both handhelds and mobiles.

One signicant problem primarily affecting mobiles but also affecting handhelds

is the lack o standards between devices. For game developers, this means generally

ocusing on a single manuacturer or platorm, thus shrinking their potential market

size. Developers o iPhone games, or instance, cannot easily convert these games

to work on a Blackberry, just as PSP games cannot be easily ported to the Nintendo

DS. Another signicant problem is size o audio les, particularly or mobile games,

since these games are downloaded rather than cartridge-based and, thereore, play-

ers may not want to pay or the bandwidth or be able to download the massive le

sizes required.

Tere have also been several advances that have come in the last year or so that will

help to make handheld and mobile video games a much better audio experience. Mobile

DLS (Downloadable Sounds) is a standardization specication that denes a wavet-

able instrument ormat or mobiles. Tis is essentially a sound bank o instruments or

MIDI enabled devices that improve sound quality o MIDI or mobiles. Packaged with

Mobile DLS is usually Mobile XMF (eXtensible Music Format), another specication

that enables users to use sampled sounds without large le sizes by bundling Mobile

DLS sounds into one le (MIDI Manuacturers Association ). Tere are also DSP

(sound processing) capabilities in some systems now, with the result that simulating

environments in games is becoming easier with reverb and other effects. Finally, there is

also the use o D positional audio in headphones which will also give the games a senseo space or environment by creating the illusion that sound are emanating rom or head-

ing to anywhere in the three-dimensional space.17

Perhaps most importantly, mobile game systems offer some advantages or unique

gameplay. Location-aware technologies such as GPS have led to location-based mobile

multiplayer games, or instance, in which social interaction, virtual game worlds, and

real locations merge, as well as augmented reality games that combine the real world

with a game world.18Customized content based on player location can be integrated

into a game, and alternate reality games have been created that require the player to take

photographs o particular locations and upload these to the game, relying urther on

unctionality o smartphone devices.

N

. While experts chosen or these predictions may be rom dubiously biased sources (with

the industry ofen hyping itsel), thus ar the predictions have been on the conservative

side, and all indications are that the mobile market will continue to grow at a rapid rate.

. Te iPhone sales number is an estimate based on quarterly sales numbers. Allegedly, there

have been fy million iPod ouch and iPhone sales combined. See also Norris ().




19/21


. Tese are estimates based on projected revenues. See: Global Mobile Games Market to

reach bn by (), and Projected Mobile Gaming Revenues in North America,

Europe and Asia Pacic ().

. It is interesting to note thatEtch-A-Sketchhas more recently converged with the game mar-ket and released digital games, with sound effects, such as Etch-A-Sketch: Knobbys Quest

game or PC and Mac.

. From the Namco Mobile website, https://www.namcogames.com/mobile/[https://www.

namcogames.com/mobile/] (accessed March , ).

. Not relevant to our discussion o sound, but an LED is an array o light-emitting diodes,

whereas an LCD is a liquid crystal display. A VFD, mentioned later, is a Vacuum Flourescent

display, closer in technology to LEDs than LCDs. LEDs and VFDs are amiliar to consum-

ers through VCR and microwave displays, digital clocks and watches, and so on.

. For a copy o the yer, see http://www.atarihq.com/dedicated/touchme.php[http://www.

atarihq.com/dedicated/touchme.php] (access March , ).

. Elements o this section are drawn rom Collins ().. A National Semiconductor COP -bit microcontroller with bytes o internal ROM

and -bit internal RAM locations.

. echnically, it is reerred to as a PAPU, a pseudo-audio-processing unit. It is not a separate

circuit or unit rom the main processor, although or programming purposes it is separate.

. An emulator to convert Sega MOD les to PSG sound using this chip can be ound on the

Kontechs website, http://modpsg.kontechs.de/[http://modpsg.kontechs.de/] (accessed

March , ).

. See Collins (:).

. See Synthopia().

. A playable web version Beat Camp demo is on http://www.patapon-game.com/[http://

www.patapon-game.com/] (accessed March , ).. From the Real ime Audio Adventures website, http://www.rtaudioadventures.com/

About.html[http://www.rtaudioadventures.com/About.html] (accessed March , ).

. Joshua Ostrowalker, interview with author, Ottawa, Ontario, October , .

. See Curtes ().

. See Vanhanen ().

R

Baer, Ralph H. n.d. Te Simon Story, http://www.dieterkoenig.at/ccc/english/se_story_

simon.htm (accessed March , ).

Boris, Daniel. . Entex Adventurevision echnical Specs V., http://www.atarihq.com/

danb/les/AdvechSpecs.pd (accessed March , ).

Casual Games Association. . Casual Games Market Report, http://www.casual-

gamesassociation.org/research_news.php (accessed October , ).

Collins, Karen. . Flat wos and the Musical Aesthetic o the Atari VCS. Popular Musicology

Online, http://www.popular-musicology-online.com/ (accessed March , ).

. . Game Sound: An Introduction to the History, Teory and Practice o Video Game

Music and Sound Design. Cambridge: MI Press.

Console Passion. . Atari Lynx. http://www.consolepassion.co.uk/atari-lynx.htm (accessed

October , ).




20/21

KAREN COLLINS

Curtes, Mike. . Editorial: Why Mobile Games Need Better Audio, Games on Deck, http://

www.gamesondeck.com/eature//editorial_why_mobile_games_need_.php (accessed

March , ).

Domin, M. and Schick, Bastian. . Lynx-Documentation, http://www.geocities.com/SiliconValley/Byte//lynx/lynxdoc.html (accessed March , ).

Drescher, Peter. . Could Mobile Game Audio BE More Annoying?! OReilly Digital

Media, http://digitalmedia.oreilly.com////could-mobile-game-audio-be-more-

annoying.html (accessed March , ).

Duryee, ricia. . U.S. Mobile Game Industry Worth Billion; Will Soar to . Billion in

,Moco News Net, October , http://www.moconews.net/entry/-us-mobile-game-i

ndustry-worth--billion-will-soar-to--billion-in-/ (accessed March , ).

El Akkad, Omar. . Apple, RIM wage app wars, Globe and Mail, October , http://www.

theglobeandmail.com/report-on-business/apple-rim-wage-app-wars/article/

(accessed October , ).

Electronic Games. . Stand-Alone Preview. Handheld Museum, http://www.handheldmu-seum.com/BooksMagazines/Mag-EG_/EG_.htm (accessed March , ).

Entex Industries Inc. . Adventure Vision (original manual), http://zappa.brainiac.com/

cdyer/instructions/adv.pd (accessed March , ).

Erickson, racy. . Sound off: iPhone gamings audio problem, Pocket Gamer, http://www.

pocketgamer.co.uk/r/iPhone/iPhone/eature.asp?c= (accessed March , ).

Gevaryahu, Jonathan. . Nintendo(M) Gameboy sound system (PAPU) guide, http://

www.joshuawise.com/~nightmare/GBSOUND.txt (accessed March , ).

Global Mobile Games Market to reach bn by . . Indiantelevision.com , July , http://

www.indiantelevision.com/headlines/yk/july/july.htm (accessed October , ).

Graf, Kris. . Analyst: iPhone o Drive Mobile Game Market Sales o . Billion

By , Gamasutra, October , http://www.gamasutra.com/php-bin/news_index.php?story= (accessed October , ).

Grossman, Lev. . Te op Everything o . ime Magazine, November , http://

www.time.com/time/specials//top/article/,,__,.

html (accessed March , ).

Hetherington, Janet. . Upwardly Mobile Gaming, http://www.awn.com/articles/gam-

ing/upwardly-mobile-gaming (accessed October , ).

IGDA Casual Games Special Interest Group. . Casual Games White Paper, http://

www.igda.org/casual/IGDA_CasualGames_Whitepaper_.pd (accessed October

, ).

iPhone Footprint. . Sound o Joy: Audio Content and iPhone Games. November , http://

www.iphoneootprint.com///sound-o-joy-audio-content-and-iphone-games/

(accessed March , ).

Mattel oy Fair Catalogue. . Handheld Museum,http://www.handheldmuseum.com/

Mattel/Cat/Cat.htm (accessed March , ).

. . Handheld Museum, http://www.handheldmuseum.com/Mattel/Mattel/

Mattel_.htm (accessed March , ).

MIDI Manuacturers Association. . About DLS Format or Mobile Devices, http://www.

midi.org/about-midi/dls/abtmdls.shtml (accessed March , ).

Nokia Corporation. . From Beeps to Soundscapes: Designing Mobile Game

Audio, http://sw.nokia.com/id/ebda-ea-b-e-b/From_




21/21


Beeps_o_Soundscapes_Designing_Mobile_Game_Audio_v__en.pd (accessed

March , ).

Norris, Ashley. . wo Billion Apps Downloaded, million iPhone/touches soldland-

mark day or Apple, ech Digest, September . http://www.techdigest.tv///two_bil-lion_app.html (accessed October , ).

OConnor, Alice, and Nick Breckon. . Nintendo DS Reaching Gender Harmony,

ShackNews, March , http://www.shacknews.com/onearticle.x/ (accessed October

, ).

Te Online Reporter. . Men twice as likely to download mobile games but game

play is gender neutral. May , http://www.theinternetpatrol.com/males-are-tw

ice-as-likely-to-download-a-game-than-women (accessed October , ).

Projected Mobile Gaming Revenues in North America, Europe and Asia

Pacic. . GoRumors.com, March , http://techcrunchies.com/

projected-mobile-gaming-revenues-in-north-america-europe-and-asia-paciic/

(accessed October , ).Stilphen, Scott. n.d. DP Interviews Mark Lesser, Digitpress, http://www.digitpress.com/

library/interviews/interview_mark_lesser.html

Synthopia. . Nintendo DS Music Programs. June , http://www.synthtopia.com/cont

ent////-nintendo-ds-music-programs/ (accessed March , ).

Vanhanen, Anssi. . Mobile Games Business. Inopical Evolution Paths o Mobile Multimedia

Services: Proceedings o the Research Seminar on elecommunications Business (ed. Sakari

Luukkainen) Helsinki University o echnology Publications in elecommunications Sofware

and Multimedia, http://www.tml.hut./Opinnot/-.//Proceedings_.pd

(accessed March , ).

Wagner, James Au. . iPod ouch Cutting Into Handheld Game Market, op Analyst

Says. Te Apple Blog, August , http://theappleblog.com////ipod-touch-cutting-into-handheld-game-market-top-analyst-says/ (accessed October , ).


Date post:	02-Jun-2018
Category:	Documents
Upload:	carlos-lopez-charles
View:	219 times
Download:	0 times

COLLINS KAREN Mobile Phone Music in Video Games Corrected_karen

Documents