HASKINS LABORATORIES

THE SCIENCE OF THE SPOKENAND WRITTEN WORD

)))))) Haskins Laboratories

)))))) Haskins Laboratories

MISSIONSTATEMENT

Haskins Laboratories is anindependent, international,multidisciplinary community of researchers conductingbasic research on spoken andwritten language. Exchangingideas, fostering collaborations,and forging partnershipsacross the sciences, itproduces groundbreakingresearch that enhances ourunderstanding of—and revealsways to improve or remediate—speech perception andproduction, reading andreading disabilities, and humancommunication.

2

4

8

12

16

20

24

28

32

38

42

The Science of the Spoken and Written Word

Speech, Script, and Communication: A Primer

On the Forefront of an Exciting Field

A Record of Theoretical and Technological Innovation

Independent and Interdisciplinary

Integrity and Objectivity

Enhancing the Quality of Our Lives

Teaching Reading Is Rocket Science

International Impact

Decades of Discovery

Board of Directors, Administration, and Contact Information

8

12

16

20

24

28

32

Talking and understanding what others saycomes naturally to every healthy child.Children rapidly learn to speak andunderstand others, and can do so with noformal training. Literacy is a very differentmatter. Many individuals as well as entiresocieties do not read or write. Althoughmost people take language for granted,understanding the nature of speech and itsrelationship to literacy is anything butsimple. How do we acquire, produce, andunderstand speech, which is our birthrightby biological evolution? How do we achieveliteracy, which is a cultural artifact? Whatbridges these dual domains of language?Exploring such questions opens a windowon the inner workings of the mind.

Answering these fascinating questions is ofmore than scientific interest to those whocannot take language for granted.

2

Disorders, disease, and trauma impair somepeople’s ability to speak and/or understandthe speech of others. Disabilities andinadequate education prevent many morefrom learning to read and write. The scienceof the spoken and the written word promisesto help these people participate more fullyin their humanity and our society.

Haskins Laboratories has been at theforefront of this research for seventy years.It is the nation’s leading independent,multidisciplinary community of scientistsstudying speech, language, and reading. Itstheoretical and technological breakthroughsare continually advancing the science of thespoken and the written word, and thepractical applications of its discoveries areimproving human communication.

The characters thatappear above andthroughout this bookare drawn from thephonetic alphabetand various writtenlanguages' alphabets.

3

A PRIMER SPEECH, SCRIPT, AND COMMUNICATION:

4

Birds, bees, and even poorly educated fleas communicate with each otherby song, flight-dance, or the twitch of antennae, but only humans possessthe gift and the power of language. Speaking comes almost as naturallyto us as breathing. Indeed, that’s in part what it is: modulating the air weexhale from our lungs with our larynx, palate, jaw, tongue, and lips to formvowels and consonants. Speech is so integral to our identity that it mayhave emerged simultaneously with the origin of Homo sapiens some200,000 years ago or even earlier, when an upright ancestor emitted asound more potentially meaningful than a chimpanzee’s grunt.

Healthy infants typically start babbling at six to eight months and beginto utter words at twelve to fifteen months, sentences a few months later.In a child’s third year, these sentences become fluent. Six-year-olds knowan average of 13,000 words. By the time we graduate from high school,our vocabularies have burgeoned to approximately 60,000 words.

As inevitably as we acquire our native language, we lose the capacity to master another one easily. By the age of ten to twelve months, infants’ability to distinguish some sounds that are not salient in the languagespoken around them begins to diminish. A Japanese baby, for instance,no longer registers the difference between English’s “l”s and “r”s. Bypuberty, our brains have lost the plasticity that would enable us tospeak a foreign language without a telltale accent.

Where in the cerebral cortex is our language facility concentrated? A critical region for language is clustered around the Sylvian fissure inthe left hemisphere of most people’s brains (including those of amajority of left-handed people), which is why most of us perceivespeech a few milliseconds earlier and more accurately through our rightear (which is connected to the left hemisphere), while many people hearmusic more acutely through their left ear.

5

Functional Magnetic Resonance Imaging (fMRI) usually reveals a flushof activity in the left hemisphere when people read or converse. One of the fascinations of speech for many scientists (and non-scientists as well) is that it opens a window on what goes on inside our heads.

“Personally,” wrote Noam Chomsky, the father of modern linguistics, “I am primarily intrigued by the possibility of learning something, fromthe study of language, that will bring to light inherent properties of thehuman mind.”

The basic building blocks of speech are phonemes. The ”b,” short “a,”and “g” that form the word “bag,” for example, are phonemes. A changein a phoneme can create a meaning change (“rag,” “bog,” “bat”). Englishconsists of about four dozen such phonemes. In contrast, the AmericanIndian language Mura has 11 phonemes. The click language XuÚ! has 141.

It has proven very difficult to reduce phonemes to acoustic properties,as evidenced by the difficulty that computer scientists have experiencedin creating useful speech recognition systems. Moreover, there isevidence that phonemes consist of more than sounds. If you listenthrough headphones to a voice saying “ba” as you watch a video of aface saying “da,” the visual information trumps the acoustic informationand you hear “da.” This phenomenon is called the “McGurk effect” afterone of its discoverers, Harry McGurk.

Words are even more confounding than the syllables that composethem. Harvard psychologist and bestselling author Steven Pinker writes,“In the speech sound wave, one word runs into the next seamlessly;there are no little silences between spoken words the way there are white spaces between written words.”

Research conducted at Haskins Laboratories over seventy years hasconvinced scientists in many disciplines to consider the sounds and wordsthat constitute speech not as discrete, disembodied acoustical entities

6

but, rather, as physical events or gestures, the overlapping actions of ourlarynx, palate, tongue, jaw, and lips. Speech is a complex neurological andphysiological system. Understanding the system of speech can help ussynthesize, recognize, and improve it.

Writing makes speech visible. Although it is based on the spoken word, it consists of symbols rather than actions. Many of these symbolsoriginated in pictographs. Turn the letter “A” upside down and you canstill make out an ox head; it takes a little more imagination to discernthe tent flap in a “D.” But the letters in written languages becamesymbols of sounds: phonemes in English and Finnish, entire syllables inMayan and Japanese, a combination of both in Korean.

The written word appeared much later than the spoken word. Theearliest known script, the cuneiform that the Sumerians incised on claytablets, is only 4,000 to 5,000 years old. Borrowing characters from thePhoenicians, the ancient Greeks created the first fully alphabetic systemof writing, memorialized in the very word, “alphabet” (alpha, beta…).

The relatively recent invention of written languages and the abundance of non-literate societies and individuals indicate that reading is not abiological, evolutionary imperative, like speech, but a cultural acquisition.We don’t go to school to learn to talk, but some kind of education isnecessary to grasp that the letters in our alphabet approximate phonemesand that b-a-g spells “bag.” Some people do not learn this lesson easilybecause they suffer from dyslexia or other learning disabilities, poor teaching, or both.

At Haskins Laboratories, psychology, physiology, linguistics, neuroscience,cognitive science, and computer science are illuminating the connections(and disconnections) between the spoken and the written word.

7

Carol A. Fowler

8

Sophisticated behavioral research, imaging technologies that peer inside the brain, and computer models that simulate mentaland physiological activities are enabling researchers to makespectacular advances in the science of the spoken and thewritten word. Haskins Laboratories is on the forefront of thisexciting field, deepening our understanding of humanity andcivilization and helping people to participate more fully in both.

OF AN EXCITING FIELDON THE FOREFRONT

9

As Haskins Laboratories entersits eighth decade, Dr. Carol A.Fowler is proud that its researchis more robust and timely thanever. “We’re making significantstrides in understanding howpeople use language to co-operate and get things done inthe world,” says the private,non-profit laboratory’s Presidentand Director of Research.

Dr. Louis Goldstein, a SeniorScientist at Haskins, shares herconviction: “We have, or are closeto having, the tools we need toexplore the fundamental nature oflanguage forms in speech andprint: functional MRI, ultrasound,magnetometers, eye-trackers, anddynamic mathematical and computer models of motion,change, and networks.”

Dr. Goldstein, Chairman of Yale University’s LinguisticsDepartment, and Dr. Fowler, aProfessor of Psychology at theUniversity of Connecticut andAdjunct Professor of Psychologyand Linguistics at Yale, repre-sent the two universities withwhich Haskins Laboratories hasbeen affiliated since 1970. MostHaskins researchers hold dualappointments at these or otheruniversities. Such arrangementsinsure the Laboratories againstinsularity by continually expos-ing Haskins researchers to thenew and different ideas of fellowfaculty members, and providethe scientists with eager students.

“You want to have students,because they’re the people whoare going to make the nextgeneration of discoveries,” Dr. Goldstein says. Independence,he explains, gives Haskins “moreinformality, flexibility, and cama-

raderie.” That independence anddecades of dedicated researchhave made Haskins Laboratories,in the words of Steven Pinker,“Speech Central.” (A chronologi-cal account of discoveries madeat Haskins Laboratories appearson pages 38-41.)

One research area of particularinterest to Dr. Goldstein is slipsof the tongue. While many psy-chologists have speculatedabout what goes on inside aperson’s head when he or shemakes a vocal slip and says“poffee cot,” for example,instead of “coffee pot,” Dr.Goldstein has studied whatgoes on in the vocal tract whensuch slips occur. In addition toshedding light on the dynamicsof speech production, his inves-tigations, he hopes, will benefitpeople with hearing disorders,aphasia, and other impairmentsby “providing key informationabout what speech actually isand helping them to speakmore comprehensibly. We willunderstand a lot more aboutourselves, and about how todeal with people whose com-munications are disordered.”

Dr. Fowler is especially inter-ested in the ways imitation andinteraction influence communi-cation. “Language is somethingthat happens between people,”she says. “When they talk toeach other their dialects andintonations begin to converge as they try literally to get onthe same wavelength.” Weoften mirror each other’s ges-tures as well, crossing our legsafter the person we’re speakingwith does, for instance, orswaying in sync.

Dr. Fowler has shown that this“postural sway,” measured bysensors attached to experimen-tal subjects’ hips, can occur dur-ing a cooperative conversationeven when participants cannotsee each other. Now she isdesigning experiments to dis-cern what happens when peopleare speaking under competitiverather than cooperative circum-stances. She is also using func-tional MRI to trace neuronalactivity when people hear a word,see it being spoken, say it tothemselves, or silently mouth it.

Both scientists are united inbelieving that language is adynamic, public phenomenon.“To turn what Chomsky said onits head,” Dr. Goldstein explains,“language is more like anorganism than an organ.” Dr. Fowler elaborates: “Theforms a language assumescome partly from ourselves, butalso from our interactions withother speakers. That’s one rea-son why it’s continually chang-ing. Teenagers want to soundlike each other, not their par-ents. Bill Clinton talked about‘growing’ the economy, andsuddenly everyone was usingthe verb in a way they neverhad before.”

Thanks to research conductedat, and inspired by, HaskinsLaboratories, people are under-standing the spoken and writtenword in ways they never didbefore. “The research here isrevolutionary,” says Dr. Goldstein.

“This is where it all began,” says Dr. Fowler. And where it continues.

10

‘The research here is revolutionary.’

11

Louis Goldstein

12

Haskins Laboratories has a long history of technological andtheoretical innovation, from creating the first prototype of areading machine for the blind to developing the landmarkconcept of phonemic awareness as a critical preparation forlearning to read. Its record of achievement, past and present,is compelling evidence that it will continue to conductcutting-edge research in the future.

TECHNOLOGICAL INNOVATIONA RECORD OF THEORETICAL AND

12

13

Catherine T. Best Philip E. Rubin

‘You can’t predict the future,

but you can bet on it. And

Haskins has a long track record

of scientific achievement.’14

15

Although basic research is at theheart of Haskins Laboratories’mission, the project thatlaunched its investigations of thespoken and written word waspractical and applied. DuringWorld War II, Haskins researchersDrs. Franklin S. Cooper and AlvinM. Liberman began work on areading machine for blindedveterans that translated printedtext into audible signals. Theirfirst attempts linked a uniquesound to each letter and faileddismally. This convinced themthat speech is uniquely able toconvey language, and that theyhad to acquire a deeperknowledge of speech in order todevelop a successful readingmachine. Their investigations andthose of their colleague, Dr.Ignatius G. Mattingly, eventuallyresulted in a prototype thatconverted typescript intosynthetic speech and “read” a“Dear Abby” newspaper columnto residents of a VeteransAdministration BlindRehabilitation Center.

This research also led to arevolutionary new theoreticaland empirical understanding ofspeech as a motor or gesturalactivity, as well as to a newapproach to teaching reading.

“Adding to our basic knowledgeis absolutely essential totechnological progress andimprovements in the clinicalrealm,” says Dr. Philip E. Rubin.“You can’t know at the outsetwhat the commercial or clinicalapplications of your researchwill be, but ultimately it will beapplied.”

After three “very hectic andvery rewarding” years spentdirecting the National ScienceFoundation’s Division ofBehavioral and CognitiveSciences, Dr. Rubin recentlyreturned to Haskins to becomeits CEO and Vice President. Heis also a research affiliate ofYale’s Psychology Departmentand an adjunct professor in theDepartment of Surgery,Otolaryngology, at the YaleUniversity School of Medicine.

What originally attracted Dr.Rubin to Haskins Laboratorieswas its unusual combination oftheoretical inquiry andtechnological innovation. Hehimself has added to thoseinnovations over the years bydeveloping, with Dr. PaulMermelstein, and othercolleagues, the first articulatorysynthesizer that could be usedas an interactive tool for testingthe relationship between theproduction and perception ofspeech (“Think of speech as aballet,” he explains, “and of thisas a tool to orchestrate it overtime.”), and by creating thesinewave synthesizer. Dr. Rubin,a former rock and roll guitarist,used frequencies derived fromhuman speech to drive a musicsynthesizer he had created andcame up with a new way to testhow people perceive anddifferentiate speech.

Dr. Catherine T. Best, a HaskinsSenior Scientist who recentlyleft Wesleyan University tobecome a Chair in Psycho-linguistic Research at theUniversity of Western Sydney’s

MARCS Auditory Laboratoriesin Australia, has used both thearticulatory synthesizer and thesinewave synthesizer in her ownresearch. When she wasworking on her doctorate atMichigan State University,studying the responses ofinfants to the sounds of speech,she made several trips to NewHaven, often driving all night,because Haskins was one of theonly facilities with thecomputers and speechsynthesizers her experimentsrequired. “Most of the tools wehave today for manipulatingnatural speech are derived froma foundation that was laiddown here,” she says.

Among other subjects, Dr. Beststudies how our experience ofour own language affects ourperception of other languages,from Japanese to the clicks ofZulu. Her research may provepertinent in today’s globaleconomy when instantaneouscommunications and increasedimmigration expose manypeople to more languages.

“You can’t predict the future,”Dr. Rubin says, “but you can bet on it. And Haskins has along track record of scientificachievement.”

16

Vincent Gracco David J. Ostry

17

Haskins Laboratories enjoys a distinct advantage as anindependent research center. Crossing the boundaries ofuniversities as well as disciplines, it has created a critical mass of full-time, part-time, and visiting psychologists, linguists,neuroscientists, physicists, engineers, and other specialists who share information and insights and frequently collaborate as they study important problems such as speech perception and production, reading, and dyslexia that are too complex forresearch conducted from a single perspective ever to resolve.

AND INTERDISCIPLINARYINDEPENDENT

18

Drs. David J. Ostry and VincentGracco are colleagues atHaskins Laboratories and McGillUniversity, but there theresemblance ends. Dr. Ostry is an engineer and neuro-physiologist who studies therole of “somatosensory input” inhuman speech and movement.Such feedback from nerve cells,for instance, enables manypeople who become deaf asadults to continue to talkintelligibly for many years. Dr.Gracco is a speech pathologistwith a particular interest instuttering. Their differencesexemplify the interdisciplinarynature of Haskins research.

Indeed, Dr. Gracco says it was“the inherent interdisciplinarityof speech pathology” thatattracted him to the field andthen to Haskins Laboratories.“Speech is such a complicatedprocess,” he explains. “Tounderstand what exactly isgoing wrong in a patient, youhave to know something aboutthe nervous system, aboutphysiology, about behavior,about language, evenmathematics, statistics,engineering, and medicalsubspecialties.”

“By making room for everyonein the same building, Haskinsgets a lot of things done,” addsDr. Ostry. “There’s a remarkableconcentration of people here.”This concentration presentsunusual opportunities forcollaboration. With Haskins Vice

President of Research Dr. Douglas H. Whalen, forinstance, Dr. Ostry has usedultrasound to measure tonguemovement. “It’s a kind ofproject I could only have doneat Haskins.” Another device thathe employs at the Laboratoriesapplies pressure to the lowerjaw in a manner that modifiessomatosensory feedback whilehaving minimal audible effect.Dr. Ostry’s experiments holdpromise for helping thehearing- and speech-impairedspeak more clearly andeffectively.

Dr. Gracco is using functionalMRI and other imagingtechnologies to study the brainanatomy and activity ofstutterers. “There’s somethingdifferent about the way theyuse their brains,” he says. “Theanatomical differences in thebrain are consistent withfunctional differences.” Studiessuggest there is a genetic linkor predisposition to stuttering,and there tends to be morespeech activity in the righthemisphere of a stutterer’sbrain than in those of mostnon-stutterers. Treatment tendsto shift activity toward the lefthemisphere, but no therapy is more than 75 percentsuccessful, according to Dr.Gracco, who wants to useimaging technology to test the efficacy of cures.

Marveling at the large numberof scientists who work at

Haskins during the course of ayear, more than fifty of themsenior researchers, Dr. Graccosays, “It’s impossible to findanother place that has such acritical mass.”

19

‘By making room for everyonein the same building, Haskinsgets a lot of things done. There’s a remarkableconcentration of people here.’

Haskins Laboratories is dedicated to basic research and hardscience, from developing the first interactive software replicatingthe interplay of the tongue, lips, jaw, and palate in human speech tousing functional Magnetic Resonance Imaging to study blood flowsin the brain as a child reads. This grounding in objective research,which Haskins scientists pursue wherever it leads, gives the work ofthe Laboratories special credibility at a time when controversysurrounds many issues concerning language and reading.

AND OBJECTIVITYINTEGRITY

20

21

Kenneth R. Pugh

‘I can’t say enough about

the Haskins Labs’ scientific

integrity,’ says an official at the

National Institute of Child Health

and Human Development, calling

Haskins ‘a national treasure.’

22

When Dr. Alvin Libermanheaded Haskins Laboratoriesduring the 1970s and ‘80s, hewould ask researchers daily,“Made any discoveries today?”“And you’d damned well betterhave an answer,” recalls SeniorScientist Dr. Kenneth R. Pugh, a psychologist who is also a Research Scientist in theDepartment of Pediatrics at Yale University School of Medicine.

Scientific zeal, accompanied byequally scientific scrupulousness,still prevails at the Laboratories.“There is a fundamentalempiricism at Haskins,” says Dr.Pugh, “a willingness to be ledby the data, and an absolute joyin pursuing the truth for its ownsake. The diversity of opinionsand disciplines here createchecks and balances.”

Dr. G. Reid Lyon, Chief of theChild Development and HumanBehavior Branch of theNational Institute of ChildHealth and HumanDevelopment (NICHD), agrees.“I can’t say enough about the Haskins Labs’ scientificintegrity,” says Dr. Lyon, whorelied heavily on Haskins’sreading research to win WhiteHouse support for science-based educational methodsand calls Haskins “a nationaltreasure.” Although educationis frequently a political andcultural battleground, Dr. Lyonsays, “Science should be abovethe fray.”

Determined to ground thestudy of the human mind inneurobiology, Dr. Pughdesigned a battery ofexperiments and measures forusing functional MRI to studybrain activity in people withreading disabilities. “It was avery heady time and extremelyhigh-risk,” he says, “because wedidn’t know if we would be ableto measure anythingmeaningful.” This pioneeringwork has been highly influentialand widely adopted, yet Dr.Pugh cautions, “We’re still inkindergarten when it comes to understanding the functionsof different areas of the brain,even though I map it endlessly.”

Now Dr. Pugh, with Haskinscolleagues Drs. Hollis S.Scarborough and RebeccaSandak, is undertaking agroundbreaking collaborationwith the Kennedy KriegerInstitute, a Baltimore-basedresearch, clinical, andeducational facility for childrenand young adults withneurological disorders anddevelopmental disabilities, and with the EducationalTesting Service. They arepooling resources and expertiseto assess three differentapproaches to treatingadolescent reading disabilities,using functional MRI, behavioraltesting, and computer analysisto monitor students’ progress.Dr. Pugh is especially excited by the project because littleresearch has been conducted

in adolescent readingdisabilities. “People seldomteach reading to 12-to-17-year-olds, and they’re falling throughthe cracks.”

Kennedy Krieger President and CEO Dr. Gary W. Goldsteincalls the study, which is fundedby the National Institutes ofHealth and the Department ofEducation, “a naturalcollaboration, and a veryproductive learning experiencefor us all. We’re bridging thegap between educationalresearch and neuroscience.After all, that’s what teachersdo: they rearrange people’sminds. Education is appliedneuroscience.”

23

24

Douglas H. Whalen

The basic research of Haskins Laboratories continually yieldspractical applications that enhance the quality of our lives.Work currently underway at the Laboratories may contribute to earlier medical diagnoses, for example, to more effectivetreatment of stuttering, to new methods of teaching andlearning a second language, and to the creation of more reliablevoice-recognition systems for security and other purposes.

OF OUR LIVESENHANCING THE QUALITY

25

Dr. Douglas H. Whalen is ideallysuited for his job. “I’m restless,”says the Vice President ofResearch at Haskins Laboratories.“I’m interested in everythingthat goes on here.” His primaryarea of concentration is basicresearch in the perception andproduction of speech, which isfraught with implications forand applications in everydaylife. “What seems like a verytechnical, theoretical debatehas real-world significance,” heexplains, noting that his investi-gations may lead to fine-tuningthe frequencies of cochlearimplants or improving speech-recognition technology. Today’sspeech-recognition systems, Dr.Whalen observes, suffer from“the curse of the 90 percent.”Mistaking one word out of tencan impede communicationrather than enhance it, he says,and “it’s taken them forever tobecome only 90 percent accurate.”

Dr. Whalen and Haskins SeniorScientist Dr. Khalil Iskarouswere using ultrasound to studythe movement of the tonguewhen a colleague, Dr. CarolGracco, made a serendipitoussuggestion exemplifying thecreative collaborations Haskins fosters. The speechpathologist set them thinkingabout Parkinson’s disease. “The tongue is all muscle,” Dr.Whalen says, “and more like anoctopus’s tentacle than anyother human muscle.”

Parkinson’s is characterized by muscular rigidity. Wouldultrasound reveal anythingabout its effect on the tongue?“It was amazingly clear,” Dr.Whalen says. “People withParkinson’s look as if they wereshoving a ball around theirmouth even if you can’t hearthe difference.” Their findingsare extremely preliminary, butDr. Whalen hopes thatultrasound may prove a usefuldiagnostic tool, and that theefficacy of various treatmentsof Parkinson’s might be gaugedby the extent to which theyliterally loosen a patient’stongue.

In another collaboration, Dr.Whalen and Haskins ResearchAffiliate Dr. Julia R. Irwin beganexamining gender differencesin people’s responses to theMcGurk effect. Women aremore prone than men to hearwhat they see, registering thesilent syllable formed by lips on film even when it differsfrom the syllable that is audiblethrough headphones. Autism is more prevalent among males,and Dr. Whalen and Dr. Irwin’sexperiments have yielded someevidence that autistic children,who tend not to look atspeakers’ faces, do not displaythe McGurk effect. Now theyhope to study whether suchinsusceptibility might providean early diagnosis of autism.Somehow Dr. Whalen has also

found the time to establish and oversee the EndangeredLanguage Fund, whichdispenses grants forcommunity-based researchprojects to record and preservenative languages. Today, Dr.Whalen points out, thanks to the field recordings of earliergenerations of linguists, manylanguages can be considereddormant rather than dead.Three to four hundred NativeAmerican languages were oncespoken in California, forexample, and the descendantsof some tribes are trying torevive them on the basis offield recordings. FromOklahoma, where Dr. Whalengrew up and has sponsoredwork on a Cheyenne-Arapahoreservation, to western Siberia,where researchers haverecorded the last surviving epicsinger in the little-knownlanguage of Shor, theEndangered Language Fund,like Haskins Laboratories, isactive worldwide.

26

‘What seems like a very

technical, theoretical debate

has real-world significance.’27

28

Susan A. Brady

In 2000, Dr. Susan A. Brady and another Haskins colleague, the late Dr. AnneFowler, launched the Haskins Early Reading Success (ERS) Initiative, aprofessional development program for elementary teachers in low-performing schools funded by the U.S. Department of Education. Dr. Brady, aProfessor of Psychology at the University of Rhode Island, felt frustrated thatthe knowledge about reading that researchers at Haskins Laboratories andelsewhere had painstakingly accumulated over decades was not getting intothe hands and heads of teachers. “There’s a chasm between the researchworld and the educational world,” Dr. Brady says, “that we must bridge.”

IS ROCKET SCIENCE TEACHING READING

29

One of her mentors at Haskins,the late Dr. Isabelle Liberman,“cared passionately aboutreading instruction and remedi-ation,” Dr. Brady explains. “Shewanted research to impinge onpractice. In turn, I have long felta dual responsibility to conductresearch on reading and to seethat insights from researchreach the classroom.” For Dr.Brady there was also a “personalhook.” Her younger brother,who eventually became anengineer without attendingcollege, is dyslexic. “I watchedhim have a difficult time inschool learning to read, andspelling is still a challenge.”

Early Reading Success helpedteachers develop students’ability to recognize that spokenwords consist of phonemicsegments and to identify thephonemes in spoken words andsyllables. Such phonemeawareness is necessary forunderstanding what the lettersin the alphabet stand for and isan essential component ofpreparation for learning to read.In its very first year, EarlyReading Success raised from 30to 50 percent the proportion ofchildren in participating schoolswho entered first grademeeting the benchmark forrequisite early reading skills.“One of the things we have toovercome is low expectationsfor poor children,” Dr. Bradysays. “Another is the belief thatkindergarten should not haveacademic goals; it is critical to

give students the foundationthey need for learning to read.”

The concept of phonemeawareness, pioneered atHaskins, is a major contributionto education. Dr. Reid Lyon atthe National Institute of ChildHealth and Human Developmentcalls the discovery of phonemeawareness “a national and eveninternational accomplishmentthat has literally savedchildren’s lives.”

The Haskins reading program iseducating the scientistsinvolved as well as teachers andschoolchildren. “Working withour team of mentors in theschools, our understanding ofwhat teaching reading requireskeeps growing,” says Dr. Brady,who believes it is more complexthan most advocates of wholelanguage instruction on the onehand or of traditional phonicson the other hand realize. “Ourapproach entails phonemeawareness, phonics, fluency,vocabulary development, andcomprehension. Teachingreading is rocket science,” shesays, quoting a colleague, Dr.Louisa Moats. “Every year I’mlearning more about readingdevelopment and teachingreading. This shapes researchquestions. The cross-talkbetween research and teachingbenefits both sides.”

Building on ERS, Dr. Brady,together with Dr. Margie Gillis,has undertaken a larger project,

Mastering Reading Instruction.Funded by the U.S. Departmentof Education’s Institute ofEducation Science, they arecarefully comparing methods ofprofessional development todetermine the key elements fortraining first-grade teachers tobe expert at teaching childrento read. “Given the low readingachievement of more than athird of the elementary studentsin the U.S.,” Dr. Brady says, “it iscrucial to determine how togive teachers the knowledgeand skills they need to help allchildren learn to read adequately.”

30

The discovery of phoneme

awareness is ‘a national

and even international

accomplishment that has

literally saved children’s lives.’

31

The reach of Haskins Laboratories is global. Its scientists are affiliated with universities and research institutions throughout the world, and it trains postdoctoral fellows from many countries.Together the Laboratories’ researchers and alumni are advancingthe science of the spoken and the written word internationally.

IMPACTINTERNATIONAL

32

33

Ram Frost

‘Haskins has an enormous

amount of intellectual credit

abroad. For me, coming here is

wonderful because I think it’s

the best lab in the world.’34

“I make my pilgrimage toHaskins at least twice a year,”says Dr. Ram Frost, who comesall the way from Jerusalem,where he is a Professor ofPsychology at HebrewUniversity and heads theLaboratory for VerbalInformation Processing.

As a postdoctoral fellow atHaskins Laboratories, the Israeliscientist conducted landmarkresearch in how the disparatewriting systems of English,Hebrew, and Serbo-Croatianaffect the way people read andwrite. His findings suggestedthat reading English, whoseletters represent phonemes,Hebrew, whose letters typicallyrepresent syllables but seldomspecify vowels, and Serbo-Croatian, which is highlyunusual because manyspeakers use both the Cyrillicand Roman alphabets, requiresdifferent cognitive strategiesand therefore differentinstructional methods. Now Dr. Frost is extending his investigations to Arabic, as well.

Dr. Frost considers hislaboratory “an auxiliary lab ofHaskins in some senses,” and it facilitates Haskins scientists’research in Semitic languagessuch as Hebrew and Arabic.“Haskins people travel to otherlabs around the world to doresearch,” he says. “Theexchange is bidirectional, and

this multinational web is uniqueto Haskins.”

Don’t just take Dr. Frost’s wordfor this. Gordon Ramsay, aBritish electronic engineer andcomputer scientist, came toHaskins as a ResearchAssociate after working atl’Institut de la CommunicationParlée in Grenoble, France.“There’s a constant flow ofpeople coming throughHaskins,” says Ramsay, “andthey travel enormous distancesto come here. People don’t dothat at other labs. Haskins hasan enormous amount ofintellectual credit abroad. Forme, coming here is wonderfulbecause I think it’s the best labin the world.”

Dr. Frost’s work is a primeexample of Haskins’sinternational impact. Deploringthe “total chaos in readingresults” in Israeli schools, hefought “a lonely fight” againstthe Ministry of Education thathe eventually won. Parliamentformed a committee on whichhe served, and subsequently heheaded a task force that revisedall the materials for readinginstructions in primary schools.“In two years the entire systemof teaching reading in Israel haschanged,” he says. “I managedto import to Israel therevolution we saw in America.”In this revolution as in so muchelse, Haskins Laboratories hasplayed an influential role.

35

AFFILIATIONSINTERNATIONAL

36

Haskins Laboratories’ global community of scientists includes thefollowing:

Cinzia AvesaniCNR (National Research Center)Italy

Harald BaayenMax Planck Institute for PsycholinguisticsNetherlands

Ocke BohnUniversity of AarhusDenmark

Brian ByrneUniversity of New EnglandAustralia

Celine Yueh-ch’in ChangNational Tsing Hua UniversityTaiwan

Edda FarnetaniCNR (National Research Center)Italy

Anatol G. FeldmanUniversity of MontrealCanada

Bryan GickUniversity of British ColumbiaCanada

Paul GribbleUniversity of Western OntarioCanada

Pierre HalléNational Center for Scientific ResearchFrance

Daisy HungNational Chung-ChengUniversityTaiwan

Peter KellerMax Planck Institute for Human Cognitive and Brain ScienceGermany

Alexei KochetovSimon Fraser UniversityCanada

Aleksandar KosticUniversity of BelgradeYugoslavia

Rafael LaboissiereMax Planck Institute for Human Cognitive and Brain ScienceGermany

Jun Ren LeeNational Chung-ChengUniversityTaiwan

Yang LeeGyeongsang NationalUniversitySouth Korea

Theraphan LuanghongkumChulalongkum UniversityThailand

Jorge LuceroUniversity of BrasiliaBrazil

Sudaporn LuksaneeyanawinChulalongkum UniversityThailand

Heikki LyytinenUniversity of JyväskyläFinland

Guy MadisonUppsala UniversitySweden

Kevin MunhallQueens UniversityCanada

Wolfgang PrinzMax Planck Institute for Human Cognitive and Brain SciencesGermany

Ronice QuadrosFederal University of Santa CatarinaBrazil

Daniel RecasensUniversitat Autònomade BarcelònaSpain

Joaquin RomeroUniversity of Rovira and VirgilSpain

Vittorio SanguinetiUniversity of GenoaItaly

Hua ShuBeijing Normal UniversityChina

Masataka SuzukiKinjo Gakuin UniversityJapan

Kalaya TingsabadhChulalongkum UniversityThailand

Ovid TzengNational Chung-ChengUniversityTaiwan

Eric Vatikiotis-BatesonUniversity of British ColumbiaCanada

Mario VayraUniversity of Siena at ArezzoItaly

Andreas WohlschlägerMax Planck Institute for Human Cognitive and Brain SciencesGermany

Yi XuUniversity College, LondonEngland

B. YegnanarayanaIndian Institute of TechnologyIndia

Elisabeth ZetterholmUmea UniversitySweden

37

38

Scores of researchers have con-tributed to scientific break-throughs at Haskins Laboratoriesover the past seventy years. All ofthem are indebted to the pioneer-ing work and inspiring leadershipof Caryl P. Haskins, Franklin S.Cooper, and Alvin M. Liberman.

1930sCaryl Haskins and Franklin Cooperestablish Haskins Laboratories in1935. Affiliated with HarvardUniversity, MIT, and Union College inSchenectady, NY, Haskins conductsresearch in microbiology, radiationphysics, and other fields inCambridge, MA and Schenectady.

In 1939 the Laboratories moves itscenter to New York City. SeymourHutner joins the staff to set up aresearch program in microbiology,genetics, and nutrition. Thedescendant of this program isnow part of Pace University inNew York City.

1940sThe U.S. Office of ScientificResearch and Development, underVannevar Bush, asks HaskinsLaboratories to evaluate and developtechnologies for assisting blindedWorld War II veterans.

Experimental psychologist AlvinLiberman joins the Laboratories toassist in developing a “soundalphabet,” an auditory Braille, as itwere, to represent the letters in atext for use in a reading machinefor the blind. Cooper and Libermanfind, however, that because theear’s ability to resolve a rapidsequence of discrete sounds intoits components is limited, noacoustic code they devise can con-vey text at more than one-tenththe typical rate of speech. Guidingresearch questions now become:

Why is speech so much moreeffective than other acoustic sig-nals? How do we speak so fast?How does speech evade limits onthe temporal resolving power ofthe ear? How is reading related tospeech perception? And, moregenerally, is there some special,perhaps biologically ordained, rela-tion between speech and thestructure of language?

The conclusions of this and otherresearch at the Laboratoriesappear in Blindness: ModernApproaches to the UnseenEnvironment, edited by co-investi-gator Paul Zahl. This influentialbook, published in 1950, identifiesscientific and technical obstaclesthat must be overcome to developpractical devices to assist blindmobility and reading.

Luigi Provasoli joins theLaboratories to set up a researchprogram in marine biology. Theprogram moves to Yale Universityin 1970 and disbands withProvasoli’s retirement in 1978.

1950sCooper invents the PatternPlayback, a machine that convertspictures of the acoustic patterns ofspeech back into sound. With thisdevice Liberman, Cooper, andPierre Delattre (later joined byKatherine Safford Harris, LeighLisker, and others) discoveracoustic cues for perception ofphonetic segments (consonantsand vowels). They find that seg-ments are not usually isolated bitsin the speech stream, and that cuesvary widely with context due tocoarticulation, that is, to the over-lapping actions of larynx, softpalate, tongue, jaw, and lips withinand across syllables. Liberman,Cooper, and Delattre conclude that

the perception of phonetic seg-ments is more simply related toarticulation than to acoustic signals.They propose a “motor theory” ofspeech perception to resolve theacoustic complexity: we perceivespeech, they hypothesize, bylearned associations betweenspeech sounds and sensory feed-back from their articulation.

Liberman, Harris, and colleagues,working with synthetic speech, dis-cover that listeners discriminate agiven acoustic difference betweenconsonants that belong in differentcategories more easily than theydiscriminate the same differencebetween consonants in the samecategory. Dubbed “categorical per-ception” and initially believed pecu-liar to speech, the phenomenoninspires years of research by PeterEimas, Michael Studdert-Kennedy,David Pisoni, and others at Haskinsand elsewhere. Today, though cate-gorical perception is no longerseen as peculiar to speech, itsexperimental paradigm retains utili-ty as a measure of phonologicalskills in young children.

Liberman, aided by FrancesIngemann and others, organizesthe results of the work on cues intoa groundbreaking set of rules forspeech synthesis by the PatternPlayback.

1960sCooper and Harris, working withPeter MacNeilage, are the firstresearchers in the U.S. to use elec-tromyographic techniques, pio-neered at the University of Tokyo,to study the neuromuscular organi-zation of speech. They discoverthat relations between muscleactions and phonetic segments areno simpler or more transparentthan relations between acoustic

OF DISCOVERYDECADES

39

A schematicrendering of thePattern Playbackmachine, whichconvertspictures of the acousticpatterns ofspeech back into sound

signals and phonetic segments.

Lisker and Arthur Abramson lookfor simplification at the level ofarticulatory action in the voicing ofcertain contrasting consonants (/b/,/d/, /g/ vs. /p/, /t/, /k/). They showby acoustic measurements in elevenlanguages and by cross-languageperceptual studies with syntheticspeech that many acoustic proper-ties of voicing contrasts arise fromvariations in voice onset time, that is,in the relative phasing of the onsetof vocal cord vibration and the endof a consonant. Their work is widelyreplicated and elaborated, here andabroad, over following decades.

Donald Shankweiler and Studdert-Kennedy introduce dichotic listen-ing into speech research, present-ing different nonsense syllablessimultaneously to opposite ears.They demonstrate dissociation ofphonetic (speech) and auditory(nonspeech) perception by findingthat phonetic structure devoid ofmeaning is an integral part of lan-guage, typically processed in theleft cerebral hemisphere. Theirwork is replicated and developed inmany laboratories over followingyears.

Alvin Liberman, Cooper,Shankweiler, and Studdert-Kennedysummarize and interpret fifteenyears of research in “Perception ofthe Speech Code,” still among themost cited papers in the speech literature. It sets the agenda formany years of research at Haskinsand elsewhere by describingspeech as a code in which speak-ers overlap (or coarticulate) seg-ments to form syllables. Theseunits last long enough to beresolved by the ear of a listener,who recovers segments from sylla-bles by means of a specialized

decoder in the brain’s left hemi-sphere that is formed from over-lapping input and output neuralnetworks—a physiologicallygrounded “motor theory.”

Haskins acquires its first computerand connects it to a speech syn-thesizer designed and built by theLaboratories’ engineers. IgnatiusMattingly, with British collaboratorsJohn N. Holmes and J. N. Shearme,adapts the Pattern Playback rulesto write the first computer pro-gram for synthesizing continuousspeech from a phonetically spelledinput. A further step toward areading machine for the blind com-bines Mattingly’s program with anautomatic look-up procedure forconverting alphabetic text intostrings of phonetic symbols.

1970sHaskins Laboratories completes themove to New Haven, CT, begun in1969, and enters into affiliationagreements with Yale Universityand the University of Connecticut.

Recognizing the Laboratories’unique facilities for analysis andsynthesis of speech, the NationalInstitutes of Health defray the costsof sharing the facilities with investi-gators from other institutions—support that continues for nearlytwenty years.

Harris, working with FrederickaBell-Berti, Gloria Borden, and oth-ers, demonstrates electromyo-graphically how the precise phas-ing and layering of articulatoryactions give rise to segmentaloverlap, and thus to the acousticphenomena of coarticulation.

Isabelle Liberman, Shankweiler, andAlvin Liberman team up withMattingly to study the relation

between speech perception andreading, a topic implicit in theLaboratories’ research programsince the 1940’s. They develop theconcept of “phonemic awareness,”the knowledge that would-be read-ers must have of the phonemicstructure of their language if theyare to learn to read. Under thebroad rubric of the “AlphabeticPrinciple,” this concept is the coreof the Laboratories’ program ofreading pedagogy today.

Patrick Nye joins the Laboratoriesto lead a team including Cooper,Jane Gaitenby, George Sholes, andGary Kuhn in work on the readingmachine. The project culminateswhen the addition of an opticaltypescript reader enables investi-gators to assemble the first auto-matic text-to-speech readingmachine. By the end of the decadethe technology has advanced tothe point where commercial con-cerns assume the task of designingand manufacturing readingmachines for the blind.

Working with Bruno Repp, VirginiaMann, Joanne Miller, DouglasWhalen, and others over the nextdecade or so, Alvin Liberman con-ducts a series of innovative experi-ments to clarify and deepen theconcept of a speech mode of per-ception. These experiments moveaway from the cue as a static prop-erty of the acoustic signal towardthe cue as a dynamic index ofarticulatory action.

Experiments by Peter Bailey, JamesCutting, Michael Dorman, QuentinSummerfield, and others castdoubt on the validity of the“acoustic cue” as a unit of percep-tual function. Building on theseexperiments, Philip Rubin developsthe sinewave synthesis program

40

used by Robert Remez, Rubin,Pisoni, and colleagues. Theseresearchers show that listeners canperceive continuous speech, withouttraditional speech cues, from a pat-tern of three sinewaves that trackthe changing resonances of thevocal tract. Their work paves theway for a view of speech as adynamic pattern of trajectoriesthrough articulatory-acoustic space.

Rubin, Thomas Baer, PaulMermelstein, and colleagues devel-op Mermelstein’s anatomically sim-plified vocal tract model into thefirst articulatory synthesizer thatcan be controlled in a physicallymeaningful way and used for inter-active experiments.

1980sStudies of different writing systemsover the next two decades supportthe controversial hypothesis that all reading necessarily activates thephonological form of a word before,or at the same time as, its meaning.Work includes experiments byGeorge Lukatela, Michael Turvey,Leonard Katz, Laurie Feldman, Ram Frost, and others in theRoman and Cyrillic alphabets ofSerbo-Croatian, by Shlomo Bentin,Frost, and Katz in Hebrew, and by Mattingly and Feldman inChinese.

Several researchers undertake todevelop compatible theoreticalaccounts of speech production,speech perception and phonologi-cal knowledge:

• Carol Fowler proposes a “directrealism” theory of speech percep-tion: listeners perceive gestures notby means of a specialized decoder,as in the motor theory, but becauseinformation in the acoustic signalspecifies the gestures that form it.

• Inspired by Turvey’s earlier workon “action theory,” Carol Fowler,Rubin, Remez, and Turvey proposea theory of speech production inwhich phonetic goals (such asclosing the lips, raising the tongueor opening the vocal cords) areachieved by transient, special-pur-pose organizations of the articula-tors, termed “coordinative struc-tures” or “synergies.”

• Scott Kelso and colleaguesdemonstrate functional synergies inspeech gestures experimentally.When one articulator in a synergyis perturbed (when the jaw istugged down, for instance, as thelips close to form /b/), other artic-ulators (in this instance the lips)automatically compensate toachieve lip closure.

• Elliot Saltzman develops adynamical systems theory of syner-getic action and implements thetheory as a working model ofspeech production, in whichactions of the articulators are ges-tures that form and release con-strictions in the vocal tract.

• Linguists Catherine Browman andLouis Goldstein develop the theoryof “articulatory phonology,” inwhich gestures are the basic unitsof both phonetic action andphonological knowledge. The asso-ciated “linguistic gestural model”generates appropriately phasedpatterns of gesture for words inEnglish. These “gestural scores,”assigned dynamic values by theSaltzman model, drive the articula-tory synthesizer of Rubin andMermelstein to produce intelligiblespeech.

Alvin Liberman and Mattinglyrevise and update the motor theo-ry, recasting it in an explicitly bio-

logical frame. They posit a special-ized “phonetic module,” encom-passing both production and per-ception, analogous in somerespects to modules for soundlocalization in the bat, the barnowl, and humans. The revisedmotor theory remains viable,though controversial, today.

Giuseppe Cossu, Isabelle Liberman,and Shankweiler are among thefirst to present evidence that diffi-culties in acquiring phonemeawareness and ensuing problems inword recognition characterizereading disability across differentlanguages that use an alphabet.

Shankweiler, Stephen Crain, Mann,and Paul Macaruso present evi-dence that language comprehen-sion difficulties associated withreading disability are typicallybased on processing limitations,not deficiencies of grammaticalknowledge.

Bell-Berti shows that vocal tractconfigurations underlying a givenphonological contrast (consonantvoicing, for instance) entail active(or passive) engagement of all thearticulators, not only those effect-ing the contrast.

Borden and Harris publish SpeechScience Primer, a graduate andadvanced undergraduate introduc-tion to speech science. First pub-lished in 1980 and later revised incollaboration with LawrenceRaphael, the book is now in itsfourth edition.

1990sHarris and Bell-Berti show that thecohesion of gestures forming cer-tain phonetic segments (tongueand lip gestures in English /u/, forexample) rests on their invariant

OF DISCOVERYDECADES

41

phasing with respect to one anoth-er. This finding is consistent withthe hypothesis that segments ariseas units of phonetic function byintegrating established gesturalroutines.

Kenneth Pugh is among the first sci-entists to use functional MagneticResonance Imaging (fMRI) to revealbrain activity associated with read-ing and reading disabilities.

Continuing research begun in the1980s, Catherine Best elaborateson the finding that infants beginlife able to distinguish the soundsof many languages, but within littlemore than six to ten months tendto lose the capacity to discriminatesome sound contrasts not presentin the language spoken aroundthem. (Intriguingly, they retain theability to discriminate others.) Shedevelops the “direct realist”Perceptual Assimilation Model topredict the effects of languageexperience on speech perception inboth infants and adults.

Rubin, Goldstein, Mark Tiede, andcolleagues design a radical revisionof the articulatory synthesis model.Their three-dimensional model ofthe vocal tract permits researchersto replicate fMRI images of actualvocal tracts and the articulations ofdifferent speakers. Whalen,Goldstein, Rubin, and colleaguesextend this work over the nextdecade to study the relationbetween speech production andperception.

Weijia Ni, Pugh, Shankweiler, andcolleagues at Yale develop novelapplications of neuroimaging tomeasure brain activity associatedwith understanding sentences.With Einar Mencl, they are alsoamong the first to extend this

method to the study of individuals.Shankweiler, Susan Brady, AnneFowler, and others explore whetherweak memory and perception inpoor readers are tied specifically tophonological deficits. Evidencerejects broader cognitive deficitsunderlying reading difficulties andraises questions about impairedphonological representations indisabled readers.

Alvin Liberman publishes Speech:A Special Code, reprinting twenty-five key Haskins papers from thepast forty-five years with an intro-ductory essay describing theirintellectual origins and theoreticalimplications.

2000sAnne Fowler and Brady launch theEarly Reading Success Initiative, ademonstration project examiningthe efficacy of professional devel-opment in reading instruction forteachers of children in kindergartenthrough second grade.

Whalen and Khalil Iskarous pioneerthe pairing of ultrasound, whichmonitors articulators that cannot beseen, and Optotrak, an opto-elec-tronic position-tracking device thatmonitors visible articulators, torecord vocal tract activities less inva-sively than other technologies. Theresulting images give a fairly com-plete picture of the vocal tract inaction, opening the door to researchon the links between production andperception that has hitherto beentoo cumbersome or costly.

David Ostry explores the neurologi-cal underpinnings of motor controlby adapting a robot arm to influ-ence jaw movement. The “Phantom”robot arm tracks the jaw—andapplies forces to it—in three dimen-sions in real time, allowing exami-

nation of the control of the jawduring speech and other activities.The Mastering Reading Instructionprogram, a large-scale experimen-tal project led by Brady and MargieGillis, focuses on professionaldevelopment in reading instructionfor first grade teachers. Funded bythe U.S. Department of Education’snew Institute of Education Science,the project applies thirty years ofHaskins research on reading acqui-sition and reading difficulties tostudy ways to train teachers ineffective methods of readinginstruction.

Studdert-Kennedy and Goldsteinpropose a theory of the evolutionof phonetic capacity. From chil-dren’s speech errors and patternsof phonological development, theyargue that a neuroanatomically dif-ferentiated vocal tract coevolvedwith vocal imitation, a capacityunique among primates to humans.

David Braze and Shankweilerdevelop an eye movement labora-tory for investigating readingprocesses in normal and disabledreaders. Eye movement recordingsare now being studied in coordina-tion with brain activity measures.

In March of 2005, Haskins movesto new quarters with 23,000square feet on the ninth floor of300 George Street in New Haven,having outgrown the capacity of270 Crown Street, where it residedfor 35 years. The new facilities pro-vide additional state-of-the art lab-oratories, including an Infant Lab.The ribbon-cutting ceremony isscheduled for May 9, 2005, and a70th anniversary symposiumplanned for the 2005-2006 aca-demic year.

HASKINS LABORATORIESBOARD AND ADMINISTRATION

DirectorsPatrick W. Nye (Chairman)Stephen R. AndersonJon Butler, ex officioClaudia CarelloCarol A. FowlerKatherine S. HarrisFred Maryanski, ex officio Joanne L. MillerLloyd N. MorrisettRobert E. RemezDonald ShankweilerKenneth N. Stevens

AdministrationCarol A. Fowler,President and Director of Research

Philip E. Rubin,Vice President and Chief Executive Officer

Douglas H. Whalen,Vice President of Research

Betty DeLise,Treasurer

Yvonne Manning-Jones,Corporate Secretary

Lyn D. TraverseDirector of Development and Communication

300 George StreetNew Haven, CT 06511Tel: +1 203.865.6163 Fax: 203.865.8963www.haskins.yale.edu

This book was produced by Haskins Laboratoriesand Jan Krukowski & Company.

42