5pSWb1: Experience with computerized speech-perception

training (SPATS-ESL) for speakers of other languages

learning English

James D. Miller, Roy Sillings, Charles S. WatsonResearch Department

Communications Disorders Technology, Inc.Bloomington, IN 47401 (www.comdistec.com)

and

Isabelle Darcy and Kathleen Bardovi-HarligSecond Language Studies

Indiana UniversityBloomington, IN 47405

SPATS-ESL

Speech Perception Assessment and Training System for ESL

learners

(Described yesterday.. Poster 4pSW26)

SPATS-ESL in Brief

A. Trains learners to identify the important syllable constituents of English: 45 onsets, 28 nuclei, & 36 codas.

B. Trains learners to quickly recognize spoken sentences.

SPATS-ESL in Brief

SPATS-ESL trains listeners in four tasks:

1. Identification of syllable onsets2. Identification of syllable nuclei3. Identification of syllable codas4. Recognition of sentences

Development of SPATS-ESL

• Originally for hearing-impaired speakers of English.

• Modified for ESL learners

based on class-room experience with 80 ESL students.

ESL Learners in Trials

L1 N %Arabic 5 6

Cantonese 1 1Japanese 5 6Kazahk 2 3Korean 27 34

Mandarin 5 6Mongolian 1 1Portuguese 8 10

Spanish 8 10Taiwanese 13 16

Turkish 1 1Vietnamese 4 5

Sum 80 100

• At least 12 L1s were represented.

• Some were enrolled in an Intensive English Program.

• Some were graduate students in degree programs.

• TOEFL (pbt) scores varied from below 500 to well over 600.

Data Analyses

• Trends for all 80 ESL students have been reviewed.

• Longitudinal analyses and fitting of learning curves is underway for the 30 ESL students that used the program in near final form.

• Summaries of completed analyses are shown. (All results are consistent with the data shown.)

Major Findings

• All have significant perceptual problems.• Improvement on all tasks is proportional to

practice time• Order of difficulty: 1) Nuclei, 2) Sentences, 3)

Codas, and 4) Onsets.• Near-native performance on all tasks is

achieved in ~(15-35) hours of SPATS-ESL training.

Longitudinal Analyses of Progress

1. Find Percent Correct (PC) and total time in SPATS practice (all kinds) at the end of each run as shown for syllable constituents at each cumulative level.

2. (Example on next slide on the next slide.)

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Pe

rcen

t C

orr

ec

t (P

C)

Total Hours of SPATS-ESL Training

Native Range (94-100)

C. L

ev

el

1

C. L

ev

el 2

C. L

ev

el 3

C. Level 4

Perceptual Indentificationof Syllable Constituents

Prior to Adjustment

Longitudinal Analyses of Progress

1. Adjust percent-correct scores (PCs) by multiplying by the proportion of items in the set. PCs for Level 1 are multiplied by 0.25; PCs for Level 2 are multiplied by 0.50; PCs for Level 3 are multiplied by 0.75; and those for Level 4 are multiplied by 1.0. See the next slide.

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Native Range (94-100)

Perceptual Indentificationof Syllable Constituents

Data Adjustment Illustration

Pe

rcen

t C

orr

ec

t (P

C)

Total Hours of SPATS-ESL Training

Longitudinal Analyses of Progress

1. Fit an exponential learning curve as shown on the next slide.

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Native Range (94-100)

Perceptual Indentificationof Syllable Constituents

Data Adjustment Illustration

Pe

rce

nt

Co

rre

ct

(PC

)

Total Hours of SPATS-ESL Training

Longitudinal Analyses of Progress

1. Estimate time to near-native performance.

2. Sample and summary data are shown for Onsets, Codas, Sentences, and Nuclei on the following series of slides

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Pe

rcen

t C

orr

ec

t (P

C)

Total Hours of SPATS-ESL Training

Perceptual Indentificationof 45 Syllable Onsets

Client 1502L1 Korean

Native Range (94-100)

0

1

2

3

4

5

6

7

8

0 4 8 12 16 20 24 28 32 36 40 44

Nu

mb

er

of

Le

arn

ers

Total Hours of SPATS-ESL Training for PC = 90%

OnsetsMean = 9.3 Hrs.

N = 19

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Pe

rfo

rman

ce S

core

Total Hours of SPATS-ESL Training

Perceptual Indentificationof 36 Syllable Codas

Client 303L1 Vietnemese

Native SpeakerRange (88 -100)

0

1

2

3

4

5

6

0 4 8 12 16 20 24 28 32 36 40 44

Nu

mb

er

of

Le

arn

ers

Total Hours of SPATS-ESL Training for PC = 85%

CodasMean = 11.4 Hrs.

N = 18

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Native Speaker Range (90-100)

Eff

ec

tive

Perc

en

t C

orr

ec

t (e

PC

)

Total Hours of SPATS TraIning

SNRs5, 0, -5, -10, -15

Sentence LearningClient 1403L1 Arabic

0

1

2

3

4

0 4 8 12 16 20 24 28 32 36 40 44

Nu

mb

er o

f L

earn

ers

Total Hours of SPATS-ESL Training for PC = 90%

SentencesMean = 15 Hrs.

N = 10

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Total Hours of SPATS-ESL Training

Pe

rce

nt

Co

rre

ct

(PC

)

Perceptual Identification of 28 Syllable Nuclei

Client 800L1 Taiwanese

Native Range (94-100)

0

1

2

3

4

5

0 4 8 12 16 20 24 28 32 36 40 44

Nu

mb

er

of

Le

arn

ers

Total Hours of SPATS-ESL Training for PC = 90%

NucleiMean = 18.4 Hrs.

N = 19

0

1

2

3

0 4 8 12 16 20 24 28 32 36 40 44

All Four TasksM = 24.4 Hrs.

N = 10

Nu

mb

er

of

Le

arn

ers

Total Hours of SPATS-ESL Training for Near-Native PC

Onsets Codas Sentences Nuclei A llMean 9.3 11.4 15.0 18.4 24.4

Median 6.2 8.2 15.1 17.0 23.0N 19 18 10 19 10

Total Hours to Near-Native Performance

Users’ Subjective Evaluations

Do you think that other ESL programs should use SPATS? 90% yes

Would you recommend SPATS to friends or relatives learning English? 90% yes

Did drill on speech sounds improve your participation in everyday conversation? 80% yes

Did sentence work improve your participation in every day conversation? 85% yes

Will SPATS training improve your pronunciation of English? 90% yes

Summary

English learners with a basic knowledge of spoken and written English have significant problems perceiving spoken English.

The unique training provided by SPATS-ESL appears to enable the ESL learner to approach the perceptual skill of native speakers.

By reasonable inference and user reports, SPATS training enables the ESL learner to acquire more knowledge of English through conversation and oral instruction and provides a necessary foundation for the improvement of pronunciation through self monitoring and instruction.

The EndFollow developments at

www.comdistec.com

The following slides are

extras.

Supplemental Class Fall 2007

• SPATS • Original HI Nucleus Seq.• Shift to ESL Nucleus Seq.

Late in Sess.• Onsets: 4 trials/item• Nuclei: 4 trials/item• Codas: 4 trials/item• Sent. Task: Adaptive*• Max. Classes: 4 per

week/7 weeks

Japanese 1Korean 4

Mandarin 3Spanish 2

Taiwanese 3Vietnamese 1

Sum 14

Class Make UP

Degree Students at IU

IEP Class Spring II 2008

• SPATS Almost Default • ESL Nucleus Seq.• Onsets: 4 trials/item• Nuclei: 6 trials/item• Codas: 5 trials/item• Sent: Random(5 to -15)

• Max. Classes: 2 per week/5 weeks

Class Make UP

IEP Students at IU

Arabic 2Korean 7

Portuguese 1Spanish 1

Taiwanese 1Turkish 1

Vietnamese 1Sum 14

SPATS-ESL in Brief (Con’t)

C. For each constituent type, training progresses from:

Cumulative Level 1: 25% of the items, to

Cumulative Level 2: 50% of the items, to

Cumulative Level 3: 75% of the items, to

Cumulative Level 4: 100% of the items.

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Total Hours of SPATS-ESL Training

Eff

ecti

ve P

erc

ent

Co

rrec

t (e

PC

)Native SpeakerRange (90-100)

Sentence LearningClient 1502L1 Korean

SNR AdaptiveCorrected to 0 dB Ave.

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Native SpeakerRange (90-100)

Sentence LearningClient 1503

L1 PortugueseSNRs

5, 0, -5, -10, -15

Eff

ec

tive

Perc

en

t C

orr

ec

t (e

PC

)

Total Hours of SPATS-ESL Training

Onsets Codas Sentences Nuclei A llMean 9.3 11.4 15.0 18.4 24.4

Median 6.2 8.2 15.1 17.0 23.0Min 2.6 3.5 3.0 6.5 11.0Max 34.0 33.0 29.0 40.0 40.0

N 19 18 10 19 10

Total Hours to Near-Native Performance