• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
Linguistic Resources for Meeting Recognition
Meghan Glenn, Stephanie Strassel Linguistic Data Consortium
{mlglenn, [email protected]}
http://www.ldc.upenn.edu/Projects/NISTMeet
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
Scope of Work• Training data
– (Pre-publication) distribution• Conference room test data
– Transcription• Careful• Quick
– Comparison and analysis• Infrastructure
– XTrans Toolkit• Features for meetings
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
RT-05S Training Datadistributed by LDC
• (Pre-publication*) distribution via e-corpus to RT-05 participants
• All available from www.ldc.upenn.edu/CatalogTitle Speech Transcripts Volume Domain
Fisher English Part 1 LDC2004S13 LDC2004T19 750+ hours CTS
*Fisher English Part 2 LDC2005S13 LDC2005T19 750+ hours CTS
ICSI Meeting Corpus LDC2004S02 LDC2004T04 72 hours Meeting
ISL Meeting Corpus LDC2004S05 LDC2004T10 10 hours Meeting
NIST Meeting Pilot Corpus LDC2004S09 LDC2004T13 13 hours Meeting
*TDT4 Multilingual Broadcast News Corpus LDC2005S11 LDC2005T16 300+ hours BN
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
RT-05S Evaluation Datatranscribed by LDC
• Conference room data– Ten meeting sessions, 12 minutes each– Contributed by five sites– Multiple recording conditions for each session
• Primarily business meeting content– Transcribers report it was faster, easier and more
interesting to transcribe than RT-04 meeting eval data
• All data carefully transcribed (CTR)• Half of data quickly transcribed (QTR)
– for contrastive study
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
CTR Process• Using IHM channels
– One exception – participant on speakerphone
• 1st pass: manual segmentation – Turns breath groups– 3-8 seconds per segment, designed for ease of transcription only
• ~10 ms padding around each segment boundary• No segmentation or transcription of isolated speaker noise
• 2nd pass: initial verbatim transcription– No time limit– Goal is to “get everything right”
• 3rd pass: verify existing transcription and timestamps, add additional markup– Indicate proper names, filled pauses, noise, etc.– Revisit difficult sections
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
• Additional QC pass by lead transcriber– Using mixed IHM recordings and/or SDM– Merge individual transcripts
• Speaker assignment• Transcription accuracy, completeness• Markup consistency• Spell check• Syntax (format) check
– Check consistency and accuracy of names, acronyms, terminology
– Check silence (untranscribed) regions for missed speech using customized tool
CTR Quality Control
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
QTR Process• 0th pass: automatic audio segmentation
– Pause detection algorithm– No manual correction
• 1st pass: verbatim transcription– Limited to five times real time– Goal is to “get the words right” only
• No special markup, orthography or capitalization• No extra time spent on difficult sections (e.g., disfluencies)
• QC pass: minimal, semi-automated– Spell check– Format check– No check of transcript content, consistency of
names/terms, etc.
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
CTR vs. QTR QTR CTR
word substitutions (e.g., and instead of %um) careful word transcription
no indication of speaker re-starts, disfluencies
indication of speaker restarts, dis-fluencies
lacking some punctuation, capi-talization standard punctuation, capitalization
lacks special markup (for filled pauses, acronyms, mispro-nounced words, etc.)
contains special markup (for filled pauses, acronyms, mispronounced words, etc.)
misinterpreted acronyms acronyms verified
transcription
misinterpreted, inconsistent transcription of technical jargon technical jargon verified
isolated breaths segmented (captured by AutoSegmenter)
no isolated breaths captured (in accordance with task specifica-tion)
words dropped out of segment careful word segmentation – no missed words
segmentation
split words (1- it 2-‘s) no split words (it’s)
transcription rates 5 times real time per channel 15-20+ times real time per channel
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
ExampleQTR CTR
789.010 791.500: For the three people I had um788.050 791.500: Okay, yeah for the threepeople I had %um
792.510 793.440: again Carl ((Laurens))794.320 795.380: outstanding grade795.700 796.350: good uh
[missed]796.900 798.550: ~G ~R ~E scores. You know.
799.220 799.760: um 799.175 799.950: %um800.470 800.980: Wright State.801.650 803.310: he wants to continue at WrightState.804.580 805.360: That's his preference. 804.400 805.800: that's his preference school.
810.950 813.900: kind of interesting in hiswrite up because he said he wanted to %uh814.800 816.725: design ^Luke ^Skywalker'shand.817.600 818.525: It's like wow. {laugh}
820.490 821.250: said when whenever821.510 825.080: Whenever someone asks himwhat he wants to do or what he's doing that's theeasiest way to describe it.
819.325 825.250: Because you know every --he said when -- whenever -- whenever someoneasks him what he wants to do, what he's doingthat's the easiest way to describe it.
792.375 796.400: again ^Carl ^Lawrence %um outstanding grades. Good %uh
800.450 804.150: ^Wright State student, hewants to continue at ^Wright State, so that's --
806.650 809.400: %um In biomedicalengineering and %uh
[missed]
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
Unique Challenges• Many speakers = takes longer to transcribe!
– Impact of overlapping speech, even using IHM audio• Varying levels of speaker participation
– Often no speech but other speaker/background noise• Meeting content
– All over the map, from games to technical meetings• Lack of customized transcription tools
– Existing tools optimized for • 1-channel, multispeaker per channel (BN)• 2-channel, one speaker per channel (CTS)
– Needed: a tool that merges features of each• Arbitrary number of channels, speakers• Easily move between mixed and individual signal playback
– Access to video would also help disambiguate
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
XTrans• Multipurpose speech annotation tool • Multilingual, multi-platform
– written in Python– AGTK infrastructure
• Customized task modules– Careful transcription
• Specialized QC functions– Quick transcription
• Timed mode– “Metadata” annotation
• Structural features, speaker diarization– Correction mode
• e.g., correct automatic transcript or QTR CTR– Comparison and adjudication of multiple transcripts– Allows video input
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
One Channel View
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
One Speaker, MultiChannel
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
MultiSpeaker, MultiChannel
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
PanelView
• RT-05 Meeting Recognition Workshop, MLMI - Edinburgh, July 13 2005
Adjudication Mode
