Date post: | 04-Jan-2016 |
Category: |
Documents |
Upload: | meryl-austin |
View: | 218 times |
Download: | 0 times |
Mapping
Workshop on Universal Reporting Parameters for the Speech
of Individuals with Cleft Palate
Washington DC, April/May 2004
Tara Whitehill, Hong Kong
Proposed Three-Stage Plan
• Evaluating: The usual speech evaluation by the local speech specialist in cleft palate.
• Mapping: Conversion from the local speech evaluation to the universal reporting measures.
• Reporting: Universal parameters that describe the cardinal features of speech in individuals with cleft lip/palate and related craniofacial anomalies regardless of the individual’s primary language.
Already covered
• Evaluation (range of protocols and procedures; key issues)
• Parameters (proposal; key issues)
• Focus: mapping
Fanny LAW Sze Kuen
International Parameters for Cleft Palate Speech: A Pilot Study
Honours dissertation
Bachelor of Science (Speech & Hearing Science)
University of Hong Kong
May 2004
Supervisor: Dr. T. L. Whitehill
DRAFT - in progress
• 6 sets of client forms from 5 centres
• 5 clients each set = 30 client forms
• Original forms only (no recordings, no additional data such as transcriptions, unless provided on original form)
• Two groups of “mappers”– 5 “experts” - members of WG who supplied
forms
– 5 students - little/no previous experience cleft
• Used “current” version of parameters and guidelines
• Students given session to familiarize/
clarify terminology• Mapped from client forms onto parameters
form• Identifying information masked (client, examiner,
centre)
• Provided feedback on ease/difficulty of mapping each parameter– Either for client form or by set (centre)
• Weighted scores - not included in this study
“Outcome measures”
• Agreement– Within expert group– Within student group– Across all mappers
• Feedback (ease/difficulty)
Why use students?
• In future, unlikely mappers will be naïve
• However, “expert” group was restricted to individuals who had developed and debated the parameters
Agreement
• A big problem was “missing data”• OK if client form stated MD (or WNL or NAD),
but what if didn’t? - assume OK? tick MD?
• Calculated
(a) absolute agreement (Condition 1)
(b) disregarding “missing data” (Condition 2)
Trial Results amongexperts
Results amongstudents
Agreement and explanation
1 1 1 1 1 1 1 1 1 1 Agreed among experts, students and allmappers in both Condition 1 and 2.
2 1 1 1 1 2 2 2 2 2 Agreed among experts, and students in bothCondition 1 and 2
Not agreed among all mappers in bothconditions
3 1 MD 2 1 1 2 1 1 1 Not agreed among any groups in eitherconditions
4 1 MD 1 MD 1 1 1 1 1 Agreed among students in Condition 1 Agreed among experts in Condition 2 Agreed among all mappers in Condition 2
5 MD MD NA 2 MD 2 2 2 MD Trial for experts was deleted as there would be
no agreement when the MD and NA wereexcluded.
Agreed among students in Condition 2 Agreed among all mappers in Condition 2
Key: MD = missing data, NA = no response, 1 = WNL/Absent, 2 = Present
Table 1. Examples to illustrate how agreement was defined
Primary Parameters
Hypernasality
Agreement• Experts Condition 1 - 60%
Condition 2 - 63%• Students Condition 1 - 73%
Condition 2 - 73%• All Condition 1 - 57%
Condition 2 - 60%
Not much missing data
Primary Parameters
HypernasalityFeedback/Issues to consider
• Some difficulties when no. scale points differed (e.g., client form - 7 pt. scale; parameters form - 4 pt. scale)
• In such cases, mappers reported difficulty, and agreement decreased
• Implication: explicit instructions?• Another difficulty: “mild to moderate” or “moderate to
severe” (students selected less severe and experts more severe!)
• “Set B” - definition of “mild” = “evident but acceptable”. One expert rater mapped onto WNL.
Primary Parameters
Hyponasality
Agreement• Experts Condition 1 - 53%
Condition 2 - 96%• Students Condition 1 - 87%
Condition 2 - 100%• All Condition 1 - 50%
Condition 2 - 97%
Excellent, when missing data is accounted for
Primary Parameters
HyponasalityFeedback/Issues to consider
• Missing data
• Otherwise, excellent
• (binary scale)
Primary Parameters
Audible Nasal Emissionwith/without Nasal Turbulence
Level One (absent, present, MD)
• Experts Condition 1 - 63%
Condition 2 - 87%
• Students Condition 1 - 77%
Condition 2 - 80%
• All Condition 1 - 57%
Condition 2 - 70%
Level Two (subcategories)
• Experts Condition 1 - 23%
Condition 2 - 60%
• Students Condition 1 - 30%
Condition 2 - 38%
• All Condition 1 - 17%
Condition 2 - 27%
Primary Parameters
Audible Nasal Emissionwith/without Nasal Turbulence
Feedback/Issues to consider• Low agreement for subcategories• “phoneme specific” - e.g. “audible for plosive and /s/”• Confusion about pervasiveness e.g. “mild but
consistent”, “phoneme specific and frequent”• Results of mirror test (if not audible in speech)• (NE w/wout nasal turbulence - not raised here)
Primary Parameters
Weak Oral Pressures
Agreement• Experts Condition 1 - 17%
Condition 2 - 75%• Students Condition 1 - 13%
Condition 2 - 62%• All Condition 1 - 0%
Condition 2 - 46%
Primary Parameters
Weak Oral Pressures
Feedback/Issues to consider• All 9 mappers found this parameter difficult to map• Primarily because of lack of data on client forms - no
form had specific category for this• Mappers needed to derive the information from other
information; particularly difficult for students• Also, students unclear about definition
Primary Parameters
Substitution ErrorsLevel One (absent, present, MD)
• Experts Condition 1 - 57%
Condition 2 - 73%
• Students Condition 1 - 73%
Condition 2 - 85%
• All Condition 1 - 57%
Condition 2 - 70%
Level Two (subcategories)
• Experts Condition 1 - 10%
Condition 2 - 30%
• Students Condition 1 - 10%
Condition 2 - 15%
• All Condition 1 - 10%
Condition 2 - 20%
• Agreement not bad for level one, but very poor for subcategories; missing data was not the main problem
Primary Parameters
Substitution Errors
Feedback/Issues to consider• All forms included level one (present/absent/MD)• Subcategories: most forms had patterns, but few
included frequency information • Some patterns which expert listeners considered
‘related to cleft palate’ not included here (e.g. nasal substitution/realization’, ‘double articulation’)
• Confusion between this parameter and ‘Developmental Delay or Other Articulation/Phonological Errors’ (even for experts) e.g. omission, backing /t/ -> [k].
Secondary Parameters
Errors related to dentition, occlusion, palatal vault configuration
Agreement• Experts Condition 1 - 13%
Condition 2 - 69%• Students Condition 1 - 30%
Condition 2 - 59%• All Condition 1 - 7%
Condition 2 - 53%
Better when missing data taken into consideration, but still pretty poor
Secondary Parameters
Errors related to dentition, occlusion, palatal vault configuration
Feedback/Issues to consider• No specific category for this on client forms;
information had to be extracted from other information
• Some forms (Sets B, E, F) included categories such as ‘lateralization’, ‘palatalization’, ‘dentalization’ - but still not clear whether these due to organic factors or “Other Articulation/Phonological Errors” - even for experts
• Errors related to hearing problem? fistula?
Secondary Parameters
Developmental Delay or Other Articulation/Phonological Errors
Agreement• Experts Condition 1 - 16%
Condition 2 - 72%• Students Condition 1 - 30%
Condition 2 - 50%• All Condition 1 - 10%
Condition 2 - 43%
Again, better when missing data taken into account, but still pretty poor
Secondary Parameters
Developmental Delay or Other Articulation/Phonological Errors
Feedback/Issues to consider• Mappers needed to make judgement regarding
whether errors were related to cleft or not • Confusion with both “Substitution Errors” and “Errors
related to dentition..”• Client forms rarely specified cause or explanation of
error patterns - mappers needed to judge• Special cases e.g. “no oral consonants”
Secondary Parameters
Voice/Laryngeal Disorder
Agreement• Experts Condition 1 - 56%
Condition 2 - 90%• Students Condition 1 - 63%
Condition 2 - 97%• All Condition 1 - 60%
Condition 2 - 96%
Agreement was good, especially when missing data was taken into account
Secondary Parameters
Voice/Laryngeal Disorder
Feedback/Issues to consider• Similar to hyponasality - mappers were familiar with
term• When data was provided on client form, it was clear• Binary choice; no subcategories
Global RatingSeverity/Intelligibility
• Not decided at time of this study
• Only 2 of the 6 sets included such data
Conclusions
• Reliability lower among students than experts
• Experts were those who developed form
• Who will be mapping in the future?