Eliciation Techniques and Data Collection Sonja Eisenbeiss [email protected]February 2013 Criteria for Method Selection: Reliability and Validity • Reliability: The consistency of a measurement/test • Test-re-test reliability • Inter-rater reliability Strict control of variables, test situation, etc. • Validity: the degree to which the test measures what it intends to measure (e.g. children’s knowledge of genitives or possessives vs. their ability to focus) • Ecological validity: capturing the use of an ability as closely to real-life use as possible Natural test situation with minimal extra demands Types of Methods • spontaneous speech / naturalistic data • "classic" elicitation • stimulus-based elicitation • experiments NOTE: Depending on the sub-field of linguistics (fieldwork, language acquisition research in different contexts,…) , the term “elicitation” is used in VERY different ways. The 1
Transcript
Eliciation Techniques and Data CollectionSonja Eisenbeiss
Criteria for Method Selection: Reliability and Validity
• Reliability: The consistency of a measurement/test• Test-re-test reliability• Inter-rater reliability
Strict control of variables, test situation, etc.
• Validity: the degree to which the test measures what it intends to measure (e.g. children’s knowledge of genitives or possessives vs. their ability to focus)
• Ecological validity: capturing the use of an ability as closely to real-life use as possible
NOTE: Depending on the sub-field of linguistics (fieldwork, language acquisition research in different contexts,…) , the term “elicitation” is used in VERY different ways. The common denominator is: some effort has been made to get people to talk (in a particular way or about a particular topic).Data can be collected cross-sectionally, longitudinally, or at fixed intervalls
• External interference is limited to the fact that the ongoing communicative event (discussion, play situation,…) is being observed and/or recorded.
• This may be known or unknown to the participants
• PRO• most “natural” data type
• analysis not limited to target phenomena• no age limitations
• CONTRA• lack of data for low-frequency phenomena
• lack of negative evidence (i.e. based on such data one does not know which structures are ungrammatical for the speaker)
• hardly any control about form and content
"Classic" Elicitation• Participants who have some general awareness of the researchers aims are asked to
provide information about their language• Types:
• Translation from a lingua franca into the target language• Back-translating utterances provided by other speakers
• Manipulating utterances (e.g. turning them into questions, negating them)• Grammaticality/acceptability judgements
• very controlled data type• provides data for low frequency phenomena
• can provide negative evidence (grammaticality judgments) and meta-linguistic information
• CONTRA
• less “natural” than observed or staged communicative events or elicitation games• analysis often limited to target phenomenon (dependent on type of elicitation)
• age limitations
2
• often limited to one or a few speaker and no/few controls for inter-speaker variation and cross-situational variation (vs. experiments)
Stimulus-Based Elicitation (Games)• creation of contexts for low-frequency phenomena using stimuli (pictures, real life
objects, videos, etc.), often in a game-like setting• e.g. eliciting complete sentences with to give in a " feeding game": participants have
to feed toy animals and explain which food they would like to give to which animals• often used with children that are too young for experiments or as supplements to
(quasi-)naturalistic data or experiments• PRO
• more controlled than other com. events.• more “natural” than experiments
• provide data for low frequency phenomena• analysis not limited to target phenomena
• CONTRA• age limitations, depending on the task
• lack of negative evidence
Experiments• controlled manipulation of one or more variables (e.g. syntactic construction: active
vs. passive sentence) in a fixed setting and with a fixed procedure• e.g. elicited production experiments: participants are asked to provide a specific
morphological form of a given word (e.g. the plural forms cars and mice of the nouns car and mouse)
• PRO
• very controlled data type• provide data for low frequency phenomena
• can provide negative evidence (grammaticality judgement experiments)• CONTRA
• less “natural” than observed or staged communicative events or elicitation games• analysis limited to target phenomenon
• age limitations
3
Cross-sectional vs. Longitudinal vs. Interval Designs• longitudinal: a series of recordings of one or more speakers over an extended
period of timePRO• appropriate for studies of development • representative for the individual speaker
CONTRA• less representative for the variety• long recording period
• cross-sectional: recordings of several speakers one point in time per speakerPRO• short recording period
• representative for the variety under studyCONTRA• less representative for the individual speaker
• not direct investigation of development• interval/mixed: several blocks or continuous series of recordings, for several
individual learnersor for several groups of learners
example I (age in yrs;ms): 1 recording per yearGroup I (3 children) 1;0 2;0 3;0 4;0 Group II (3 children) 5;0 6;0 7;0 8;0 Group III (3 children) 9;0 10;0 11;0 12;0
example II: 1 recording per week, 3 childrenI: 1;0-2;0 II: 2;0-3;0 III: 3;0-4;0
PRO• large age range in limited recording time• representative for speaker AND variety
CONTRA• no analysis of individual development across the entire time span
4
Study Design and Method• Naturalistic sampling and semi-structured elicitation can be used in longitudinal
studies. • Experiments can lead to the development of task-specific or stimulus-specific
strategies (low validity). They can thus not be used in longitudinal designs (or only at the end).
• Experiments require larger groups of participants (typically at least about 20 participants).
Combining Methods• It is advisable to provide converging evidence, using a combination of different data
collection methods.• If little is known about the acquisition of a phenomenon, one can start with one or
two sessions with naturalistic sampling and semi-structured elicitation:• to build up a relationship with participants• to test knowledge of words in experimental stimuli• to use information about spontaneous “errors” to fine-tune experiments (e.g.
grammaticality judgment).• Naturalistic samples can be collected “around” blocks of elicitation/experiments, e.g.
while (un)packing toys.
Using Elicitation Games to overcome the Lack of Data in Spontaneous SpeechConstructions in spontaneous speech can differ widely in their frequency and distribution:
Table 1: A German First Language (L1) corpus (Eisenbeiss 2003)child age files (w. elicit.) utterancesAND 2;1 1 1.4500ANN 2;4-2;9 6 1.977CAR 3;6 1 1.795HAN 2;0-2;8 8 1.399LEO 1;11-2;11 15 (all) 4.383 (4.383)MAT 2;3-3;6 18 1.978SVE 2;9-3;3 15 (10) 3.811 (2814)total 1;11-3;6 64 (20) 16.793 (7.197)
5
Table 2: Contexts for Different Types of Noun Phrase Elements (Eisenbeiss 2003)Noun phrase with context for
number % of utterances correlation with mean length of utterance, in words (MLUw)
Article 2.646 28 0.489; sign.
adjective + article 249 3 0.274; n.s.
possessive ‘s 19 <1 -0.097; n.s.
• In Table 2, you can see that some NP types were found to be comparatively frequent and became more frequent with increasing utterance length (MLU).
• BUT: Some noun phrase types (e.g. with contexts for adjective+article or ‘s) :• were rare• occurred in some files, but not in others• did not become more frequent over time, even though children’s utterances
got longer over time: there was no correlation between the relative frequency of a particular utterance type and the mean length of utterance for the respective recording.
• In recordings with semi-structured elicitation games (represented by black filled symbols in Figure 1), a higher percentage of utterances involve a context for a possessive s-marker than in recordings with spontaneous speech (represented by unfilled symbols). For Leonie, elicited and non-elicited data for the same child in the same recording period and MLU-range (Leonie) were available. Spontaneous speech data contained NO s-contexts, but such contexts were found in the elicited data from the same period.
Figure 1: The Frequency of Possessive s-Contexts in Naturalistic and Elicited Data
6
MLU
4.54.03.53.02.52.01.51.0
% a
ller a
naly
sier
bare
n Ä
ußer
unge
n 6
5
4
3
2
1
0
KORPUS
sve
sve
mat
leo
leo
han
car
ann
and
Stimulus-Based Elicitation: Interactional Setting
• Director/matcher (or “confederate description”):A “director” describes a scene/object etc. and a “matcher” who is not able to see this scene/object, has to recreate it.E.g.: The matcher has to build a toy house identical to the one created by the director.
• Speaker/Listener: A speaker provides information for someone who does not have access to this information. E.g.: The speaker retells a story (s)he heard/read while the listener was not in the room.
• Co-Players:All participants are involved in a game and provide each other with information to co-ordinate their actions. E.g.: The players are involved in a construction or puzzle game where not everyone has access to all pieces.
Stimulus-Based Elicitation: Target Type
• broad-spectrum: generally encouraging participants to speak
7
% of analysable utterances
• frog story: a picture book w/o words usedto elicit narratives (Berman & Slobin, 1994)
• interview techniques• re-telling of videos or stories
• form-focused: the use of a particular form or constructione.g. a game where children have to find the possessions of animals and describe the combination using an adnominal possessive construction like the horse's saddle
• meaning-focused: the linguistic encoding of a particular function or meaning (which can be encoded in different ways)• e.g. “circle of dirt”: a picture book w/owords used to elicit descriptions of part-
whole relationships and actions affecting (body) parts (Eisenbeiss, McGregor, & Schmidt 1999)
• “cut-and-break”: video stimulus created for cross-linguistic studies of “separation and material destruction” events (Bohnemeyer, Bowerman and Brown 2001)
• see p. 13 for References and download information.
8
3 Tasks (Eisenbeiss 2009)
The Bag Task: broad-spectrum This co-player task involves a bag with pockets for blocks and animals of different sizes and colours. The bag has pockets that match the animals in colour and have coloured buttons, ties, etc.; and children frequently refer to colours, sizes and locations when they ask other players to help them hide or find animals in the pockets or to attach the pockets to the Velcro-strips on the bag.As an alternative, one can use sets of small bags that are not attached as pockets to a big bag, but can be carried in a big bag. This requires fewer sewing skills and allows for a more flexible use of different bags.
The Picture-Pairing Task: form-focused or meaning-focusedThis director-matcher task is similar to the game “memory”, but the pictures are matched - not identical. For very young children, memory load can be reduced by visible pictures.Variants• contrast-oriented:
The child has to describe what is on the two pictures and to say whether they match (e.g. small green bananas vs. large yellow bananas)
• combination-oriented: The child is asked to form a phrase or an utterance with components that are depicted on the matching pictures (e.g. the hairdresser’s sheep or the sheep is helping the hairdresser).
• Form vs. meaning-based• The task can be used to focus on particular forms (e.g. constructions with the
verb help or noun phrases with adjectives).• It can be used to investigate how children encode particular meanings. For
instance, picture pairs of owners and their possessions can be used to study how children talk about possessions (e.g. the queen's crown vs. the crown of the queen?).
9
Figure 2: Sample Pictures for the Picture-Pairing Task
The Puzzle-Task: form-focused or meaning-focused• In this co-player task, children have to describe contrasting pictures on a puzzle
board, adults find the matching pieces, and the children then put them into the matching cut-out on the puzzle-board.
• The puzzle board involves exchangeable pictures and puzzle pieces (see Figure 3)• It can be used to elicit particular forms (e.g. or to elicit the linguistic encoding of
particular meanings (e.g. actions on body-parts, see Figure 4).
10
Figure 3: The Picture-Pairing Task
11
Figure 4: Some Sample Pictures for the Puzzle Task
12
Some Freely Available Elicitation Stimuli
The BOWPED Stimulus Bowerman, Melissa & Eric Pederson. 1992. Topological relations picture series. In Stephen C. Levinson (ed.), Space stimuli kit 1.2: November 1992, 51. Nijmegen: Max Planck Institute for Psycholinguistics.http://fieldmanuals.mpi.nl/volumes/1992/bowped/
The Caused Positions StimulusHellwig, Birgit & Friederike Lüpke. 2001. Caused positions. In Stephen C. Levinson & N.J. Enfield (eds.), Manual for the field season 2001, 126-128. Nijmegen: Max Planck Institute for Psycholinguistics.http://fieldmanuals.mpi.nl/volumes/2001/caused-positions/
The Circle of Dirt StimulusEisenbeiss, S., McGregor, B. & Schmidt, C. (1999). Story book stimulus for the elicitation of external possessor constructions and dative constructions ('the circle of dirt'). In David Wilkins (ed.), Manual for the 1999 Field Season, 140-144. Nijmegen: Max Planck Institute for Psycholinguistics.http://fieldmanuals.mpi.nl/volumes/1999/story-book-stimulus-possessor-dative-constructions
The Cut-and-Break StimulusBohnemeyer, J., Bowerman, M. & Brown, P. (2001). Cut and break clips. Manual’for the field season 2001, 90--96.http://fieldmanuals.mpi.nl/volumes/2001/cut-and-break-clips/
See the following webpages for more downloads of stimuli:http://fieldmanuals.mpi.nl http://experimentalfieldlinguistics.wordpress.com/experimental-materials/ http://www.iris-database.org/iris/app/home/index
