CLT Conference HeerlenRon Salden, Ken Koedinger, Vincent Aleven, & Bruce McLaren(Carnegie Mellon University, Pittsburgh, USA)Does Cognitive Load Theory account for the beneficial effects of worked examples in tutored problem solving?
Worked examples and tutored problem solving
Worked examples mostly investigated in untutored problem solving environments
Cognitive Tutor is Intelligent Tutoring System provides step-by-step guidance during complex problem solving practice
Worked examples and tutored problem solving
Cognitive Tutor provides a tougher control condition of tutored problem solving
It is this tutored part that in our view reduces extraneous load And, sometimes, increases germane load
Longstanding tradition in improving students learning
Grounded in cognitive theory (ACT-R, Anderson & LeBire, 1998)
Methods for reducing WM loadScaffolding or prompting of sub-goalsStep by step feedback & hints (i.e., guided learning)
Use cognitive model of student thinking
Many full-year classroom evaluations show improved math competence (Koedinger & Aleven, 2007)Cognitive Tutors
Cognitive TutorsElements that reduce extraneous cognitive load.
*Pittsburgh Science of Learning Center *1a. Corrective feedbackStandard Cognitive Tutor: control condition
Pittsburgh Science of Learning Center
*Pittsburgh Science of Learning Center *Standard Cognitive Tutor: control condition1b. Implicit positive feedback
Pittsburgh Science of Learning Center
2. Stepwise hints:last hint level is bottom-out hint problem fading into exampleStandard Cognitive Tutor: control condition
3. Problem sub-goals are givenStandard Cognitive Tutor: control condition
*Pittsburgh Science of Learning Center *Standard Cognitive Tutor: control condition4. Students self explanationThis feature is not so much about reducing extraneous load but about increasing germane load
Pittsburgh Science of Learning Center
StudiesShih et al (Geometry)McLaren et al (Chemistry)Salden et al (Geometry)
Not addressed in this talkAnthony et al (Algebra) obtained similar results as the other three studies
Does Cognitive Load Theory explain beneficial effects of examples in tutored problem solving?
Shih et alStudy: re-analyzing prior study (Aleven & Koedinger, 2002)Logged response data on bottom-out hint usage
One type of Gaming the system behaviorCan be hint abuse due to students skipping abstract hints to obtain the concrete answerCan also be helpful when bottom-out hints act as worked examples
Shih et alDeveloped a model to distinguish between good student use of bottom-out hints from bad student use of bottom-out hints
Two key elements of model are time spent on:Reflecting about prior step (after bottom-out hint) Thinking about next step (prior to next action)
Subtraction method to isolate reflection (self-explanation) timeUse other data, when bottom-hints are not requested, to estimate next step time
Shih et al resultsHigh correlation of time spent reflecting on bottom-out hint with learning (pre-to-post gain)
Spending time on hints is beneficial to learning for all students
Difference between students hint usage:Good usage = spending more time on bottom-out hintBad usage = spending less time on bottom-out hint
Thus students who study bottom-out hint as worked example obtain higher learning gains
McLaren et alConducted three studies comparingTutored Alone vs. Worked Examples + Tutored
Examples are alternated with isomorphic problems
Stoichiometry Tutor: control condition
Worked Example conditionStudents watch video of a worked example plus do prompted self-explanations following the example:
McLaren et al resultsNo differences on posttest performance
BUT, students in Examples condition did learn more efficiently, using 21% less time to finish same problem set
Salden et alConducted lab and classroom study comparing:Tutored problem solvingFixed example fadingAdaptive example fading
Adaptive fading based on students self-explanations of the example stepsStudents who self explain well receive fewer examples than students who self explain poorly
*Pittsburgh Science of Learning Center *Standard Cognitive Tutor: control condition
Pittsburgh Science of Learning Center
Salden et al resultsLab study:Adaptive fading condition needed fewer examples than fixed fading conditionAdaptive fading > both fixed conditions on posttest and delayed posttest
Classroom study:Adaptive fading condition needed fewer examples on several theorems than fixed fading conditionAdaptive fading > problem solving on delayed posttest
Summary of resultsShih et al: Students can effectively use bottom-out hints as worked examples and achieve higher learning gains
McLaren et al: Students working with examples can complete learning phase needing 21% less time while obtaining the same learning outcomes
Salden et al: Students learning from adaptively faded examples obtained higher immediate and delayed posttest performance
Fourth study by Anthony et al (using Algebra Tutor): Students who learned with examples attained better long term retentionAlso measured mental effort: examples = tutored problem solving
Does CLT explain these beneficial effects of worked examples in tutored problem solving?Cognitive Tutor is a harder control condition than untutored environmentsStudents can effectively use bottom-out hints as worked examples
The tutoring seems to reduce possible extraneous cognitive loadAnthony study even showed no difference in mental effort between control and experimental condition
Stepwise feedback & hints, self-explanation prompts geared to increase germane cognitive load
Does CLT explain these beneficial effects of worked examples in tutored problem solving?Possible explanations
Without the information (guidance) provided by examples, students waste time tackling new skills during problem solving
McLaren study: examples lead to same learning gains but needed 21% less time
Two Freiburg lab studies: examples lead to same learning gains needing roughly 17.5% and 25% less time
Motivation
Goal of understanding v. performing (Shih et al)
Frustration after unsuccessful solution attempt
Where is the cognitive load?
Questions?
By providing students with the sub-goals (the angles they need to find), students do not need to engage in means-ends and store a goal stack in working memory. Just as in completion problems, this feature of some (but not all) cognitive tutor units reduces or eliminates the extraneous cognitive load that problem solving typically produces.*Worth making explicit that this feature is not about reducing extraneous load, but about increasing generative processing or germane load. (And that past studies have shown that it enhances transfer -- even though it means fewer practice problems in the same time.)*Ken: You may want to note that there are other kinds of gaming the system behavior, particularly systematic guessingneed to say, model distinguishes between these two based only on information naturally available to the tutor, i.e., students' problem-solving actions in the tutor interface, and the tutor's interpretation of these actions based on its cognitive model***On the time issue, it isn't load in the sense of sub-goaling or anything else creating a dual task that distracts from learning. Instead, at some points the learner just doesn't know or doesn't remember what to do. That's the cause of the floundering and thus wasted time. It isn't load. It isn't simultaneous with what might otherwise be a productive learning event as a load explanation requires.*
