Post on 05-Apr-2018
transcript
7/31/2019 Metacognition and Learning in Adulthood
1/22
Developmental Testing Service, LLC
35 South Park Terrace, Northampton, MA 01060 T Phone F Phone Email http:devtestservice.com
Metacognition and learning in
adulthood
Prepared in response to tasking from ODNI/CHCO/IC Leadership Development Office
by Dr. Theo L. Dawson1, Developmental Testing Service, LLC
Saturday, August 23, 2008
7/31/2019 Metacognition and Learning in Adulthood
2/22
Developmental Testing Service, LLC
Metacognition 2
Metacognition and learning in
adulthood
Contents
7/31/2019 Metacognition and Learning in Adulthood
3/22
Developmental Testing Service, LLC
Metacognition 3
Metacognition and learning in
adulthood1
What is metacognition?
Metacognition is thinking about thinking. Metacognitive skills are usually
conceptualized as an interrelated set of competencies for learning and thinking,
and include many of the skills required for active learning, critical thinking,
reflective judgment, problem solving, and decision-making. Adults whose
metacognitive skills are well developed are better problem-solvers, decision makers and critical thinkers,are more able and more motivated to learn, and are more likely to be able to regulate their emotions
(even in difficult situations), handle complexity, and cope with conflict. Although metacognitive skills, once
they are well-learned, can become habits of mind that are applied in a wide variety of contexts, it is
important for even the most advanced adult learners to flex their cognitive muscles by consciously
applying appropriate metacognitive skills to new knowledge and in new situations.
According to Flavell (1979), who coined the term, metacognition is a regulatory system that includes (a)
knowledge, (b) experiences, (c) goals, and (d) strategies. Metacognitive knowledge is stored knowledge
or beliefs about (1) oneself and others as cognitive agents, (2) tasks, (3) actions or s trategies, and (4)
how all these interact to affect the outcome of any intellectual undertaking. Metacognitive experiences are
conscious cognitive or affective experiences that concern any aspect of an intellectual undertaking.
Knowledge is considered to be metacognitive (as opposed to simply cognitive) if it is used in a strategic
manner to meet a goal. According to Sternberg {, 1986 #14899) it is "figuring out how to do a particular
task or set of tasks, and then making sure that the task or set of tasks are done correctly" (p. 24).
Some of the concepts commonly associated with the metacognition literature are shown in Table 1.
1Theo L. Dawson, Ph.D. (UC Berkeley, 1998), is a respected cognitive developmental psychologist. Her work focuses
on the description of learning sequencesthe actual pathways through which people learn complex concepts and
skillsand the design of developmental assessments of these skills. She has worked closely with members of the IC
since 2002, when she, along with her colleague Dr. Kurt Fischer (Harvard Graduate School of Education), conducted
a study of the sequences through which critical thinking, leadership reasoning, and decision making skills develop in
adulthood. Since then, her work with the IC has involved assisting with the design of leadership curricula, evaluating
the extent to which curricula support leader development, and helping to define skill levels for a range of leadership
and analyst standards.
7/31/2019 Metacognition and Learning in Adulthood
4/22
Developmental Testing Service, LLC
Metacognition 4
Table 1: Metacognition concepts
Concept Description
Metacognitive knowledgeabout persons
Includes a persons beliefs about intra-individual differences, inter-individual differ-ences, and universals of cognition
Metacognitive knowledge
about tasks
The information available to apply to a cognitive activity and an individuals knowl-
edge about the task demands of a given situation
Metacognitive knowledgeabout strategies
Awareness of and beliefs about available strategies
Level of conscious aware-ness of metacognitiveknowledge
Retrieval and construction of metacognitive knowledge can be either conscious orunconscious
Limits of metacognitiveknowledge
Can be accurate or inaccurate, may not be activated when needed, may not havemuch influence when it is activated, and may not have a beneficial effect when it isinfluential
Duration of metacognitiveexperiences
Can be momentary or lengthy, as when we are consciously grappling with a chal-lenging problem
Occurrence of metacogni-
tive experiences
Most likely to occur when one is engaged in intentional, reflective intellectual activi-
ties such as problem-solving and learningEffects of metacognitiveexperiences
Can lead one to establish new goals and to revise or abandon old ones, can causeone to add to ones existing metacognitive knowledge base, and can activatestrategies that would otherwise have remained inactivated
Memory-monitoring, self-regulation, meta-reasoning,consciousness/awareness,and auto-consciousness/self-awareness
Other terms for metacognition
Transfer Using a metacognitive skill learned in one context to solve a problem in anothercontext.
Cognition A general term for thinking, sometimes distinguished from metacognition, which is
thinking about thinking.
Ill-structured problems Ill-structured problems are open-ended, with unclear goals, multiple solutions, andno right answerslike many of the problems we face in the workplace
Double loop learning Learning how the way one defines and solves problems can be a source of prob-lems (Argyris, 1991).
Models of metacognition
The term metacognition is both a general term for thinking about thinking, and a
term used by a particular group of researchers to describe their field. There are
numerous models of metacognition, far too many to describe here. To complicate
things further, metacognition is a central component of several skill sets that are
central to education and the workplace, including (1) reflective judgment, (2) cri tical thinking, (3) decision
making, and (4) problem solving. (Some researchers argue that these are components of metacognition.)
Table 1 provides brief descriptions of some of the more prominent metacognitive models.
7/31/2019 Metacognition and Learning in Adulthood
5/22
Developmental Testing Service, LLC
Metacognition 5
Table 2: Models of metacognition
Finding Implication Reference
Self regu-lated learning(lower levels)
Self-regulated learners approach educational tasks withconfidence, diligence, and resourcefulness; are awarewhen they know a fact or possess a skill and when they donot; proactively seek out information when needed andtake the necessary steps to master it; find a way to suc-ceed even when they encounter obstructions; view learn-ing as a systematic and controllable process; accept re-sponsibility for their achievement outcomes; and monitorthe effectiveness of their learning methods or strategies.Self-regulated learning strategies include self-evaluation,organization and transformation, goal setting and planning,information seeking, record keeping, self-monitoring, envi-ronmental structuring, giving self-consequences, rehears-ing and memorizing, seeking social assistance, and re-
viewing. (paraphrased from Zimmerman, 1990) In additionto metacognition, motivation and behavior are consideredto be components of self-regulated learning.
(J. G. Borkowski & Thorpe, 1994;Como, 1986; Ghatala, 1986;Schloemer & Brenan, 2006; Barry J.Zimmerman, 1990)
Note: Watch out for applications ofthis model that focus exclusively onteaching students more effectiveways to memorize facts.
7/31/2019 Metacognition and Learning in Adulthood
6/22
Developmental Testing Service, LLC
Metacognition 6
Finding Implication Reference
Critical think-ing (all lev-els)
The Foundation for Critical Thinking(http://criticalthinking.org)
This model includes standards ofthought (clarity, accuracy, rele-vance, logicalness, breadth, preci-
sion, significance, completeness,fairness, and depth), which are ap-plied to the elements of thought(shown on the left), resulting in thedevelopment of intellectual virtues.
This is an iterative cycle, which isrepeated as reasoning increases incomplexity over the course of de-velopment.
Critical thinking and creative thoughtare linked in this model (Paul &Elder, 2006).
Reflectivejudgment
The reflective judgment model is a very well-researchedmodel of the development of thinking about thinking.
(Kitchener & Fischer, 1990; Kitch-ener & King, 1990; P. K. Wood,1997)
Double looplearning (up-per levels)
Argyris brings metacognition to the workplace by highlight-ing the importance of becoming conscious of the conse-quences of ones actions. He uses real-life examples fromthe workplace to demonstrate methods for teaching upperlevel managers to think effectively about their thinking.
{Argyris, 1991 #14906; Argyris,1982 #14625;
7/31/2019 Metacognition and Learning in Adulthood
7/22
Developmental Testing Service, LLC
Metacognition 7
Metacognition research
Table 3: Metacognition and learning
Finding Implication Reference
Metacognitive skills develop. They can be learned. (Baer, Hollenstein, Hofstetter,Fuchs, & Reber-Wyss, 1994;Brown, Bransford, Ferrara, & Cam-pione, 1983; John H. Flavell, 1979;Ruth Garner & Alexander, 1989)
Adults often fail to monitor theirthinking.
Adults can benefit from metacogni-tive training.
(J. H. Flavell, 1981; Ruth Garner &Alexander, 1989; Glenberg, Wilkin-son, & Epstein, 1982)
Students who have been taughtmetacognitive (self-regulated learn-ing) skills learn better than studentswho have not been taught theseskills.
It is possible to produce betterlearners by teaching metacognitiveskills.
(J. Borkowski, Carr, & Pressely,1987; Bransford, Sherwood, Vye, &Rieser, 1986; Carr, Kurtz, Schnei-der, Turner, & Borkowski, 1989; R.Garner, 1990; Hascher & Oser,1995; Mace, Belfiore, & Hutchinson,2001; Pressley & Ghatala, 1990; B.
J. Zimmerman & Schunk, 2001)
Students with good metacognitiveskills are better critical thinkers,problem-solvers, or decision makersthan students who are not.
It is possible to produce better criti-cal thinkers, problem-solvers, anddecision makers by teaching meta-cognitive skills.
(Bransford et al., 1986; Ewell-Kumar, 1999; Heath, 1983)
People whose thinking is morecomplex tend to have better meta-cognitive skills.
Metacognition and cognitive com-plexity are related.
(Swanson & Hill, 1993; Vukman,2005)
In most people, the development ofcognitive complexity progresses atdifferent rates in different knowledgedomains, depending upon experi-ence and learning in particular do-
mains.
Since cognitive complexity andmetacognition are related, we mightexpect metacognition to be moreadvanced in more developed knowl-edge domains.
(Fischer & Pruyne; Fischer, Yan, &Stewart)
Cognitive development involvesboth knowledge acquisition and(largely unconscious) knowledgestructuring. (If there is no knowledgeto organize, then there is no devel-opment.)
Because metacognitive skills in-volve the conscious structuring ofknowledge, they are likely to bemore developed in areas of greaterknowledge.
(Bransford et al., 1986)
Content knowledge is more easilyaccessed in real-world situations ifstudents learn how new conceptsand procedures can function astools for solving relevant problems.
Learning environments should in-clude opportunities for students toreflectively apply new concepts andtools in real-world contexts.
(Glaser, 1984)
Both content knowledge and meta-cognitive skills are essential for
learning.
Learning may be enhanced wheninstruction (1) provides explicit con-
tent knowledge while (2) askingstudents to use metacognitive skillsto operate on that knowledge.
(Bransford et al., 1986; Perkins,1987)
Metacognitive training can increasestudents self-confidence and senseof personal responsibility for theirown development.
Increased self-confidence and asense of increased personal re-sponsibility may provide motivationfor learning.
(McCombs & Marzano, 1990;Schunk, 1990)
7/31/2019 Metacognition and Learning in Adulthood
8/22
Developmental Testing Service, LLC
Metacognition 8
Finding Implication Reference
Metacognitive training can increasestudents motivation to learn.
Training in metacognitive skills mayenhance students sense of selfefficacy, thus increasing their moti-vation to learn.
(Bandura, 1986; Hofer & Yu, 2003;Sperling, Howard, Staley, & DuBois,2004)
Moving adult learners to a point of
acknowledging that old routines nolonger work as well as new,
instructed ones takes time andmany demonstrations of the superi-ority of the new routines.
Metacognitive strategies should beembedded in assignments andclassroom activities across the cur-riculum at every level of instruction
(Ruth Garner & Alexander, 1989)
Intelligent novices can use generalmetacognitive skills to figure outhow to obtain knowledge in an un-familiar domain.
Once adults have gained expertiseand learned how to use a range ofmetacognitive skills in one domain,they can use some of their meta-cognitive skills to more rapidly learnin another domain.
(Bruer, 1993; Mathan & Koedinger,2005); Garner, 1989 #14918}
Students receiving intelligent novicefeedback acquire a deeper concep-
tual understanding of domain princi-ples and demonstrate better transferand retention of skills over time thanstudents who do not receive suchfeedback.
Teachers or intelligent tutors cansupport the use of existing meta-
cognitive strategies in new knowl-edge areas by providing feedbackthat reminds students to employmetacognitive strategies they haveused in familiar knowledge areas.
(Mathan & Koedinger, 2005)
When students perceive an empha-sis on mastery goals in their class-room, they report using more meta-cognitive learning strategies.
Classrooms in which covering thecontent is emphasized over under-standing can deprive students of theopportunity to learn and masterlearning skills.
(Ames & AfIher, 1988)
Use of concept maps helped adultstudents develop thinking skills,promoted growth in understandingtheir learning processes, and fos-
tered understanding of knowledgeconstruction.
Concept mapping, used well, is auseful metacognitive skill.
(Daley, 2002)
Repeated experiences of dyadicdiscussions within the classroomimproved reasoning skills (over con-trols).
Active engagement in thinkingabout a topic enhances the qualityof reasoning about that topic.
(Kuhn, Shaw, & Felton, 1997)
Informal learning is enhanced inmanagers who employ a wide rangeof metacognitive strategies.
Training in the use of metacognitivestrategies may increase informallearning in less metacognitively so-phisticated managers.
(Enos, Kehrhahn, & Bell, 2003)
Students in problem based learningclassrooms have been found tohave higher levels of intrinsic goal
orientation, task value, use of elabo-ration learning strategies, criticalthinking, metacognitive self-regulation, effort regulation, and/orpeer learning compared with con-trol-group students.
Problem based learning environ-ments may enhance metacognitiveskills relative to conventional in-
structional environments.
(Sungur & Tekkaya, 2006)
7/31/2019 Metacognition and Learning in Adulthood
9/22
Developmental Testing Service, LLC
Metacognition 9
Finding Implication Reference
Gifted learners have been found toemploy fewer metacognitive strate-gies than less gifted students.
Gifted learners, because they learneasily, may not need to employmetacognitive strategies to excel.This could result in reasoning defi-cits in later life.
(Dresel & Haugwitz, 2005)
Table 4: Metacognition and critical thinking
Concept Resources
Critical thinking is the ability to (1) identify and formulate important questions andproblems; (2) gather and assess information; (3) test proposed conclusionsagainst relevant criteria and standards; (4) think within alternative systems ofthought, assessing their assumptions, implications and practical consequences;and (5) communicate effectively, without appealing to logical fallacies or manipu-lating others.
The Foundation for CriticalThinking(http://criticalthinking.org)
Table 5: Metacognition and reflective judgment
Description Implications Source
Reflective judgment is metacogni-tion.
(Hofer, 2004)
Beliefs about learning significantlyimpact the quality of learning strate-gies and learning outcomes in gen-eral.
Students whose reflective judgment skillsare more developed are likely to be betterlearners.
(B. K. Hofer, 1999; Barbara K.Hofer, 2000; Paulsen & Feld-man, 2005; Schommer, 1990,1993; Schommer, Crouse, &Rhodes, 1992)(Brten &Stromso, 2005; Muis, 2007; P.Wood & Kardash, 2002)
7/31/2019 Metacognition and Learning in Adulthood
10/22
Developmental Testing Service, LLC
Metacognition 10
Figure 1: Model of metacognitive skills and conditions for their development and use
7/31/2019 Metacognition and Learning in Adulthood
11/22
Developmental Testing Service, LLC
Metacognition 11
Table 6: Metacognition and problem solving
Finding Implications Source
Effective problem solving dependson the nature and organization ofthe knowledge available to individu-als.
Students whose reasoning skills are more de-veloped are likely to be better learners.
Bransford, 1986 #6747;Chi, 1988 #982}
Psychological biases interfere withproblem-solving
Psychological biases may be amelioratedthrough metacognition
(Duchon, Ashmos, &Dunegan, 1991)
Table 7: Metacognition and psychological biases
Unconscious knowledge structuring takes place every time we learn something. These natural structuring
processes have limitations that (1) are ineffective in some situations or (2) produce cognitive biases.
Metacognition can be employed to counter these biases.
Note: Although it is clearly part of the subtext of much research in metacognition (especially critical
thinking), the subject of psychological bias is rarely raised explicitly.
Bias Implications Source
Overestimateexplanatoryknowledge
People tend to overestimate the depth of their explanatory knowl-edge (how well they understand concepts), which can produce deci-sion errors.
(Mills & Keil, 2004)
Table 8: Metacognition and mindfulness
Finding or comment Source
Mindfulness begins by bringing awareness to current experienceobserving and at-
tending to the changing field of thoughts, feelings, and sensations from moment tomomentby regulating the focus of attention. This leads to a feeling of being very alertto what is occurring in the here-and-now. It is often described as a feeling of being fullypresent and alive in the moment. (p. 232) Mindfulness is further defined by an orienta-tion to experience that [involves] making a commitment to maintain an attitude ofcuriosity about where the mind wanders. (p. 233)
(Bishop et al., 2004)
Mindfulness can be thought of as creating an optimally receptive state for new learningand experience, increasing the likelihood that appropriate metacognitive skils will beselected and employed.
(Garland, 2007; Tho-mas, 2006)
Mindfulness practice requires the activation of metacognitive knowledge, monitoring,and control.
(Wells, 2005)
7/31/2019 Metacognition and Learning in Adulthood
12/22
Developmental Testing Service, LLC
Metacognition 12
Figure 2: Metacognitive skills are an important component of interpersonal intelligence
7/31/2019 Metacognition and Learning in Adulthood
13/22
Developmental Testing Service, LLC
Metacognition 13
Teaching and learning metacognition
Figure 3: Learning model for the workplace, emphasizing motivation, assessment, and metacognition (adapted from
http://www.bbi.school.nz/philosophy/learningtolearn.html)
Table 9: Teaching and learning metacognitive skills: The research
Finding Reference
Metacognitive skills can be taught. (J. Borkowski et al.,1987; Bransford et al.,1986; R. Garner,1990; Hascher & Oser,1995)
Metacognitive skills learned in one context are not automatically transferred to anothercontext.
(Ericsson, Chase, &Faloon, 1980)
Metacognitive skills are learned and applied more effectively in supported active learn-
ing contexts than in direct instruction contexts.
An application of metacognition to diversity issues. Could be useful as a starting placefor discussion.
(Byars-Winston &Fouad, 2006)
The role of formative assessment in self-regulated learning. (Nicol & MacFarlane-Dick, 2006)
7/31/2019 Metacognition and Learning in Adulthood
14/22
Developmental Testing Service, LLC
Metacognition 14
Table 10: Teaching and learning metacognitive skills: ActivitiesActivity Description Resource
Identifying cogni-tive errors
When students monitor their learning, they can become aware ofpotential problems, including errors in encoding, operations, andgoals. Errors in encoding include missing important data or notseparating relevant from irrelevant data. Errors in operations in-clude failing to select the right sub-skills to apply or failing to dividea task into subparts. For example, some math students will jumpright to what they think is the final calculation to get the desiredanswer. Errors in goal seeking include misrepresenting the task andnot understanding the criteria to apply. Problems with cognitive loadinclude being unable to handle the number of sub-skills necessaryto do a task, or not having enough automatic, internalized sub-skills.
A good way to discover what kind of errors managers are making intheir thinking processes is to have them unpack their thinking byexplaining, step by step, how they are approaching a given task.This not only allows the instructor to diagnose possible errors, itprovides managers with an opportunity to describe their thinkingprocesses, which, by itself, develops their metacognitive abilities.
(Nickerson, Perkins, &Smith, 1985)
Strategic learning Working in groups, have students generate questions about thecontent being learned and attempt to answer them.
Have students produce written or verbal summaries of course texts.
Require students to create examples, make analogies, and explainrelationships between concepts. Then, engage students in the useof organizing strategies like concept maps, network representa-tions, and other graphic organizers.
(Simpson & Nist, 2000)
Self-regulatorymetacognitivequestions
These questions are designed to follow instruction on a particulartopic and precede instruction on the next topic: (a) "What did I learnabout this topic?" (monitoring) (b) "With what did I have difficulty?"(monitoring) (c) "What types of things can I do to deal with this diffi-culty?" (problem solving-planning) (d) "What specific action(s) am Igoing to take this week to solve any difficulties?" (planning)
Before assigning this task for the first time, instructors should workthrough an example with the group.
(McInerney, McInerney,& Marsh, 1997)
7/31/2019 Metacognition and Learning in Adulthood
15/22
Developmental Testing Service, LLC
Metacognition 15
Activity Description Resource
Cromley summa-rizes the researchfindings, thentranslates theminto practical ac-
tivities for theclassroom. This isan excellent re-source for adulteducators.
Skills need to be taught in the context in which they will be used.For example, if students are learning to lead in a high security situa-tion, they need to practice confronting the kinds of issues that comeup in high security situations, not just "general" skills.
Reading skills are subject-specificunderstanding what you read in
a technical report does not guarantee that you will read well in an-other subject area.
Problem-solving skills in one subject area are different from those inother areas. Problem-solving skills need to be taught separately foreach subject.
Since problem-solving skills do not automatically transfer from onesubject area to another, instructors need to show students how totransfer these skills and give them lots of practice.
Students need more and better mental models of the world in orderto learn and master new information and skills.
Thinking skills, such as inferring unstated facts, need to be taughtexplicitly in the classroom; they do not develop on their own (exceptin a very few students). These strategies need to be practiced over
and over again.Most adult learners have a very limited number of strategies forunderstanding new material or solving problems. Teaching themmore strategies can help them learn much better.
Learning lasts when the student understands the material, not justmemorizes it.
Information needs to be presented in small chunks so that workingmemory can process it.
Students need immediate practice to move information from work-ing memory to long-term memory.
It is impossible to remember without associating new informationwith what you already know.
(Cromley, 2000)
Authors provide
an overview ofscaffolding and itseffects on learn-ing.
(From the abstract) This paper proposes an expanded conception
of scaffolding with four key elements:(i) scaffolding agency expert, reciprocal, and self-scaffolding;
(ii) scaffolding domain conceptual and heuristic scaffolding;
(iii) the identification of self-scaffolding with metacognition; and
(iv) the identification of six zones of scaffolding activity; each zonedistinguished by the matter under construction and the relative posi-tioning of the participant(s) in the act of scaffolding.
(Holton & Clarke, 2006)
7/31/2019 Metacognition and Learning in Adulthood
16/22
Developmental Testing Service, LLC
Metacognition 16
Figure 4: Model of formative assessment and self-regulated learning
7/31/2019 Metacognition and Learning in Adulthood
17/22
Developmental Testing Service, LLC
Metacognition 17
Table 12: Metacognition fads
Fad Claim Comments Source
Learningstyles
Different people havedifferent learning styles(that can be measuredwith survey-type instru-ments) and these havean impact on learning.
Adjusting teaching style to accommo-date different learning styles has notbeen shown to impact learning. Moreo-ver, catering to learning preferencesmay limit versatility, having a negativeeffect on learning over time.
(Cromley, 2000; Curry, 1990;Stahl, 2002)
Emotionalintelligence
Bringing intelligence tobear upon emotion.
Much of the data necessary for demon-strating the unique association betweenEI and work-related behavior appears toreside in proprietary databases, pre-venting rigorous tests of the measure-ment devices or of their unique predic-tive value. For those reasons, anyclaims for the value of EI in the worksetting cannot be made under the scien-tific mantle (p. 411)
{Landy, 2005#14980}
Resources
Table 11: Metacognition curricula
Resource Description Level Resources
IDEAL problemsolving: Identify,Define, Explore,Act, and Look,and Learn.
The IDEAL problem solving system is based on evidencethat successful problem-solvers actively attempt to (a)identify problems that others may have overlooked; (b)develop at least two sets of contrasting goals for any prob-lem and define them explicitly; (c) explore strategies andcontinually evaluate their relevance to their goals; (d) an-ticipate the effects of strategies before acting on them; and(e) look at the effects of their efforts and learn from them.
(p. 3)
Pre-management
(Bransford, Hay-nes, Stein, Lin, &NashvilleRead,1998)
TV411a na-tional televisionseries aimed atadult learners
Focuses, in part on helping adults manage their own learn-ing using metacognitive skills.
Pre-management
(D'Amico &Capehart, 2001)
Curricula to en-hance studentshigher ordercognitive skills
A handbook offering a number of activities designed foradult learners. Many of these are appropriate for pre-managers, especially if they are contextualized (free fromEric).
Pre-management
(Carman &Askov, 1994)
Cognitive Strat-egy Instruction(CSI)
An instructional approach that has been shown to enhancelearning by emphasizing the development of thinking skillsand processes. The objective is to enable all students tobecome more strategic, self-reliant, flexible, and productive
learners.
Pre-management
(Scheid, 1993)
Using criticalthinking to gainknowledge andunderstanding
A useful resource for students of crit ical thinking in theworkplace.
Pre-management
http://www.unisanet.unisa.edu.au/Resources/nursing/Critical%20thinking/Critical%20thinking.htm
7/31/2019 Metacognition and Learning in Adulthood
18/22
Developmental Testing Service, LLC
Metacognition 18
Resource Description Level Resources
Designing meta-cognitive activi-ties
Several research-based suggestions for creating learningactivities with a strong metacognitive component
(Lin, 2001)
Table 12: Metacognition measures
Resource Description Resources
Lectical ReflectiveJudgment Assessment(LRJA01)
An online assessment of reflective judgment skill (for man-agers). Provides reliable developmental scores in levelincrements, as well as personalized feedback, includingdevelopmentally informed learning suggestions. Intended forpersonal use, coaching, and course evaluations.
http://devtestservice.com
Reasoning About Cur-rent Issues Test (RCI)
The RCI is an online assessment of the capacity to recog-nize and endorse statements that reflect the attributes ofreflective thinking. It is not an assessment of reflectivejudgment skill, and it is not intended as an assessment ofindividuals, but can be used to assess group trends.
http://www.umich.edu/~ref-judg/reasoningaboutcurrentissuestest.html
International CriticalThinking Basic Con-cepts and Understand-ing Online Test
An assessment of students knowledge about critical think-ing conceptsthe extent to which they have learned theseconcepts as they are presented in Elder and Pauls model. Itis not an assessment of critical thinking ability. Reliabilityinformation is not provided.
http://www.criticalthinking.org/courses/critical_thinkg_test1.cfm
International CriticalThinking Test
A pen and paper assessment of critical thinking skill that canbe adapted to any subject area. Scoring is done by instruc-tors, and is based on rubrics.
http://www.criticalthinking.org/assessment//ICAT-info.cfm
Motivated Strategiesfor Learning Question-naire (MSLQ)
Based on the idea of self-regulated learning (SRL), this self-report instrument is designed to assess motivation and useof learning strategies by college students. The motivationscales tap into three broad areas: (1) value (intrinsic andextrinsic goal orientation, task value), (2) expectancy (con-trol beliefs about learning, self-efficacy); and (3) affect (test
anxiety). The learning strategies section is comprised ofnine scales which can be distinguished as cognitive, meta-cognitive, and resource management strategies. The cogni-tive strategies scales include (a) rehearsal, (b) elaboration,(c) organization, and (d) critical thinking. Metacognitivestrategies are assessed by one large scale that includesplanning, monitoring, and regulating strategies. Resourcemanagement strategies include (a) managing time andstudy environment; (b) effort management, (c) peer learning,and (d) help-seeking. (abstract)
(Pintrich, Smith, Garcia, &Mckeachie, 1993)
7/31/2019 Metacognition and Learning in Adulthood
19/22
Developmental Testing Service, LLC
Metacognition 19
Resource Description Resources
Structure of ObservedLearning Outcomes(SOLO)
Method for evaluating learning/reasoning outcomes thatinvolves 5 levels of learning. Is often used to scaffold thedevelopment of scoring rubrics. Rubrics are high inferenceand inter-rater agreement is difficult to maintain. Nonethe-less, when inter-rater agreement is properly controlled, the
rubrics are reliable enough for course evaluations (not usu-ally for individual evaluation).
1. Pre-structural
Learners acquire bits of unconnected information, whichhave no organization and make no sense.
2. Uni-structural
Learners make simple and obvious connections, but showlittle evidence that their significance has been grasped.
3. Multi-structural
Learners make a number of connections, but meta-connections between them are missed, as is their signifi-cance for the whole.
4. Relational
Learners appreciate the significance of the parts in relationto the whole.
5. Extended abstract
Learners make connections both within and beyond the sub-ject area, showing they are able to generalize and transferthe principles and ideas.
(Biggs & Collis, 1982)
7/31/2019 Metacognition and Learning in Adulthood
20/22
Developmental Testing Service, LLC
Metacognition 20
References
Ames, C., & AfIher, J. ( 1988). Achievement goals in the classroom: Students' learning strategies and motivationprocesses. Journal of Educational Psychology., 80, 260-267.
Argyris, C. (1991). Teaching smart people how to learn. Harvard Business Review, May-June.Baer, M., Hollenstein, A., Hofstetter, M., Fuchs, M., & Reber-Wyss, M. (1994). How d expert and novice writers differ
in their knowledge of the writing process and its regulation (metacognition) from each other, and what are the
differences in metacognitive knowledge between writers of different ages? Paper presented at the AnnualMeeting of the American Educational Research Association.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall, EnglewoodCliffs, NJ.
Biggs, J., & Collis, K. (1982). Evaluating the quality of learning: The SOLO taxonomy (structure of the observedlearning outcome). New York: Academic Press.
Bishop, S. R., Lau, M., Shauna, S., Carlson, L., Anderson, N. D., Carmody, J., et al. (2004). Mindfulness: A proposedoperational definition. Clinical Psychology: Science and Practice, 7(3), 230-241.
Borkowski, J., Carr, M., & Pressely, M. (1987). "Spontaneous" strategy use: Perspectives from metacognitive theory.Intelligence, 11, 61-75.
Borkowski, J. G., & Thorpe, P. K. (1994). Self-regulation and motivation: A life-span perspective onunderachievement at any point during the life span (B. P. D & A. E. L, Trans.). In B. J. Z. Dale H. Schunk (Ed.),Self-regulation of learning and performance: Issues and educational applications. (pp. 45-73). U Notre Dame,Notre Dame, IN, US: Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US.
Bransford, J. D., Haynes, A. F., Stein, B. S., Lin, X., & NashvilleRead, N. (1998). The IDEAL workplace: Strategies forimproving learning, problem solving, and creativity.
Bransford, J. D., Sherwood, R., Vye, N. J., & Rieser, J. (1986). Teaching thinking and problem solving. AmericanPsychologist, 41(10), 1078-1089.
Brten, I., & Stromso, H. (2005). The relationship between epistemological beliefs, implicit theories of intelligence,and self-regulated learning among Norwegian postsecondary students. British Journal of EducationalPsychology, 75, 539565.
Brown, A. L., Bransford, J. D., Ferrara, R. A., & Campione, J. C. (1983). Learning, remembering, and understanding.In J. H. Flavell & E. M. Markman (Eds.), Carmichael's manual of child psychology(Vol. 1, pp. 77-166 ). NewYork: Wiley.
Bruer, J. (1993). Schools for Thought: A science of learning in the classroom . Cambridge: MIT Press.Byars-Winston, A. M., & Fouad, N. A. (2006). Metacognition and multicultural competence: Expanding the culturally
appropriate career counseling model. Career Development Quarterly, 54, 187-201.Carman, P. S., & Askov, E. N. (1994). Development of a curriculum to enhance adult learners higher order skills.
University Park, PA: Pennsylvania State Universityo. Document Number)
Carr, M., Kurtz, B. E., Schneider, W., Turner, L. A., & Borkowski, J. G. (1989). Strategy acquisition and transferamong German and American children: Environmental influences on metacognitive development. DevelopmentalPsychology, 25, 765-771.
Como, L. (1986). The metacognitive control components of self-regulated learning. Contemporary EducationalPsychology, 11, 333-346.
Cromley, J. (2000). Learning to think: Learning to learn. Washington, DC: National Institute for Literacyo. DocumentNumber)
Curry, L. (1990). One critique of the research on learning styles. Educational Leadership, 48, 50-56.D'Amico, D., & Capehart, M. A. (2001). Letting Learners Lead: Theories of Adult Learning and TV411. Focus on
Basics, 5(B), 35-41.Daley, B. J. (2002). Facilitating learning with adult students through concept mapping. Journal of Continuing Higher
Education, 50(1), 21-32.Dresel, M., & Haugwitz, M. (2005). The relationship between cognitive abilities and self-regulated learning: Evidence
for interactions with academic self-concept and gender. High ability studies, 16(2), 201-218.Duchon, D., Ashmos, D., & Dunegan, K. J. (1991). Avoid decision making disaster by considering psychological bias.
Review of Business, 13.Enos, M. D., Kehrhahn, M. T., & Bell, A. (2003). Informal learning and the transfer of learning: How managers
develop proficiency. Human Resource Development Quarterly, 14(4), 369-.Ericsson, K., Chase, W., & Faloon, S. (1980). Acquisition of a memory skill. Science and Engineering Ethics, 208,
1181-1182.Ewell- Kumar, A. (1999). The influence of metacognition on managerial hiring decision making: Implications for
management development Dissertation Abstracts International Section A: Humanities and Social Sciences,59(10-A).
Fischer, K. W., & Pruyne, E. Reflective thinking in adulthood: Development, variation, and consolidation. In J. Demick(Ed.), Handbook of Adult Development.
7/31/2019 Metacognition and Learning in Adulthood
21/22
Developmental Testing Service, LLC
Metacognition 21
Fischer, K. W., Yan, Z., & Stewart, J. B. Adult cognitive development: Dynamics in the developmental web. In J.Valsiner & K. Connoly (Eds.), Handbook of developmental psychology(pp. 491-516). Thousand Oaks, CA: Sage.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry.American Psychologist, 34(10), 906-911.
Flavell, J. H. (1981). Cognitive monitoring. In W. P. Dickson (Ed.), Children's oral communication skills (pp. 35-60).New York: Academic.
Garland, E. L. (2007). The meaning of mindfulness: A second-order cybernetics of stress, metacognition, and coping.
Complementary Health Practice Review, 12(1), 15-30.Garner, R. (1990). When children and adults do not use learning strategies: Toward a theory of settings. Review of
Educational Research, 60, 517-529.Garner, R., & Alexander, P. A. (1989). Metacognition: Answered and unanswered questions. Educational
Psychologist, 24(2), 143-158.Ghatala, E. S. (1986). Strategy-monitoring training enables young learners to select effective strategies. Educational
Psychologist, 21, 43-54.Glaser, R. (1984). Education and thinking: The role of knowledge.American Psychologist, 39, 93-104.Glenberg, A. M., Wilkinson, A. C., & Epstein, W. (1982). The illusion of knowing: Failure in the self-assessment of
comprehension. Memory & Cognition, 10, 597-602.Hascher, T. A., & Oser, F. (1995). Promoting autonomy in the workplace--A cognitive-developmental intervention.
Paper presented at the Annual Meeing of the American Educational Research Association.Heath, S. B. (1983). Ways with words: Language, life, and work in communities and classrooms. Cambridge, MA:
Cambridge University Press.Hofer, B. K. (2004). Epistemological understanding as a metacognitive process: Thinking aloud during online
searching. Educational Psychologist, 39(1), 43-55.Hofer, B. K., & Yu, S. L. (2003). Teaching self-reglated learning through a "learning to learn" course. Teaching in
Psychology, 30(1), 30-33.Holton, D., & Clarke, D. (2006). Scaffolding and metacognition. International Journal of Mathematical Education in
Science and Technology,, 37(2), 127-143.Kitchener, K. S., & Fischer, K. W. (1990). A skill approach to the development of reflective thinking. Contributions to
Human Development, 21, 48-62.Kitchener, K. S., & King, P. M. (1990). The reflective judgment model: ten years of research. In M. L. Commons, C.
Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer & J. D. Sinnott (Eds.),Adult development(Vol. 2, pp. 62-78).New York: Praeger.
Kuhn, D., Shaw, V., & Felton, M. (1997). Effects of dyadic interaction on argumentative reasoning. Cognition andInstruction, 15(3), 287-315.
Lin, X. (2001). Designing metagognitive activities. Educational Technology, Research, & Development, 49, 23-40.Mace, F. C., Belfiore, P. J., & Hutchinson, J. M. (2001). Operant theory and research on self regulation. In B. J.
Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement(pp. 39-65). Mahwah,NJ: Eribaum.Mathan, S., & Koedinger, K. R. (2005). Fostering the intelligent novice: Learning from errors with metacognitive
tutoring. Educational Psychologist, 40(4), 257-265.McCombs, B. L., & Marzano, R. J. (1990). Putting the self into self-regulating learning: The self as agent in integrating
will and skill. Educational Psychologist, 25(51-69).McInerney, V., McInerney, D. M., & Marsh, H. W. (1997). Effects of metacognitive strategy training within a
cooperative group learning context on computer achievement and anxiety: An aptitude treatment interactionstudy. Journal of Educational Psychology., 89(4), 686-695.
Mills, C. M., & Keil, F. C. (2004). Knowing the limits of one's understanding: The development of an awareness of anillusion of explanatory depth. Journal of Experimental Child Psychology, 87, 1-32.
Muis, K. R. (2007). The role of epistemic beliefs in self-regulated learning. Educational Psychologist, 42(3), 173-190.Nickerson, R. S., Perkins, D. N., & Smith, E. E. (1985). The teaching of thinking. Hillsdale, NJ: Erlbaum.Nicol, D. J., & MacFarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven
principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Paul, R., & Elder, L. (2006). Critical thinking: The nature of critical and creative thought. Journal of DevelopmentalEducation, 30(2), 34-35.
Perkins, D. (1987). Knowledge as design: Teaching thinking through content. In J. B. R. Sternberg (Ed.), Teachingthining skills: Theory and practice. New York: W. H. Freeman.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & Mckeachie, W. J. (1993). Reliability and predictive validity of theMotivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-813.
Pressley, M., & Ghatala, E. S. (1990). Self-regulated learning: Monitoring learning from text. EducationalPsychologist, 25, 9-33.
Schloemer, P., & Brenan, K. (2006). Journal of Education for Business, 81-87.
7/31/2019 Metacognition and Learning in Adulthood
22/22
Developmental Testing Service, LLC
Metacognition 22
Schunk, D. H. (1990). Goal setting and self-efficacy during self-regulated learning. Educational Psychologist, 25, 71-86.
Simpson, M. L., & Nist, S. L. (2000). An update on strategic learning: Its more than textbook reading strategies.Journal of Adolescent and Adult Literacy, 43, 528-541.
Sperling, R. A., Howard, B. C., Staley, R., & DuBois, N. (2004). Metaognition and self-regulated learning constructs.Educational Research and Evaluation, 10(2), 117-139.
Stahl, S. A. (2002). Different strokes for different folks? In L. Abbedutto (Ed.), Taking sides: Clashing on controversial
issuesin educational psychology(pp. 98-107). Guilford, CT: McGraw-Hill.Sungur, S., & Tekkaya, C. (2006). Effects of problem-based learning and traditional instruction on self-regulated
learning. Journal of Educational Research, 99(5), 307-317.Swanson, L., & Hill, G. (1993). Metacognitive aspects of moral reasoning and behavior.Adolescence, 28, 711-735.Thomas, D. C. (2006). Domain and development of cultural intelligence: The importance of mindfulness. Group &
Organization Management, 31(1), 78-99.Vukman, K. B. (2005). Developmental differences in metacognition and their connections with cognitive development
in adulthood. Journal of Adult Development, 12(4), 211-221.Wells, A. (2005). Detached mindfulness In cognitive therapy: A metacognitive analysis and ten techniques. Journal of
Rational-Emotive and Cognitive-Behavior Therapy, 23(4), 337-355.Wood, P., & Kardash, C. A. (2002). Critical elements in the design and analysis of studies of epistemology. In B. K.
Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing(pp. 231260). Maswah, NJ: Lawrence Erlbaum.
Wood, P. K. (1997). A secondary analysis of claims regarding the Reflective Judgment interview: Internalconsistency, sequentiality and intra-individual differences in ill-structured problem-solving. In Higher education:Handbook of theory and research (Vol. 12, pp. 243-312). New York: Agathon.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. EducationalPsychologist, 25(1), 3-17.
Zimmerman, B. J., & Schunk, D. H. (2001). Reflections on theories of self-regulated learning and academicachievement. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement:Theoretical perspectives (pp. (pp. 289-307). Eribaum: Mahwah, NJ.