Date post: | 24-Dec-2015 |
Category: |
Documents |
Upload: | augustine-shields |
View: | 213 times |
Download: | 0 times |
Assessing the Assessing the Development of Critical Development of Critical
Thinking Skills in Thinking Skills in Undergraduate Undergraduate
Psychology StudentsPsychology Students
Phil Dunwoody & Mark McKellop
Partially funded by grants from the Teagle Foundation, the Council of Independent Colleges/Collegiate Learning Assessment Consortium, and the James J. Lakso Center for the
Scholarship of Teaching & Learning at Juniata College.
What is Critical Thinking?What is Critical Thinking?
What is Critical Thinking?What is Critical Thinking?
American Philosophical Association’s “Delphi Report”
(Facione, 1990)
CT is “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based.”
What is Critical Thinking?What is Critical Thinking?
CT is multidimensional construct – it is both a disposition & a learned skill
(Halpern, 1998, 2003)
CT has general & discipline-specific aspects with limited generalizability
(Lehman & Nisbett, 1990; Toplak & Stanovich, 2002; Williams et al., 2004)
Psychological Critical Thinking (PCT)Psychological Critical Thinking (PCT)
American Psychological Association“Guidelines for the Undergraduate
Psychology Major” (APA, 2007)
◦Goal 3: Critical Thinking Skills in Psychology. Students will respect and use critical and creative
thinking, skeptical inquiry, and, when possible, the scientific approach to solve problems related to behavior and mental processes.
Documenting & Assessing our Teaching◦Departmental Assessment Plan◦Mapping our POE
Psychological Critical ThinkingPsychological Critical Thinking
PCT is “the ability to judge the plausibility of specific assertions, to weigh evidence, to assess the logical soundness of inferences, to construct counter-arguments and alternative hypotheses.” (Nickerson et al, 1985)
CT skills in psychology include:◦ Argument analysis and evaluation◦ Methodological reasoning◦ Statistical reasoning◦ Causal reasoning◦ Focusing and clarifying questions (Bensley & Murtaugh, 2012)
Psychological Critical Thinking Psychological Critical Thinking Inventory (PCTI)Inventory (PCTI)
Our assessment/measure of PCT will◦be relatively brief◦be (relatively) easily scored◦evaluate “applied” critical thinking skills (can one “think like a behavioral scientist?”)
PCTI focuses on following aspects of PCT: Appropriate evaluation of expert behavioral research
(i.e., peer-reviewed journal articles) Appropriate evaluation of non-expert behavioral claims
(i.e., media reports) Identifying reasoning faults in faux-novice designed
behavioral research
Validating the PCTIValidating the PCTI
California Critical Thinking Skills TestWatson-Glaser Critical Thinking AppraisalLawson’s Psychological Critical Thinking
ExamCollegiate Learning AssessmentPsychology Area Concentration
Achievement Test (PACAT)National Survey of Student Engagement
(NSSE)
ParticipantsParticipants
Participants◦Intro Psych: 96 (Spring, 2011)◦Senior Psych Capstone: 39 (Spring & Fall, 2011
and Spring, 2012)◦Pre and Post Research Methods: 29 (Fall, 2011
and Spring, 2012) Pre and Post with CAT currently
Descriptive Stats PCTIDescriptive Stats PCTI
Popular Press◦Possible range: 0-10 points◦Observed range=2-8, Mean=5.9, SD=1.7
Critique Faux-Novice Study◦Possible range: 0-7 points◦Observed range= 0-6, Mean=2.9, SD=1.3
Peer-review Articles◦Possible range: 0-7 points◦Observed range= 0-6, Mean=3.28, SD=1.3
Total PCTI◦Possible range: 0-24◦Observed range= 6-20.5, Mean=12.1, SD=2.7
Hypothesis 1Hypothesis 1
Graduating psychology seniors will score higher on PCTI than freshman in Introductory Psychology.
Popular Press◦ Intro: M=5.19, SD=1.48◦ Capstone: M=6.55, SD=1.99◦ Significant: t(91)=-3.72, p<0.001, d=0.78
Critique Faux-Novice Study◦ Intro: M=2.48, SD=1.19◦ Capstone: M=3.88, SD=1.26◦ Significant: t(91)=-5.27, p<0.001, d=1.14
Peer-review Articles◦ Intro: M=3.03, SD=1.37◦ Capstone: M=3.34, SD=1.45◦ Significant: t(91)=-1.02, p=0.31
Total PCTI◦ Intro: M=10.70, SD=2.34◦ Capstone: M=13.77, SD=2.76◦ Significant: t(91)=-5.64, p<0.001, d=1.20
Hypothesis 2Hypothesis 2
Students in Research Methods should score higher at the end of the semester than at the beginning of the semester (alternate form).
Popular Press◦ Pre: M=6.52, SD=1.46◦ Post: M=7.30, SD=1.32◦ Significant: t(26)=-2.09, p=0.05, d=0.56
Critique Faux-Novice Study◦ Pre: M=3.26, SD=1.10◦ Post: M=3.81, SD=1.27◦ Significant: t(26)=-1.86, p=0.07, d=0.46
Peer-review Articles◦ Pre: M=3.67, SD=1.30◦ Post: M=3.93, SD=1.75◦ Significant: t(26)=-0.55, p=0.59
Total PCTI◦ Pre: M=13.44, SD=2.68◦ Post: M=15.04, SD=2.49◦ Significant: t(26)=-2.39, p=0.02, d=0.62
PCTI Convergent ValidityPCTI Convergent Validity
PCTI with measures of generic PCTI with measures of generic (abstract) critical thinking(abstract) critical thinking
PCTI with Lawson’s Psychological PCTI with Lawson’s Psychological Critical Thinking ExamCritical Thinking Exam
PCTI with measures of educational PCTI with measures of educational achievementachievement
PCTI with measures of Psychology PCTI with measures of Psychology Knowledge (PACAT)Knowledge (PACAT)
ConclusionConclusion
Our PCTI has good face validity and shows clear differences not only between Freshman and Seniors, but also within one course.
The PCTI does not correlate much with more traditional abstract measures of CT.
The PCTI clearly correlates with the psychology knowledge-based PACAT.
The PCTI also clearly correlates with general measures of academic achievement such as GPA, SAT V, and SAT M.
Intelligence tests also correlate with these same measures.
So…
Is critical thinking just another Is critical thinking just another term for intelligence? term for intelligence?
"[Intelligence is] the aggregate, or global capacity to act purposefully, think rationally, and deal effectively with the environment. It is global because it characterizes the individual's behavior as a whole; it is an aggregate because it is composed of elements or abilities which, though not entirely independent, are qualitatively differentiable.”
◦David Wechsler, 1938
ReferencesReferences
American Psychological Association. (2007). APA guidelines for the undergraduate psychology major. Washington, DC: Author. Retrieved from www.apa.org/ed/resources.html
Bensley, D. A., & Murtagh, M. P. (2012). Guidelines for a scientific approach to critical thinking assessment. Teaching of Psychology, 39, 5-16.
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Millbrae, CA: The California Academic Press.
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Dispositions, skills, structure training, and metacognitive monitoring. American Psychologist, 53, 449–455.
Halpern, D. F. (2003). Thought and knowledge: An introduction to critical thinking. Mahwah, NJ: Erlbaum.
Lehman, D. R., & Nisbett, R. E. (1990). A longitudinal study of the effects of undergraduate training on reasoning. Developmental Psychology, 26, 952-960.
Nickerson, R. S., Perkins, D. N., & Smith, E. E. (1985). The teaching of thinking. Hillsdale, NJ: Lawrence Erlbaum.
Toplak, M. E., & Stanovich, K. E. (2002). The domain specificity and generality of disjunctive reasoning: Searching for a generalizable critical thinking skill. Journal of Educational Psychology, 94, 197-209.
Williams, R. L., Oliver, R, & Stockdale, S. (2004). Psychological versus generic critical thinking as predictors and outcome measures in a large undergraduate human development course. The Journal of General Education, 53, 37-58.