Assessing Critical Thinking in Vital Signs
A look at reasoning in arguments for species identification
Ashley Young, Thanh Le, Jon Shemwell, and Dan Capps
University of Maine
Sarah Kirn and Christine Voyer Gulf of Maine Research Institute
Gulf of Maine Research Institute Science. Education. Community.
Learning Goals
1. Increase student interest in/engagement with science
2. Increase students’ skill at working collaboratively, including giving and receiving peer review
3. Increase students’ skill at thinking critically, specific
Category Skills A. Content Knowledge 1. Master core academic content
2. Acquire, apply and expand knowledge
B. Cognitive Strategies 3. Think critically and solve complex problems 4. Communicate effectively
C. Learning Behaviors 5. Work collaboratively 6. Learn how to learn
http://www.hewlett.org/programs/education-program/deeper-learning
Hewlett Deeper Learning Framework
What is Critical Thinking?
Thinking or problem solving that (is)
• Self directed • Addresses novel problems • Utilizes good reasoning
Willingham, D. T. (2007). Critical thinking: Why is it so hard to teach? American Educator, 31(2), 8.
Critical Thinking in the K-12 Framework
1. Asking questions and defining problems 2. Developing and using models 3. Planning and carrying out investigations 4. Analyzing and interpreting data 5. Using mathematics, information and computer technology, and
computational thinking 6. Constructing explanations and designing solutions 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information
“Critical thinking is required, whether in developing and refining an idea (an explanation or a design) or in conducting an investigation. The dominant activities in this sphere are argumentation and critique…” (p. 46).
What is an argument?
Claim Evidence Reasoning
I found Eastern Hemlock
The needle has racing stripes
Hemlock needles have racing stripes
McNeill & Krajcik, 2012
Levels of Reasoning in Arguments
Ruling out alternative explanations is hard!
Insufficient evidence No evidence or evidence does not go with claim Confirming evidence Includes some evidence that is relevant to the claim Confirming and falsifying evidence Includes some evidence that is relevant to the claim AND considers and rules out alternative explanations
Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994-1020.
Data Summary Chose 3 species to focus on based on level of support for falsification reasoning on the species card.
Multiflora rose (low support)
Broad-leafed cattail (intermediate support)
Rock crab (high support)
• In total, coded the reasoning level of 226 written responses • Some responses were confirmed by experts; others were not • Data ranged from 2009 - 2012
invasive upland, Loranger MS & Tim seed, Waynflete Gertrud K, www.flickr.com Photo by Curt Brown, GMRI
Multiflora Rose
• Lowest level of support for considering and ruling out alternative species
• Suggests several roses are similar, but does not clearly identify the distinguishing characteristics of these alternatives
Broad-leafed Cattail
• Middle level of support for considering and ruling out alternative species
• States one particular alternative species to consider and clearly describes the characteristics of the Broad-Leaved in comparison to the other species.
Rock Crab
• Highest level of support for considering and ruling out alternative species
• Clearly describes the attributes of an alternative species in comparison to the Rock Crab • Pictures the Jonah and Rock Crabs side by side
Coding
0 = Insufficient evidence 1 = Confirming evidence 2 = Confirming and falsifying evidence
Multiflora rose (low support)
Broad-leafed cattail (intermediate support)
Rock crab (high support)
invasive upland, Loranger MS & Tim seed, Waynflete Gertrud K, www.flickr.com Photo by Curt Brown, GMRI
Coding examples from the Rock crab
The rock crab has nine spines from the eye to the side of the shell.
The legs are very long and the tips are very pointy.
The shell is a reddish brown color.
This crab has ten spines and the back end of the shell is straight(ish) instead of being more circler. The Johan Crab has a more circler shell and the Rock Crab has a more heart-shaped looking shell.
The shell on this crab is yellow with tiny purple dots, which would lead me to believe that this is a Rock Crab.
The belly of this crab is white and its legs have long pointed tips.
Confirming evidence:
Confirming and falsifying evidence:
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
100%
Low support (rose) n = 136
Intermediate support (cattail) n = 58
High support (crab) n = 20
Confirming and falsifying evidence
Confirming evidence
With more support, reasoning is more sophisticated… but it’s still a challenge!
invasive upland, Loranger MS & Tim seed, Waynflete Gertrud K, www.flickr.com Photo by Curt Brown, GMRI
Also Looked at Reasoning Level vs. Writing Level (Flesch-Kincaid Score)
Confirming evidence
Confirming and falsifying evidence
Writ
ing
leve
l (F-
K)
No statistically significant relationship between reasoning level and writing level
Error bars + 1 SE
Writ
ing
Leve
l (F-
K)
Cattail writing level is higher than crab
0
1
2
3
4
5
6
7
Broad-leaved cattail Rock crab
The reason that crab reasoning is higher than cattail is probably not due to differences in students’ writing ability
Error bars + 1 SE
Summary of Results
• Falsification reasoning was less frequent than we expected
• Reasoning level was related to the level of support on species cards
• Reasoning level was not related to writing level
Further Questions
• How much is the “crab effect” learning to reason, and how much is it an artifact of the species card?
• Would reasoning learned in crab investigations transfer to rose and cattail?
• In cattail investigations, how close are students getting to falsification reasoning?
• Should we expect falsification reasoning in middle school?
• How can levels of reasoning help frame pedagogy in student-scientist partnerships like Vital Signs?
Thank You
www.vitalsignsme.org Sarah Kirn Sarah Morrisseau Christine Voyer
Ashley Young, Thanh Le, Jon Shemwell, and Dan Capps
Play Paper
More questions:
1. Does previous experience in Vital Signs lead to more higher level reasoning?
2. Do some teachers yield more higher level reasoning than others?
3. Broad-leaved cattail (not found)
1. Brazilian elodea (not found)
2. Yellow floating heart (found)
Trip to a freshwater pond in Gardiner, Maine:
Higher reasoning level is not statistically related to groups’ number of species investigated or number of Vital Signs trips
Species studied:
Some teachers yielded more higher level reasoning than others
Reasoning level
Teacher No evidence
Confirming evidence
Confirming and falsifying
evidence N
% Confirming and falsifying
evidence A 0 1 1 2 50% B 0 14 3 17 18% C 0 1 1 2 50% D 0 2 2 4 50% E 0 1 2 3 67% F 0 4 1 5 20% G 0 13 0 13 0% H 0 2 0 2 0% I 0 1 0 1 0% J 1 0 0 1 0% K 0 2 0 2 0% L 0 1 0 1 0% M 0 1 0 1 0% N 0 1 0 1 0% O 0 1 0 1 0% P 0 1 0 1 0% Q 0 1 0 1 0% R 0 1 0 1 0%
Broad- leaved cattail
Spec
ies n
umbe
r
Teacher
A B C D E F G H I J K L M N O P Q R
This effect does not seem to be due to teacher experience in Vital Signs
Broad-leaved cattail
Spec
ies n
umbe
r
Teacher
A B C D E F G H I J K L M N O P Q R
High proportion of higher level
reasoning
Broad-leaved cattail
This effect does not seem to be due to teacher experience in Vital Signs
