Assessing Critical Thinking in Vital...

Post on 23-Mar-2018

215 views 0 download

transcript

Assessing Critical Thinking in Vital Signs

A look at reasoning in arguments for species identification

Ashley Young, Thanh Le, Jon Shemwell, and Dan Capps

University of Maine

Sarah Kirn and Christine Voyer Gulf of Maine Research Institute

Gulf of Maine Research Institute Science. Education. Community.

Learning Goals

1. Increase student interest in/engagement with science

2. Increase students’ skill at working collaboratively, including giving and receiving peer review

3. Increase students’ skill at thinking critically, specific

Category Skills A. Content Knowledge 1. Master core academic content

2. Acquire, apply and expand knowledge

B. Cognitive Strategies 3. Think critically and solve complex problems 4. Communicate effectively

C. Learning Behaviors 5. Work collaboratively 6. Learn how to learn

http://www.hewlett.org/programs/education-program/deeper-learning

Hewlett Deeper Learning Framework

What is Critical Thinking?

Thinking or problem solving that (is)

• Self directed • Addresses novel problems • Utilizes good reasoning

Willingham, D. T. (2007). Critical thinking: Why is it so hard to teach? American Educator, 31(2), 8.

Critical Thinking in the K-12 Framework

1. Asking questions and defining problems 2. Developing and using models 3. Planning and carrying out investigations 4. Analyzing and interpreting data 5. Using mathematics, information and computer technology, and

computational thinking 6. Constructing explanations and designing solutions 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information

“Critical thinking is required, whether in developing and refining an idea (an explanation or a design) or in conducting an investigation. The dominant activities in this sphere are argumentation and critique…” (p. 46).

What is an argument?

Claim Evidence Reasoning

I found Eastern Hemlock

The needle has racing stripes

Hemlock needles have racing stripes

McNeill & Krajcik, 2012

Levels of Reasoning in Arguments

Ruling out alternative explanations is hard!

Insufficient evidence No evidence or evidence does not go with claim Confirming evidence Includes some evidence that is relevant to the claim Confirming and falsifying evidence Includes some evidence that is relevant to the claim AND considers and rules out alternative explanations

Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994-1020.

Data Summary Chose 3 species to focus on based on level of support for falsification reasoning on the species card.

Multiflora rose (low support)

Broad-leafed cattail (intermediate support)

Rock crab (high support)

• In total, coded the reasoning level of 226 written responses • Some responses were confirmed by experts; others were not • Data ranged from 2009 - 2012

invasive upland, Loranger MS & Tim seed, Waynflete Gertrud K, www.flickr.com Photo by Curt Brown, GMRI

Multiflora Rose

• Lowest level of support for considering and ruling out alternative species

• Suggests several roses are similar, but does not clearly identify the distinguishing characteristics of these alternatives

Broad-leafed Cattail

• Middle level of support for considering and ruling out alternative species

• States one particular alternative species to consider and clearly describes the characteristics of the Broad-Leaved in comparison to the other species.

Rock Crab

• Highest level of support for considering and ruling out alternative species

• Clearly describes the attributes of an alternative species in comparison to the Rock Crab • Pictures the Jonah and Rock Crabs side by side

Coding

0 = Insufficient evidence 1 = Confirming evidence 2 = Confirming and falsifying evidence

Multiflora rose (low support)

Broad-leafed cattail (intermediate support)

Rock crab (high support)

invasive upland, Loranger MS & Tim seed, Waynflete Gertrud K, www.flickr.com Photo by Curt Brown, GMRI

Coding examples from the Rock crab

The rock crab has nine spines from the eye to the side of the shell.

The legs are very long and the tips are very pointy.

The shell is a reddish brown color.

This crab has ten spines and the back end of the shell is straight(ish) instead of being more circler. The Johan Crab has a more circler shell and the Rock Crab has a more heart-shaped looking shell.

The shell on this crab is yellow with tiny purple dots, which would lead me to believe that this is a Rock Crab.

The belly of this crab is white and its legs have long pointed tips.

Confirming evidence:

Confirming and falsifying evidence:

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Low support (rose) n = 136

Intermediate support (cattail) n = 58

High support (crab) n = 20

Confirming and falsifying evidence

Confirming evidence

With more support, reasoning is more sophisticated… but it’s still a challenge!

invasive upland, Loranger MS & Tim seed, Waynflete Gertrud K, www.flickr.com Photo by Curt Brown, GMRI

Also Looked at Reasoning Level vs. Writing Level (Flesch-Kincaid Score)

Confirming evidence

Confirming and falsifying evidence

Writ

ing

leve

l (F-

K)

No statistically significant relationship between reasoning level and writing level

Error bars + 1 SE

Writ

ing

Leve

l (F-

K)

Cattail writing level is higher than crab

0

1

2

3

4

5

6

7

Broad-leaved cattail Rock crab

The reason that crab reasoning is higher than cattail is probably not due to differences in students’ writing ability

Error bars + 1 SE

Summary of Results

• Falsification reasoning was less frequent than we expected

• Reasoning level was related to the level of support on species cards

• Reasoning level was not related to writing level

Further Questions

• How much is the “crab effect” learning to reason, and how much is it an artifact of the species card?

• Would reasoning learned in crab investigations transfer to rose and cattail?

• In cattail investigations, how close are students getting to falsification reasoning?

• Should we expect falsification reasoning in middle school?

• How can levels of reasoning help frame pedagogy in student-scientist partnerships like Vital Signs?

Thank You

www.vitalsignsme.org Sarah Kirn Sarah Morrisseau Christine Voyer

Ashley Young, Thanh Le, Jon Shemwell, and Dan Capps

Play Paper

More questions:

1. Does previous experience in Vital Signs lead to more higher level reasoning?

2. Do some teachers yield more higher level reasoning than others?

3. Broad-leaved cattail (not found)

1. Brazilian elodea (not found)

2. Yellow floating heart (found)

Trip to a freshwater pond in Gardiner, Maine:

Higher reasoning level is not statistically related to groups’ number of species investigated or number of Vital Signs trips

Species studied:

Some teachers yielded more higher level reasoning than others

Reasoning level

Teacher No evidence

Confirming evidence

Confirming and falsifying

evidence N

% Confirming and falsifying

evidence A 0 1 1 2 50% B 0 14 3 17 18% C 0 1 1 2 50% D 0 2 2 4 50% E 0 1 2 3 67% F 0 4 1 5 20% G 0 13 0 13 0% H 0 2 0 2 0% I 0 1 0 1 0% J 1 0 0 1 0% K 0 2 0 2 0% L 0 1 0 1 0% M 0 1 0 1 0% N 0 1 0 1 0% O 0 1 0 1 0% P 0 1 0 1 0% Q 0 1 0 1 0% R 0 1 0 1 0%

Broad- leaved cattail

Spec

ies n

umbe

r

Teacher

A B C D E F G H I J K L M N O P Q R

This effect does not seem to be due to teacher experience in Vital Signs

Broad-leaved cattail

Spec

ies n

umbe

r

Teacher

A B C D E F G H I J K L M N O P Q R

High proportion of higher level

reasoning

Broad-leaved cattail

This effect does not seem to be due to teacher experience in Vital Signs