Thinking Evolutionarily: The Evidence Base

8/3/2019 Thinking Evolutionarily: The Evidence Base

http://slidepdf.com/reader/full/thinking-evolutionarily-the-evidence-base 1/40

ThinkingEvolu-onarily:

TheEvidenceBase

RossNehm

AssociateProfessor,Educa5on&EEOB

TheOhioStateUniversity



Outlineoftoday’stalk

• 1.Bird’seyeviewoftheevidencebase

• 2.Thinkingevolu5onarily:whatweknowaboutnovicetoexpertreasoning

paKerns

• 3.Recentdiscoveriesaboutnovices’evolu5onarythinkingpaKerns

• 4.Implica5onsofpriorresearchfor‘evolu5onacrossthecurriculum’



Today’stalk


• 2.Thinkingevolu5onarily:noviceto

expertreasoningpaKerns


• 4.Implica5onsforevolu5onacrossthecurriculum



Lowlevelsofevolu5onaryknowledgeandhigh

levelsofmisconcep-onsareubiquitous

Generalpublic(e.g.,Brooks,2001;Newport,2004)

Highschoolstudents(e.g.,Demastesetal.,199)

Undergraduatestudents(e.g.,Bishop&Anderson,1990)

Undergraduatebiologymajors(e.g.Nehm&Reilly2007);

Scienceteachers(e.g.,Nehm&Schonfeld,2007;Nehmetal.,2009).

Medicalstudents(e.g.,Brumby,1984)

I n c r

e a s i n g e d u c a t i on

Source:Nehm&Schonfeld(2007)



Moreeduca5on,samemisconcep5ons

Increasing education

Nehm&Schonfeld(2007)

Biology teacher misconceptions:

Nature of Science Secondary students College undergraduates Teachers

Theories become facts when they are well-supported

Evolution can't be "proven"

Evolution can't be refuted by any observation

For evolution to be true it must be observed

Evolution is weak because it is a theory

Evolution

Chance cannot be a factor in the origin of complex traitsNo fossil species found between humans and "apes"

Mutations are harmful and cannot give rise to new traits

Humans and dinosaurs coexisted

Fossil record lacks intermediates

Natural selection

Use and disuse explains the appearance/disappearance of traits

Traits appear only when they are needed

Populations develop new traits rather than individuals

When sight is lost other senses evolve to be more sensitive

Mutations are caused by mutagenic substances in environment Change is caused by the environment

Deadman & Kelly (1978);

Demastes, Settlage, & Good

(1995); Hallden (1988); Settlage

(1994).

Johnson & Peeples (1987);

Sinclair, Pendarvis, & Baldwin

(1997)

Author & Sheppard (2003);

Eve and Dunn (1990),

Sharmann et al. (2003),

Sharmann and Harris (1992)

Eve and Dunn (1990);

Zuzovsky (1994); Author &

Sheppard (2003)

Jimenez (1992); Greene

(1990); Zuzovsky (1994)

Bishop & Anderson (1990);

Brumby (1979, 1984); Dagher &

BouJaoude (1997); Jensen and

Finley (1995, 1997); Sinclair &

Pendarvis (1997/8); Sinclair,

Pendarvis, & Baldwin (1997)

Also documented in samples of:

Johnson & Peeples (1987);

Sinclair, Pendarvis, & Baldwin

(1997)

Ryan & Aikenhead (1992)

Deadman & Kelly (1978),

Lawson & Worsnop (1992)



Whatworkstoaddresstheseproblems?

Typesofevidence

• RCT(Randomizedcontroltrials)(causa5on,possiblegeneraliza5on)

• Interven5onswithcomparisongroups(causalimplica)ons,possiblegeneraliza5on)

• Interven5onwithnocomparisongroup(pre-postchange;associa5ons)

• Surveyresearch(associa5ons)

• Casestudies,interviews,qualita5veresearch(variableiden5fica5on,possibleassocia5ons)

C a u s a l c l a i m s



Research studies in evolution education

Intervention studies with control groups

(subject randomization)

Intervention studies with comparison

groups

200

Intervention studies

Evolution education studies sampled

6

24

Data from Nehm, 2006

0



Interven5onstudies



Intervention efficacy tests depend on

measurement quality and task authenticity

Less…

Assessmentofthemosteasilymeasured

knowledgeelements

…fragmentsofisolatedknowledge

…recogni5onofideas,explana5ons

…mul5plechoice

More…

Assessmentofthemostvaluableskillsand

performances

…knowledgeselec5on,organiza5on,and

assembly

…communica5onofideas,explana5ons

…constructedresponse

NRC(2001:):‘‘[a]ssessmentsneedtoexaminehowwellstudentsengagein

communica5veprac5cesappropriatetoadomainofknowledgeandskill,what

theyunderstandaboutthoseprac5ces,andhowwelltheyusethetools

appropriatetothatdomain’’



Interven5onassessments:scienceteachers

Ha,Nehm,andBaldwin,inreview



Part1Summary:

Problemsarewell-established;solu5onsarenot

• Researchhasestablishedkeyvariablesthatshouldbeinves5gatedandmanypossiblebeneficialinstruc5onalinterven5ons

• …butwelackrobust,generalizable,causalclaims

rela5ngtopar5cularpedagogicalstrategiesandinterven5ons.

• …welackmeasurementinstrumentsthatmeetbasicqualitycontrolstandards(i.e.,AERA,1996)(Nehm&Schonfeld,2008)andcaptureauthen5cdisciplinaryprac5ces.

• ..welackconsistentapplica5onofmeasurementinstrumentsacrossdifferentpopula5ons (applesandorangesmeasurementissues).



Today’stalk


• 2.Thinkingevolu5onarily:

progressionsfromnovicetoexpert





Thinkingevolu5onarily:

progressionsfromnovicetoexpert

• Howdodifferentgroups(e.g.,novicetoexpert)think

aboutthesameproblemsusingperformance-based

measures(explainingevolu5onarychange).

• Arecentlycompletedlarge-scalestudyof>400

individuals,fromnon-majorcollegestudentstofull

professorsofevolu5onarybiology(Nehm&Ha,in

prepara5on)revealsusefulinsightsintolearningevolu5on.



Competency: explaining evolutionary change

Beingabletoclearlyandlogicallyexplainscien5ficphenomenafreeof

naïveideasisacorecompetency

thatishighlyvaluedbyscien5stsand

educatorsbutisdifficulttomeasure.

Studentshaveaharder5me

explainingevolu5onarychange(in

wri5ngororally)thanrecognizingaccuratescien5ficelementsofan

explana5ononamul5ple-choicetest

(NehmandSchonfeld,2008).



Views of competence

Identifying the appropriate

elements of an evolutionaryexplanation (MC tests)

Building a robust and functioningevolutionary explanation (CR)

Knowing the parts and tools needed to

assemble furniture does not mean that you can actually built it effectively.



Naïve model Mixed model Scientific model

“One day there was a mutation

[Variation] that produced apoison. The poisonous snail

was better able to producemore offspring [Differential

survival] in the environmentpassing on his trait [Heredity].”

“One day snails had to [Need]

a mutation for poison[Variation]. The poisonous

snail had gradually adapted totheir environment [Adapt] so

the population of the snailincrease [Change of pop.].”

“One day snails had to have a

poison in order to fightpredator [Need]. Then,

environmental pressure thencaused them to have poison

[Pressure]. Therefore, they allhas changed into poisonoussnails [Essentialism].”

Competenceandmul5ple-choicetests:

asecondconcern



0

2

4

6

8

10

eyconcept Misconcep5on

0

20

40

60

80

4 3 2

0

10

20

30

40

50

2~3

Scientificmodels

Mixedmodels

Naïvemodels

2-3 core concepts

(%)

(%)

**

Key concepts Misconceptions

**

Non-majors Majors Advanced majors Experts

** p<0.01

A

C

B

**

** D

0

2

4

6

8

10

1Non-adaptive concepts

(%)

N=428(107eachgroup);fromNehm&Ha,inprepara5on



0

10

20

30

40

50

60

70

Year 1 Year 2 Year 3 Year 4

Differential survivalVariance

Teleology

Adapt Heredity

Limited resource

Change of pop.

CompetitionIntentionality

(%)

Pressure

Energy Use/disuse

Differential survival

Variance

Teleology

Adapt

Heredity

Limited resource

Change of pop.

CompetitionIntentionality

Use/disuse

Energy


Variance

Teleology

Adapt

Heredity

Limited resource

Change of pop.

Use/disusePressure

Competition

Energy


Variance

Teleology

Heredity

Limited resource

Change of pop.

Use/disuse

Non-majors Majors Advancedmajors

Expert

Languageofevolu5onaryexplana5on(n=428)

N=428

(107each

group);fromNehm

&Ha,in

prepara5on



40%

30%

20%10%

5%

Non-majorsR

V

D

T

MajorsR

V

D

T

Advancedmajors

R

D

V

T

ExpertsR

V

D

T

P

Scientific concept

V: VariabilityH: Heritability

D: Differential survival

Naïve idea

T: Need/GoalU: Use/disuse

I: IntentionalityA: Adapt

E: EnergyP: Pressure

N=107each

group

Nehm&Ha,in

prepara5on



WhataccountsforthesepaKerns?

• Whatfactorsarecausingdifferencesinnovice

andexpertevolu5onaryreasoningpaKerns?

P bl l i h i th i d i



Explanatory

model “A”

Expert biological

thinking

Problem solving research in other science domains

Explanatorymodel “N”

Explanatorymodel “B”

Explanatory

model “C”

Item a

Item b

Item c

Novice biological

thinking



Cognitiveresource

Description Consistency*of Expert use

Consistency*

of Noviceuse

Key concept 6 Differential survival 70% 36%

Key concept 1 Causes of variation 60% 4%

Key concept 2 Heritability 60% 8%

Naïve concept 1 Needs drive change n/a 0%

Naïve concept 2 Pressure forces change n/a 0%

Naïve concept 3 Use an disuse explain change n/a 0%

Naïve concept 4 Acclimation = adaptation n/a 0%

Naïve concept 5 Inheritance of acquired traits n/a 0%

Naïve concept 6 Intentionality explains change n/a 0%

Natural selection elements were linked consistently (more coherent) inexperts, but haphazardly applied in novices.

Naïve ideas absent in experts, no coherence whatsoever in novices

Nehm & Ridgway, in press

Coherence in evolutionary explanation

N i bi l i l



Explanatorymodel “N”

Explanatory

model “A”

Explanatorymodel “B”

Explanatory

model “C”

Item a

Item b

Item c

Expert biological

thinking

Novice biological

thinkingCoherencehypothesis

• Novices solveproblems using

concrete surfacefeatures.

• Experts solve

problems usingdomain principles

(e.g., naturalselection).

• Significant

coherence

characterizes experts,multiple explanatory

models characterize

novices.

Nehm&Ridgway,inpress,



Part2Summary:

Expertsandnovices“see”evolu5ondifferently

• Surfacefeaturepercep5onsaccountfordifferencesinproblemsolvingperformancebetweennovicesandexperts.

• Whilestudentsknowtheelementsofthetheoryofnaturalselec5on,theydonotusetheseelementstogetherinaconsistentmanneracrossdifferentproblems.

• Evenaercomple5nganevolu5oncourse,only0%ofstudentshave“expert-like”percep5onofevolu5onaryproblems.

• Asstudentsprogressthroughbiology,wedoliKletohelpthemreasonacrosscases.



Today’stalk

• 1.Theknowledgebase:Evidence-basedevolu5oneduca5on







ItemResponse

Structure

SurfaceFeatures

CognitiveResources

(conceptual

procedural

analytical,

factual)

Key concepts

Naïve ideas

Cognitive biases

Cueing, Framing

Contextualproblem space

Recruitment

Theory: Contextualized reasoning

S t o r a g e

P r o c e s s

C o n t e x t

A n s w

e r

1

2

3

4

5

6

7

I

T

E

U

A

P

Nehm&Ridgway,inpress,Nehm,inprepara5on



Whichsurfacefeaturesareproblema5c?

• Scale(intraspecific,interspecific)

– (Nehm&Ha,2011)

• Polarity(traitgain,traitloss) – (Nehm&Ha,2011)

• Taxon(animal,plant)

– (Opfer,Nehm,Ha,etal.2011)

• Familiarity(Doddervs.Rose)

– (Opfer,Nehm,Ha,etal.2011)



Experimentaldesign

• Par5cipantrandomiza5on

• Largesamplesofnovices(>200par5cipants)

• Controlofallaspectsoflanguageanditemfeatures

• Manipula5onofoneevolu5onaryproblemfeature

• Nehm&Ha(2011)JRST



G ai n

L o s s

G ai n

L o s s

G ai n

L o s s

Plant Animal Bacteria

Scale,polarity,andfamiliarityeffects

onevolu5onaryreasoning

S l i ifi i ifi



0

2

4

6

8

EGALT-B EGALT-W

A c

c u r a t e e l e m e

n t s

Between species Between populationsof same species

Gain

Loss

Nehm&Ha

(2011)

Scale:intraspecificvs.interspecific

Polarity:traitgainvs.traitloss

(Same:organismsandtraits)

S l i t ifi i t ifi



Scale:intraspecificvs.interspecific

Polarity:traitgainvs.traitloss

(Same:organismsandtraits)

0

2

4

6

8

EGALT-B EGALT-W

M i s

c o n c e p t i o n s

Between Within

Gain

Loss

Nehm&Ha

(2011)

F ili it (P i i R )



0.0

0.4

0.8

1.2

1.6

2.0

Plant Animal

K e y c o n c e p t s

Taxa

Familiar

Unfamiliar

Familiarity effects:

plants vs. animals

Familiarity(Prosimianvs.Rose)

Taxon(Plantvs.animal)

(Same:interspecifictraitgain)

Opfer,Nehm,Ha,etal.(2011)

A Within species B Between species



A. Within species B. Between species

C. Trait Gain D. Trait loss

V V

V V

H

H H

H

D D

D D

Scientific concept

V: Variability

H: HeritabilityD: Differential survival

Naïve idea

N: Need/GoalU: Use/disuse

I: IntentionalityA: Adapt

E: EnergyP: Pressure

N N

N N

P P

P P

E

E E

E

A A

A A I I

I I

U U

U U

p<0.01 (+)

p<0.05 (+)

p<0.01 (-)

p<0.05 (-)



Part3Summary:

ProblemsurfacefeaturesdeservemajoraKen5on

• Specificsurfacefeaturesofevolu5onaryproblemsplayahugerolehownovicesthinkaboutevolu5on.

• Biologyeduca5oncanbemorepreciseininstruc5onaltargets.



Today’stalk

• 1.Theknowledgebase:Evidence-basedevolu5oneduca5on







Pointstoconsider1:

• Studentsappeartoprogressfromnaïvemodelsmixedmodelsscien5ficmodels,andprogressisveryslow(2%ofadvancedmajors—studentswhohave

completedanevolu5onclassandaddi5onalcoursework--havemixedmodels).

• Learningevolu5onischaracterizedbyaddingscien5ficideastonaïveideas,andyetmostassessmentsdon’tallowthisop5on.

Points to consider 2



Pointstoconsider2:

• Thesurfacefeaturesofevolu5onaryproblemsplayahugerolehow

novicesthinkaboutevolu5on(Nehm&Ridgway,inpress).

• Misconcep5onsaresurface-featurespecific,soinstruc5onalexamplesmustbechosencarefully.

• Taxon(animal/plant),traitchangepolarity(gain/loss),scale(withinvs.

betweenspecies),andfamiliaritypresentuniquereasoningchallengesforstudents(Nehm&Ha,2011;Opfer,Nehm,Ha,etal.,2011).



Takehomepoints3:

• Assessmentsofcompetencymustincludeauthen5cproduc5ontasks,suchasexplaininghowevolu5onarychangeoccurs,notjustfragmentedknowledgeselec5ontasks(Nehm&

Schonfeld2010).

• Evolu5onassessmentsmustbedevelopedthatmeetqualitycontrolstandardsestablishedbytheeduca5onalmeasurementcommunity(AERAetal.,1999);otherwise,robustclaims(causalorotherwise)cannotbemade.



Takehomepoints4:

• Aminorityofresearchstudiesinevolu5oneduca5oninvolveinterven5onswithcomparisongroups,limi5ngrobust

guidelinesforprac5ce.

• Whileevidencecomesinmanyforms,causalclaimsmustbeestablished;RCTstudies,or

interven5onstudieswithcomparisongroups,aredesperatelyneeded.



Thankyou

• NSFCAREERprogram;NSFCCLIprogram;NSFTUESprogram;NSFREESEprogramforresearchsupport

• Numerouscollaboratorsandstudents,par5cularlyMinsuHa,IrvinSchonfeld,HendrikHaer5g,LeahReilly,MeghanRector,fortheirimportantcontribu5ons.

• Papers:www.nehmlab.org

Date post:	07-Apr-2018
Category:	Documents
Upload:	national-academies-of-science-engineering-and-medicine
View:	223 times
Download:	0 times

Thinking Evolutionarily: The Evidence Base

Documents