Developmental changes in probabilistic reasoning: The role of cognitive capacity, instructions,...

This article was downloaded by: [Universitaets und Landesbibliothek]On: 15 December 2013, At: 16:30Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Thinking & ReasoningPublication details, including instructions for authorsand subscription information:http://www.tandfonline.com/loi/ptar20

Developmental changes inprobabilistic reasoning: Therole of cognitive capacity,instructions, thinking styles, andrelevant knowledgeFrancesca Chiesi a , Caterina Primi a & Kinga Morsanyi ba Department of Psychology , University of Florence ,Florence, Italyb School of Psychology, University of Plymouth ,Plymouth, UKPublished online: 09 Aug 2011.

To cite this article: Francesca Chiesi , Caterina Primi & Kinga Morsanyi (2011)Developmental changes in probabilistic reasoning: The role of cognitive capacity,instructions, thinking styles, and relevant knowledge, Thinking & Reasoning, 17:3,315-350, DOI: 10.1080/13546783.2011.598401

To link to this article: http://dx.doi.org/10.1080/13546783.2011.598401

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressedin this publication are the opinions and views of the authors, and are not theviews of or endorsed by Taylor & Francis. The accuracy of the Content shouldnot be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions,claims, proceedings, demands, costs, expenses, damages, and other liabilities

http://www.tandfonline.com/loi/ptar20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/13546783.2011.598401

http://dx.doi.org/10.1080/13546783.2011.598401

whatsoever or howsoever caused arising directly or indirectly in connectionwith, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expresslyforbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Developmental changes in probabilistic reasoning: The

role of cognitive capacity, instructions, thinking styles,

and relevant knowledge

Francesca Chiesi1, Caterina Primi

1, and Kinga Morsanyi

2

1Department of Psychology, University of Florence, Florence, Italy2School of Psychology, University of Plymouth, Plymouth, UK

In three experiments we explored developmental changes in probabilisticreasoning, taking into account the effects of cognitive capacity, thinking styles,and instructions. Normative responding increased with grade levels andcognitive capacity in all experiments, and it showed a negative relationshipwith superstitious thinking. The effect of instructions (in Experiments 2 and 3)was moderated by level of education and cognitive capacity. Specifically, onlyhigher-grade students with higher cognitive capacity benefited from instruc-tions to reason on the basis of logic. The implications of these findings forresearch on the development of probabilistic reasoning are also discussed.

Keywords: Cognitive development; Dual-process theories; Heuristics andbiases; Instruction manipulation; Probabilistic reasoning.

Researchers in the heuristics and biases tradition (started by Kahneman &Tversky, 1972, 1973; Tversky & Kahneman, 1974) have demonstrated thatpeople often overestimate the importance of vivid, readily availableknowledge, and disregard basic logical or probabilistic rules. Theseviolations of normative rules result in a large, and well-documented,array of biases (for summaries of the literature see e.g., Baron, 2000;

Correspondence should be addressed to Kinga Morsanyi, University of Geneva 40 bd du

Pont d’Arve, 1205 Geneve, Switzerland. E-mail: [email protected]

Kinga Morsanyi is now working at the University of Geneva. We would like to thank Simon

J. Handley, Jonathan St. B. T. Evans, David Over, Maxwell J. Roberts, and two anonymous

reviewers for their comments on earlier drafts of this paper.

THINKING & REASONING, 2011, 17 (3), 315–350

� 2011 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business

http://www.psypress.com/tar DOI: 10.1080/13546783.2011.598401

Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Gilovich, Griffin, & Kahneman, 2002; Stanovich, 2004). According todual-process theories, mental functioning can be characterised by twodifferent types of process that have different functions and differentstrengths and weaknesses (e.g., Brainerd & Reyna, 2001; Epstein, 1994;Evans & Over, 1996; Stanovich, 1999). We will follow Evans (2006) inusing the terms Type 1 and Type 2 processes to describe these separateaspects of cognitive functioning. Type 1 processes (which are sometimescalled heuristic; see e.g., Evans, 2006; Klaczysnki, 2004) are considered tobe autonomous because their execution is rapid and mandatory when thetriggering stimuli are encountered, they do not require much cognitiveeffort, and they can operate in parallel. Type 1 processing is the defaultbecause it is cognitively economical, its output is not consciously generatedbut seems to ‘‘pop’’ into consciousness (Sloman, 1996), and people ‘‘feel’’intuitively that heuristic responses are right (Epstein, 1994; Thompson,2009). Indeed, Type 1 processing often leads to normatively correctresponses (e.g., Evans, 2003; Stanovich & West, 1998). On the other hand,in some cases Type 1 processes lead to systematic biases and errors. Bycontrast, Type 2 processes are relatively slow and computationallyexpensive, they are available for conscious awareness, serial, and oftenlanguage based. Type 2 processes are also often associated with the use ofnormative rules and logical responding, although this is not necessarily thecase (see e.g., Evans, 2006).

To demonstrate the role of the two types of process in reasoning,consider the following problem. Imagine that in order to win a prize youhave to pick a red marble from one of two urns (Urn A and B). Urn Acontains 20 red and 80 blue marbles, and Urn B contains 1 red and 9 bluemarbles. When you respond to the task you can compare the ratio ofwinning marbles in each urn (20% vs 10%), which requires some time,mental effort, and computations, or you can simply rely on the feeling/intuition that it is preferable to pick from the urn with more ‘‘favourableevents’’. In this example both processes cue the normatively correct answer(that is, Urn A). On the other hand, it is possible to set up a task whereType 1 and Type 2 reasoning cue different responses. For example, if youcan choose between picking a marble from an urn containing 10 red and90 blue marbles, or from an urn containing 2 red and 8 blue marbles, thefeeling/intuition that it is preferable to pick from the urn with more‘‘favourable events’’ results in a normatively incorrect choice.

When Type 1 and Type 2 processes do not produce the same output,Type 1 processes usually cue responses that are normatively incorrect and,according to dual-process theorists (e.g., Stanovich, 1999) one of the mostcritical functions of Type 2 processes in these cases is to interrupt andoverride Type 1 processing. However, this does not always happen. In thecase of a conflict between intuitions and normative rules even educated

316 CHIESI, PRIMI, MORSANYI

Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

adults will predominantly produce heuristic responses (e.g., Klaczynski,2001a). On the other hand, a number of studies found evidence that peoplewith higher cognitive capacity will be more likely to produce normativelycorrect responses. According to Stanovich and West (2008) this is becausecarrying out slow, sequential, and effortful (Type 2) computations, whilesimultaneously inhibiting quick, low-effort, and intuitively compelling Type1 responses requires considerable cognitive resources. Thus thinking errorsare expected to decrease with increasing cognitive ability, and, in fact,several studies have found evidence that people with higher cognitivecapacity are less inclined to use reasoning heuristics inappropriately (seeStanovich & West, 2000, for a review). In addition to this correlationalevidence, studies using a dual-task manipulation also demonstrated thatcertain reasoning biases increase when participants’ working memory isburdened with a cognitive load (e.g., De Neys 2006a, 2006b). Finally, peoplewho are more inclined to think harder about problems (i.e., people withhigher need for cognition; see Cacioppo & Petty, 1982), and people who areinstructed to reason logically about problems (e.g., Epstein, Lipson,Holstein, & Huh 1992; Ferreira, Garcia-Marques, Sherman, & Sherman,2006) typically show improved normative performance. Thus these resultssupport the notion that investing more cognitive effort in the reasoningprocess increases normative responding.

Based on this evidence one could expect that reasoning performanceshould improve with development, as cognitive capacity steadily increaseswith age. Indeed, a number of studies reported this pattern (e.g., Handley,Capon, Beveridge, Dennis, & Evans, 2004; Kokis, Macpherson, Toplak,West, & Stanovich, 2002). On the other hand, some studies reported anincrease in heuristic responses with development, which was (in some cases)accompanied by a decrease in normative responses (e.g., Davidson, 1995; DeNeys, 2007; Jacobs & Potenza, 1991; Morsanyi & Handley, 2008), or a U-shaped or inverted U-shaped pattern (e.g., Chiesi, Gronchi, & Primi, 2008; orsee Osman & Stavy, 2006, for a review). Finally, some studies found thatwhereas performance increased with age on some tasks, there was an age-related decrease in normative responses on other tasks in the samepopulation (e.g., De Neys & Vanderputte, 2011; Fischbein & Schnarch,1997).

One obvious reason for these mixed results is that in developmentalsamples the available knowledge base changes rapidly with age andeducation. According to Klaczynski and colleagues (e.g., Jacobs &Klaczynski, 2002; Klaczynski, 2001b; Klaczynski, 2009) social, motivational,and affective influences, as well as prior beliefs, greatly affect the way peoplereason about problems. As a result, there is no clear relationship betweenchildren’s abstract reasoning ability, their cognitive capacity and theirreasoning performance. Klaczynski and colleagues also emphasise that an

DEVELOPMENT OF PROBABILISTIC REASONING 317

Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

increasing number of heuristics are acquired over the course of development,and the use of these heuristics becomes more prevalent with age (for a similarapproach see also, e.g., Brainerd & Reyna, 2001; Reyna & Brainerd, 1992;Reyna & Ellis, 1994; Reyna & Farley, 2006). Thus, in this view, knowledgeand experiences can affect reasoning performance both positively andnegatively.

Given that, according to these theorists, no general predictions can bemade about the relationships between development, knowledge, cognitiveability, and reasoning performance developmental studies usually focus oncomparisons between different age groups without taking into accountparticipants’ cognitive capacity and their thinking styles. Similarly, studiesthat explore the impact of education on probabilistic reasoning (e.g.,Fischbein & Schnarch, 1997; Lehman, Lempert & Nisbett, 1988) usually donot investigate the interactions between level of education, cognitivecapacity, and thinking styles. By contrast, studies with adults typicallyfocus on the effect of cognitive ability, cognitive load, and thinking styles.For this reason it is hard to reconcile or even compare the findings oftheorists who investigate age- or education-related changes in reasoningperformance, and those who investigate the effects of individual differencesin cognitive abilities and cognitive styles.

It should be noted here that in a recent review paper Stanovich, Toplak,and West (2008) emphasised the role of both relevant knowledge andcognitive capacity (and the interactions between these factors) in thedevelopment of reasoning skills (see more on this in the GeneralDiscussion). However, although Stanovich and colleagues (2008) offer auseful framework for investigating the interplay between these factors,they do not make specific predictions regarding age-related changes basedon their model. Thus one important aim of the present series ofexperiments is to combine the methods of developmental and adultdual-process theorists; that is, to investigate developmental changes, andthe effects of cognitive capacity and thinking styles simultaneously in thesame population.

Heuristics and biases tasks typically pit heuristic responses (based onbeliefs, and everyday experiences) against a normative rule. In these casesexperiences and knowledge play an important role in shaping heuristicresponses (e.g., De Neys & Vanderputte, 2011; Kokis et al., 2002). On theother hand, the awareness of normative rules also increases with age. Thatis, whereas an increasing number of heuristics is acquired through thecourse of development (e.g., Klaczynski, 2009), the available knowledge ofnormative rules and strategies also increases. Probabilistic rules andmathematical abilities related to probabilistic reasoning are learned andconsolidated through education, and although the ability to computeratios start to develop earlier (Kreitler & Kreitler, 1986) the relevant rules


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

for sound probabilistic reasoning are mostly acquired during adolescence.1

Many theorists (e.g., Fischbein, 1987) emphasise that the rules ofprobabilistic reasoning are virtually impossible to infer from our everydayexperiences, which are hopelessly ‘‘messy’’ (e.g., people win every week atthe national lottery, despite its extremely low likelihood; Borovcnik &Peard, 1996), or the actual patterns of probabilistic outcomes resemblemore what could be predicted based on the fallacies of probabilisticreasoning than on the relevant normative rules (see Hahn & Warren,2009).

In Experiment 1 we investigated developmental changes in probabilisticreasoning, as well as the interactions between grade levels, and children’scognitive ability and cognitive styles. Grade levels are necessarily highlycorrelated with children’s age. However, we hypothesised that they should bebetter predictors of children’s probabilistic reasoning performance, aspersonal experiences about probability and general cognitive maturationare not expected to be related to probabilistic reasoning ability. At least it isnot expected that higher-ability children would be more able to infer the rulesof probability without formal instruction. Instead the relevant rules and skillsfor probabilistic reasoning are acquired through education (see above). Thusin the present studies we considered children’s grade level as an indicator oftheir knowledge regarding probability. Nevertheless, it should be noted thatknowledge in other areas also increases with grade levels, and this could alsocontribute to a potential increase in children’s probabilistic reasoningperformance with education (see more on this issue in the GeneralDiscussion).

We measured two separate aspects of children’s cognitive styles. One wastheir need for cognition (i.e., the tendency to rely on effortful reasoning whensolving problems), which we expected to be positively related toprobabilistic reasoning (e.g., Kokis et al., 2002). The other measure wasthe superstitious thinking scale. Superstitions and belief in luck have beenfound to have a detrimental effect on probabilistic reasoning performance(see Toplak, Liu, Macpherson, Toneatto, & Stanovich, 2007). Additionally,in Experiments 2 and 3 we also used instruction manipulations; that is, weasked half of our participants to reason from the point of view of a perfectlylogical person, whereas the other half of the group was asked to reasonintuitively (Experiment 2) or they were given no instructions (Experiment 3).The purpose of these instructions was to manipulate the effort thatparticipants invest in the reasoning process. Increased cognitive effort

1Italian children first learn about the notion of probability at the age of 9 (in grade 4). Then,

in Grade 7, they learn about some important concepts underpinning probabilistic reasoning

(e.g., proportions, percentages, etc.). Finally, in Grade 8 probabilistic reasoning is taught in

relation to scientific topics (e.g., the basics of genetics).


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

should facilitate participants’ reliance on the normative rules of probability.Whereas in Experiments 1 and 2 we compared the effects of grade levelswithin a secondary school sample, in Experiment 3 we compared theperformance of students with a much greater difference between theireducational backgrounds (i.e., students at the start of their secondaryeducation and university students).

As a measure of our participants’ probabilistic reasoning ability, weemployed a number of different well-established heuristics and biases tasksrelated to probabilistic thinking. In order to stress the relevance of the rulesof probability, it was emphasised in all problems that the outcome wasgenerated through a random selection process (e.g., coin toss, raffle, etc.).This is very important as the normative rules of probability only apply whena random selection process is in place (see e.g., De Neys & Vanderputte,2011; Gigerenzer, Hell, & Blank, 1988).

EXPERIMENT 1

In this experiment the role of grade levels in probabilistic reasoning wasinvestigated controlling for individual differences in superstitious thinking(Toplak et al., 2007), cognitive ability, and need for cognition (see, e.g.,Kokis et al., 2002). We expected that less-superstitious children, regardlessof grade level, should perform better than more-superstitious ones sincetheir judgements about the likelihood of events should be less affected byimpressions or feelings related to luck, superstitions, or belief in theparanormal. Moreover, we expected children with higher cognitive capacity,and with higher need for cognition, to perform better, as they are able toinvest greater cognitive resources in solving the tasks. The main researchquestion was whether we would find any difference between higher andlower grade children in their probabilistic reasoning performance once wecontrolled for the effects of cognitive ability and cognitive styles. Based onthe concept that the rules of probability can only be acquired througheducation (e.g., Borovcnik & Peard, 1996; Fischbein, 1987; Hahn &Warren,2009) we expected this to be the case.

Method

ParticipantsParticipants were 302 students in grade 6 (n¼ 82, 43 males; mean age: 11.8yrs, SD¼ 0.51), grade 7 (n¼ 95, 49 males; mean age: 12.7 yrs, SD¼ .55), andgrade 8 (n¼ 121, 68 males; mean age: 13.7 yrs, SD¼ .57). They wereenrolled in Italian junior high schools that serve families from the lowermiddle to middle socioeconomic classes. All grade 6–8 students in the school


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

were invited to participate, their parents were given information about thestudy, and their permission was requested.

Materials

Probabilistic reasoning questionnaire. The questionnaire consisted of sixdifferent probabilistic reasoning tasks adapted from the heuristics and biasesliterature (see Table 1 for the source of each task, and the normativeprinciples required to solve them; the tasks can be found in Appendix A; theproportion of correct responses for each problem is also reported inAppendix C in Table S2). There were three response options in the case ofeach task, and children were given either 1 (correct) or 0 (incorrect) point foreach response. As in previous studies (Kokis et al., 2001; Toplak et al., 2007;West, Toplak & Stanovich, 2008) the scores on the six probabilisticreasoning tasks were summed to form a composite score.

TABLE 1Tasks employed in the study

Source Rule Bias

Task 1 Kahneman, Slovic, &

Tversky, 1982

likelihood of independent and

equiprobable events

p(A)¼ p(A/B)

gambler’s fallacy

Task 2 Kahneman, Slovic, &

Tversky, 1982

likelihood of strings of

independent and

equiprobable events

random similarity

bias

Task 3 Denes-Raj & Epstein,

1994

ratios computation and

comparison:

f1/n15¼4 f2/n2

ratio bias

Task 4 Kanheman & Tversky,

1973

likelihood of one event:

p(A)¼ f/n

base-rate fallacy

Task 5 Tversky & Kahneman,

1983

conjunction rule:

p(A)4 p(A&B)

conjunction

fallacy

Task 6 Green, 1982 likelihood of one event

p(A)¼ f/n

equiprobability

bias

Task 7 Tversky & Kahneman,

1974

departures from population are

more likely in small samples

sample size

neglect

Task 8 Stanovich & West, 2003 likelihood of independent

events

p(A)¼ p(A/B)

random similarity

bias

Task 9 Kirkpatrick & Epstein,

1992

ratios computation and

comparison:

f1/n15¼4 f2/n2

ratio bias

Task 10 Konold, 1989 likelihood of compound

events

equiprobability

bias

Summary of the tasks employed in the study with reference to the origin, requested rule, and

related heuristics and biases.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Individual differences measures

Cognitive ability. Set I of the Advanced Progressive Matrices (APM;Raven, 1962) was administered as a short form of the Raven’s StandardProgressive Matrices (see also Morsanyi & Holyoak, 2010; Nathaniel-Jameset al., 2004). Set I consists of 12 matrices, and participants have to select thecorrect response out of eight possible response options. Set I is usually usedas a practice and screening set for the full test, and it draws on all theintellectual processes sampled on the full test (although it does not extend tothe highest levels of complexity). Before administering the test, the first threematrices of Set A were administered as practice items. Test adaptation to theItalian population was made using IRT analysis procedures, which attestedadequate validity and reliability (Ciancaleoni, Primi, & Chiesi, 2010).Cronbach’s alpha for this measure was .78.

Need for cognition. Participants filled in the 9-item need for cognitionquestionnaire (see Appendix B) adapted by Kokis et al. (2002) from the 18-item adult scale (Cacioppo, Petty, Feinstein, & Jarvis, 1996). Cacioppo andPetty (1982) described the need for cognition as individual differences in thetendency to engage in and enjoy effortful cognitive activity. Example itemsare: ‘‘I like hard problems instead of easy ones’’ (positively scored), ‘‘I liketo do jobs where I don’t have to think at all’’ (negatively scored). The Italianversion of the scale was obtained using a forward-translation method, andvalidated through a sample of 470 sixth to eighth grade students (mean age12.8 yrs). The unidimensional structure of the scale was attested throughconfirmatory factor analysis, and the scale showed a good internalconsistency (alpha¼ .80; Chiesi & Primi, 2008).

Superstitious thinking. The 8-item children scale (Kokis et al., 2002, seeAppendix B) was used. Scale items concerned belief in paranormal events,superstition, and luck. Example items are: ‘‘I have things that bring meluck’’ (positively scored), ‘‘I do not believe in any superstitions’’ (negativelyscored). The Italian version of the scale was obtained using a forward-translation method and validated through a sample of 469 sixth to eighthgrade students (mean age 12.8 yrs). The unidimensional structure of thescale was attested through confirmatory factor analysis, and the scaleshowed acceptable internal consistency (alpha¼ .74; Chiesi, Donati, Papi, &Primi, 2010).

ProcedureParticipants completed the battery of questionnaires in a single sessionduring school time. Tasks were collectively administered and presented in apaper and pencil version, and children had to work through them


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

individually. The questionnaires were given in the same order for eachparticipant apart from the order of the Need for Cognition Scale and theSuperstitious Thinking Scale, which were counterbalanced. The studentsworked through the probabilistic reasoning problems first (the six problemswere presented in a fixed order) then they filled in the two questionnaires,and finally they solved the Raven’s Matrices. The session took about 35–40minutes altogether.

Results

Scores on the Raven’s Matrices ranged from 2 to 12 (M¼ 8.11, SD¼ 2.55,skewness¼ –.45; kurtosis¼ –.58). Mean scores across grade levels aredisplayed in Table S1 in Appendix C. Descriptive indices showed that caseswere concentrated on the high values of the distribution. Six participants (lessthan 2% of the sample) who had scores below 3 (corresponding to scoresmore than two standard deviations below the mean) were considered outliersand were excluded from the sample. We did this because an IQ score that istwo standard deviations below the sample mean would correspond to an IQof 70 or less (i.e., a mild intellectual disability). As our participants were frommainstream schools, we suspected that these scores actually reflecteddisengagement with the tasks. As we made important predictions regardingthe role of cognitive capacity in reasoning, we decided to exclude participantswhose measured scores were likely to be very different from their true scores.

Table 2 displays the correlations between the variables in the studymeasuring individual differences in age (and grade levels), cognitive ability,need for cognition, and superstitious thinking. Age was not correlated withany other individual differences variables. Cognitive ability (M¼ 8.26,SD¼ 2.40) was positively related to grade levels and need for cognition(M¼ 27.63, SD¼ 6.81) and negatively related to superstitious thinking

TABLE 2Experiment 1: Intercorrelations

1 2 3 4 5

1. Age in months

2. Cognitive Ability .07

3. Need for Cognition 7.04 .20**

4. Superstitious Thinking 7.03 7.22** 7.15*

5. Probabilistic Reasoning .15* .30** .09 7.22**

6. Grade .83** .20** 7.03 7.07 .23**

Intercorrelations between age, cognitive ability, need for cognition, superstitious thinking, and

probabilistic reasoning performance in Experiment 1.

*p5 .05. **p5 .01.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

(M¼ 21.64, SD¼ 6.30). Finally, a significant but small in size negativecorrelation was found between need for cognition and superstitiousthinking.

Performance on the probabilistic reasoning scale (M¼ 2.90, SD¼ 1.21)was positively related to grade levels and cognitive ability, and negativelyrelated to superstitious thinking, but it was unrelated to need for cognition.A significant but small in size correlation was also found with age.

To examine the effect of grade levels (6, 7, and 8) on probabilisticreasoning, a one-way ANCOVA was run in which cognitive ability andsuperstitious thinking were used as covariates (the result of the sameanalysis without the inclusion of the covariates is reported in Appendix C inTable S3). The effect of grade was significant, F(2, 289)¼ 5.16, p5 .01,Zp2¼ .04, once both the significant effects of cognitive ability, F(1,

289)¼ 16.42, p5 .001, Zp2¼ .05, and superstitious thinking, F(2,

289)¼ 8.24, p5 .01, Zp2¼ .03, were partialled out. Post-hoc comparisons

showed that grade 8 (M¼ 3.23, SD¼ 1.20) differed significantly both fromgrade 6 (M¼ 2.60, SD¼ 1.22, p5 .01), and grade 7 (M¼ 2.79, SD¼ 1.11,p5 .05).

Discussion

The main aim of this study was to investigate the role of grade levels inchildren’s probabilistic reasoning performance. Specifically, we expectedthat higher grade children would be more competent in solving probabilisticreasoning tasks, as they possess more probability-related knowledge.This conjecture was supported by the results of the ANCOVA. After takinginto account the effects of superstitious thinking and cognitive ability,grade levels continued to have a significant effect on children’s reasoningperformance.

These results are in line with previous studies (e.g., Handley et al., 2004;Kokis et al., 2002; Stanovich & West, 1999), which indicate that thetendency for analytic processing increases with increasing cognitive ability.That is, children with higher cognitive ability performed better, regardless oftheir grade levels and their susceptibility to superstitious thinking. Kahne-man and Frederick (2002) proposed that people with higher cognitive abilityare more likely to possess the relevant mathematical and probabilistic rules,as well as to recognise the applicability of these rules in particular situations.In fact, cognitive ability was positively related to need for cognition (thetendency to engage in and enjoy effortful thinking) and negatively related tosuperstitious thinking (the tendency to reason on the basis of erroneousbeliefs regarding luck and chance). Need for cognition itself was alsonegatively correlated with superstitious thinking. Thus our findings are inline with the argument of Kahneman and Frederick (2002). However,


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

additionally our results point to developmental changes in probabilisticreasoning ability. Lower grade children, regardless of cognitive ability,performed more poorly on the tasks than higher grade children. In fact, theperformance of grade 6 children was only slightly above chance, and evengrade 8 children got only half of the problems right on average.

Finally, superstitious thinking (see Toplak et al., 2007) deterioratesprobabilistic reasoning performance, and this happens independently ofother relevant individual differences variables. That is, more-superstitiouschildren performed worse than less-superstitious ones, regardless of gradeand cognitive ability. In sum, Experiment 1 demonstrated an effect of age/level of education in probabilistic reasoning, which was independent of theeffect of individual differences in cognitive ability, and superstitiousthinking.

EXPERIMENT 2

Experiment 1 provided evidence for a change in probabilistic reasoningability as a function of grade levels, after controlling for the effects ofcognitive ability and superstitious thinking. One potential explanation forthis improvement of probabilistic reasoning skills is that although bothyounger and older children possess relevant normative knowledge, youngerchildren are less able to resist tempting heuristics, or they are less able torecognise situations when the implementation of a particular rule isrequired. It is possible that in Experiment 1 some children gave incorrectresponses, despite possessing knowledge about the relevant rules. In fact,there is some evidence that even those adults who give incorrect responses toconflict tasks, experience a conflict between their intuitions and logic,although they might not be consciously aware of this (e.g., De Neys,Vartanian, & Goel, 2008). Similarly, children may possess more knowledgeabout the normative rules concerning probabilities than their answers show.That is, some children who end up giving an incorrect response might beaware of the relevant norms, and they might spend some time evaluating thedifferent response options. Thus giving an incorrect response does notnecessarily imply that children simply ignore the normative option. Anotherpossibility (which is in line with the claims of educational theorists whoemphasise the role of education in probabilistic reasoning; e.g., Fischbein,1987) is that younger children simply do not possess the relevant rulesrequired to solve the problems. Thus, they give heuristic responses bydefault.

One possible way of distinguishing between reasoning errors that arisefrom a lack of relevant knowledge (either not possessing the relevant rule, ornot recognising the applicability of the rule in a particular situation), andthose that are the result of participants’ not investing enough effort into


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

implementing the rule properly is to use different instructional conditions.Dual-process theories predict that increasing cognitive effort (that is,increasing the amount of Type 2 processing) should lead to an increase innormative responding. In line with this conjecture, instructions provided toparticipants are related to the magnitude of reasoning biases (e.g., the beliefbias; see Evans, Newstead, Allen, & Pollard, 1994). In a study conductedwith university students, Ferreira et al. (2006) found that instructions to beintuitive vs rational oriented the tendency for using heuristic vs rule-basedreasoning when participants solved base-rate, conjunction fallacy, and ratiobias tasks. Similarly, Klaczynski (2001a) reported that ‘‘framing’’ instruc-tions to reason ‘‘like a perfectly logical person’’ boosted the performance ofadolescents on ratio bias problems. Moreover, Klaczynski (2001a) alsofound that instructions had an effect on both adolescents’ and adults’performance.

On the other hand, Klaczynski and Cottrell (2004) proposed thatmetacognition only starts to shape reasoning around mid-adolescence.Metacognition is related to the tendency to inhibit heuristically cuedresponses, and also to consider alternatives, and it is mostly independent ofcognitive abilities (Klaczynski, 2005). Metacognitive skills are not alwaysused or fully developed even in adults. Nevertheless, these skills make adifference in how people reason, as the consideration of alternatives reducesthe tendency to rely on shortcuts and default responses. Thus a possibleexplanation for the developmental change in Experiment 1 is that highergrade children displayed more developed metacognitive skills, which made itpossible for them to make better use of their relevant knowledge.

Experiment 2 was aimed at investigating the effect of instructions onchildren’s probabilistic reasoning performance, taking into account gradelevels, cognitive capacity, and superstitious thinking. We measuredprobabilistic reasoning ability through a set of 10 tasks (including the 6problems used in Experiment 1). Experiment 1 showed that probabilisticreasoning ability increased with level of education. We hypothesised that ifthis increase was the result of children’s increasing knowledge of the relevantrules, then instructions should make little if any difference for lower gradestudents (whose errors stem from a lack of relevant knowledge). By contrast,instructions to reason rationally should increase the number of normativeresponses in the case of higher grade students who possess more relevantknowledge. On the other hand, if the developmental changes were the resultof children’s developing metacognitive skills, then younger children shouldbenefit more from the instructions, as this would encourage them toconsider multiple possibilities, and to try and apply normative rules whensolving the problems.

Based on the results of Experiment 1 we expected that superstitiousthinking will decrease normative responding, whereas higher cognitive


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

ability will be related to better probabilistic reasoning performance.Moreover, drawing on recent findings with adults (Evans, Handley, Neilens,Bacon, & Over, 2009; Morsanyi, Primi, Chiesi, & Handley, 2009) weexpected to find a greater effect of instructions in the case of participantswith higher cognitive ability. According to Kahneman and Frederick (2002)higher-ability people are both more likely to possess the relevant rules and atthe same time more able to recognise the need to use them. As a result, theywill benefit more from investing more effort into reasoning. In order to testthese predictions, in the present study we investigated the effect of gradelevel, individual differences in cognitive ability and superstitious thinking,and instructional set within a single experiment.

Method

ParticipantsParticipants were 400 students in grade 6 (n¼ 144, 77 males; mean age: 11.7yrs, SD¼ .49), grade 7 (n¼ 130, 72 males; mean age: 12.6 yrs, SD¼ .54), andgrade 8 (n¼ 126, 60 males; mean age: 13.7 yrs, SD¼ .59). They wereenrolled in Italian junior high schools that serve families from the lowermiddle to middle socioeconomic classes. All grade 6 to 8 students in theschool were invited to participate, their parents were given informationabout the study and their permission was requested. None of theseparticipants had taken part in Experiment 1.

Materials

Probabilistic reasoning questionnaire. The questionnaire consisted of 10different probabilistic reasoning tasks (4 tasks were added to the 6 tasksdescribed in Experiment 1) selected from the heuristics and biases literature(see Appendix A for the tasks, and Table 1 for the source of the tasks, andthe normative principles required to solve them). As in Experiment 1children were given 1 point for each normatively correct response, and theirscores across tasks were collapsed to form an overall normative score.

Individual differences measures. Cognitive ability and superstitiousthinking were measured as described in Experiment 1.

Design and procedureThe procedure was the same as in Experiment 1 except that participantswere given one of two different instructions (based on Klaczynski, 2001a;and Ferreira et al., 2006). In the intuitive condition participants were told:‘‘Please answer the questions on the basis of your intuition and personal


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

sensitivity’’. In the rational condition, participants were told: ‘‘Please answerthe questions taking the perspective of a perfectly logical and rational person’’.As in Ferreira et al. (2006), a between-participants design was used whereparticipants were randomly assigned to the intuitive (n¼ 200) or to therational (n¼ 200) condition. The number of children from each grade levelwas similar in the intuitive/rational groups.

Results

Scores on the Raven’s Matrices ranged from 0 to 12 (M¼ 8.33, SD¼ 2.37,skewness¼7.82; kurtosis¼7.24). Mean scores on the Raven test acrossgrade levels are displayed in Table S1 in Appendix C. As in Experiment 1,participants (n¼ 8) who had scores of less than 3 were excluded from thesample as outliers. As in the previous experiment, cognitive ability(M¼ 8.48, SD¼ 2.15) and superstitious thinking (M¼ 21.27, SD¼ 6.10)did not correlate with age (r¼ .09, p¼ .08; r¼7.03, p¼ .56, respectively). Asmall in size negative correlation was found between cognitive ability andsuperstitious thinking (r¼7.12, p5 .05).

Two cognitive ability groups were created by using the median (9) of theRaven’s Matrices score as a cut-off: students below the median formed thelow to medium cognitive ability group (n¼ 187, M¼ 6.65, SD¼ 1.53), andstudents at or above the median formed the high cognitive ability group(n¼ 205, M¼ 10.15, SD¼ 1.02).

To examine the effect of instructions (intuitive vs rational), cognitiveability (low vs high), and grade (6, 7, and 8) on probabilistic reasoning, a26 26 3 ANCOVA was run in which superstitious thinking was used as acovariate (see Table 3 for descriptive statistics; the result of the sameanalysis without controlling for the effect of superstitious thinking isreported in Appendix C in Table S3). The effect of superstitious thinking

TABLE 3Experiment 2: Probabilistic reasoning scores

Instructions

Grade 6

(n¼ 140)

Grade 7

(n¼ 125)

Grade 8

(n¼ 125)

M (SD) M (SD) M (SD)

Low ability

High ability

Intuitive 3.73 (1.27) 3.63 (1.33) 4.57 (1.73)

Rational 3.95 (1.78) 3.44 (1.29) 4.13 (1.20)

Intuitive 4.59 (1.48) 4.42 (1.48) 4.43 (1.32)

Rational 3.74 (1.48) 5.11 (1.56) 5.18 (1.36)

The means and standard deviations (in brackets) of the probabilistic reasoning score across

groups defined on instruction, grade, and cognitive ability in Experiment 2.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

was significant, F(1, 390)¼ 13.27, p5 .001, Zp2¼ .03. Once this significant

effect was partialled out, the main effect of instructions was not significant,F(1, 390)5 1. However, there was a main effect of cognitive ability, F(2,390)¼ 17.55, p5 .001, Zp

2¼ .04, and grade, F(2, 389)¼ 5.06, p5 .01, Zp2¼ .03.

Post-hoc comparisons showed that grade 8 (M¼ 4.58, SD¼ 1.42) differedsignificantly from grade 6 students (M¼ 3.97, SD¼ 1.53, p5 .01), whereasgrade 7 students (M¼ 4.25, SD¼ 1.57) did not differ significantly from theother two groups. The interaction between cognitive ability and grade wassignificant, F(2, 390)¼ 3.79, p5 .05, Zp

2¼ .02. This indicated that thedifference between high- and low-ability children was greater in the case ofhigher grade children. There was also a significant interaction betweencognitive ability, grade, and instructions, F(2, 390)¼ 5.44, p5 .01, Zp

2¼ .03.This was because higher ability, higher grade children benefited more from theinstruction to reason logically than lower grade and/or lower ability children(see below). No other interactions were significant.

We analysed these effects further by running a 2 (instructions) x 3 (grade)ANCOVA (as before, superstitious thinking was used as a covariate)separately for the high and low cognitive ability groups. For the low-abilitygroup, the results indicated that the effect of superstitious thinking wassignificant, F(1, 185)¼ 3.79, p5 .05, Zp

2¼ .02. Once this significant effect waspartialled out, the main effect of grade, F(2, 184)¼ 4.56, p5 .01, Zp

2¼ .03,was still significant. Post-hoc comparisons showed that grade 8 (M¼ 4.31,SD¼ 1.45) significantly differed (p5 .01) from grade 7 (M¼ 3.53,SD¼ 1.30), whereas grade 6 students (M¼ 3.83, SD¼ 1.52) did notsignificantly differ from the other two groups. There was no effect ofinstructions, F(1, 185)5 1, and no interaction between grade andinstructions, F(2, 184)5 1. That is, in the lower-ability group only theeffects of superstitious thinking and grade levels reached significance.

In the high-ability group the effect of superstitious thinking wassignificant, F(1, 204)¼ 10.40, p5 .01, Zp

2¼ .05. Once this significant effectwas partialled out, the main effect of grade, F(2, 203)¼ 3.10, p5 .05,Zp2¼ .05, was still significant. Post-hoc comparisons showed that grade 6

(M¼ 4.15, SD¼ 1.53) differed significantly (p5 .05) both from grade 7(M¼ 4.77, SD¼ 1.55) and grade 8 (M¼ 4.79, SD¼ 1.39), grade 7 and 8 didnot significantly differ from each other. The main effect of instructions wasnot significant, F(1, 204)5 1, whereas the interaction between grade andinstructions was significant, F(2, 203)¼ 5.55, p5 .01, Zp

2¼ .05. That is,higher grade students were able to follow the instructions and their answerschanged in the expected direction (i.e., they performed better in the rationalcondition than in the intuitive condition). On the other hand, grade 6students’ answers also changed as a result of the instructions, but in the‘‘wrong’’ direction (i.e., they gave more normative responses when they wereasked to be more intuitive and vice versa).


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Discussion

Experiment 2 replicated and extended some of the findings of the previousexperiment. Once taking into account the effects of cognitive ability andsuperstitious thinking, grade levels continued to have a significant effect onprobabilistic reasoning (at least, grade 8 students performed significantlybetter than grade 6 students, whereas grade 7 students’ performance wasstatistically indistinguishable from that of the other groups). Moreover, theresults also confirmed the role of superstitious thinking in probabilisticreasoning. It can be argued that those who do not possess the relevant rulesmight use beliefs about luck and chance as a substitute for it, andsuperstitious thinking can prevent the use of normative rules even in peoplewho possess relevant knowledge.

The main aim of the present study was to experimentally manipulate themental effort that participants invest in solving the tasks in order to betterunderstand the cognitive underpinnings of the developmental changes inprobabilistic reasoning. Children were asked to answer the questions eitherrationally or intuitively. The impact of instructions was moderated by anumber of factors (i.e., we did not find a main effect of instructions).Specifically, only higher grade children with high cognitive capacitybenefited from investing effort into thinking ‘‘like a perfectly logicalperson’’. The most straightforward explanation for this is that children onlybenefit from thinking hard if they possess the relevant rules (which is morelikely in the case of higher grade students). On the other hand, cognitiveability played a crucial role as well. Similarly to recent findings with adultsamples (Evans et al., 2009; Morsanyi et al., 2009), only participants withhigher cognitive ability were able to follow the instructions. A possibleexplanation for this is that giving instructions is effectively a dual-taskmanipulation whereby participants have to maintain the instructions in theirworking memory while simultaneously carrying out reasoning tasks.Moreover, higher cognitive capacity makes it easier to recognise thesituations where the normative rules of probability should be implemented.

Interestingly, when asked to be rational, lower grade, high-abilitystudents gave fewer normative responses than when they were told to beintuitive. This shows that they take into account the instructions, since theiranswers change depending on the experimental condition, but this changewas not in the expected direction. A possible reason for this is that becauselower grade students do not possess relevant knowledge, they cannot use itto answer normatively, even if they try harder. According to some theorists(e.g., Hahn & Warren, 2009) deriving the rules of probability from personalexperiences is most likely to lead to/confirm biases. However, given that theinteraction was driven by high ability grade 6 students’ in the intuitivecondition giving more normative responses than grade 6 students in any


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

other condition, it is also possible that although high-ability lower gradestudents had a rudimentary grasp of some of the relevant rules, they wereunable to fully justify the application of these rules in the context of thepresent tasks. Thus, when thinking hard about the problems, they ended uprelying on some salient cues, which made it possible for them to generatejustifications/rationalisations.

In sum, the findings of this study suggest that individual differences incognitive capacity moderate the ability to comply with instructions, but onlyhigher grade students, who were presumably more familiar with the relevantrules, were able to perform better through investing more effort into solvingthe problems. By contrast, lower grade students did not benefit frominstructions to be logical. These findings support the claim that probabilisticreasoning ability crucially depends on the acquisition of relevant knowledge.On the other hand, these findings do not support a metacognitive account ofdevelopmental changes in probabilistic reasoning which would attribute thebetter performance of higher grade students to a general tendency to becomemore reflective and, thus, more rational with age. Although metacognitiveskills might improve during adolescence in the sense that adolescents’thinking becomes more flexible with age, this might also mean that they donot adhere to rules as rigidly as children, and, as a result, they become moresensitive to task context. In fact, in a recent review Klaczynski (2009) alsoemphasised predominantly this aspect of the development of metacognitiveskills in adolescence. Nevertheless, the present experiment exploreddevelopmental changes within a narrow age range. A comparison betweengroups with a greater difference between their level of education and theirmetacognitive skills would allow for a better test of these accounts.

EXPERIMENT 3

The aim of our third experiment was to compare the performance of highergrade junior high school students (i.e., year 7 and 8 students who showedbetter performance on the tasks), and a group of university students. Fromthe developmental literature we know that adults as well as children displaybiases in probabilistic reasoning (e.g., Chiesi et al., 2008; De Neyes & VanGelder, 2008; Fishbein & Schnarch, 1997; Klaczynski, 2001a, 2001b; Kokiset al., 2002; Reyna & Brainerd, 2008). We expected that university studentswould have a more consolidated knowledge of probabilistic rules, and theywould also be better able to recognise the need to use these rules. At thesame time, we predicted that they would display more developedmetacognitive skills. As a result, we expected them to perform better thansecondary school students, although even educated adults are prone toerrors and biases when they solve probabilistic reasoning problems (e.g.,Kahneman & Tversky, 2000), presumably because people tend to rely on


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

low-effort, heuristic reasoning by default (e.g., Evans & Over, 1996;Stanovich, 1999). As in the previous experiment we implemented aninstruction manipulation, where students were either given no instructions,or they were asked to reason like a perfectly logical and rational person. Aspeople tend to rely on low-effort, intuitive processing by default, weexpected that giving no instructions would lead to a relatively highproportion of heuristic responding in both groups.

We predicted that university students would show a better overallperformance on the tasks. However, we wanted to discriminate betweenpotential accounts for this developmental change. Based on the findings ofthe previous studies we expected an effect of cognitive capacity. Moreover, ifuniversity students reason better mainly because of their increased knowl-edge of the relevant rules, then instructions to reason logically should widenthe gap between the performance of the two groups, as this shouldencourage university students to rely on their advanced knowledge and skillseven more. This would be in line with the results of Experiment 2. However,if the difference between groups is mainly based on an increasedspontaneous tendency in the older group to observe normative rules, thenthe instructions should help younger students to overcome this disadvan-tage, and to display relatively better performance in the logical condition.

METHOD

ParticipantsParticipants were 97 students in grade 7 and 8 (57 males; mean age: 13.1 yrs,SD¼ 0.92) enrolled in Italian junior high schools. None of these studentstook part in any of the previous studies. The university students (n¼ 60, 31males; mean age: 24.5 yrs, SD¼ 3.70) were enrolled in different degreeprogrammes (psychology, educational sciences, biology, and engineering) atthe University of Florence. All participants were volunteers and they did notreceive any reward for their participation in this study.

Materials

Probabilistic reasoning questionnaire. The same questionnaire consistingof 10 probabilistic reasoning problems was used as in Experiment 2.

Individual differences measures

Cognitive ability. Adults were administered the Advanced ProgressiveMatrices – Short Form (APM-SF; Arthur & Day, 1994). The APM-SFconsists of 12 items selected from the 36 items of the APM-Set II (Raven,


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

1962). Participants have to choose the correct response out of eight possibleoptions. Consistently with the long form of the test, three items—derivedfrom Set I of the APM—were used for practice before completing the APM-SF. Test adaptation was made using IRT analysis procedures, whichattested adequate validity and reliability (Primi, Ciancaleoni, Galli & Chiesi,2010). Cronbach’s alpha for this measure was .79. As in Experiments 1 and2, children worked through Set 1 of the Raven’s Advanced ProgressiveMatrices. Both groups solved 12 problems altogether. (Note that the twogroups were tested on different items.)

Superstitious thinking. Both children and adults were administered thescale described in Experiment 1.

Design and procedureThe procedure was the same as in Experiment 2 except that participants inone condition (n¼ 81) were given rational instructions, whereas noinstructions were given in the other condition (n¼ 76).

Results

To examine the effect of instructions (rational vs none), and educationalstage (junior high school vs university) on probabilistic reasoning, a 26 2ANCOVA was run in which superstitious thinking was used as a covariate(we did not include cognitive ability in this analysis, as the tests used tomeasure cognitive capacity in the two age groups were different, althoughwe assumed that university students had higher cognitive ability). The resultof the same analysis without the inclusion of the covariates is reported inAppendix C in Table S3. The effect of superstitious thinking was significant,F(1, 156)¼ 10.71, p5 .01, Zp

2¼ .07. Once this significant effect was partialledout, the main effect of instructions was not significant, F(1, 156)5 1,whereas a significant effect of educational stage, F(1, 156)¼ 11.30, p5 .01,Zp2¼ .07, was found, indicating that university students (M¼ 5.95,

SD¼ 1.30) performed better than junior high school students (M¼ 4.60,SD¼ 1.57). The interaction between instructions and educational stage,F(1, 156)¼ 5.44, p5 .05, Zp

2¼ .04, was significant (see Table 4 fordescriptives). This indicated that university students performed better thanhigh school students when they were instructed to respond logically,whereas no difference between educational groups was found when studentswere given no instructions.

As in Experiment 2, cognitive ability groups were created by dividingthe samples using the median of the Raven’s Matrices score as cut-off.Mean scores across grade/educational levels are displayed in Table S1 inAppendix C. For both age groups the median score was 9: students below


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

the median formed the low to medium cognitive ability group (n¼ 78,junior high school: n¼ 50, M¼ 6.50, SD¼ 1.50; university: n¼ 28,M¼ 5.17, SD¼ 1.87), and students at or over the median formed thehigh cognitive ability group (n¼ 79, junior high school: n¼ 47, M¼ 10.38,SD¼ 1.03; university: n¼ 32, M¼ 10.16, SD¼ 1.39). In order to investi-gate the interactions between instructions and cognitive ability within eacheducational level, a 2 (instruction)6 2 (cognitive ability) ANCOVA (withsuperstitious thinking as a covariate) was run separately for junior highschool and university students.

Concerning junior high school students, the results indicated that theeffect of superstitious thinking was significant, F(1, 96)¼ 12.01, p5 .001,Zp2¼ .12. A correlational analysis indicated that probabilistic reasoning

performance was weakened by superstition, r(94)¼7.31. Once thesignificant effect of superstitious thinking was partialled out, the maineffect of cognitive ability was still significant, F(1, 96)¼ 10.50, p5 .01,Zp2¼ .10, (low: M¼ 4.12, SD¼ 1.55, high: M¼ 5.10, SD¼ 1.43). The main

effect of instructions was not significant, F(1, 96)¼ 2.68, p¼ .11, and therewas no interaction between cognitive ability and instructions, F(2, 184)52.That is, higher cognitive ability students performed better both when theywere instructed to answer rationally and when no instructions were given.Moreover, there was no difference between the two instruction conditionsregardless of students’ cognitive capacity, (i.e., whether students were askedto be rational or not, they performed the same way).

In the case of university students the results indicated that the effect ofsuperstitious thinking was not significant, F(1, 594)5 1. There was also noeffect of cognitive ability, F(1, 59)5 1, whereas the main effect ofinstructions was significant, F(1, 59)¼ 4.64, p5 .05, Zp

2¼ .09, (rational:M¼ 6.33, SD¼ 1.27, no instruction: M¼ 5.58, SD¼ 1.23). No interactionbetween cognitive ability and instructions was found, F(1, 59)5 1. That is,university students, regardless of their superstitious thinking score andcognitive ability, showed better performance when instructed to be rationalthan when they were given no instructions.

TABLE 4Probabilistic reasoning across groups

Instruction

Junior high

(n¼ 97)

University

(n¼ 60)

M (SD) M (SD)

Rational 4.45 (1.53) 6.39 (1.27)

None 4.76 (1.61) 5.58 (1.23)

The means and standard deviations (in brackets) of the probabilistic reasoning score across

groups defined on instruction and educational stage.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Discussion

The aim of the present experiment was to investigate higher grade junior highschool and university students’ performance in situations in which logicalthinking was explicitly encouraged or where students were given noinstructions to rely on effortful thinking. As in the previous experiments,individual differences in cognitive ability and superstitious thinking were alsotaken into account. Based on the results of Experiment 2 we expected that dueto their more consolidated knowledge of the rules of probability, and alsobecause of their higher cognitive ability (which makes it easier for them torecognise the need to apply the relevant normative rules, and to actuallyimplement these rules) university students would generally perform betterthan high school students. In addition we expected that, for the same reason,university students would also be better able to follow instructions to reasonrationally, and through investing more effort, they would be better able toimplement the relevant rules. The results supported these predictions.University students performed better, and we found a significant interactionbetween educational stage and instructions. That is, there was a bigger effectof instructions in the case of university students who presumably possess morerelevant knowledge about probabilistic reasoning, and also have highercognitive ability than high school students. Similarly to Experiment 2, we didnot find a main effect of instructions, or an interaction between cognitiveability and instructions in the case of high school students. These resultsconfirmed once more the role of cognitive capacity and level of education inprobabilistic reasoning. In addition, we found that university students wereless influenced by superstitions than high school students.

Our results also hint at some developmental changes in the factors affectingprobabilistic reasoning ability. In the younger group students’ performancewas characterised by an interplay between superstitious thinking andcognitive capacity (and level of education—although this was not investigatedin the present study). Higher ability students and less-superstitious students inthis group performed better. By contrast, university students’ performancedepended mainly on instruction conditions, and there was no effect ofsuperstitious thinking or cognitive ability. Once we controlled for the effect ofsuperstitious thinking, university students in the no instructions condition didnot perform any better than high school students. On the other hand,university students were able to give more normative responses wheninstructed to reason logically, and this was independent of their cognitiveability (presumably because they all had the necessary capacity to follow theinstructions, and to be able to implement the relevant rules). According toKlaczynski (e.g., 2009) around mid-adolescence children reach a stage in theirdevelopment when metacognitive skills start to play an important role in theirreasoning performance. Our findings showed that, in fact, university students


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

were more flexible in their reasoning, and showed greater changes in theirperformance as a result of instructions. On the other hand, university studentsdisplayed no preference for effortful thinking relative to the younger group.Indeed, when no instructions were given, the two groups performed equallywell, despite the obvious advantage of university students in terms ofeducation and cognitive capacity.

In sum, these results suggest that it is unlikely that younger students’overall poorer performance on the probability tasks would stem from theirreduced tendency to spontaneously observe relevant normative rules. Infact, when they were not instructed to reason logically, they still performedjust as well as when they were instructed to do so. This is similar to thefinding of Experiment 1, where individual differences in need for cognitionwere unrelated to reasoning performance, possibly because students(regardless of their general attitude towards effortful thinking) tried hardto solve the tasks, even without explicit instructions to do so. Nevertheless,without sufficient cognitive capacity, and without being aware of some ofthe relevant normative rules, younger students showed an overall poorerperformance than higher grade students. On the other hand, higher gradestudents and university students showed a more flexible performance, andwere more responsive to different instructions. Arguably, this developmentalchange is driven by increases in cognitive resources and relevant knowledge(at least in part).

GENERAL DISCUSSION

In three experiments we investigated developmental changes in secondaryschool and university students’ probabilistic reasoning ability. We alsoexamined the interactions between level of education, cognitive capacity,and the investment of conscious effort in determining reasoning perfor-mance. The novelty of this project lies in combining the methods ofdevelopmental/educational theorists with the individual differences ap-proach employed by dual-process theories of reasoning. This also made itpossible to contrast some of the predictions of these approaches regardingprobabilistic reasoning.

Dual-process theories (e.g., Evans & Over, 1996; Stanovich, 1999) place agreat emphasis on the role of cognitive capacity and thinking dispositions inshaping reasoning performance. These theorists often ignore the role ofrelevant knowledge in reasoning (for a discussion on this issue see alsoEvans, 2010), possibly because of the repeated finding that even educatedadults are prone to biases and fallacies (e.g., Gilovich et al., 2002). Althoughdevelopmental dual-process theories (e.g., Brainerd & Reyna, 2001;Klaczynski, 2009) recognise the role of knowledge in reasoning development,and they also use this to explain the inconsistencies in the developmental


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

patterns of reasoning performance, their studies have not combineddevelopmental research with an investigation into the individual differencescorrelates of reasoning performance. Similarly, educational theoristsinvestigated the role of education and the acquisition of relevant knowledgein probabilistic reasoning (e.g., Borovcnik & Peard, 1996; Fischbein, 1987;Fong, Krantz & Nisbett, 1986) without taking into account the effects ofthinking styles and cognitive capacity.

Although dual-process theorists emphasise the crucial role of conscious,effortful (Type 2) reasoning in normative performance, and thus, the role ofcognitive capacity in reasoning, recently it emerged (Stanovich & West,2008) that in many cases high-ability people are not any less susceptible tocertain reasoning biases than lower ability people. To account for thisfinding, Stanovich and West (2008; see also Stanovich et al., 2008) suggestedthat the Type 2 override process crucially depends on whether people detectthe conflict between their intuitions and their knowledge about relevantnormative rules. These rules, procedures, and strategies derived from pastlearning experiences have been referred to as mindware (Perkins, 1995). Theconcept of mindware was adopted by Stanovich and colleagues (Stanovich& West, 2008) in their recent model of the role of knowledge in producingnormative responses to reasoning problems, which we describe next.

According to this model, if people do not possess the necessaryknowledge to produce a normatively correct response, their errors derivefrom a mindware gap (i.e., missing knowledge). However, even if peopledetect the conflict between their intuitions and a normative rule, and thusrelevant knowledge is available, they can still produce a normativelyincorrect response. In this case their errors result from an override failure:different alternatives are produced and there is an attempt to override Type1 processing, but this attempt fails, usually because people do not have thenecessary cognitive capacity to inhibit beliefs, feelings, and impressions, andthe same time implement the appropriate normative rules (e.g., De Neys,Schaeken, & d’Ydewalle, 2005; Handley et al., 2004).

Errors based on override failures and errors based on mindware gaps arerelated: a well-learnt rule not appropriately applied is a case of an overridefailure; a missing or poorly compiled rule is a case of a mindware gap.Finally, some acquired mindware can be the direct cause of reasoningerrors. In this case the failure is related to contaminated mindware. There arevarious mechanisms that can lead to ‘‘contamination’’. Toplak et al. (2007)propose that a good candidate for contaminated mindware in the case ofprobabilistic reasoning could be superstitious thinking.

The model of Stanovich and colleagues provides a theoretical frameworkfor reconciling the educational and dual-process approaches. Moreover, thepresent results are entirely consistent with this model, as we will explainbelow. In Experiment 1 we investigated the effect of grade levels on junior


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

high school students’ probabilistic reasoning ability, while controlling forthe effects of cognitive capacity, and superstitious thinking. We found thatgrade levels accounted for a significant proportion of variance inprobabilistic reasoning once the significant effects of cognitive ability andsuperstitious thinking were removed. The effect of grade levels might beattributed to increases in probability knowledge with education, thussupporting the notion that relevant knowledge (i.e., mindware) plays animportant role in probabilistic reasoning.

In Experiment 2 we increased the number of reasoning problems, and wealso manipulated experimentally the effort that children invested in solvingthe problems by providing them with instructions to reason rationally/intuitively. Instructions to reason rationally have been found to increasenormative performance (e.g., Ferreira et al., 2006; Klaczynski, 2001a),especially in the case of higher ability participants (e.g., Morsanyi et al.,2009). We hypothesised that if older children reason better because of theirimproved metacognitive skills (i.e., their increased tendency to overridedefault response tendencies) then instructing children to reason effortfullyshould close the gap between younger and older children. However, if thedevelopmental differences are mostly based on differences between theavailable relevant knowledge of lower and higher grade students theninstructions should make little if any difference. In fact, as cognitive capacityincreases with age, older children should benefit more from the instructions.In the terms of Stanovich and colleagues’ (2008) model, the abovepredictions can be reformulated the following way. Increasing the mentaleffort that participants invest into reasoning will reduce the number oferrors, if these errors stem from override failures. However, instructionsshould not affect the number of errors stemming from mindware gaps. As inthe previous experiment, in Experiment 2 we found main effects of grade,cognitive ability, and superstitious thinking. That is, after controlling for theeffect of cognitive ability and superstitious thinking, the effect of grade levelswas still significant. However, the most important findings of this study werethe significant interaction between grade and cognitive ability, and thesignificant grade by cognitive ability by instruction interaction. Theseindicated that only higher grade children with higher cognitive capacitybenefited from instructions to reason on the basis of logic. Thus, in line withthe predictions of educational theorists, and the Stanovich and West (2008)model, grade levels (which presumably reflected the effect of relevantknowledge) played a significant role in children’s reasoning performance,and this effect was present independently of the effect of cognitive ability,and cognitive effort. Although higher ability students (regardless of gradelevel) were sensitive to the instructions, in the case of lower grade studentsgiving instructions to reason logically did not boost performance (in fact, itdecreased reasoning performance). Thus investing more effort only helped


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

when participants’ errors predominantly stemmed from override failures(i.e., in the case of participants with more relevant knowledge).

In Experiment 3 we tested the same predictions as in Experiment 2 with adifferent group of students. Instead of comparing the performance ofchildren at different grade levels within a junior high school sample, wecompared the performance of high school and university students. Weexpected that (corresponding to the three-way interaction between gradelevel, cognitive ability, and instructions in the previous study) instructionsshould have a stronger effect on university students’ reasoning performance,as they possess more relevant knowledge than high school students, and thesame time they have higher cognitive capacity. This prediction wasconfirmed. We also replicated the finding of the previous experiments thathigh school students’ performance was influenced by their cognitive abilityand their susceptibility to superstitious thinking. Finally, in the case ofuniversity students there was no effect of cognitive ability or superstitiousthinking, only an effect of instructions. The reason for this is probably thatin this group, students all possessed sufficient knowledge, and had thenecessary cognitive capacity to follow the instructions, and the same timethey were minimally affected by superstitious thinking. The finding thatthe effect of instructions was universally present in the older studentpopulation, whereas in the younger group it only emerged under specificcircumstances (i.e., given sufficient knowledge, and cognitive capacity) canpossibly explain the repeated finding of Klaczynski (see e.g., 2009) that therole of metacognitive interference increases during adolescence. Our resultssuggest that this might not be due to the emergence of an entirely new ability(i.e., metacognition), although participants did seem to show more flexibilityin their performance with development. However, this flexibility is likely toresult from the interplay between increased cognitive capacity and increasesin knowledge and skills during adolescence. Thus, as the number of errorsstemming from mindware gaps and mindware contamination decreases, andthe same time the available cognitive capacity increases, the influence ofthinking dispositions increases with development.

Although in our interpretation we place a great emphasis on the role ofrelevant knowledge in reasoning development, an important weakness of thepresent experiments is that we did not use an independent measure ofprobability knowledge as an index of relevant knowledge. Grade level issurely associated with probability knowledge. However, it is also likely to berelated to other relevant knowledge and skills, such as familiarity with themultiple-choice question format, reading skills, and text comprehension.Thus, based on the present studies alone, there is no way of telling exactlyhow much of the observed effects of educational level are due to relevantskills in probabilistic reasoning. On the other hand, an effect of grade levelswas consistently found in all of these experiments. Thus even with a less-


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

than-perfect measure of probability knowledge we were able to demonstratethe expected role of education. Nevertheless, future studies shouldimplement more direct measures of relevant knowledge, or manipulate theknowledge of participants by assessing their knowledge of new skillsacquired during the experiment (i.e., participants should be provided withrelevant novel knowledge in a training session, as part of the experiment).

Increases with grade levels in reasoning performance could also be relatedto the development of executive functions (EFs), and the maturation of thefrontal lobes. Indeed, the problems used in the present study pit heuristicand rule-based reasoning against each other, and the ability to ignore salientbut irrelevant information might be necessary for children to be able toovercome the tendency to give heuristic responses (see e.g., Handley et al.,2004). Different EF components show distinct developmental trajectories,and some continue to develop into young adulthood. However, inhibitionskills reach mature levels by adolescence (Huizinga, Dolan, & van derMolen, 2006). Thus, whereas the development of inhibition skills might havecontributed to the differences between lower and higher grade secondaryschool students, the differences between higher grade secondary schoolstudents and university students are unlikely to be related to differences inthe ability to inhibit the effect of salient distractors, as this should be similaracross the two groups. Given that our comparisons between grade levels inour secondary school sample (in Experiment 2), and between secondaryschool and university students (in Experiment 3) yielded similar results, it isunlikely that age-related changes in inhibition skills played a major part inthe developmental patterns that we observed. It should also be noted thatinhibition skills are usually not considered or measured separately in studieslooking at the individual differences correlates of reasoning performance.Instead it is assumed that the measures of working memory or fluidintelligence tap into the ability to actively maintain and manipulateinformation in the presence of conflicting or distracting information (whichhas also been referred to as cognitive decoupling; see e.g., Stanovich, 2006).

Another potential criticism is related to our adopting a dual-processperspective. Although dual-process theories are very popular in manydifferent areas of psychological science, including social, developmental,and cognitive psychology, these theories are not without controversy (see,e.g., Keren & Schul, 2009; Osman & Stavy, 2006). The main reason that weadopted a dual-process perspective in this paper is because probabilisticreasoning performance is almost always considered to emerge from aninterplay between intuitive processes and rule-based reasoning. In fact, mostof the relevant literature on these biases is embedded in a dual-processperspective. Our findings are clearly very important for dual-process theorists(such as Stanovich & West, 2008) who developed their model of the role ofmindware in reasoning in order to be able to accommodate findings regarding


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

the relative independence of thinking biases and cognitive capacity with adual-process framework. Indeed, taking into account the role of knowledgeand skills in reasoning helps dual-process theorists to respond to some of thecriticism raised by dual-process sceptics (e.g., Osman & Stavy, 2006).Nevertheless, our studies can simply be taken as an analysis of the interactionsbetween knowledge, cognitive ability and thinking dispositions, and theseresults can inform researchers in the fields of reasoning and cognitivedevelopment, whether they adopt a dual-process perspective or not.

A final issue that we would like to briefly address is participants’ potentialinterpretation of the problems and the experimental instructions (see moreon this in Stenning & van Lambalgen, 2008). Classic problems in theheuristics and biases tradition are often criticised on the basis that theycreate an unnatural conflict between pragmatic/interpretative processes andthe application of normative rules derived from probability theory or logic(see, e.g., Hertwig & Gigerenzer, 1999). In Experiments 2 and 3, not only theproblems themselves but also the instructions to reason intuitively, logically(or the lack of instructions) were open to participants’ interpretation. Infact, following the instruction to be logical is only possible for participantswho understand the concept of logic (i.e., they possess another pieceof relevant knowledge), and clarifying the concept of logic has beendemonstrated to lead to better reasoning performance (e.g., Schmidt &Thompson, 2008). Thus it is possible that lower grade students were less ableto give normative responses and to follow the instructions to be logicalbecause they were less clear about what logicality entailed in this particularcontext. However, in the present study being logical simply meant toimplement particular relevant rules (i.e., it did not depend on participants’understanding of the difference between pragmatic and logical interpreta-tions of quantifiers, for example), and thus we expect that those studentswho were aware of the relevant rules also perceived these to be logical. Infact, only two problems (problems 4 and 5) raise the issue of a potentialconflict between pragmatic and logical interpretations.

Overall the present studies lend support to the claim of developmentaldual-process theorists (e.g., Brainerd & Reyna, 2001; Klaczynski, 2009) thatcognitive capacity per se is insufficient to explain developmental changes inreasoning performance, because normative responding crucially depends onparticipants’ beliefs, as well as their relevant knowledge. However, althoughthis claim has been made repeatedly, it has not been tested experimentally.These theorists also failed to generate specific hypotheses about theinteractions between knowledge, thinking dispositions, and cognitivecapacity. Our findings are in line with the claim that cognitive capacity andconscious effort will only be good predictors of normative reasoning ifstudents possess the relevant knowledge to solve a task, and the implementa-tion of relevant knowledge requires sustained effort, and the ability to resist


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

tempting heuristic response options (see Stanovich & West, 2008). Whenchildren do not possess relevant normative knowledge (i.e., mindware), wecan expect this to lead to the counterintuitive developmental patterns oftenreported in the literature, such as no change or even increases in heuristicresponding with development (see e.g., Klaczynski, 2009 for a review). Fillingthe ‘‘mindware gap’’ in developmental dual-process theories offers a key to abetter understanding of the development of children’s reasoning.

REFERENCES

Arthur, W., & Day, D. (1994). Development of a short form for the Raven Advanced

Progressive Matrices test. Educational and Psychological Measurement, 54, 395–403.

Baron, J. (2000). Thinking and deciding (3d ed.). Cambridge, UK: Cambridge University Press.

Borovcnik, M., & Peard, R. (1996). Probability. In J. Kilpatrick (Ed.), International handbook of

mathematics education (pp. 371–401). Dordrecht: Kluwer.

Brainerd, C. J., & Reyna, V. F. (2001). Fuzzy-trace theory: Dual processes in memory, reasoning,

and cognitive neuroscience. Advances in Child Development and Behavior, 28, 49–100.

Brainerd, C. J., & Reyna, V. F. (2004). Fuzzy-trace theory and memory development.

Developmental Review, 24, 396–439.

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social

Psychology, 42, 116–131.

Cacioppo, J.T., Petty, R. E., Feinstein, J. A., & Jarvis, W. B. G. (1996). Dispositional

differences in cognitive motivation: The life and times of individuals varying in need for

cognition. Psychological Bulletin, 119, 197–253.

Chiesi, F., Donati, M., Papi, C., & Primi, C. (2010). Misurare il pensiero superstizioso nei

bambini: validita e attendibilita della Superstitious Thinking Scale (Measuring superstitious

thinking in children: Validity and reliability of the Superstitious Thinking Scale). Eta

Evolutiva, 97, 5–14.

Chiesi, F., Gronchi, G., &, Primi, C. (2008). Age-trend related differences in task involving

conjunctive probabilistic reasoning. Canadian Journal of Experimental Psychology, 62(3),

188–191.

Chiesi, F., & Primi, C. (2008). Le proprieta psicometriche della versione italiana della Need for

Cognition Scale – Short Form (Psychometric properties of the Italian version of the Need

For Cognition Scale – Short Form). Paper presented at the Congresso Nazionale AIP-

Sezione di Psicologia Sperimentale, Padova.

Ciancaleoni, M., Primi, C., &, Chiesi, F. (2010). Exploring the possibility to use Set I of

Advanced Progressive Matrices as a short form of Standard Progressive Matrices. Testing,

Psychometrics, Methodology in Applied Psychology, TPM, 17(2), 91–98.

Davidson, D. (1995). The representativeness heuristics and the conjunction fallacy in children’s

decision making. Merrill Palmer Quarterly, 41(3), 328–346.

Denes-Raj, V., & Epstein, S. (1994). Conflict between intuitive and rational processing: When

people behave against their better judgment. Journal of Personality and Social Psychology,

66, 819–829.

DeNeys, W. (2006a). Dual processing in reasoning: Two systems but one reasoner.

Psychological Science, 17, 428–433.

DeNeys, W. (2006b). Automatic-heuristic and executive-analytic processing in reasoning:

Chronometric and dual task considerations. Quarterly Journal of Experimental Psychology,

59, 1070–1100.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

De Neys, W. (2007). Development of decision making: The case of the Monty Hall Dilemma. In

J. A. Elsworth (Ed.), Psychology of Decision Making in Education, Behavior, and High Risk

Situations (pp. 271–281). Hauppage, NY: Nova Science Publishers.

De Neys, W., Schaeken, W., & d’Ydewalle, G. (2005). Working memory and everyday

conditional reasoning: Retrieval and inhibition of stored counterexamples. Thinking &

Reasoning, 11, 349–381.

De Neys, W., & Van Gelder, E. (2008). Logic and belief across the life span: The rise and fall of

belief inhibition during syllogistic reasoning. Developmental Science, 12, 123–130.

De Neys, W., & Vanderputte, K. (2011). When less is not always more: Stereotype knowledge

and reasoning development. Developmental Psychology, 47, 432–441.

De Neys, W., Vartanian, O., & Goel, V. (2008). Smarter than we think: When our brains detect

that we are biased. Psychological Science, 19, 483–489.

Epstein, S. (1994). An integration of the cognitive and psychodynamic unconscious. American

Psychologist, 49, 709–724.

Epstein, S., Lipson, A., Holstein, C., & Huh, E. (1992). Irrational reactions to negative

outcomes: Evidence for two conceptual systems. Journal of Personality and Social


Evans, J. St. B. T. (2003). In two minds: Dual process accounts of reasoning. Trends in

Cognitive Science, 7, 454–459.

Evans, J. St. B. T. (2006). The heuristic-analytic theory of reasoning: Extension and evaluation.

Psychological Bulletin, 13(3), 378–395.

Evans, J. St. B. T. (2010). Intuition and reasoning: A dual-process perspective. Psychological

Inquiry, 21, 313–326.

Evans, J. St. B. T., Handley, S. J., Neilens, H., & Over, D. E. (2009). The influence of cognitive

ability and instructional set on causal conditional inference. Quarterly Journal of

Experimental Psychology, 63, 892–909.

Evans, J. St. B. T., Newstead, S. E., Allen, J. L., & Pollard, P. (1994). Debiasing by instruction:

The case of belief bias. European Journal of Cognitive Psychology, 6(3), 263–285.

Evans, J. St. B. T., & Over, D. E. (1996). Rationality and reasoning.Hove, UK: Psychology Press.

Ferreira, M. B., Garcia-Marques, L., Sherman, S. J., & Sherman, J. W. (2006). Automatic and

controlled components of judgment and decision making. Journal of Personality and Social


Fischbein, E. (1987). Intuition in science and mathematics. Dordrecht, The Netherlands: Reidel.

Fischbein, E., & Schnarch, D. (1997). The evolution with age of probabilistic, intuitively based

misconceptions. Journal for Research in Mathematics Education, 28(1), 96–105.

Fong, G. T., Krantz, D. H., & Nisbett, R. E. (1986). The effects of statistical training on

thinking about everyday problems. Cognitive Psychology, 18, 253–292.

Gigerenza, G., Hell, W., & Blank, H. (1988). Presentation and content: The use of base-rates as

a continuous variable. Journal of Experimental Psychology: Human Perception and

Performance, 14, 513–525.

Gilovich, T., Griffin D., & Kahneman, D. (Eds.). (2002). Heuristics and biases: The psychology

of intuitive judgment. Cambridge, UK: Cambridge University Press.

Green D. R. (1982). Probability concepts in 11–16 year old pupils (2nd ed). Loughborough, UK:

Centre for Advancement of Mathematical Education in Technology, College of Technology.

Hahn, U., & Warren, P. A. (2009). Perceptions of randomness: Why three heads are better than

four? Psychological Review, 116, 454–461.

Handley, S., Capon, A., Beveridge, M., Dennis, I., & Evans, J. St. B. T. (2004). Working

memory, inhibitory control and the development of children’s reasoning. Thinking and

Reasoning, 10, 175–195.

Hertwig, R., & Gigerenzer, G. (1999). The ‘conjunction fallacy’ revisited: How intelligent

inferences look like reasoning errors. Journal of Behavioral Decision Making, 12, 275–305.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Huizinga, M., Dolan, C. V., & van der Molen, M. W. (2006). Age-related change in executive

function: Developmental trends and a latent variable analysis. Neuropsychologia, 44, 2017–

2036.

Jacobs, J. E., & Klaczynski, P. A. (2002). The development of decision making during

childhood and adolescence. Current Directions in Psychological Science, 4, 145–149.

Jacobs, J. E., & Potenza, M. (1991). The use of judgment heuristics to make social and object

decision: A developmental perspective. Child Development, 62, 166–178.

Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in

intuitive judgement. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and

biases: The psychology of intuitive judgment (pp. 49–81). New York: Cambridge University

Press.

Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and

biases. New York: Cambridge University Press.

Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness.

Cognitive Psychology, 3, 430–454.

Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Bulletin,

80, 237–251.

Kahneman, D., & Tversky, A. (Eds.). (2000). Choices, values and frames.New York: Cambridge

University Press and the Russell Sage Foundation.

Keren, G., & Schul, Y. (2009). Two is not always better than one: A critical evaluation of two-

system theories. Perspectives on Psychological Science, 4, 533–550.

Kirkpatrick, L. A., & Epstein, S. (1992). Cognitive-experiential self-theory and subjective

probability: Further evidence for two conceptual systems. Journal of Personality and Social


Klaczynski, P. A. (2001a). Framing effects on adolescent task representations, analytic and

heuristic processing, and decision making: Implications for the normative/descriptive gap.

Journal of Applied Developmental Psychology, 22, 289–309.

Klaczynski, P. A. (2001b). The influence of analytic and heuristic processing on adolescent

reasoning and decision making. Child Development, 72, 844–861.

Klaczynski, P. A. (2004). A dual-process model of adolescent development: Implications for

decision making, reasoning, and identity. In R. V. Kail (Ed.), Advances in child development

and behavior vol. 31 (pp. 73–123). San Diego, CA: Academic Press.

Klaczynski, P. A. (2005). Metacognition and cognitive two-process model of decision making

and its development. In J. E. Jacobs and P. A. Klaczynski (Eds.), The development of

decision making in children and adolescents (pp. 39–76).

Klaczynski, P. A. (2009). Cognitive and social cognitive development: Dual-process research

and theory. In J. B. St. T. Evans & K. Frankish (Eds.), In two minds: Dual processing and

beyond (pp. 265–292). Oxford, UK: Oxford University Press.

Klaczynski, P., & Cottrell, J. (2004). A dual-process approach to cognitive development: The case

of children’s understanding of sunk cost decisions. Thinking and Reasoning, 10, 147–174.

Kokis, J. V., MacPherson, R., Toplak, M. E., West, R. F., & Stanovich, K. E. (2002). Heuristic

and analytic processing: Age trends and associations with cognitive ability and cognitive

styles. Journal of Experimental Child Psychology, 83, 26–52.

Konold, C. (1989). Issues in assessing conceptual understanding in probability and statistics.

Journal of Statistics Education 3(1) http://www.amstat.org/publications/jse/v3n1/konold.html

Kreitler, S., & Kreitler, H. (1986). Development of probability thinking in children 5 to 12 years

old. Cognitive Development, 1, 365–390.

Lehman, D. R., Lempert, R. O., & Nisbett, R. E. (1988). The effects of graduate training on

reasoning: Formal discipline and thinking about everyday-life events. American Psychol-

ogist, 43, 431–442.

Morsanyi, K., & Handley, S. J. (2008). How smart do you need to be to get it wrong? The role

of cognitive capacity in the development of heuristic-based judgment. Journal of

Experimental Child Psychology, 99, 18–36.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

http://www.amstat.org/publications/jse/v3n1/konold.html

Morsanyi, K., & Holyoak, K. J. (2010). Analogical reasoning ability in autistic and typically

developing children. Developmental Science, 13, 578–587.

Morsanyi, K., Primi, C., Chiesi, F., & Handley, S. (2009). The effects and side-effects of

statistics education. Psychology students’ (mis-)conceptions of probability. Contemporary

Educational Psychology, 34, 210–220.

Nathaniel-James, D. A., Brown, R. G., Maier, M., Mellers, J., Toone, B., & Ron, M. A. (2004).

Cognitive abnormalities in schizophrenia and schizophrenia-like psychosis of epilepsy.

Journal of the Neuropsychiatry and Clinical Neurosciences, 16, 472–479.

Osman, M., & Stavy, R. (2006). Intuitive rules: From formative to developed reasoning.

Psychonomic Bulletin & Review, 13, 935–953.

Perkins, D. N. (1995). Outsmarting IQ: The emerging science of learnable intelligence. New

York: Free Press.

Primi, C., Ciancaleoni, M., Galli, S., & Chiesi, F. (2010). Applying IRT to the Advanced

Progressive Matrices – Short Form. Paper presented at the 7th Conference of the

International Test Commission, Hong Kong.

Raven, J. C. (1962). Advanced progressive matrices. London: Lewis & Co. Ltd.

Reyna, V. F., & Brainerd, C. J. (1992). A fuzzy-trace theory of reasoning and remembering:

Paradoxes, patterns, and parallelism. In A. Healy, S. Kosslyn, & R. Shiffrin (Eds.), From

learning processes to cognitive processes: Essays in honor of William K. Estes (pp. 235–259).

Hillsdale, NJ: Lawrence Erlbaum Associates Inc.

Reyna, V. F., & Brainerd, C. J. (2008). Numeracy, ratio bias, and denominator neglect

in judgments of risk and probability. Learning and Individual Differences, 18, 89–107.

Reyna, V. F., & Ellis, S. C. (1994). Fuzzy-trace theory and framing effects in children’s risky

decision making. Psychological Science, 5, 275–279.

Reyna,V.F.,&Farley, F. (2006).Risk and rationality in adolescent decision-making: Implications

for theory, practice, and public policy. Psychological Science in the Public Interest, 7, 1–44.

Schmidt, J., & Thompson, V. A. (2008). ‘‘At least one’’ problem with ‘‘some’’ formal reasoning

paradigms. Memory & Cognition, 36, 217–229.

Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin,

119, 3–22.

Stanovich, K. E. (1999). Who is rational? Studies of individual differences in reasoning. Mahwah,

NJ: Lawrence Erlbaum Associates Inc.

Stanovich, K. E. (2004). The robot’s rebellion: Finding meaning in the age of Darwin. Chicago:

Chicago University Press.

Stanovich, K. E. (2006). Fluid intelligence as cognitive decoupling. Behavioral and Brain

Sciences, 29, 139–140.

Stanovich, K. E. (2008). Rationality and the reflective mind: Toward a tri-process model of

cognition. New York: Oxford University Press.

Stanovich, K. E. (2009). Distinguishing the reflective, algorithmic and autonomous minds: Is it

time for a tri-process theory? In J. Evans & K. Frankish (Eds.), In two minds. Dual-processes

and beyond (pp. 55–88). Oxford, UK: Oxford University Press.

Stanovich, K. E., & West, R. F. (1998). Who uses base rates and P(D/*H)? An analysis of

individual differences. Memory & Cognition, 26, 161–179.

Stanovich, K. E., & West, R. F. (1999). Discrepancies between normative and descriptive

models of decision making and the understanding acceptance principle. Cognitive


Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the

rationality debate. Behavior Brain Science, 23, 645–726.

Stanovich, K. E., & West, R. F. (2003). Evolutionary versus instrumental goals: How

evolutionary psychology misconceives human rationality. In D. E. Over (Ed.), Evolution

and the psychology of thinking (pp. 171–230). Hove, UK: Psychology Press.

Stanovich, K. E., & West, R. F. (2008). On the relative independence of thinking biases and

cognitive ability. Journal of Personality and Social Psychology, 94, 672–695.


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Stanovich, K. E., Toplak, M. E., & West, R. F. (2008). The development of rational thought: A

taxonomy of heuristics and biases. Advances in child development and behaviour, 36, 251–285.

Stenning, K., & van Lambalgen, M. (2008). Human reasoning and cognitive science. Cambridge,

MA: MIT Press.

Thompson, V. A. (2009). Dual process theories: A metacognitive perspective. In J. Evans & K.

Frankish (Eds.), In two minds: Dual processes and beyond. Oxford, UK: Oxford University

Press.

Toplak, M., Liu, E., Macpherson, R., Toneatto, T., & Stanovich, K. E. (2007). The reasoning

skills and thinking dispositions of problem gamblers: A dual-process taxonomy. Journal of

Behavioral Decision Making, 20, 103–124.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases.

Science, 185, 1124–1131.

Tversky, A., & Kahneman, D. (1983). Extentional versus intuitive reasoning: The conjunction

fallacy in probability judgment. Psychological Bulletin, 90, 293–315.

West, R. F., Toplak, M. E., & Stanovich, K. E. (2008). Heuristics and biases as measures of

critical thinking: Associations with cognitive ability and thinking dispositions. Journal of

Educational Psychology, 100, 930–941.

APPENDIX A

Probabilistic thinking assessment

Correct responses are marked with *; for task 6 the correct response is thatstatement b has to be more likely than statement a.

1. A fair coin is flipped five times, each time landing with tails up; TTTTT.What is the most likely outcome if a coin is flipped a sixth time?

a. Tailsb. Headsc. Heads and Tails are equally likely.*

2. A fair coin is tossed six times. Which of the following sequence ofoutcomes is the most likely result of six flips of the fair coin? (H:Heads, T: Tails)

a. THHTHTb. HTHTHTc. Both sequences are equally likely*

3. Two containers, labelled A and B, are filled with red and blue marblesin the following quantities. Container A contains 100 marbles (65 redand 35 blue). Container B contains 10 marbles (6 red and 4 blue).Each container is shaken vigorously. After choosing one of the


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

containers, you must draw out a marble (without peeking, of course).If you draw a blue marble, you win 10 Euros. Which container givesyou a better chance of drawing a blue marble?

a. Container A (with 65 red and 35 blue marbles)b. Container B (with 6 red and 4 blue marbles)*c. Equal chances from each container

4. A reporter interviewed 4 footballers and 8 singers. One of them isDavid.David earns a lot of money. He likes sports a lot. He exercises everyday and he is well-built. He likes working as a team and he sticks tohis daily schedule. He goes to bed early at night and he avoidssmoking and drinking alcohol. Which of the following statements isthe most likely to be true?

a. David is a footballerb. David is a singer*c. It is equally likely for David to be a footballer or a singer

5. A health survey was conducted in a sample of 100 adult malesof all ages and occupation. Mr F. was selected by chance from thelist of participants. Mr F. is a manager in an internationalindustrial unit. He had a very busy life for a long time, he oftenhad business dinners at night, and he rarely had a holiday.Recently he had to stop working for a short time. He startedworking again but he’s less engaged. He used to go to the park forjogging, but now he prefers to go for a walk. Order the followingstatements from the least likely to be true (1) to the most likely tobe true (3).

a. _____Mr F. is over 55 and has had a heart attackb. _____Mr F. is over 55c. _____Mr F. has a big family

6. A mathematics class has 13 boys and 16 girls in it. The teacher does araffle. Each pupil’s name is written on a slip of paper. All the slips areput in a hat. The teacher picks one slip without looking. Which of thefollowing results is most likely?

a. The winner is more likely to be a boy than a girlb. The winner is more likely to be a girl than a boy*c. The winner is just as likely to be a girl as a boy


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

7. In a town there are two hospitals. In the big one (Hospital A) thereare 100 births a day, on average. In the small one (Hospital B) thereare, on average, about 10 births a day. Usually half of all newbornsare girls and half of them are boys. So, the likelihood of giving birthto a boy is about 50%. Nevertheless, there are days on which morethan 50% of the babies born are boys, and there are days on whichfewer than 50% of the babies born are boys. On a particular day,which hospital is more likely to record 70% or more male births?

a. Hospital A (with 100 births a day)b. Hospital B (with 10 births a day)*c. The two hospitals are equally likely to record such an event

8. A bag contains 20 blue marbles and 10 green marbles. You have todraw out a marble six times. Each time the drawn marble has to bereplaced in the bag (you always draw a marble from 30 marbles, 20blue and 10 green). Imagine that you receive 10 Euros for each timeyou correctly predict the colour of the drawn marble. Which of thefollowing is the best choice to win as much possible?

a. To predict randomly ‘‘blue’’ or ‘‘green’’b. To predict any combination of 4 ‘‘blue’’ and 2 ‘‘green’’c. To always predict ‘‘blue’’

9. Two decks, labelled A and B, are composed of cards with a star (starcards) and cards without any figure (white cards) on the down side.Deck A contains 100 cards, 80 white and 20 with a star. Deck Bcontains 10 cards, 8 white and 2 with a star. After choosing one of thedecks, you must draw out a card (without peeking, of course). If youdraw a star card, you win 10 Euros. Which deck gives you a betterchance of drawing a star card?

a. Deck A (with 80 white and 20 star cards)b. Deck B (with 8 white and 2 star cards)c. Equal chances from each deck*

10. Five faces of a fair dice are painted black, and one face is paintedwhite. The dice is rolled six times. Which of the following results ismost likely?

a. To get five Blacks and one White*b. To get six Blacksc. Same chance of getting six Blacks or five Blacks and one White


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

APPENDIX B

Measures of thinking dispositions

These scales were developed and first used by: Kokis, J., Macpherson, R.,Toplak, M., West, R. F., & Stanovich, K. E. (2002). Heuristic and analyticprocessing: Age trends and associations with cognitive ability and cognitivestyles. Journal of Experimental Child Psychology, 83, 26–52.

On each scale the participants have to respond to each item on thefollowing 4-point scale: (1) strongly agree, (2) agree, (3) disagree, (4)strongly disagree.

Superstitious Thinking

1. I have things that bring me luck.2. The number 13 is unlucky.3. It is bad luck to have a black cat cross your path.4. Opening an umbrella inside can bring you bad luck.5. I don’t believe in luck.6. It’s a good idea to look at your horoscope every day.7. Horoscopes can be useful in making personality judgments.8. I do not believe in superstitions.(R)

Need for Cognition

1. I like hard problems instead of easy ones.2. I like to be in charge of a problem that needs lots of thinking.3. I try to avoid problems that I have to think about a lot. (R)4. I like to spend a lot of time and energy thinking about something.5. I like to do things that I’ve learned well over and over, so that I don’t

have to think about it any more. (R)6. It’s really cool to figure out a new way to do something.7. I’m not interested in learning new ways to think. (R)8. I like to do jobs that make me think hard.9. I like to do jobs where I don’t have to think at all. (R)

APPENDIX C

Supplementary tables


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

TABLE S3ANOVA analyses

F Sig. Partial eta squared

Experiment 1

Grade levels 8.11 5 .001 .05

Experiment 2

Grade levels 5.32 5 .05 .03

Instructions 5 1 – –

Cognitive ability 20.77 5 .001 .05

Cognitive ability6Grade 3.54 5 .05 .02

Cognitive ability6Grade6 Instructions 5.77 5 .01 .03

Experiment 3

Level of education 32.23 5 .001 .17

Instructions 5 1 – –

Level of education6 Instructions 5.00 5 .05 .03

ANOVA analyses to examine the effect of grade levels (in Experiments 1, 2, and 3), instructions

(Experiments 2 and 3) and cognitive ability (Experiment 2), and the interactions between these

factors.

TABLE S1Mean Raven scores across the different age groups in Experiments 1, 2, and 3

Grade 6 Grade 7 Grade 8 University

M (SD) M (SD) M (SD) M (SD)

Experiment 1 7.98 (2.39) 8.11 (2.40) 8.79 (2.30) –

Experiment 2 8.11 (2.14) 8.70 (2.13) 8.67 (2.15) –

Experiment 3 – 8.43 (2.25) 8.33 (2.40) 7.79 (2.99)

Note that university students were administered a different 12-item short form of the Raven

APM.

TABLE S2The proportion of correct responses for each probabilistic reasoning problem (SD in

parentheses)

Problem Experiment 1 Experiment 2 Experiment 3

1 .64 (.48) .65 (.48) .76 (.43)

2 .64 (.48) .64 (.48) .72 (.45)

3 .44 (.50) .43 (.49) .61 (.49)

4 .03 (.18) .08 (.27) .11 (.32)

5 .51 (.50) . 64 (.50) .47 (.50)

6 .63 (.48) .69 (.46) .49 (.40)

7 – .22 (.41) .27 (.45)

8 – .30 (.46) .43 (.50)

9 – .41 (.49) .63 (.48)

10 – .37 (.48) .31 (.46)


Dow

nloa

ded

by [

Uni

vers

itaet

s un

d L

ande

sbib

lioth

ek]

at 1

6:30

15

Dec

embe

r 20

13

Date post:	18-Dec-2016
Category:	Documents
Upload:	kinga
View:	212 times
Download:	0 times

Developmental changes in probabilistic reasoning: The role of cognitive capacity, instructions,...

Documents