Educational/cognitive technology: how to use the evidence Cargo cult science

TECHNOLOGIES IN EDUCATION

Why being interested?

EVIDENCE-BASED ATTITUDE

Fair test: pay attention to…

Evidence about cognitive effects & learning…

The trouble with transfer and generalization

NEURO- & TECHNO-MYTHS

ECC 2012-13

Cargo cult science

•  1964 Feynman participates to a Commission for evaluation of math teaching manuals •  New math

•  Feynman is greatly disappointed both by new math and by the outcomes of the Commission’s work

•  And by education and psychology as sciences that he considered cargo cult sciences or pseudoscience

ECC 2012-13

¤  Feynman 1974

¤  There are big schools of reading methods and mathematics methods, and so forth, but if you notice, you'll see the reading scores keep going down--or hardly going up in spite of the fact that we continually use these same people to improve the methods. There's a witch doctor remedy that doesn't work. It ought to be looked into; how do they know that their method should work?

ECC 2012-13

¤  All experiments in psychology are not of this type, however. For example, there have been many experiments running rats through all kinds of mazes, and so on--with little clear result. But in 1937 a man named Young did a very interesting one. He had a long corridor with doors all along one side where the rats came in, and doors along the other side where the food was. He wanted to see if he could train the rats to go in at the third door down from wherever he started them off. No. The rats went immediately to the door where the food had been the time before

¤  The question was, how did the rats know, because the corridor was so beautifully built and so uniform, that this was the same door as before? Obviously there was something about the door that was different from the other doors. So he painted the doors very carefully, arranging the textures on the faces of the doors exactly the same. Still the rats could tell. Then he thought maybe the rats were smelling the food, so he used chemicals to change the smell after each run. Still the rats could tell. Then he realized the rats might be able to tell by seeing the lights and the arrangement in the laboratory like any commonsense person. So he covered the corridor, and still the rats could tell.

¤  He finally found that they could tell by the way the floor sounded when they ran over it. And he could only fix that by putting his corridor in sand. So he covered one after another of all possible clues and finally was able to fool the rats so that they had to learn to go in the third door. If he relaxed any of his conditions, the rats could tell.

ECC 201-132

¤  Now, from a scientific standpoint, that is an A-number-one experiment. That is the experiment that makes rat-running experiments sensible, because it uncovers the clues that the rat is really using--not what you think it's using. And that is the experiment that tells exactly what conditions you have to use in order to be careful and control everything in an experiment with rat-running.

¤  I looked into the subsequent history of this research. The next experiment, and the one after that, never referred to Mr. Young. They never used any of his criteria of putting the corridor on sand, or being very careful. They just went right on running rats in the same old way, and paid no attention to the great discoveries of Mr. Young, and his papers are not referred to, because he didn't discover anything about the rats. In fact, he discovered all the things you have to do to discover something about rats. But not paying attention to experiments like that is a characteristic of cargo cult science."

ECC 201-132

¤  In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they've arranged to imitate things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land.

ECC 2012-13

¤  So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land.

¤  Yet these things are said to be scientific. We study them. And I think ordinary people with commonsense ideas are intimidated by this pseudoscience. A teacher who has some good idea of how to teach her children to read is forced by the school system to do it some other way--or is even fooled by the school system into thinking that her method is not necessarily a good one. Or a parent of bad boys, after disciplining them in one way or another, feels guilty for the rest of her life because she didn't do "the right thing," according to the experts.

¤  A great deal of their difficulty is, of course, the difficulty of the subject and the inapplicability of the scientific method to the subject. Nevertheless it should be remarked that this is not the only difficulty. That's why the planes didn't land--but they don't land.

ECC 2012-13

Educational/cognitive technology: how to use the evidence Cargo cult science

TECHNOLOGIES IN EDUCATION

Why being interested?

EVIDENCE-BASED ATTITUDE

Fair test: pay attention to…

Evidence about cognitive effects & learning…

The trouble with transfer and generalization

NEURO- & TECHNO-MYTHS

ECC 2012-13

Reasons for being concerned by technologies in education

¤  Digital revolution ¤  Digital natives/Digital literacy ¤  Technomyths ¤  The impact of technologies on cognition & the human nature

¤  Educational technologies ¤  Educational methods that exploit or are inspired by

technologies

¤  Technology’s inspired educational interventions ¤  Video games ¤  Multi-media interactive technologies

ECC 2012-13

1. The digital revolution

ECC 2012-13

¤  Claims that the use of technologies

changes our mind/nature

¤  Digital natives/Generation Y have special skills

¤  Are more intelligent

¤  Stupid

¤  Violent

¤  Addicted

ECC 2012-13

…cognitive impact of technologies

¤  Prensky 2001

¤  What should we call these “new” students of today? Some refer to them as the N-[for Net]-gen or D-[for digital]-gen. But the most useful designation I have found for them is Digital Natives. Our students today are all “native speakers” of the digital language of computers, video games and the Internet.

¤  So what does that make the rest of us? Those of us who were not born into the digital world but have, at some later point in our lives, become fascinated by and adopted many or most aspects of the new technology are, and always will be compared to them, Digital Immigrants.

ECC 2012-13

¤  Prensky 2001

¤  Today’s students have not just changed incrementally from those of the past, nor simply changed their slang, clothes, body adornments, or styles, as has happened between generations previously. A really big discontinuity has taken place.

¤  Computer games, email, the Internet, cell phones and instant messaging are integral parts of their lives.

¤  It is now clear that as a result of this ubiquitous environment and the sheer volume of their interaction with it, today’s students think and process information fundamentally differently from their predecessors. These differences go far further and deeper than most educators suspect or realize.

¤  “Different kinds of experiences lead to different brain structures, “ says Dr. Bruce D. Perry of Baylor College of Medicine … it is very likely that our students’ brains have physically changed – and are different from ours – as a result of how they grew up.

¤  But whether or not this is literally true, we can say with certainty that their thinking patterns have changed.

ECC 2012-13

¤  Media Awareness Network Canada

¤  Established and internationally accepted definitions of digital literacy are generally built on three principles:

¤  the skills and knowledge to use a variety of digital media software applications and hardware devices, such as a computer, a mobile phone, and Internet technology;

¤  the ability to critically understand digital media content and applications;

¤  and the knowledge and capacity to create with digital technology.

ECC 2012-13

2. Teaching machines

¤  1900: Educational film

¤  1924: Teaching Machines(Pressey)

¤  1954: Teaching Machines (Skinner)

¤  1966: Sesame Street

¤  1967: Logo Turtles (Papert)

¤  1970s-1990s: Computer based training/learning CBT/L

¤  1970s-1990s: Computer aided instruction CAT

¤  E-learning

ECC 2012-13

ECC 2012-13

ECC 2012

… fair evaluation of effects

¤  Theoretical understanding of the principles (and limits)

¤  Empirical, experimental evaluation of the effects

ECC 2012-13

3. Technology-inspired education

ECC 2012-13

36 good learning principles embedded in video games (Gee 2005)

¤  Shaffer, Squire, Halverson, Gee 2004

¤  Will video games change the way we learn? ¤  We argue here for a particular view of games—and of learning—as

activities that are most powerful when they are personally meaningful, experiential, social, and epistemological all at the same time. From this perspective, we describe an approach to the design of learning environments that builds on the educational properties of games, but deeply grounds them within a theory of learning appropriate for an age marked by the power of new technologies.

¤  We argue that to understand the future of learning, we have to look beyond schools to the emerging arena of video games.

¤  We suggest that video games matter because they present players with simulated worlds: worlds which, if well constructed, are not just about facts or isolated skills, but embody particular social practices. Video games thus make it possible for players to participate in valued communities of practice and as a result develop the ways of thinking that organize those practices.

ECC 2012-13

… evaluation of effects

ECC 2012-13

… fair evaluation of claims

¤  playing is intrinsically motivating, because one plays for the fun of it and not because one has to. ¤  contradiction: even if games are for fun, if one has to play a game for

learning, the game is no more just for fun.

¤  playing makes learning fun and effortless ¤  opposed to school learning, which is considered as boring and effortful ¤  Unfair comparison: the kind of learning that is proposed at school can

hardly not be effortful because it concerns skills that do not come naturally to us

¤  good games are motivating because they are concrete, multi-modal, interactive, and involve the player learner in first person actions ¤  Sure it works better?

ECC 2012

Educational/cognitive technology: how to use the evidence Cargo cult science

TECHNOLOGIES IN EDUCATION

Why being interested?

EVIDENCE-BASED ATTITUDE

Fair test: pay attention to…

Evidence about cognitive effects & learning…

The trouble with transfer and generalization

NEURO- & TECHNO-MYTHS

ECC 2012-13

Fair tests: ay attention to…

¤  Causality vs correlation ¤  (Rosser et al, 2007)

ECC 2012-13

¤  Equivalence between experimental and control group ¤  Gopher 1994

ECC 2012-13

¤  Active controls ¤  (Robertson 2009)

ECC 2012-13

¤  Expected effect ¤  (Owen et al 2010)

ECC 2012-13

¤  Polarized, ideological debates & the power of anecdotes

ECC 2012-13

¤  Cognitive training in the elderly: memory, problem solving, rapid visual identification ¤  Ball et al. 2002

¤  Alzheimer ¤  Papp et al 2009

¤  Brain Training ¤  Owen et al 2010 ¤  Bavelier, Green & Dye

2011

ECC 2012-13

Evidence about cognitive effects & learning

¤  Video-games & Visuo-spatial attention

¤  Green & Bavelier 2008

¤  Bavelier, Green & Dye 2010

¤  Boot et al 2008

¤  Boot et al 2011

ECC 2012-13

The trouble with transfer and generalization

ECC 2012-13

¤  Willingham 2010

¤  Concrete cases and variation is not enough

ECC 2012-13

¤  Bransford et al 2000, p. 44

¤  In one of the most famous early studies comparing the effects of "learning a procedure" with "learning with understanding," two groups of children practiced throwing darts at a target underwater (Scholckow and Judd, described in Judd, 1908; see a conceptual replication by Hendrickson and Schroeder, 1941).

¤  One group received an explanation of refraction of light, which causes the apparent location of the target to be deceptive. The other group only practiced dart throwing, without the explanation. Both groups did equally well on the practice task, which involved a target 12 inches under water.

¤  But the group that had been instructed about the abstract principle did much better when they had to transfer to a situation in which the target was under only 4 inches of water.

¤  Because they understood what they were doing, the group that had received instruction about the refraction of light could adjust their behavior to the new task.

ECC 2012-13

¤  Bransford et al. 2000, p. 52

¤  A general wishes to capture a fortress located in the center of a country. There are many roads radiating outward from the fortress. All have been mined so that while small groups of men can pass over the roads safely, a large force will detonate the mines. A full-scale direct attack is therefore impossible. The general's solution is to divide his army into small groups, send each group to the head of a different road, and have the groups converge simultaneously on the fortress. Students memorized the information in the passage and were then asked to try another task, which was to solve the following problem

ECC 2012-13

¤  You are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that may be used to destroy the tumor. If the rays reach the tumor all at once and with sufficiently high intensity, the tumor will be destroyed, but surrounding tissue may be damaged as well. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the healthy tissue?

ECC 2012-13

¤  Few college students were able to solve this problem when left to their own devices.

¤  However, over 90 percent were able to solve the tumor problem when they were explicitly told to use information about the general and the fortress to help them.

¤  …. Despite the relevance of the fortress problem to the tumor problem, the information was not used spontaneously—the connection between the two sets of information had to be explicitly pointed out.

ECC 2012-13

¤  Bransford et al. 2000, p. 23

¤  In one study, a chess master, a Class A player (good but not a master), and a novice were given 5 seconds to view a chess board position from the middle of a chess game. After 5 seconds the board was covered, and each participant attempted to reconstruct the board position on another board. This procedure was repeated for multiple trials until everyone received a perfect score. On the first trial, the master player correctly placed many more pieces than the Class A player, who in turn placed more than the novice: 16, 8, and 4, respectively.

¤  However, these results occurred only when the chess pieces were arranged in configurations that conformed to meaningful games of chess. When chess pieces were randomized and presented for 5 seconds, the recall of the chess master and Class A player were the same as the novice—they placed from 2 to 3 positions correctly.

ECC 2012-13

¤  “Ericsson et al. (1980) worked extensively with a college student for well over a year, increasing his capacity to remember digit strings (e.g., 982761093 …). As expected, at the outset he could remember only about seven numbers. After practice, he could remember 70 or more… How? Did he develop a general skill analogous to strengthening a "mental muscle?" No, what happened was that he learned to use his specific background knowledge to "chunk" information into meaningful groups. The student had extensive knowledge about winning times for famous track races, including the times of national and world records. For example 941003591992100 could be chunked into 94100 (9.41 seconds for 100 yards). 3591 (3 minutes, 59.1 seconds for a mile), etc. But it took the student a huge amount of practice before he could perform at his final level, and when he was tested with letter strings, he was back to remembering about seven items.” (Bransford et al. 2000, p. 40)

ECC 2012--13

Educational/cognitive technology: how to use the evidence Cargo cult science

TECHNOLOGIES IN EDUCATION

Why being interested?

EVIDENCE-BASED ATTITUDE

Fair test: pay attention to…

Evidence about cognitive effects & learning…

The trouble with transfer and generalization

NEURO- & TECHNO-MYTHS

ECC 2012-13

Is technology making us more stupid/ intelligent?

ECC 2012-13

¤  Carr 2008 ¤  "Dave, stop. Stop, will you? Stop, Dave. Will

you stop, Dave?” So the supercomputer HAL pleads with the implacable astronaut Dave Bowman in a famous and weirdly poignant scene toward the end of Stanley Kubrick’s 2001: A Space Odyssey… “Dave, my mind is going,” HAL says, forlornly. “I can feel it. I can feel it.”

¤  I can feel it, too. Over the past few years I’ve had an uncomfortable sense that someone, or something, has been tinkering with my brain, remapping the neural circuitry, reprogramming the memory. My mind isn’t going—so far as I can tell—but it’s changing. I’m not thinking the way I used to think. I can feel it most strongly when I’m reading.

¤  Chabris & Simon 2010

¤  The alarmists cite the concept of "neural plasticity" and talk of technology "rewiring" the brain to convince us that the new distractions make us not just less willing but less able, on a physiological level, to focus.

¤  ….The appeals to neural plasticity, backed by studies showing that traumatic injuries can reorganize the brain, are largely irrelevant.

¤  The basic plan of the brain's "wiring" is determined by genetic programs and biochemical interactions that do most of their work long before a child discovers Facebook and Twitter. There is simply no experimental evidence to show that living with new technologies fundamentally changes brain organization in a way that affects one's ability to focus.

¤  Of course, the brain changes any time we form a memory or learn a new skill, but new skills build on our existing capacities without fundamentally changing them. We will no more lose our ability to pay attention than we will lose our ability to listen, see or speak.

ECC 2012-13

ECC 2012-13

Two neuro-myths

Considerations

¤  There's a logic in this apparent limitation of the brain ¤  an infinitely malleable brain that would change a wealth of

configurations for each new acquisition would risk to loose useful capacities just because of a new acquisition in a completely different domain

¤  A certain level of modularity and segregate learning effects seem to be justified, in addition to be widely demonstrated in many studies about perceptual, motor and cognitive training.

¤  The limits of transfer are a big preoccupation for educators ¤  education is meaningful only when it transfers towards

ecological situations – that is outside the classroom or away from the video game console: in the real life

ECC 2012

…and technomyths

¤  Technomyth

¤  Google generation has different mindset/skills

¤  Use of internet

¤  Attention/multi-tasking

¤  Risks with technomyts

¤  Illusion of understanding

¤  Illusion of attention

¤  Feeling confortable, secure, and skilled

ECC 2012-13