Interpreting one’s subjective experience: Heuristics and biases

Robert A. BjorkUniversity of California, Los Angeles

Festschrift in Honor of Douglas L. NelsonUniversity of South Florida

Tampa FloridaTampa, FloridaMarch 21-22, 2008

Interpreting subjective indices of learningInterpreting subjective indices of learning and performance

Perceptual fluency;Perceptual fluency;Retrieval fluency; Fluency of induction;y ;Fluency of association;

Foresight biasInflation of conditional predictionsInflation of conditional predictions

Experience-based vs. theory-based predictions of one’s own forgetting or learning;Egocentrism in social communication

Concluding comments on our subjective experience as teachers.

Important qualification

Subjective experience is not always misleading, it is sometimes even the best basis for judgmentsj g

Example: Jacoby and Kelley (1987)p y y ( )

b ll ( )Jacoby & Kelley (1987)

FSCAR ?????

vs.

FSCAR SCARF

J b Bj k & K ll (1994)Jacoby, Bjork, & Kelley (1994)

“Subjective experience like the publicSubjective experience, like the public media, is unavoidable, serves useful functions and is not to be fully trusted”functions, and is not to be fully trusted

Prelude: Interpreting objective indices ofPrelude: Interpreting objective indices of performance

L i fLearning versus performanceWhat we can observe is performance; What we must infer is learning;…and the former is an unreliable guide to the latter.

Conditions of instruction that make performance improve rapidly often fail to support long-term retention and transfer, whereas

Conditions of instruction that appear to create difficulties for the learner, slowing the rate of apparent learning, often optimize long-term retention and transfer

We are susceptible to being fooled by our own currentWe are susceptible to being fooled by our own current performance

Example: Interleaved versus blocked practice

Blocked versus random/interleaved practiceSimon & Bjork (2001)j ( )

Simon & Bjork (2001)

HighActual

EErrorRandom

Low Blocked

Practice 24hrs

Simon & Bjork (2001)

HighActual Predicted Retention

EErrorRandom

Random

Low Blocked

Practice 24hrs Practice 24hrs

Simon & Bjork (2001)

HighActual Predicted Retention

EErrorRandom

Random

Low Blocked Blocked

Practice 24hrs Practice 24hrs

Rohrer & Taylor (2007)

Wedge V= Spheroid V= 2

2 πhr3

4 2 πhr3

Spherical Cone V= Half Cone V= 32 2 πhr

37 2 πhr

3 3

63%

89%100

ect

60%

age

Cor

re

20%

Per

cent

0Mixed Blocked Mixed Blocked

Practice Final Test

Blocked versus interleaved practiceSte-Marie, Clark, Findlay, & Latimer (2004)

Interpreting and misinterpreting perceptualInterpreting and misinterpreting perceptual fluency or familiarity

Heuristic value

Misattributions and illusionsMisinterpreting the cause of perceptual fluencyMisinterpreting the cause of perceptual fluencyMisinterpreting the meaning of perceptual fluencyIllusions of competence

Example: Reder (1987, 1988)

(Reder, 1987, 1988)

“What is the term in golf for scoring one underWhat is the term in golf for scoring one under par?”

Interpreting and misinterpreting retrievalInterpreting and misinterpreting retrieval fluency

Heuristic value

Misattributions and illusionsInterpreting performance as learning; illusions of p g p g;competenceEgocentrism in instruction and social communicationIncomplete/faulty models of ourselves asIncomplete/faulty models of ourselves as learners/remembers

Example: Benjamin, Bjork, & Schwartz (1998)

B j i Bj k & S h t (1998)Benjamin, Bjork, & Schwartz (1998)

Phase 1: 20 (easy) general-knowledge questionsE. g., “Who was the first president of the United States?”Participants asked to:

(a) hit ENTER as soon as the answer “came to mind” (latency recorded);(b) say the answer;(c) predict the likelihood they would be able--at the end of the(c) predict the likelihood they would be able--at the end of the experiment--to free-recall having given that answer.

Phase 2: Distracting activity (spatial/map task)Phase 3: Final testPhase 3: Final test

Free recall: Write down as many of the 20 answers you gave earlier as you can;(Original questions were not shown again)( g q g )

I t ti fl f i d ti /l iInterpreting fluency of induction/learning

Kornell & Bjork (in press). Abstracting concepts and patterns: Is “spacing the g p p p genemy of induction”?Lesson on the reliability, or lack thereof, of researchers’ intuitionsresearchers’ intuitions.

Interleaving/spacing versus blocking/massingg p g g g

Interleaved/spaced: items re-studied after other items

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this pictureare needed to see this picture. are needed to see this picture.

Blocked/massed: items studied in succession

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Gentoo

Where’s the Gentoo?

Inductive learningInductive learning

• The ability to generalize concepts and categoriesThe ability to generalize concepts and categories through exposure to multiple exemplars.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

Hypothesis

• Blocking/massing allows the learner to notice characteristics that unify a categorycharacteristics that unify a category

• Interleaving/spacing makes doing so difficultGentoo Gentoo Gentoo Gentoo

Gentoo Lachesis Reinhard Gentoo

“Spacing is the friend of recall but the enemy of induction ”enemy of induction.

-Ernst Rothkopf

Method: phases

1. Instructions2 Study2. Study3. Distractor4. Test5. Questionnaire5. Questionnaire

Kornell & Bjork (in press, Psychological Science)

InstructionsInstructionsIn this experiment you’re going to look at some beautiful paintings. To start, you’ll be shown 72 paintings for 3 seconds each. The paintings will be by twelve artists, with six pictures per artist. Try to learn to recognize which artistsix pictures per artist. Try to learn to recognize which artist painted which picture based on their style.

Later you’ll be shown 48 new paintings which you haven’tLater, you ll be shown 48 new paintings, which you haven t seen before. You’ll have to identify who painted each one.

For example, if there were only two artists, named Al and Barb, you’d be shown paintings by Al and Barb, and later, you’d be shown new paintings and asked who paintedyou d be shown new paintings and asked who painted them, Al or Barb.

Design

• Two within-subject conditions: massed & spacedp

M S S M M S S M M S S M

M S S M M S S M M S S M

Massed blockMassed block

Lewis

Lewis

Lewis

Lewis

Lewis

Lewis

M S S M M S S M M S S M

Spaced blockSpaced block

Pessani

Wexler

Schlorff

Stratulat

Hawkins

Mylrea

M S S M M S S M M S S MM

Lewis LewisLewisLewis Lewis Lewis

Pessani StratulatSchlorffWexler Juras Mylrea

Pessani StratulatSchlorff WexlerJuras Mylrea

Hawkins HawkinsHawkinsHawkins Hawkins Hawkins

TestFeedback

Test

0 8

0.7

0.8

Spaced

0.6

Spaced

0.4

0.5Massed

0.3

0.21 2 3 4

Test BlockTest Block

Results

Actual Responses

11

Actual Responses

0.6

0.8

uenc

y0.6

0.8

uenc

y

0.2

0.4Fr

eq

0.2

0.4

Freq

0Massed The same Spaced

0Massed The same Spaced

0.8A

0 5

0.6

0.7 SpacedA

Spacing vs. massing manipulation within

0.3

0.4

0.5

Massed

pparticipants

0.2

0.7

0.8

SpacedB

Spacing vs massing

0.5

0.6

0.7 Spaced Spacing vs. massing manipulated betweenparticipants

0 2

0.3

0.4Massed

participants

0.21 2 3 4

Test Block

Why a spacing effect?

• The test depended on recalling name-style associationsy– Spacing is the friend of recall

• Experiment 2Experiment 2– Final test: Yes/No recognition test for

paintings by studied artistspaintings by studied artists

MethodInstructions

Study

Distractor

Test Only one change: Recognition test

Questionnaire

Recognition test

Recognition Test

Studied vs. Distractor ArtistsHawkins: Hawkins Distractors:

Results (n=80)

0.70.80.9

MassedSpaced

0.40.50.60.7 Spaced

0.10.20.30

0Hit FA

Response

Results

I litIn reality29% 24% 48%

Responses80% 10% 10%80% 10% 10%

Differentiation hypothesisDifferentiation hypothesis• Original hypothesis: Blocking/massing highlights similarities

LewisLewis Lewis Lewis

• New hypothesis: Interleaving/spacing highlights differences

LewisLewis Lewis Lewis

LewisLewis Schlorff Hawkins

Interpreting and misinterpreting fluency of te p et g a d s te p et g ue cy oassociation

Foresight biasKoriat and Bjork (2005, 2006)

Prediction-inflation biasesN l D d l d G d (2005)Nelson, Dyrdal, and Goodmon (2005)Koriat, Fiedler, and Bjork (2006)Maki (2007a, 2007b)( , )

Foresight bias (Koriat & Bjork 2005)Foresight bias (Koriat & Bjork, 2005)

Likelihood the second word will be given as a free associate to the first?

Lamp: LightLamp: LightFind: SeekSell: BuyCheese: CheddarCitizen: Tax

F i ht bi (K i t & Bj k 2005)Foresight bias (Koriat & Bjork, 2005)

Likelihood the second word will be given as a free associate to the first?

Lamp: Light ( 71)Lamp: Light (.71)Find: Seek (.03)Sell: Buy (.56)Cheese: Cheddar (.03)Citizen: Tax (.00)

F i ht bi (K i t & Bj k 2005)Foresight bias (Koriat & Bjork, 2005)

Likelihood the second word will be given as a free associate to the first?

Lamp: Light ( 71) Forward pairLamp: Light (.71) Forward pairFind: Seek (.03) Backward pairSell: Buy (.56) Forward pairCheese: Cheddar (.03) Backward pairCitizen: Tax (.00) “Purely a-posteriori” pair

F i h bi (K i t & Bj k 2005)Foresight bias (Koriat & Bjork, 2005)

Th t d f l t i ill i fThe tendency of learners to experience an illusion of competence during learning, resulting in inflated predictions of later recallp

Judgments of learning are made in the presence of information that is absent, but solicited, on a subsequent testSuch as the targets in cue-target paired associates (e gSuch as the targets in cue target paired associates (e.g., Cheese-cheddar)

We are unable to anticipate the test situation, when the answer will be absent, but required., qThus, “Cheese ___?____” will trigger other strong associates, such as “mouse,” “bread,” “wine,” etc., which will compete with “cheddar.”

Koriat & Bjork (2005)

Pair type Predicted recall Actual recallForward 78 79Forward .78 .79

Backward .76 .60

Purely a-posteriori

.65 .47p

Nelson, Dyrdal, & Goodmon (2005), y , ( )

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.a e eeded to see t s p ctu e

What is preexisting strength? Predicting free association probabilities, similarity ratings, and cued recall probabilitiesprobabilities, similarity ratings, and cued recall probabilities (Nelson, Dyrdal, & Goodmon, 2005)

INSTRUCTIONS TO PARTICIPANTS:INSTRUCTIONS TO PARTICIPANTS:“Some word pairs are more related than others. For example, mostpeople would say that CAT DOG is more strongly related than GUARD DOG. The purpose of this study is to collect ratings on a large numberDOG. The purpose of this study is to collect ratings on a large number of people in order to determine whether such ratings can predict recall. If DOG were recently studied, would CAT be a better memory cue than GUARD? Such results will be important for understanding how memory

kworks.“What we want you to do today is rate each pair of words that you aregiven on a scale of 1-7 in terms of their degree of mutual association or relatedness If the first word readily calls the second word to mind or ifrelatedness. If the first word readily calls the second word to mind or if you can find an easy way to relate the two words to each other, thengive the pair a higher rating. Conversely, give the pair a lower ratingwhen they seem weakly associated or related.

Nelson, Dyrdal, & Goodmon (2005): R lt d di iResults and discussion

“Forward and backward strength emerged as the onlyForward and backward strength emerged as the only significant predictor of the ratings.”

“… both backward (r = .30) and forward (r = .20) strength were correlated with the ratings;correlated with the ratings;“… the ratings explained 8.8% … of the variance in cued recall”

“ i ti i il it ti t i fl d b“… associative similarity ratings are not … influenced by several predictors that influence cued recall, including mediated strength, target activation strength, target

tit t th d t t f ”competitor strength, and target frequency.”

(Maki, 2007)

JAM = 50.3 + 0.36 FSGR2 = 0.884

1224567

Inflation of conditional predictions f f p(Koriat, Fiedler, & Bjork, 2006)

Conditional prediction: “What is the probability that event Y will occur under condition X?

E.g., “What is the likelihood that if Iran stops supplying crude oil, g , p pp y g ,the price of a barrel will exceed $110”?

Prediction-inflation bias: Owing to a backward activation process, conditional predictions will tend to be inflated,

The stated outcome, whose likelihood is to be assessed, brings to the fore aspects of the condition that support the occurrence of thefore aspects of the condition that support the occurrence of the outcome.Example: Predicting free-association probabilities

Inflation of conditional predictions f f p(Koriat, Fiedler, & Bjork, 2006)

INSTRUCTIONS TO PARTICIPANTS:“In a word-association ask, a person is presented with a word and is asked to say as fast as he can the first word that comes to mind in response to the presented wordin response to the presented word.

“You will be presented with pairs of words. For each pair we would like you to estimate the percentage of people who would say thelike you to estimate the percentage of people who would say the second word (on the right) as the first response to the stimulus word (on the left)

; Hebrew is written from right to left).“You will find a blank line next to each word pair. Write on that line a number between 0 and 100 that reflects the percentage of people who, according to your opinion, would say the second word (the right one) in response to the first word (the left one).”g ) p ( )

90

100

Actual

60

70

80 Predicted

40

50

60

10

20

30

0Zero Low High

Associative Level

Koriat, Fiedler, & Bjork (2006)Experiment 5

80

90ActualPredicted- Control

Experiment 5

60

70

Predicted- ControlPredicted- Experimenta

40

50

20

30

0

10

U B F

Associative Directio

Koriat, Fielder, & Bjork (2006)Experiment 6

90

100Actual

Experiment 6

70

80

90Predicted- ControlPredicted- Experimenta

50

60

20

30

40

0

10

20

U B F

Associative Directio

How are memory judgments made?

Theory-basedDeliberate

Experience-basedAutomatic

In general…“… forgetting happens.”

E.g., Encoding fluencyRetrieval fluency

“… studying causes learning.”

et e a ue cyCue familiarity

Objective and subjective learning curvesObjective and subjective learning curves

Koriat, Bjork, Sheffer, & Bar (2004): Predicting one’s own forgettingone s own forgetting

Kornell & Bjork (in progress): Predicting one’sKornell & Bjork (in progress): Predicting one s own learning

Koriat, Bjork, Sheffer, & Bar (2004)jPredicting One's Own Forgetting: The Role of Experience-Based and Theory-Based Processes.

“ j d t f l i (JOL) if d b i“… judgments of learning (JOL), if governed by processing fluency during encoding, should be insensitive to anticipated retention interval.”

Subjective experience: “Various mnemonic cues contribute directly to produce an immediate feeling of knowing that can serve as the basis of judgments. Thus, for example, encoding and retrieval fluency may foster a feeling of competence that can serve as [an experience-basedfoster a feeling of competence that can serve as [an experience based basis for judgments of learning].”Domain-specific knowledge retrieved from memory. “Theory-based judgments, in contrast, rely on the deliberate use of specific beliefs and j g y pinformation to form an educated guess about one's own knowledge. Thus, JOLs may utilize such rules as “memory performance should be better on recognition than on recall memory test.”

Predicting one’s own forgetting (K i t Bj k Sh ff & B 2004)(Koriat, Bjork, Sheffer, & Bar, 2004)

E i t 1Experiment 1:60 paired associates

30 related;30 unrelated30 unrelated

Participants judge, pair by pair, the likelihood they will remember that pair on a later cued-recall testRetention interval to the final test (between-subjects):

Immediately after the study phase;One day;One week

Experiment 2:Experiment 2: To-be-predicted retention interval manipulated within-subjects.

Koriat, Bjork, Sheffer, & Bar (2004)

Exp 1: Between-subjects Exp 2: Within-subjects

Koriat et al. (2004): Experiment 4c

“This experiment was an attempt to bring the matter to an absurdity: Would participants fail to take into account a year’s retention interval in comparison toaccount a year s retention interval in comparison to participants who are asked to predict performance under immediate testing?”g

Results: Experiment 4c

Retention Interval

MeasureTen minutes

One week

One year

P di t d ll 35 2 40 4 37 1Predicted recall:(Exp. 4c)

35.2 40.4 37.1

Koriat et al. (2004): Experiment 7

S E i 4 b i i k d i b f iSame as Experiment 4c, but participants asked to estimate number of items forgotten:

“In a previous experiment that we conducted, students were presented with a list of 60 word pairs such as “table-chair” and “girl-eagle” one after the other. Each word pair wasword pairs such as table chair and girl eagle one after the other. Each word pair was presented for 4 seconds. The students’ task was to study these pairs so that when presented later with the first word, they would be able to recall the second word. The memory test took place ten minutes/one week /one year later.”“In the test the first word was presented with a blank line next to it for example:In the test, the first word was presented with a blank line next to it, for example:

Table _________Girl _________

and the students were asked to recall the second word and to write it down on the line. ““We would like you to estimate how many of the word pairs the students forgot after ten minutes/one week/one year. Your estimate can range from 0 to 60 pairs. ... Write down your estimate at the appropriate space at the bottom of the next page.”

Results: Experiment 7

Retention Interval

MeasureTen minutes

One week

One year

P di t d ll 35 2 40 4 37 1Predicted recall:(Exp. 4c)

35.2 40.4 37.1

Predicted recall: 43 4 31 1 18 7Predicted recall:(Exp. 7)

43.4 31.1 18.7

Koriat et al. (2004) conclusions

“… participants can access their knowledge about forgetting…

but only when theory-based predictions are made, and then only when the notion of forgetting is accentuated …y g g

either by manipulating retention interval within individuals, orby framing recall predictions in terms of forgetting rather than remembering ”remembering.

Experiment 1:Experiment 1:Predicting one’s own learning

24 paired associates24 paired associates12 related (Hill-Valley)12 unrelated (Clemency-Idiom)

Number of study/test cycles (between-subject):Number of study/test cycles (between-subject):STSTSTSTSTSTSTSTSTST

Participants judge, pair by pair, the likelihood they will remember that pair on the first, second, third, or fourth cued-recall test cycle

B t bj tBetween-subjects“Complete this sentence: When I make my 0-100% rating, I will do it about test number ___.”

Methodological reasons why predicting one’sMethodological reasons why predicting one s own forgetting and learning might differ

Fi t th i t h b t l i dFirst, there is a match between learning and “remembering”

Analogous to Koriat et al.’s Experiment 7 in which forgetting and “f tti ” t h“forgetting” matchVersus Koriat et al.’s other experiments in which forgetting and “remembering” mismatch

S d h di ti d f th dSecond, when predictions were made for the second, third, and fourth test cycle, it was clear that there could be multiple study/test cycles, which should then trigger th t f l i d ti t th b dthe concept of learning and activate theory-based judgments

100

Actual Recall (easy) Actual Recall (hard)Predicted Recall (easy) Predicted Recall (hard)

70

80

90

100

40

50

60

70

0

10

20

30

Exp. 1 (between-subjects)

01 2 3 4Test

Exp. 1 (between subjects)

100

Actual Recall (easy) Actual Recall (hard)Predicted Recall (easy) Predicted Recall (hard)

100

Actual Recall (easy) Actual Recall (hard)Predicted Recall (easy) Predicted Recall (hard)

70

80

90

100

70

80

90

100

40

50

60

40

50

60

70

0

10

20

30

0

10

20

30

Exp. 1 (between-subjects) Exp. 2 (within-subjects)

01 2 3 4Test

01 2 3 4Test

Exp. 1 (between subjects) Exp. 2 (within subjects)

Concluding comments:Predicting one’s forgetting versus predictingPredicting one s forgetting versus predicting one’s learning

Like Koriat et al ’s resultsLike Koriat et al. s results …participants were sensitive to item differences

Unlike Koriat et al.’s results … within-subject manipulation of number of study-test cycles did not appear to j p y y ppactivate the concept of learning

Why?Perhaps the concept of learning was activated, but people have a faulty and/or incomplete mental model of learning:incomplete mental model of learning:

the benefits of study are underappreciated;and the benefits of tests as learning events are not understood (e.g., Kornell & Son, 2006)

Or perhaps people are captured by item differences, which blocks access toOr perhaps people are captured by item differences, which blocks access to what they know (or don’t know) about learning:

Tentative conjecture: Among the hierarchy of experience-based and theory-based processes, the concept of learning falls below the concept of forgetting.

Concluding comments on our subjective g jexperience as teachers

Egocentrism in social communication

Newton (1990) as a parable of teaching;

Piaget (1962) quotePiaget (1962) quote

Calvin & Hobbes

Piaget (1962)

“Every beginning instructor discovers sooner or later that his first lectures were incomprehensible because he was talking to himself, so to say, mindful only of his point of view. He realizes only gradually and with difficulty that it is not easy to place one’s self in thedifficulty that it is not easy to place one s self in the shoes of students who do not yet know about the subject matter of the course.”

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Th E dThe End

References

Baddeley A D & Longman D J A (1978) The influence of length and frequency of training session on theBaddeley, A.D., & Longman, D.J.A. (1978). The influence of length and frequency of training session on the rate of learning to type. Ergonomics, 21, 627-635. Benjamin, A. S., Bjork, R. A., & Schwartz, B. L. (1998). The mismeasure of memory: When retrieval fluency is misleading as a metamnemonic index. Journal of Experimental Psychology: General, 127, 55-68.Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing. (pp.185-205). Cambridge, MA: MIT Press. Bjork, R. A. (1999). Assessing our own competence: Heuristics and illusions. In D. Gopher and A. Koriat (Eds.), Attention and performance XVII. Cognitive regulation of performance: Interaction of theory and application (pp. 435-459). Cambridge, MA: MIT Press. pp (pp ) gBjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus fluctuation. In A. Healy, S. Kosslyn, & R. Shiffrin (Eds.), From learning processes to cognitive processes: Essays in honor of William K. Estes (Vol. 2, pp. 35-67). Hillsdale, NJ: Erlbaum. Jacoby, L. L., Bjork, R. A., & Kelley, C. M. (1994). Illusions of comprehension, competence, and remembering In D Druckman and R A Bjork (Eds ) Learning remembering believing: Enhancing humanremembering. In D. Druckman and R. A. Bjork (Eds.), Learning, remembering, believing: Enhancing human performance (pp.57-80). Washington, DC: National Academy Press.Jacoby, L.L., & Kelley, C.M. (1987). Unconscious influences of memory for a prior event. Personality and Social Psychology Bulletin, 13, 314-336.

References (continued)

Koriat, A., Bjork, R. A., Sheffer, L., & Bar, S. K. (2004). Predicting one’s own forgetting: The role of experience-based and theory-based processes. Journal of Experimental Psychology: General, 133, 643-656.Koriat, A., & Bjork, R. A. (2005). Illusions of competence in monitoring one’s knowledge during study. Journal of Experimental Psychology: Learning, Memory, Cognition, 31, 187-194.Koriat, A., Ma'ayan H., Sheffer, L., & Bjork, R. A. (2006). Exploring a mnemonic debiasing account of the underconfidence-with-practice effect. Journal of Experimental Psychology: Learning, Memory and Cognition, 32, 595-608. Koriat, A., & Bjork, R. A. (2006). Illusions of competence during study can be remedied by manipulations that enhance learners’ sensitivity to retrieval conditions at test. Memory & Cognition, 34, 959-972.that enhance learners sensitivity to retrieval conditions at test. Memory & Cognition, 34, 959 972.Koriat, A., & Bjork, R. A. (2006). Mending metacognitive illusions: A comparison of mnemonic-based and theory-based procedures. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(5), 1133-1145. Koriat, A., Fiedler, K., & Bjork, R. A. (2006). Inflation of conditional prediction. Journal of Experimental Psychology: General 135(3) 429 447Psychology: General, 135(3), 429-447.Kornell, N., & Bjork, R. A. (2007). The promise and perils of self-regulated study. Psychonomic Bulletin & Review, 14, 219–224.Kornell, N., & Bjork, R. A. (in press). Optimizing self-regulated study: The benefits—and costs—of dropping flashcards. Memory.

References (continued)

Kornell N & Bjork R A (in press) Learning concepts and categories: Is spacing the “enemy ofKornell, N., & Bjork, R. A. (in press). Learning concepts and categories: Is spacing the enemy of induction”? Psychological Science.Maki, W. S. (2007a). Judgments of associative memory. Cognitive Psychology, 54, 319-353.Maki, W. S. (2007b). Separating bias and sensitivity in judgments of associative memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 231-237.Newton, L.(1990). Overconfidence in the Communication of Intent: Heard and Unheard Melodies. Unpublished doctoral dissertation, Department of Psychology, Stanford University.Piaget, J. (1962). Play, Dreams and Imitation in Childhood. New York: Norton.Reder, L. M. (1987). Selection strategies in question answering. Cognitive Psychology, 19, 90-138.R h D & T l K (2007) Th h ffli f th ti ti bl b t l iRohrer, D., & Taylor, K. (2007). The shuffling of mathematics practice problems boosts learning. Instructional Science, 35, 481-498.Simon, D. A., & Bjork, R. A. (2001). Metacognition in motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 907-912.Ste-Marie, D. M., Clark, S. E., Findlay, L. C. & Latimer A. E. (2004). High levels of contextual interferenceSte Marie, D. M., Clark, S. E., Findlay, L. C. & Latimer A. E. (2004). High levels of contextual interference enhance handwriting skill acquisition. Journal of Motor Behavior, 36, 115-126.