Bayesian Inference in Psychology

Jeffrey N. Rouder

April, 2013

Jeffrey N. Rouder

Bayesian Inference in Psychology

Collaborators

I Richard Morey, Groningen

I Mike Pratte, Vanderbilt

I Jory Province, Mizzou

I Paul Speckman, Mizzou, Stats

I Dongchu Sun, Mizzou, Stats

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Belief That Evidence For Effects Has Been Overstated

I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals

I Several Cases of Outright Fraud

I Crisis in How We Produce, Understand, and EvaluateEvidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Belief That Evidence For Effects Has Been Overstated

I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals

I Several Cases of Outright Fraud

I Crisis in How We Produce, Understand, and EvaluateEvidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Belief That Evidence For Effects Has Been Overstated

I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals

I Several Cases of Outright Fraud

I Crisis in How We Produce, Understand, and EvaluateEvidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Belief That Evidence For Effects Has Been Overstated

I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals

I Several Cases of Outright Fraud

I Crisis in How We Produce, Understand, and EvaluateEvidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Belief That Evidence For Effects Has Been Overstated

I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals

I Several Cases of Outright Fraud

I Crisis in How We Produce, Understand, and EvaluateEvidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

The p < .05 Rule

I People Know It is Not Perfect

I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.

I Provides A Mutual-Assurance Standard

I Easy To Teach

I Feels About Right / Natural

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

The p < .05 Rule

I People Know It is Not Perfect

I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.

I Provides A Mutual-Assurance Standard

I Easy To Teach

I Feels About Right / Natural

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

The p < .05 Rule

I People Know It is Not Perfect

I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.

I Provides A Mutual-Assurance Standard

I Easy To Teach

I Feels About Right / Natural

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

The p < .05 Rule

I People Know It is Not Perfect

I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.

I Provides A Mutual-Assurance Standard

I Easy To Teach

I Feels About Right / Natural

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

The p < .05 Rule

I People Know It is Not Perfect

I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.

I Provides A Mutual-Assurance Standard

I Easy To Teach

I Feels About Right / Natural

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

The p < .05 Rule

I People Know It is Not Perfect

I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.

I Provides A Mutual-Assurance Standard

I Easy To Teach

I Feels About Right / Natural

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Crisis of Confidence

I Why the Crisis? What is to be done?

I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.

I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.

I Solution: Be good.

I Increased focus on recording all data and intent.

I Increased focus on replication

I Prudent to examine the p < .05 Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The Free Lunch

I The p < .05 Rule a “A Free Lunch” Property.

I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.

I Lack of assumptions about the alternative is the “Free-Lunch”part.

I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”

I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.

I The free lunch is rotten, and eating it contributes to thestench in the field.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The Free Lunch

I The p < .05 Rule a “A Free Lunch” Property.

I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.

I Lack of assumptions about the alternative is the “Free-Lunch”part.

I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”

I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.

I The free lunch is rotten, and eating it contributes to thestench in the field.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The Free Lunch

I The p < .05 Rule a “A Free Lunch” Property.

I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.

I Lack of assumptions about the alternative is the “Free-Lunch”part.

I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”

I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.

I The free lunch is rotten, and eating it contributes to thestench in the field.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The Free Lunch

I The p < .05 Rule a “A Free Lunch” Property.

I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.

I Lack of assumptions about the alternative is the “Free-Lunch”part.

I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”

I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.

I The free lunch is rotten, and eating it contributes to thestench in the field.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The Free Lunch

I The p < .05 Rule a “A Free Lunch” Property.

I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.

I Lack of assumptions about the alternative is the “Free-Lunch”part.

I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”

I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.

I The free lunch is rotten, and eating it contributes to thestench in the field.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The Free Lunch

I The p < .05 Rule a “A Free Lunch” Property.

I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.

I Lack of assumptions about the alternative is the “Free-Lunch”part.

I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”

I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.

I The free lunch is rotten, and eating it contributes to thestench in the field.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Outline

I I. Frequentist And Bayesian Probability

I II. What Do We Want To Know: Invariances and Effects

I III. Pay For Lunch: The Frequentist Perspective

I IV. Bayes Factor Inference

Jeffrey N. Rouder

Bayesian Inference in Psychology

Outline

I I. Frequentist And Bayesian Probability

I II. What Do We Want To Know: Invariances and Effects

I III. Pay For Lunch: The Frequentist Perspective

I IV. Bayes Factor Inference

Jeffrey N. Rouder

Bayesian Inference in Psychology

Outline

I I. Frequentist And Bayesian Probability

I II. What Do We Want To Know: Invariances and Effects

I III. Pay For Lunch: The Frequentist Perspective

I IV. Bayes Factor Inference

Jeffrey N. Rouder

Bayesian Inference in Psychology

Outline

I I. Frequentist And Bayesian Probability

I II. What Do We Want To Know: Invariances and Effects

I III. Pay For Lunch: The Frequentist Perspective

I IV. Bayes Factor Inference

Jeffrey N. Rouder

Bayesian Inference in Psychology

Outline

I I. Frequentist And Bayesian Probability

I II. What Do We Want To Know: Invariances and Effects

I III. Pay For Lunch: The Frequentist Perspective

I IV. Bayes Factor Inference

Jeffrey N. Rouder

Bayesian Inference in Psychology

Part I. Frequentist and Bayesian Probability

Jeffrey N. Rouder

Bayesian Inference in Psychology

Probability

I Kolmogorov Axioms: Probability as a relative weight.

I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.

I Needed: One more property to link relative weight touncertainty.

I Bayesian and Frequentists differ on this additional property.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Probability

I Kolmogorov Axioms: Probability as a relative weight.

I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.

I Needed: One more property to link relative weight touncertainty.

I Bayesian and Frequentists differ on this additional property.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Probability

I Kolmogorov Axioms: Probability as a relative weight.

I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.

I Needed: One more property to link relative weight touncertainty.

I Bayesian and Frequentists differ on this additional property.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Probability

I Kolmogorov Axioms: Probability as a relative weight.

I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.

I Needed: One more property to link relative weight touncertainty.

I Bayesian and Frequentists differ on this additional property.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Frequentist Interpretation of Probability

Coin Flip:

I Probability is a long-run property

I# heads

# of flips, limit of lots of flips

I Probability is a property of the coin, much like weight andarea.

I Frequentists Obligation: Get It Right In The Long Run.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Frequentist Interpretation of Probability

Coin Flip:

I Probability is a long-run property

I# heads

# of flips, limit of lots of flips

I Probability is a property of the coin, much like weight andarea.

I Frequentists Obligation: Get It Right In The Long Run.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Frequentist Interpretation of Probability

Coin Flip:

I Probability is a long-run property

I# heads

# of flips, limit of lots of flips

I Probability is a property of the coin, much like weight andarea.

I Frequentists Obligation: Get It Right In The Long Run.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Frequentist Interpretation of Probability

Coin Flip:

I Probability is a long-run property

I# heads

# of flips, limit of lots of flips

I Probability is a property of the coin, much like weight andarea.

I Frequentists Obligation: Get It Right In The Long Run.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Frequentist Interpretation of Probability

Coin Flip:

I Probability is a long-run property

I# heads

# of flips, limit of lots of flips

I Probability is a property of the coin, much like weight andarea.

I Frequentists Obligation: Get It Right In The Long Run.

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Constructive Challenge

I Dylan Byers, Politico Blogger

I Blog Entry Entitled: Nate Silver: One-term Celebrity?

I Context: Who will win the 2012 Presidential Election as seenin late October

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Constructive Challenge

I Dylan Byers, Politico Blogger

I Blog Entry Entitled: Nate Silver: One-term Celebrity?

I Context: Who will win the 2012 Presidential Election as seenin late October

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Constructive Challenge

I Dylan Byers, Politico Blogger

I Blog Entry Entitled: Nate Silver: One-term Celebrity?

I Context: Who will win the 2012 Presidential Election as seenin late October

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Constructive Challenge

I Dylan Byers, Politico Blogger

I Blog Entry Entitled: Nate Silver: One-term Celebrity?

I Context: Who will win the 2012 Presidential Election as seenin late October

Jeffrey N. Rouder

Bayesian Inference in Psychology

2012 Presidential Election Poll Average

Jeffrey N. Rouder

Bayesian Inference in Psychology

Nate Silver’s Prediction

Jeffrey N. Rouder

Bayesian Inference in Psychology

Byers’ Column

Silver is an ideologue, wants Obama to win.

I Joe Scarborough, “Nate Silver says this is a 73.6 percentchance that the president is going to win? Nobody in thatcampaign thinks they have a 73 percent chance. They thinkthey have a 50.1 percent chance of winning. And you talk tothe Romney people, it’s the same thing. Both sidesunderstand that it is close, and it could go either way. Andanybody that thinks that this race is anything but a tossupright now is such an ideologue, they should be kept away fromtypewriters, computers, laptops and microphones for the next10 days, because they’re jokes.”

I “And even then [if Obama wins], you won’t know if heactually had a 50.1 percent chance or a 74.6 percent chanceof getting there.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

Byers’ Column

Silver is an ideologue, wants Obama to win.

I Joe Scarborough, “Nate Silver says this is a 73.6 percentchance that the president is going to win? Nobody in thatcampaign thinks they have a 73 percent chance. They thinkthey have a 50.1 percent chance of winning. And you talk tothe Romney people, it’s the same thing. Both sidesunderstand that it is close, and it could go either way. Andanybody that thinks that this race is anything but a tossupright now is such an ideologue, they should be kept away fromtypewriters, computers, laptops and microphones for the next10 days, because they’re jokes.”

I “And even then [if Obama wins], you won’t know if heactually had a 50.1 percent chance or a 74.6 percent chanceof getting there.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

Byers’ Column

Silver is an ideologue, wants Obama to win.

I Joe Scarborough, “Nate Silver says this is a 73.6 percentchance that the president is going to win? Nobody in thatcampaign thinks they have a 73 percent chance. They thinkthey have a 50.1 percent chance of winning. And you talk tothe Romney people, it’s the same thing. Both sidesunderstand that it is close, and it could go either way. Andanybody that thinks that this race is anything but a tossupright now is such an ideologue, they should be kept away fromtypewriters, computers, laptops and microphones for the next10 days, because they’re jokes.”

I “And even then [if Obama wins], you won’t know if heactually had a 50.1 percent chance or a 74.6 percent chanceof getting there.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

Byers’ Column

“And even then [if Obama wins], you won’t know if he actually hada 50.1 percent chance or a 74.6 percent chance of getting there.”

I By symmetry: a 99% chance or a 1% chance

I In what sense is frequentist probability a useful concept here?No large sample limit.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Interpretation of Probabilty

I Probability is a statement of belief

I Bayesian focus is on how beliefs should change in light of data.

I Long-run properties are consequences rather than primitives.

I Probability is used to describe the observer or analyst, not thecoin (or system more generally).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Interpretation of Probabilty

I Probability is a statement of belief

I Bayesian focus is on how beliefs should change in light of data.

I Long-run properties are consequences rather than primitives.

I Probability is used to describe the observer or analyst, not thecoin (or system more generally).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Interpretation of Probabilty

I Probability is a statement of belief

I Bayesian focus is on how beliefs should change in light of data.

I Long-run properties are consequences rather than primitives.

I Probability is used to describe the observer or analyst, not thecoin (or system more generally).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Interpretation of Probabilty

I Probability is a statement of belief

I Bayesian focus is on how beliefs should change in light of data.

I Long-run properties are consequences rather than primitives.

I Probability is used to describe the observer or analyst, not thecoin (or system more generally).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

Probability is a statement of belief.

I Silver’s p = .75 means 3:1 odds.

I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.

I Probabilities may be placed on anything

I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?

What wager should a journal editor accept?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.

I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.

I Probabilities may be placed on anything

I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?

What wager should a journal editor accept?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.

I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.

I Probabilities may be placed on anything

I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?

What wager should a journal editor accept?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.

I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.

I Probabilities may be placed on anything

I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?

What wager should a journal editor accept?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.

I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.

I Probabilities may be placed on anythingI Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?

What wager should a journal editor accept?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

I Bayesian focus is on how beliefs should change in light of data.

I Bayes Rule: How to update beliefs in light of data

I Bayesian Obligation: Use Bayes’ Rule Always.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

I Bayesian focus is on how beliefs should change in light of data.

I Bayes Rule: How to update beliefs in light of data

I Bayesian Obligation: Use Bayes’ Rule Always.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesian Probability

I Bayesian focus is on how beliefs should change in light of data.

I Bayes Rule: How to update beliefs in light of data

I Bayesian Obligation: Use Bayes’ Rule Always.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Prior Beliefs: How Well Can I Shoot Free Throws?

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

Jeffrey N. Rouder

Bayesian Inference in Psychology

Prior Beliefs: How Well Can I Shoot Free Throws?

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

Uncommited

Jeffrey N. Rouder

Bayesian Inference in Psychology

Prior Beliefs: How Well Can I Shoot Free Throws?

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

Looks Good

Jeffrey N. Rouder

Bayesian Inference in Psychology

Prior Beliefs: How Well Can I Shoot Free Throws?

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

Looks Bad

Jeffrey N. Rouder

Bayesian Inference in Psychology

Posterior Beliefs: 8 Makes in 12 Tries

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

0.0 0.2 0.4 0.6 0.8 1.00

12

34

Probability of Success

Den

sity

Jeffrey N. Rouder

Bayesian Inference in Psychology

Posterior Beliefs: 8 Makes in 12 Tries

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

0.0 0.2 0.4 0.6 0.8 1.00

12

34

Probability of Success

Den

sity

Jeffrey N. Rouder

Bayesian Inference in Psychology

Posterior Beliefs: 8 Makes in 12 Tries

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Probability of Success

Den

sity

0.0 0.2 0.4 0.6 0.8 1.00

12

34

Probability of Success

Den

sity

Jeffrey N. Rouder

Bayesian Inference in Psychology

Posterior Beliefs: 8 Makes in 12 Tries

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Probability of Success

Den

sity

0.0 0.2 0.4 0.6 0.8 1.00

24

68

10

Probability of Success

Den

sity

Jeffrey N. Rouder

Bayesian Inference in Psychology

Part II: What Do We Want To Know?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Is there an effect?

Jeffrey N. Rouder

Bayesian Inference in Psychology

Simplified Example

Do women and men differ in working-memory capacity?

I If significant difference, then go publish.

I If no significant difference, then better luck next time.

I Focus is on effects, not on invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Simplified Example

Do women and men differ in working-memory capacity?

I If significant difference, then go publish.

I If no significant difference, then better luck next time.

I Focus is on effects, not on invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Simplified Example

Do women and men differ in working-memory capacity?

I If significant difference, then go publish.

I If no significant difference, then better luck next time.

I Focus is on effects, not on invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Simplified Example

Do women and men differ in working-memory capacity?

I If significant difference, then go publish.

I If no significant difference, then better luck next time.

I Focus is on effects, not on invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariances are at the heart of the physical sciences

Jeffrey N. Rouder

Bayesian Inference in Psychology

Johannes Kepler (1571-1630)

I Planets varied greatly inthe speed & direction oftheir paths through thesky.

I Kepler extracted theinvariants of celestialmotion (e.g., ellipticalorbits, equal areacircumscribed in equaltime).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Johannes Kepler (1571-1630)

I Planets varied greatly inthe speed & direction oftheir paths through thesky.

I Kepler extracted theinvariants of celestialmotion (e.g., ellipticalorbits, equal areacircumscribed in equaltime).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Johannes Kepler (1571-1630)

I Planets varied greatly inthe speed & direction oftheir paths through thesky.

I Kepler extracted theinvariants of celestialmotion (e.g., ellipticalorbits, equal areacircumscribed in equaltime).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariances At The Heart of Science

I Conservation Laws: e.g., F = MA implies F1M1

= F2M2

in acommon gravitational field.

I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.

I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariances At The Heart of Science

I Conservation Laws: e.g., F = MA implies F1M1

= F2M2

in acommon gravitational field.

I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.

I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariances At The Heart of Science

I Conservation Laws: e.g., F = MA implies F1M1

= F2M2

in acommon gravitational field.

I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.

I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariances At The Heart of Science

I Conservation Laws: e.g., F = MA implies F1M1

= F2M2

in acommon gravitational field.

I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.

I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Hot-Hand Phenomena

I Gilovich, Vallone, and Tversky (1985)

I Hot Hand: The outcome of a shot attempt in basketball iscorrelated with the outcome of previous attempts.

I Conclusion: No such correlation. Probability of success isinvariant to previous history.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariance in Models: Top-Down Selection

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariance in Models: Top-Down Selection

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariance in Models: Top-Down Selection

Jeffrey N. Rouder

Bayesian Inference in Psychology

Invariance in Models: Top-Down Selection

Jeffrey N. Rouder

Bayesian Inference in Psychology

Commentary

I Maybe there are no invariances (Cohen)

I Example: Planets don’t follow ellipses to arbitrary precision

I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.

I Models are not true or false, they just vary in usefulness

I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Commentary

I Maybe there are no invariances (Cohen)

I Example: Planets don’t follow ellipses to arbitrary precision

I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.

I Models are not true or false, they just vary in usefulness

I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Commentary

I Maybe there are no invariances (Cohen)

I Example: Planets don’t follow ellipses to arbitrary precision

I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.

I Models are not true or false, they just vary in usefulness

I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Commentary

I Maybe there are no invariances (Cohen)

I Example: Planets don’t follow ellipses to arbitrary precision

I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.

I Models are not true or false, they just vary in usefulness

I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Commentary

I Maybe there are no invariances (Cohen)

I Example: Planets don’t follow ellipses to arbitrary precision

I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.

I Models are not true or false, they just vary in usefulness

I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.

Jeffrey N. Rouder

Bayesian Inference in Psychology

I Lawfulness, regularity, invariance, and constraint correspondto null hypotheses.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints

I Wrong side of the null

I Can’t state evidence for a null.

I State a lack of evidence for an effect.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints

I Wrong side of the null

I Can’t state evidence for a null.

I State a lack of evidence for an effect.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints

I Wrong side of the null

I Can’t state evidence for a null.

I State a lack of evidence for an effect.

Jeffrey N. Rouder

Bayesian Inference in Psychology

The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints

I Wrong side of the null

I Can’t state evidence for a null.

I State a lack of evidence for an effect.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Researchers Often Accept The Null

HOW?

I Inspection: “Figure x shows the lack of effect”

I The p > .05 Rule

I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)

I Unprinicpled

Jeffrey N. Rouder

Bayesian Inference in Psychology

Researchers Often Accept The Null

HOW?

I Inspection: “Figure x shows the lack of effect”

I The p > .05 Rule

I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)

I Unprinicpled

Jeffrey N. Rouder

Bayesian Inference in Psychology

Researchers Often Accept The Null

HOW?

I Inspection: “Figure x shows the lack of effect”

I The p > .05 Rule

I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)

I Unprinicpled

Jeffrey N. Rouder

Bayesian Inference in Psychology

Researchers Often Accept The Null

HOW?

I Inspection: “Figure x shows the lack of effect”

I The p > .05 Rule

I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)

I Unprinicpled

Jeffrey N. Rouder

Bayesian Inference in Psychology

Researchers Often Accept The Null

HOW?

I Inspection: “Figure x shows the lack of effect”

I The p > .05 Rule

I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)

I Unprinicpled

Jeffrey N. Rouder

Bayesian Inference in Psychology

p < .05 Rule is Useless

I Suppose it is theoretically meaningful to show effects.

I Shouldn’t we use the p < .05 Rule then? No

Jeffrey N. Rouder

Bayesian Inference in Psychology

p < .05 Rule is Useless

I Suppose it is theoretically meaningful to show effects.

I Shouldn’t we use the p < .05 Rule then? No

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:

I If there is a true effect, then in the large-sample limit,Pr(reject) = 1

I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:

I If there is a true effect, then in the large-sample limit,Pr(reject) = 1

I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:

I If there is a true effect, then in the large-sample limit,Pr(reject) = 1

I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:

I If there is a true effect, then in the large-sample limit,Pr(reject) = 1

I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:I If there is a true effect, then in the large-sample limit,

Pr(reject) = 1

I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:I If there is a true effect, then in the large-sample limit,

Pr(reject) = 1I BUT, if there is no effect, then Pr(reject) = α = .05, even in

the large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistency

I A consistent method is one that leads to the right decisionalways in the large-sample limit.

I It is a minimal obligation for the interpretation of frequentistprobability.

I Significance Testing is not consistent:I If there is a true effect, then in the large-sample limit,

Pr(reject) = 1I BUT, if there is no effect, then Pr(reject) = α = .05, even in

the large-sample limit.

I Bias toward rejecting the null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistent Frequentist Inference

I Usual: α(N) = .05, N is sample size

I To Be Consistent: α(N) must go to 0 as N increases

I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power

I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.

I To compute β(N), need to specify an alternative (Pay ForLunch).

I δ, effect size, is .4

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistent Frequentist Inference

I Usual: α(N) = .05, N is sample size

I To Be Consistent: α(N) must go to 0 as N increases

I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power

I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.

I To compute β(N), need to specify an alternative (Pay ForLunch).

I δ, effect size, is .4

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistent Frequentist Inference

I Usual: α(N) = .05, N is sample size

I To Be Consistent: α(N) must go to 0 as N increases

I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power

I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.

I To compute β(N), need to specify an alternative (Pay ForLunch).

I δ, effect size, is .4

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistent Frequentist Inference

I Usual: α(N) = .05, N is sample size

I To Be Consistent: α(N) must go to 0 as N increases

I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power

I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.

I To compute β(N), need to specify an alternative (Pay ForLunch).

I δ, effect size, is .4

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistent Frequentist Inference

I Usual: α(N) = .05, N is sample size

I To Be Consistent: α(N) must go to 0 as N increases

I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power

I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.

I To compute β(N), need to specify an alternative (Pay ForLunch).

I δ, effect size, is .4

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consistent Frequentist Inference

I Usual: α(N) = .05, N is sample size

I To Be Consistent: α(N) must go to 0 as N increases

I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power

I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.

I To compute β(N), need to specify an alternative (Pay ForLunch).

I δ, effect size, is .4

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Small Change To Make Significance Testing Consistent

Sample Size

Obs

erve

d E

ffect

Siz

e N

eede

d To

Rej

ect N

ull

0.0

0.2

0.4

0.6

0.8

1.0

5 10 20 50 100 200 500 1000

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Small Change To Make Significance Testing Consistent

Sample Size

Obs

erve

d E

ffect

Siz

e N

eede

d To

Rej

ect N

ull

0.0

0.2

0.4

0.6

0.8

1.0

5 10 20 50 100 200 500 1000

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Small Change To Make Significance Testing Consistent

Sample Size

Obs

erve

d E

ffect

Siz

e N

eede

d To

Rej

ect N

ull

0.0

0.2

0.4

0.6

0.8

1.0

5 10 20 50 100 200 500 1000

α = 0.05α = min (0.05, β)α = β 5

Jeffrey N. Rouder

Bayesian Inference in Psychology

A Small Change To Make Significance Testing Consistent

Sample Size

Obs

erve

d E

ffect

Siz

e N

eede

d To

Rej

ect N

ull

0.0

0.2

0.4

0.6

0.8

1.0

5 10 20 50 100 200 500 1000

α = 0.05α = min (0.05, β)α = β 5

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips, p < .05

I Implication: Alternative is substantially more likely than thenull.

I Let q be true probability of a tailH0: q = .5H1: q > .5.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips, p < .05

I Implication: Alternative is substantially more likely than thenull.

I Let q be true probability of a tailH0: q = .5H1: q > .5.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips, p < .05

I Implication: Alternative is substantially more likely than thenull.

I Let q be true probability of a tailH0: q = .5H1: q > .5.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips, p < .05

I Implication: Alternative is substantially more likely than thenull.

I Let q be true probability of a tailH0: q = .5H1: q > .5.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips, p < .05

I Implication: Alternative is substantially more likely than thenull.

I Let q be true probability of a tailH0: q = .5H1: q > .5.

Jeffrey N. Rouder

Bayesian Inference in Psychology

If q = .5

●●●●●●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●●●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●●●●●●

460 480 500 520 540

0.00

50.

015

0.02

5

Outcome

Pro

babi

lity

of O

utco

me

●●

●●

●●●●●●●●●●

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips p ≈ .05

I If we reject the null, then for what alternative?

I Let’s look at q = .527, most favorable alternative

Jeffrey N. Rouder

Bayesian Inference in Psychology

Tail Whispering

I Does whispering “tails” to coins increase the odds of tails?

I 527 out of 1000 flips p ≈ .05

I If we reject the null, then for what alternative?

I Let’s look at q = .527, most favorable alternative

Jeffrey N. Rouder

Bayesian Inference in Psychology

q = .5 vs. q = .527

●●●●●●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●●●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●●●●●●

460 480 500 520 540

0.00

50.

015

0.02

5

Outcome

Pro

babi

lity

of O

utco

me

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●●●●●●●●

●●

●●

●●

●

●

●

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating Evidence Against The Null

I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05

1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.

2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null

I No alternative is much better than the null

I Many reasonable ones are less concordant with the data thanthe null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating Evidence Against The Null

I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05

1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.

2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative

3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null

I No alternative is much better than the null

I Many reasonable ones are less concordant with the data thanthe null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating Evidence Against The Null

I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05

1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.

2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null

I No alternative is much better than the null

I Many reasonable ones are less concordant with the data thanthe null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating Evidence Against The Null

I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05

1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.

2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null

I No alternative is much better than the null

I Many reasonable ones are less concordant with the data thanthe null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating Evidence Against The Null

I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05

1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.

2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null

I No alternative is much better than the null

I Many reasonable ones are less concordant with the data thanthe null.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating-Evidence Argument

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating-Evidence Argument

0.0 0.2 0.4 0.6 0.8 1.0

05

1015

p−Value

Den

sity

Eff.Size=.2, N=50Eff.Size=.2, N=500Null

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating-Evidence Argument

0.00 0.02 0.04 0.06 0.08

01

23

45

p−Value

Den

sity

Eff.Size=.2, N=50Null

Jeffrey N. Rouder

Bayesian Inference in Psychology

Overstating-Evidence Argument

0.00 0.02 0.04 0.06 0.08

01

23

45

p−Value

Den

sity

Eff.Size=.2, N=500Null

Jeffrey N. Rouder

Bayesian Inference in Psychology

Interim Summary

I The p < .05 Rule is unprincipled. Overstates evidence againstnull.

I Principled frequentist inference requires paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Interim Summary

I The p < .05 Rule is unprincipled. Overstates evidence againstnull.

I Principled frequentist inference requires paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Part IV: Bayes-Factor Inference

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesians place beliefs on models

I Odds

I Prior Beliefs:P(M0)

P(M1)

I Posterior Beliefs:P(M0|Data)

P(M1|Data)

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesians place beliefs on models

I Odds

I Prior Beliefs:P(M0)

P(M1)

I Posterior Beliefs:P(M0|Data)

P(M1|Data)

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesians place beliefs on models

I Odds

I Prior Beliefs:P(M0)

P(M1)

I Posterior Beliefs:P(M0|Data)

P(M1|Data)

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesians place beliefs on models

I Odds

I Prior Beliefs:P(M0)

P(M1)

I Posterior Beliefs:P(M0|Data)

P(M1|Data)

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayesians place beliefs on models

I Odds

I Prior Beliefs:P(M0)

P(M1)

I Posterior Beliefs:P(M0|Data)

P(M1|Data)

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor: A Fully Bayesian Approach

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

I Bayes factors are the updating factor

B01 =P(Data|M0)

P(Data|M1).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor: A Fully Bayesian Approach

I Use Bayes Theorem to Update Beliefs:

P(M0|Data)

P(M1|Data)=

P(Data|M0)

P(Data|M1)× P(M0)

P(M1)

I Bayes factors are the updating factor

B01 =P(Data|M0)

P(Data|M1).

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factors Change In Beliefs

P(M0|Data)

P(M1|Data)= B01 ×

P(M0)

P(M1)

I Bayes factor is the change in belief due to the data

I Bayes factor is independent of prior odds

I Ratios can be interpreted in terms of wagers.

I Prior odds are a good place to add value.

I B10 = 1/B01

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factors Change In Beliefs

P(M0|Data)

P(M1|Data)= B01 ×

P(M0)

P(M1)

I Bayes factor is the change in belief due to the data

I Bayes factor is independent of prior odds

I Ratios can be interpreted in terms of wagers.

I Prior odds are a good place to add value.

I B10 = 1/B01

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factors Change In Beliefs

P(M0|Data)

P(M1|Data)= B01 ×

P(M0)

P(M1)

I Bayes factor is the change in belief due to the data

I Bayes factor is independent of prior odds

I Ratios can be interpreted in terms of wagers.

I Prior odds are a good place to add value.

I B10 = 1/B01

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factors Change In Beliefs

P(M0|Data)

P(M1|Data)= B01 ×

P(M0)

P(M1)

I Bayes factor is the change in belief due to the data

I Bayes factor is independent of prior odds

I Ratios can be interpreted in terms of wagers.

I Prior odds are a good place to add value.

I B10 = 1/B01

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factors Change In Beliefs

P(M0|Data)

P(M1|Data)= B01 ×

P(M0)

P(M1)

I Bayes factor is the change in belief due to the data

I Bayes factor is independent of prior odds

I Ratios can be interpreted in terms of wagers.

I Prior odds are a good place to add value.

I B10 = 1/B01

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factors Change In Beliefs

P(M0|Data)

P(M1|Data)= B01 ×

P(M0)

P(M1)

I Bayes factor is the change in belief due to the data

I Bayes factor is independent of prior odds

I Ratios can be interpreted in terms of wagers.

I Prior odds are a good place to add value.

I B10 = 1/B01

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

What is P(Data|M)?

I Models have parameters: θ

I P(Data|θ) is easy. Likelihood, L(θ).

I If we define the null and alternative as specific parametervalues θ0 and θ1, then

B01 =P(Data|M0)

P(Data|M1)=

L(θ0)

L(θ1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

What is P(Data|M)?

I Models have parameters: θ

I P(Data|θ) is easy. Likelihood, L(θ).

I If we define the null and alternative as specific parametervalues θ0 and θ1, then

B01 =P(Data|M0)

P(Data|M1)=

L(θ0)

L(θ1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

What is P(Data|M)?

I Models have parameters: θ

I P(Data|θ) is easy. Likelihood, L(θ).

I If we define the null and alternative as specific parametervalues θ0 and θ1, then

B01 =P(Data|M0)

P(Data|M1)=

L(θ0)

L(θ1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

What is P(Data|M)?

I Models have parameters: θ

I P(Data|θ) is easy. Likelihood, L(θ).

I If we define the null and alternative as specific parametervalues θ0 and θ1, then

B01 =P(Data|M0)

P(Data|M1)=

L(θ0)

L(θ1)

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

Parameters can take on more than a point:

B01 =P(Data|M0)

P(Data|M1)=

∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1

I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).

I If priors are too broad, then the average will include manyparameter values with low likelihood.

I Specification of priors is paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

Parameters can take on more than a point:

B01 =P(Data|M0)

P(Data|M1)=

∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1

I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).

I If priors are too broad, then the average will include manyparameter values with low likelihood.

I Specification of priors is paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

Parameters can take on more than a point:

B01 =P(Data|M0)

P(Data|M1)=

∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1

I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).

I If priors are too broad, then the average will include manyparameter values with low likelihood.

I Specification of priors is paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

Parameters can take on more than a point:

B01 =P(Data|M0)

P(Data|M1)=

∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1

I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).

I If priors are too broad, then the average will include manyparameter values with low likelihood.

I Specification of priors is paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

Toy Problem:

I yi ∼ Normal(µ, 1)

I Null, M0 : µ = 0.

I Alternative, M1 : µ ∼ Normal(0, σ20).

I Specification of σ20 is paying for lunch.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor & Priors on Parameters

Prior Standard Deviation sigma_0

Bay

es F

acto

r B

10

0.01 0.1 1 10 100 1000 1e+05

1e−

050.

011

100

1e+

05

y = 0

y = 0.15

y = 0.22

Jeffrey N. Rouder

Bayesian Inference in Psychology

Some Properties of Bayes Factors

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consequence #1: Sample Size Considerations

Sample Size

Crit

ical

t−va

lue

5 10 20 50 100 500 2000 5000

23

45

6

B10 = 10B10 = 3B10 = 1p−value = .05

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consequence #2: Calibration in Real Data

p−Value

Bay

es F

acto

r

0.5

110

40

0.001 0.01 0.05 0.1

Jeffrey N. Rouder

Bayesian Inference in Psychology

Consequence #3: Respects Resolution of Data

●●

●

●●

●●

●● ● ●

●

●

●

Bay

es F

acto

r

5 10 20 50 100

200

500

1000

2000

5000

1000

0

2000

0

5000

0

1e+

05

0.01

0.1

1

10

Sample Size

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor

A Most Excellent Means of Model Comparison:

I Principle: Update Beliefs Rationally in Light of Data

I State evidence, avoid decisions

I Evidence is in odds, natural language in terms of wagers.

I State evidence for or against invariances, as dictated by data

I Context through prior odds

I Natural penalty for model complexity.

I Respects resolution of data; best account of data.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor Computations

I Bayes factor computations are hard because one mustintegrate out all of the parameters

I My group knows how to do it for:

I t-tests (paired and grouped)Web applet at pcl.missouri.edu

I Multiple regressionWeb applet at pcl.missouri.edu

I ANOVA (within, between, mixed)BayesFactor package for R

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor Computations

I Bayes factor computations are hard because one mustintegrate out all of the parameters

I My group knows how to do it for:

I t-tests (paired and grouped)Web applet at pcl.missouri.edu

I Multiple regressionWeb applet at pcl.missouri.edu

I ANOVA (within, between, mixed)BayesFactor package for R

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor Computations

I Bayes factor computations are hard because one mustintegrate out all of the parameters

I My group knows how to do it for:

I t-tests (paired and grouped)Web applet at pcl.missouri.edu

I Multiple regressionWeb applet at pcl.missouri.edu

I ANOVA (within, between, mixed)BayesFactor package for R

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor Computations

I Bayes factor computations are hard because one mustintegrate out all of the parameters

I My group knows how to do it for:I t-tests (paired and grouped)

Web applet at pcl.missouri.edu

I Multiple regressionWeb applet at pcl.missouri.edu

I ANOVA (within, between, mixed)BayesFactor package for R

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor Computations

I Bayes factor computations are hard because one mustintegrate out all of the parameters

I My group knows how to do it for:I t-tests (paired and grouped)

Web applet at pcl.missouri.eduI Multiple regression

Web applet at pcl.missouri.edu

I ANOVA (within, between, mixed)BayesFactor package for R

Jeffrey N. Rouder

Bayesian Inference in Psychology

Bayes Factor Computations

I Bayes factor computations are hard because one mustintegrate out all of the parameters

I My group knows how to do it for:I t-tests (paired and grouped)

Web applet at pcl.missouri.eduI Multiple regression

Web applet at pcl.missouri.eduI ANOVA (within, between, mixed)

BayesFactor package for R

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Some analysts recommend using Bayesian estimation butNOT Bayes factors.

I Focus on Kruschke as his approach is now the recommendedone for The Psychonomic Society

Jeffrey N. Rouder

Bayesian Inference in Psychology

Parameter µ

Pos

terio

r D

ensi

ty

−2 0 2 4 6

Jeffrey N. Rouder

Bayesian Inference in Psychology

Parameter µ

Pos

terio

r D

ensi

ty

−2 0 2 4 6

95% of Area

Jeffrey N. Rouder

Bayesian Inference in Psychology

Parameter µ

Pos

terio

r D

ensi

ty

−2 0 2 4 6

95% of Area

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.

I Q. How plausible does one believe the null is before collectingdata?

I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.

I Q. How are these beliefs updated in light of data.

I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.

I Q. How plausible does one believe the null is before collectingdata?

I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.

I Q. How are these beliefs updated in light of data.

I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.

I Q. How plausible does one believe the null is before collectingdata?

I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.

I Q. How are these beliefs updated in light of data.

I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.

I Q. How plausible does one believe the null is before collectingdata?

I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.

I Q. How are these beliefs updated in light of data.

I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.

I Q. How plausible does one believe the null is before collectingdata?

I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.

I Q. How are these beliefs updated in light of data.

I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Other “Bayesian” Competitors

I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.

I Q. How plausible does one believe the null is before collectingdata?

I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.

I Q. How are these beliefs updated in light of data.

I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Pay For Lunch...

I Frequentist:

I Consistent Inference is Possible When the Null and AlternativeAre Well Specified.

I Principled Frequentist Analysts Pay For Lunch.

I Bayesian:

I Proper Updating Is Possible When the Null and AlternativeAre Well Specified.

I Principled Bayesian Analysts Pay For Lunch.

I Or Don’t Eat.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Pay For Lunch...

I Frequentist:I Consistent Inference is Possible When the Null and Alternative

Are Well Specified.I Principled Frequentist Analysts Pay For Lunch.

I Bayesian:

I Proper Updating Is Possible When the Null and AlternativeAre Well Specified.

I Principled Bayesian Analysts Pay For Lunch.

I Or Don’t Eat.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Pay For Lunch...

I Frequentist:I Consistent Inference is Possible When the Null and Alternative

Are Well Specified.I Principled Frequentist Analysts Pay For Lunch.

I Bayesian:I Proper Updating Is Possible When the Null and Alternative

Are Well Specified.I Principled Bayesian Analysts Pay For Lunch.

I Or Don’t Eat.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Pay For Lunch...

I Frequentist:I Consistent Inference is Possible When the Null and Alternative

Are Well Specified.I Principled Frequentist Analysts Pay For Lunch.

I Bayesian:I Proper Updating Is Possible When the Null and Alternative

Are Well Specified.I Principled Bayesian Analysts Pay For Lunch.

I Or Don’t Eat.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Does the Free Lunch Make Us Ill?

I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.

I With the free lunch, there is a mental distance between whatwe know and what we can show.

I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Does the Free Lunch Make Us Ill?

I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.

I With the free lunch, there is a mental distance between whatwe know and what we can show.

I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Does the Free Lunch Make Us Ill?

I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.

I With the free lunch, there is a mental distance between whatwe know and what we can show.

I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Does the Free Lunch Make Us Ill?

I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.

I With the free lunch, there is a mental distance between whatwe know and what we can show.

I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Current Psychonomic Society Guidelines

I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.

I Many of the current guidelines are reasonable when takenindividually

I Unaddressed is the free-lunch problem, which is a matter ofprinciple.

I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

Current Psychonomic Society Guidelines

I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.

I Many of the current guidelines are reasonable when takenindividually

I Unaddressed is the free-lunch problem, which is a matter ofprinciple.

I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

Current Psychonomic Society Guidelines

I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.

I Many of the current guidelines are reasonable when takenindividually

I Unaddressed is the free-lunch problem, which is a matter ofprinciple.

I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

Current Psychonomic Society Guidelines

I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.

I Many of the current guidelines are reasonable when takenindividually

I Unaddressed is the free-lunch problem, which is a matter ofprinciple.

I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”

Jeffrey N. Rouder

Bayesian Inference in Psychology

My Guidelines

I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.

I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.

I Good analysts are transparent in and responsible for theirchoices.

I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

My Guidelines

I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.

I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.

I Good analysts are transparent in and responsible for theirchoices.

I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

My Guidelines

I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.

I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.

I Good analysts are transparent in and responsible for theirchoices.

I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

My Guidelines

I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.

I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.

I Good analysts are transparent in and responsible for theirchoices.

I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

My Guidelines

I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.

I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.

I Good analysts are transparent in and responsible for theirchoices.

I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.

Jeffrey N. Rouder

Bayesian Inference in Psychology

Thank You

Jeffrey N. Rouder

Bayesian Inference in Psychology