Lec02-1.p466.a15

Finish: History of JDMBegin: Linear Judgment ModelsPsychology 466: Judgment & Decision MakingInstructor: John Miyamoto 10/06/2015: Lecture 02-2Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros arent needed to view the slides. You can disable or delete the macros without any change to the presentation. 1OutlineBrief history of the psychology of decision makingCapacity limitations in human cognitive processesFour linear judgment modelsWhy are psychologists interested in linear judgment models?How can we make decisions based on a linear judgment model?Psych 466, Miyamoto, Aut '152History of Psych of Decision MakingPsych 466, Miyamoto, Aut '153History of the Psychology of Decision MakingVictorian rationality, Freudian irrationality, behaviorist arationality.Expected utility theory (Von Neumann & Morgenstern, 1944)Rational agent model of economic behaviorHeuristics and biases movement, 1970 1990 (approx.)Reactions to heuristics and biases movementEvolutionary psychology, ecological psychology, naturalistic decision making, Bayesian models of psychological processesPsychology of happinessSeparate development neuroscience of decision making (current hot topic!)

The Cognitive Approach to Judgment & Decision Making3Psych 466, Miyamoto, Aut '154The Cognitive Approach to Judgment & Decision MakingCognitive limitations limitations on human cognitive capacity affect judgment and decision makingHeuristics and biases movement: 1970 1990 (approx.)Reactions to heuristics and biases movementEvolutionary psychologyEcological psychologyNaturalistic decision makingBayesian models of psychological processesEmotion in decision processes The Standard Memory Model4The Standard Cognitive Model of Human MemoryPsych 466, Miyamoto, Aut '155Sensory RegistersH&DFig. 1.1Sensory Input BuffersWorking MemoryCentral ExecutivePhonological BufferGoalStackVisuospatial BufferLong-Term Memory5Psych 466, Miyamoto, Aut '156Sensory registers retain the sensory information for very brief periods of time.Working MemoryH&DFig. 1.1Sensory Input BuffersWorking MemoryCentral ExecutivePhonological BufferGoalStackVisuospatial BufferLong-Term Memory6Psych 466, Miyamoto, Aut '157Working memory (WM) holds a limited amount of information for 10 20 seconds. Thoughts are actively manipulated in WM. Long-Term MemorySensory Input BuffersWorking MemoryCentral ExecutivePhonological BufferGoalStackVisuospatial BufferLong-Term MemoryH&DFig. 1.17Psych 466, Miyamoto, Aut '158Long-term memory (LTM) retains information over longer periods of time. LTM interacts with WM. General Hypothesis of Cognitive ResearchSensory Input BuffersWorking MemoryCentral ExecutivePhonological BufferGoalStackVisuospatial BufferLong-Term MemoryH&DFig. 1.18Psych 466, Miyamoto, Aut '159General Hypothesis of Cognitive ResearchLimitations in working memory impose limitations on human ability to engage in complex reasoning.Decision making requires complex reasoning. Basic Message: Cognitive Limitations Produce SimplificationsH&DFig. 1.1Sensory Input BuffersWorking MemoryCentral ExecutivePhonological BufferGoalStackVisuospatial BufferLong-Term Memory9Psych 466, Miyamoto, Aut '1510Working Memory (WM) Has Severe Capacity LimitationsWM can only hold a limited number of "chunks" of information.Information is lost from WM fairly quickly (within ~20 sec.)When information is complex, people are forced to simplify the reasoning process. Simplifications can lead to distortions.EXCEPTION: Experience can teach one to integrate specific types of complex information but only in some cases. Example: Expert chess players can reason about complicated chess problems.Example: Experienced drivers can understand traffic situations that are actually very complex.

Same Slide but with Additional Comment re the Role of AttentionPsych 466, Miyamoto, Aut '1511Working Memory (WM) Has Severe Capacity LimitationsWM can only hold a limited number of "chunks" of information.Information is lost from WM fairly quickly (within ~20 sec.)When information is complex, people are forced to simplify the reasoning process. Simplifications can lead to distortions.Limited WM implies that attention plays a central role in cognition. Misallocation of attention can lead one to overlook important relevant info.Complex information processes have many points at whichcognitive limitations can exert an influence. Where Are We in this Lecture?Psych 466, Miyamoto, Aut '1512Where Are We in the Lecture?Normative and prescriptive decision models require complex representations and processingCognitive limitations cause us to simplify decisions,and this can produce errorsNEXT: How to Deal with Cognitive Complexity Intuitive clinical judgment versus statistical ModelsBrunswiks Lens Model of Human JudgmentLinear models applied to making better choicesApplications to clinical judgmentClinical vs Actuarial JdmtPsych 466, Miyamoto, Aut '1513Intuitive Judgment versus Acturial JudgmentIntuitive judgmentCombine complex information in your headMake decision based on gut feelingActuarial judgment (a.k.a. statistical model or linear model)Base decisions on a statistical decision rule. Intellectual warfare between cognitive psychology and clinical psychology. (Especially in 1950's - 1970's). Statistical Models Outperform Human JudgesExamples of Judgment ProblemsWe will only consider decisions to which intuitive judgment and actuarial judgment (statistical methods) both apply.--------------------------------------------------------------------------E.g., Clinicians attempt to identify patients with progressive brain dysfunction. Data = intellectual test resultsExperienced clinicians achieved 58% correct detection of new cases.Statistical model achieved 83% correct detection of new cases.E.g., Bank loan officer must decide which loan applications are good risks and which are bad risks. E.g., Professors must decide which applicants will do well in grad school and which will not do well. Psych 466, Miyamoto, Aut '1514Critique of Clinical Judgment What Is It?Critique of Clinical JudgmentClinical insight does it exist? Clinical judgment what is it good for?Clinical judgment what are its weaknesses?Accusation: Belief in the efficacy of intuitive clinical judgment is a cognitive conceit.

Psych 466, Miyamoto, Aut '1515General Finding: Stat Models Outperform Human JudgesPsych 466, Miyamoto, Aut '1516General Finding: Stat Models Outperform Human JudgesStatistical models almost always outperform the human judgeson clearly defined decision tasks. Human cognitive processes are good at noticing particular pieces of information. Does my friend look happy? Sad? Stressed? Irritated?Is the patient nervous? Defensive? Exhibitionistic? Human cognitive processes are not good at integrating multiple pieces of information.Can I predict how my friend will feel about a surprise party?Can the clinician predict how the patient will progress after 4 months of therapy? Implications of this Lecture / ENDPsych 466, Miyamoto, Aut '1517Implications of this TopicWe can improve human decisions by stressing what humans are good at:... noticing what are important issues that are relevant to a decision;... evaluating how good or bad is an outcome on a specific dimension;while avoiding what we are not good at: ... combining complex information in our heads.Know thyself Make better decisions Brunswik's Lens ModelPsych 466, Miyamoto, Aut '1518The Lens Model of Egon BrunswikTo-be-judged criterion = the thing you are trying to predict.

Cues = things you can observe about the criterionExamples of Judgment Problems: Explain Idea of Criterion, Cues & JudgmentFigure 3.1 of Hastie & Dawes.Judgment = the judges prediction (you are the judge)The lens model is a conceptualization of the structure of typical judgment problems. Psych 466, Miyamoto, Aut '1519

Examples of Judgment Problems

Brief Digression: Definition of Holistic JudgmentPsych 466, Miyamoto, Aut '1520Holistic JudgmentHolistic judgment - judgment of a complex trait from multiple cues by means of a single intuitive judgment as opposed to a calculation based on a formula. Example of Holistic Judgment20Psych 466, Miyamoto, Aut '1521Example of Holistic JudgmentSuppose a clinician has to decide whether a patient is suicidal and should be hospitalized. He considers patients appearance, what patient says, the patients background and record, previous interactions, etc. Eventually clinician makes a decision based on an intuitive integration of all this information. ------------------------------------------------------------------------- Note: Holistic judgment may include stages in which the judge considers component features of the decision.What makes it a holistic judgment is that the ultimate evaluation is made by the judge through an intuitive integration of all of the information about the decision. Digression on the Meaning of Holism21Psych 466, Miyamoto, Aut '1522HolismHolistic judgments are sometimes called global judgments to distinguish them from judgments that evaluate the separate cues.

Holism is sometimes written as wholism.Holistic judgment strategies are contrasted with analytical judgment strategies.

Analytical judgment strategy: Break the decision into component parts; use an explicit calculation to combine these parts. Claims Regarding Holistic & Analytical Judgment Strategies22Psych 466, Miyamoto, Aut '1523Claims Regarding Holistic & Analytical Judgment StrategiesClaim: An analytical judgment strategy will generally produce better predictions than a holistic judgment strategy. E.g., clinical judgments are more often correct if they are made analytically than if they are made holistically.E.g., your own predictions of what will happen in sports events, on the stock market, and predictions about social behavior will be more often correct if they are made analytically than if they are made holistically.Related Claim: An analytic judgment strategy is better than an intuitive judgment strategy because it provides a better way to combine complex information. Analytical versus Intuitive Judgment Strategy (cont.)Psych 466, Miyamoto, Aut '1524Do Analytical Judgment Strategies Exclude Human Intuition?

NO!A human judge is still needed to ...... decide what are the important issues in a decision;... judge how good or bad an outcome would be on a particular dimension, e.g., social impact of a decision, health impact of a decision, political impact of a decision, etc.Reminder: Standard Model of Human MemoryPsych 466, Miyamoto, Aut '1525(Reminder) General Hypothesis of Cognitive ResearchLimitations in working memory impose limitations on human ability to engage in complex reasoning.Decision making requires complex reasoning. Four Linear Judgment ModelsH&DFig. 1.1Sensory Input BuffersWorking MemoryCentral ExecutivePhonological BufferGoalStackVisuospatial BufferLong-Term Memory25Psych 466, Miyamoto, Aut '1526Four Linear Judgment ModelsUsing multiple regression on objective data for which the true state is known. Using multiple regression on judgment data where the true state is not known. SMART (Simple Multi-Attribute Rating Technique)Unit Weighting Model--------------------------------------------Except for Model 4 (unit weighting), the models do NOT describe everyday judgment. Some natural models are similar to these models. Using any of these models can improve human performance. Same Slide with Expanded Comments about Each Model26Psych 466, Miyamoto, Aut '1527Four Linear Judgment ModelsUsing multiple regression on objective data for which the true state is known. This method produces a proper linear model that is statistically the best way to generate accurate predictions. Using multiple regression on judgment data where the true state is not known. This method produces a model of the judge (abbreviated as MUD or Model of the jUDge)SMART (Simple Multi-Attribute Rating Technique)This method produces an improper linear model. Unit Weighting ModelThis method produces an improper linear model. Relevance of these Four Models to Assignment 127Relevance of these Four Methods to Assignment 1Four Linear Judgment ModelsProper Linear Model Model of the Judge (MUD)SMART (Simple Multi-Attribute Rating Technique) (a type of improper linear model)Unit Weighting (a type of improper linear model)

Assignment 1: Explain how to use a linear judgment procedure to choose 1 UW course to take. Minimize issues of major or minor requirements. Compare intuitive judgment and linear judgment models.On Assignment 1, you only need to explain 1 of 4 methods.Psych 466, Miyamoto, Aut '1528Introduction to Barons College Admission Decision Problem28Next: Explain These Four Methods on a Concrete Judgment ProblemConcrete Problem: Predicting college performance (GPA) based on information in a college application. Predictions based on a linear model how to produce themPredictions based on a model of the judge how to produce themPredictions based on SMART methodPredictions based on unit weights how to produce themAssignment 1: Pick one of these four methods. Describe this method and then discuss its strengths and weaknesses relative to other methods (including intuitive judgment. Psych 466, Miyamoto, Aut '1529Continue the Intro to Baron's College Admission Problem Psych 466, Miyamoto, Aut '1530Baron's College GPA Judgment ProblemUsing the terminology of the lens model, the GPA judgment problem looks like this:TerminologyExampleCriterion(what we want to predict)COL = College GPA of high school student Cues(these are things we know) GPA = High school GPA SAT = SAT test scores REC = recommendations (converted to ratings) ESS = essay quality (converted to a rating)Judgment(this is our prediction)PRE = Estimate (guess) of students future college GPA Table Showing Data & Variables for the College Admission Example30Psych 466, Miyamoto, Aut '1531Judgment Data Used in Baron's Chapter 20COL = college GPA. This is the criterion. This is what the judge wants to predict.SAT = SAT score; REC = judge's rating of the recommendation; ESS = judge's rating of the student's essay; GPA = high school GPA. These are the cues.

Comment re Qualitative Variables31Psych 466, Miyamoto, Aut '1532Qualitative Variables Must Be Converted To Quantitative VariablesSAT, GPA are already quantitative.REC = strength of recommending letters is qualitative; convert to quant measure by having humans rate the letters for how positive they are.ESS = quality of applicants essay is qualitative; convert to quant measure by having humans rate the essay for how good it is.

Four Methods for Predicting Future Cases32Psych 466, Miyamoto, Aut '1533Four Ways to Compute a Statistical Prediction ModelMethod 1: Multiple regression applied to existing data. Called a proper linear modelMethod 2: Multiple regression applied to a judges predictionsCalled a model of the judgeMethod 3: SMART Method with "importance" weightsCalled the SMART method or importance weighting methodMethod 4: Unit weighting modelCalled the unit weighting model or unit weighting methodMultiple Linear RegressionNextPsych 466, Miyamoto, Aut '1534Method 1: Multiple Linear RegressionMultiple linear regression is a statistical method for finding a formula that predicts the criterion from a set of data.

COL is the criterionThese are the cuesPRE = statistical prediction(predicted college GPA) Prediction Equation from the Multiple Regression34Psych 466, Miyamoto, Aut '1535Prediction EquationPRE = 0.000175SAT + 0.092REC + 0.217ESS + 1.893GPA 5.161

The regression weights for the prediction equation are underlined above:A statistics program can compute the regression weights based on the datain the table. Prediction Equation with 4 Predictor Variables

COL = the criterion = college GPA = to-be-predicted quantitySAT, REC, ESS, GPA are the cues (predictor variables)Example of a Regression Equation35See e:\p466\nts\baron.quant.jdmt.r-code.docm for R-code that computes the prediction from the linear regression.

Psych 466, Miyamoto, Aut '1536Prediction Equation with 4 Predictor VariablesPrediction EquationPRE = 0.000175SAT + 0.092REC + 0.217ESS + 1.893GPA 5.161 Example: If a high school student has SAT = 1200, REC = 3.7, ESS = 3.9, GPA = 3.2, then we predictPRE = 0.000175(1200) + 0.092(3.7) + 0.217(3.9) + 1.893(3.2) 5.161= 2.2933

COL = the criterion = to-be-predicted quantitySAT, REC, ESS, GPA are the cues (predictor variables) How to Use the Multiple Regression Model to Predict New Cases36Psych 466, Miyamoto, Aut '1537

How We Use the Linear Regression Model Linear Model Outperforms the Human JudgeStep 1: Apply multiple regression calculation to data for which the value of the criterion (COL) is known. This produces the prediction equation. Step 2: Use the prediction equation to predict the criterion (COL) for new cases where the value of the criterion is NOT known. 37See e:\p466\nts\baron.quant.jdmt.r-code.doc for R-code that computes the regression equation.

Psych 466, Miyamoto, Aut '1538Linear Model Outperforms Human JudgesIn Barons college GPA example, the statistical model makes more accurate predictions of college GPA than do expert human judges.Same finding on many other examples (see Dawes, Faust & Meehl paper)Is this surprising? Is it interesting from the standpoint of psychology?Who cares? Or who might care? Judges think that they make their judgments by means of a complex, interactive, nonlinear evaluation of the cues. Whether or not the judge's introspections are veridical, the results of these studies show that a much simpler combination of the cues can produce better predictions than the processes used by human judges. Return to Outline of ModelsPsych 466, Miyamoto, Aut '1539Four Ways to Compute a Statistical Prediction ModelMethod 1: Multiple regression applied to existing data. Called a proper linear modelMethod 2: Multiple regression applied to a judges predictionsCalled a model of the judgeMethod 3: SMART Method with "importance" weightsCalled the SMART method or importance weighting methodMethod 4: Unit weighting modelCalled the unit weighting model or unit weighting methodModel of the JudgeNextPsych 466, Miyamoto, Aut '1540Method 2: Model of the Judge Researcher has available the scores for SAT, REC, ESS and GPA.The values of the criterion (COL) are NOT available.Researcher asks the judge to make intuitive, global predictions for these cases. This produces the column labeled "JUD (next slide).

Same Slide Except JUD Column Added to TableNot AvailableNot Available40Psych 466, Miyamoto, Aut '1541Method 2: Model of the Judge Researcher has available the scores for SAT, REC, ESS and GPA.The values of the criterion (COL) are NOT available.Researcher asks the judge to make intuitive, global predictions for these cases. This produces the column labeled "JUD."

Same Slide Except Column Labeled MUD Added to TableNot AvailableNot Available41Psych 466, Miyamoto, Aut '1542Method 2: Model of the Judge Compute a regression model that predicts JUD (Model of the jUDge or MUD). Policy Capturing.

Example: MUD = (5x1018)SAT + 0.1REC + 0.1ESS + 2.0GPA 4.8 Use the MUD model to predict college GPA (COL) for these cases or future cases. Discussion of Model of the Judge

Not AvailableNot AvailableIt is an accident that in this example, JUD and MUD are identical

42See e:\p466\nts\baron.quant.jdmt.r-code.docm for R-code that computes the prediction from the model of the judge.

Psych 466, Miyamoto, Aut '1543Tuesday, October 06, 2015: The Lecture Ended HerePsych 466, Miyamoto, Aut '1544Discussion of Case 2: Model of the Judge (MUD)Empirical findings for Case 2 are the same as for Case 1MUD more accurate than the intuitive judgments of the judge. Judges think that they make their judgments by means of a complex, interactive, nonlinear evaluation of the cues. Whether or not this is really true, studies show that a much simpler combination of the cues produces better predictions. We don't need to know the value of the criterion (COL) in order to find a statistical formula (prediction equation) that can outperform the judge. . Why Does MUD Outperform the Judge?44Psych 466, Miyamoto, Aut '1545Why Does MUD Outperform the Judge?The model of the judge (MUD) is a model of the judges decisions, not of the criterion (reality). The MUD outperforms the judge because the statistical formula is consistent it treats each case by the same formula. A human judge has all sorts of random variations in his or her judgment. These random variations simply increase the inaccuracy (error) of the judge's predictions. Four Methods SMART Method is Next45