Analyzing the task Thomas R. Stewart, Ph.D. Center for Policy Research Rockefeller College of Public...

Analyzing the task

Thomas R. Stewart, Ph.D.Center for Policy Research

Rockefeller College of Public Affairs and PolicyUniversity at Albany

State University of New [email protected]

Public Administration and PolicyPAD634 Judgment and Decision Making Behavior

PAD634 (analyzing-the-task.ppt) 2

This list contains examples of questions that can be addressed in order to analyze judgment and decision problems. Not all questions are relevant for all problems. Furthermore, there may be questions not listed here that are critical for understanding a particular task.

Analyzing the task


Judgment strategies

• Judgment strategies• Model 1 Two step (lens model)

– Linear– Acquire cues, then make judgment

• Model 2 Hypothesis testing, problem solving– Iterative– Make and revise cue judgments and overall

judgments at the same time


Model 1 Two step (lens model)

Image or environment Judge cues

Cue 1

Cue 2

Cue 3

Judgment


Model 2 Hypothesis testing, problem solving

Image or environment

Cue 1

Cue 2

Cue 3

Hypothesis (tentative judgment)

Judge cues to check

hypothesisJudgment


General characteristics

• These characteristics are usually identified by interview, observation, review of descriptive materials.

• What information is available?• Is the amount of information a problem (too much or too little)?• What costs are involved in obtaining information?• Is the task dynamic or static?• Is the problem repetitive or one-time (or infrequent)?• How reliable are the data?• How much redundancy is there in the data?• How is the information obtained/displayed?



• How is the information organized?– Physically– Mentally

• Structuring the problem• Influence diagrams• Mental models



• What judgments and decisions are made?– What alternative choices are available?– Are choices continuous or discrete?



• Who is involved?– Individual– Group

• Consensus?• Hierarchy?



• What moral or ethical issues are involved?• What role does emotion play in the decision

process?• What aspects of the procedures are relevant?

– Time pressure?– Documentation?– Justification?



• How high is task uncertainty?• What are the sources of uncertainty

(aleatory, epistemic)?• What is the base rate?





• What are the opportunities for learning?– Is manipulation possible?– What feedback is available?– How do the judges know if they are

right or wrong?– What is the speed and frequency of

feedback?



• What are the opportunities for learning?– Is manipulation possible?– What feedback is available?– How do the judges know if they are

right or wrong?– What is the speed and frequency of

feedback?


Autopsies – feedback in medicine

Shojania, K. G., & Burton, E. C. (2008). The Vanishing Nonforensic Autopsy. New England Journal of Medicine, 358(9), 873-875.



• How stable are relationships over time?– Cue/Cue relationships– Cue/Criterion relationships



• What are the rewards/penalties for good/bad decisions?– False positives, false negatives, true

positives, true negatives• Analysis of errors

– What might cause errors?– What safeguards are in place to prevent

errors?– What errors have occurred in the past?


Specific characteristics

• These are important in Social Judgment Theory and the Cognitive Continuum Theory.

• They are usually difficult to ascertain without formally structuring the task and collecting data.



• Complexity of task structure– Texture of judgment scale– Number of cues– Vicarious mediation– Cue distribution– Weights– Organizing principle



• Ambiguity of task content– Availability of an organizing

principle– Feedback

• Task outcome available• Other feedback (e.g., cognitive)

– Familiarity with content



• Form of task presentation– Simultaneous or sequential?– Task decomposition– Cognitive decomposition– Type of cue data– Type of cue definition– Response time permitted or

implied


Statistical analysis of task properties

• These overlap with specific characteristics, and can only be done for repetitive judgments.

• Case 1: Cue and criterion data are available– Criterion distribution (base rate)– Cue distributions– Cue interrelations– Cue reliability– Task uncertainty/predictability

• Linear

• Nonlinear

– Weight sensitivity


Statistical analysis of task properties

• For cases 2-4, we have to rely on the judgments of experienced judges to infer task properties that cannot be directly analyzed.– Case 2: Criterion data available, but not cue data– Case 3: Cue data available, but not criterion data– Case 4: No task data available

Date post:	18-Jan-2016
Category:	Documents
Upload:	basil-baker
View:	220 times
Download:	0 times

Analyzing the task Thomas R. Stewart, Ph.D. Center for Policy Research Rockefeller College of Public...

Documents