Variables, Measurement

Issues, & Psychometrics

Chapters 5 & 7

Experimentation Experimental Hypothesis

Experiment

Experimental Procedure

Experimental Variables Variable

An event or characteristic that can take on at least two values

Independent Variable The variable in a study that is being

manipulatied

Subject Variable A characteristic. Attribute of a participatnt

Dependent VariableThe variable in a study that is observed or

measured

Extraneous Variables Variables out of control

Experimental Conditions Treatment Conditions

Levels of the independent variable (number of groups)

Common Treatment Conditions Experimental Condition: exposed to the IV Control Condition: Not exposed to the IV Placebo Control Condition

The group that is exposed to an inert substance or object that is similar to the independent variable but has no effect

Experimental Design In a well designed experiment, the only

difference between the experimental and control conditions is the independent variable

Levels of the Independent Variable Basic Experiment

Two groups: experimental and Control Condition Two levels of the independent variable

Multiple Levels of the Independent Variable Most research manipulates the IV at more than

two levels Allows us to better understand the influence of

the IV

ExamplesCaffeine on Alertness

Group 1: 8 oz coffeeGroup 2 : 24 oz coffeeGroup 3 : no coffee

Screen size on Reaction Time

Group 1: 17 inchesGroup 2: 19 inchesGroup 3 : 21 inches

Selecting Dependent Variables Behavioral Measures

Measures that investigate behavior Attitudinal Measures

Measures that assess one’s attitudes about a topic

Cognitive Measures Measures of one’s mental ability or knowledge

of a topic

Physiological Measures Measures that are biological in nature (ie, pulse

rate)

Multiple Dependent Measures Most studies assess more than one dependent

variable to increase the amount of information collected

Measuring Dependent Variables Measurement

The assignment of numerical values to a variable of interest

Levels of Measurement

Measuring Dependent Variables Restriction of Range

Scores cluster at the top or bottom of the scale Floor Effects: scores cluster at the bottom of the

scale Ceiling Effects: scores cluster at the top of the

scale

Operational Definitions Operational Definition

Procedure by which a variable is defined solely in terms of the observable procedures used to manipulate and measure it

Variables can be very broad; definition makes it more specific

Hypothetical Constructions Many variables of interest in psychology are

hypothetical constructs that cannot be directly observed

Operationalizing variables is critical for: Careful measurement Replication

Psychometrics Reliability

The consistency of a measure

Validity The accuracy of a measure

Measure assesses intended construct Measure assesses all aspcts of the intended construct Measure does not assess unintended constructs

Reliability is a necessary, yet insufficient condition for validity An unreliable measure is likely invalid A reliable measure is not a guarantee of validity

Measurement Error Observed Score

Actual score obtained from measurement observed score= true score + measurement

True Score Actual Degree of the trait of a person possesses

Measurement Error Random Error

Fluctuates with ach measurement May result in unreliable, invalid measurements

Systematic Error Present every time a measurement is made May result in reliable, yet invalid measurements

Assessing Reliability Consistency Across Time

Test-Retest Reliability

Consistency Across Forms Alternate/Parallel Forms Reliability

Inter-Rater (Inter-Observer) Reliability

Consistency Among Items Split-Half Reliability

Inter-Item Reliability (Internal Consistency)

Example: Inter-Rater Reliability Two managers rate an employee on

three skills needed for the job. Are the ratings reliable: consistent

between raters?

Example: Test-Retest Reliability A researcher develops a new

intelligence measure and wants to test its reliability. Are the scores reliable: consistent over

time?

Example: Inter-Item Reliability A researcher develops a 4-item survey

to measure brand loyalty and wants to test its reliability. Are the scores reliable: consistent among

items?

Validity of the IV Manipulation Check

An experiment is designed to test the effects of social status on assertiveness.

Types of Validity Statistical Conclusion Validity

Internal Validity

Construct Validity

External Validity

Measurement of Test Validity Content Validity

A researcher gives a test to measure personality. A group of SMEs judge it’s

content validity. Does the test content seem to be measuring personality?

Low Content Validity High Content Validity

Measurement of Test Validity Criterion-Related Validity

Construct Validity

Example: Criterion Validity ETS uses the GRE to predict students’

readiness in college. To test the validity of the GRE, scores may be compared to students’ GPAs. Is the GRE a good predictor of readiness?

Do scores on the GRE predict GPA?

Example: Criterion Validity A researcher develops a new measure

of sales aptitude and wants to test whether it is a good measure of sales performance. Can scores on the sales aptitude test predict

dollar sales?

Example: Construct Validity A researcher develops a new self-report

measure of anxiety and wants to test its construct validity. Does the test measure anxiety?