Validity and Validity and ReliabilityReliability

EAF 410EAF 410

July 9, 2001July 9, 2001

ValidityValidityDegree to which evidence Degree to which evidence

supports inferences madesupports inferences madeAppropriateAppropriateMeaningfulMeaningfulUsefulUseful

Appropriate Appropriate InferenceInference

Relevant to the studyRelevant to the study

Meaningful Meaningful InferenceInferenceSays something about Says something about the meaning of the the meaning of the information obtained information obtained through an instrumentthrough an instrument

Useful InferenceUseful InferenceHelps researchers Helps researchers make decisions related make decisions related to what they studiedto what they studied

Do we get useful Do we get useful information about information about the topic/variables?the topic/variables?

Validity depends on:Validity depends on:The amount and type The amount and type of evidence supporting of evidence supporting the interpretationsthe interpretations

Types of EvidenceTypes of EvidenceContent-related Content-related Criterion-relatedCriterion-relatedConstruct-relatedConstruct-related

Content-relatedContent-relatedAppropriate?Appropriate?Comprehensive?Comprehensive?Logical?Logical?Adequate?Adequate?

Adequacy of the samplingAdequacy of the sampling• representative of content?representative of content?

Format of the instrumentFormat of the instrument• language, directions, etc.language, directions, etc.

Who judges the validity of Who judges the validity of the content?the content?• InterpretationInterpretation

• judgementjudgement

Criterion-relatedCriterion-relatedRelationship between scores Relationship between scores

from different instrumentsfrom different instrumentsStrong relationship?Strong relationship?Accurate?Accurate?

Instrument results compared Instrument results compared with independent criterionwith independent criterion• a second test or measurementa second test or measurement

Predictive validityPredictive validity• time between measurementstime between measurements

Concurrent validityConcurrent validity• instrument and criterion instrument and criterion

measurements at same timemeasurements at same time

Validity CoefficientValidity Coefficient• r r = correlation coefficient= correlation coefficient

• degree of relationship degree of relationship between scores from between scores from instrument and criterioninstrument and criterion

• Between -1.00 and +1.00Between -1.00 and +1.00

• .00 = no relationship.00 = no relationship

Construct-relatedConstruct-relatedHow well does the How well does the

theoretical construct account theoretical construct account for or explain differences?for or explain differences?

Is the construct sufficient?Is the construct sufficient?

Three steps:Three steps:• variable measured clearly variable measured clearly

defineddefined

• hypotheses (based on theory)hypotheses (based on theory)

• hypotheses tested logically hypotheses tested logically and empiricallyand empirically

Collect variety of different Collect variety of different types of evidencetypes of evidence

ReliabilityReliabilityConsistency of the scoresConsistency of the scores

• among individualsamong individuals

• among itemsamong items

Reliable -- valid or not Reliable -- valid or not validvalid

Not reliable -- not validNot reliable -- not validValidity and reliability Validity and reliability

dependent on the context dependent on the context in which the instrument is in which the instrument is used.used.

Measurement ErrorMeasurement ErrorSame test, different scoresSame test, different scoresCaused by variety of factors:Caused by variety of factors:

• motivationmotivation

• anxietyanxiety• environmentenvironment

• etc.etc.

Reliability CoefficientReliability CoefficientReliability estimatesReliability estimatesRelationship between scores Relationship between scores

at two different times, orat two different times, orRelationship between two Relationship between two

parts of the same instrumentparts of the same instrument

Test-RetestTest-RetestSame test, same group, two Same test, same group, two

timestimesReliability coefficient Reliability coefficient

indicates relationship indicates relationship between the two scoresbetween the two scores

Equivalent-FormsEquivalent-FormsTwo different, equivalent Two different, equivalent

tests, same grouptests, same groupReliability coefficient Reliability coefficient

calculatedcalculated

Internal-ConsistencyInternal-ConsistencySingle administration of the Single administration of the

instrumentinstrumentSplit-half procedureSplit-half procedureKuder-Richardson approachKuder-Richardson approachAlpha coefficientAlpha coefficient

Split-Half ProcedureSplit-Half ProcedureScore two halves of test Score two halves of test

separatelyseparatelyDegree to which the two halves Degree to which the two halves

provide same resultsprovide same resultsReliability increases with length Reliability increases with length

of the test/instrumentof the test/instrument

Standard Error of Standard Error of MeasurementMeasurement

Index showing how much a Index showing how much a measurement would vary measurement would vary under changed circumstancesunder changed circumstances