Validity and Reliability in Quantitative Research

7/27/2019 Validity and Reliability in Quantitative Research

1/57

QuantitativeResearch

March

9,

2011


2/57

Theapproximatetruthofpropositions,

inferences,or

conclusions.


3/57

Sampling

Design

Analysis


4/57


5/57


6/57


7/57

rox ma

m a r y

o e


8/57

Userandomselection,ifpossible,ratherthan

anonrandom

rocedure

Trytoassurethattherespondentsparticipate

rateslow.

Usethetheoryofproximalsimilaritymore

effectivel b a l in techni ues like conce t

mapping


9/57


10/57


11/57


12/57

Simp eRan omSamp ing Tose ectn unitsouto

N suchthat

each

NCn hasan

equal

chance

of

e ngse ec e . sea a eo ran omnum ers,acomputerrandomnumbergenerator,ora

m c c

v c

c

m . StratifiedRandomSampling Dividethe

popu at on ntononover app nggroups .e.,

strata)

N1,

N2,

N3,

...

Ni,

such

that

N1 +

N2 +

N3 +

...

+ i = . en oas mp eran omsamp eo =n/Nineachstrata.


13/57

.

SystematicRandomSampling

numbertheunitsinthepopulationfrom1toN

k=N/n=theintervalsize

randomlyselectanintegerbetween1tok

thentakeeverykth unit

ClusterRandomSampling

randomlysampleclusters

measureall units

within

sampled

clusters

MultiStageSampling


14/57

Accidental,HaphazardorConvenienceSampling Traditional

"person

on

the

streetinterviews conducted

frequently

by

opinion.Choiceofstudentsbecauseitisconvenientisalsoan

exam leofthist eofsam lin .

PurposiveSampling

We

sample

with

a

purpose in

mind.

We

usuallywouldhaveoneormorespecificpredefinedgroups

weareseeking.Forinstance,haveyoueverrunintopeoplein

amalloronthestreetwhoarecarryingaclipboardandwho

ares opp ngvar ouspeop ean as ng eycou n erv ew

them?Mostlikelytheyareconductingapurposivesample.


15/57

PurposiveSamplingTypes

ModalInstance

Sam lin

T ical

Voter

ExpertSampling

Proportional(Proportionaltothepopulation)

onproport ona noug to ot estat st ca tests

Heterogeneity

Sampling

(Wide

Arrays) SnowballSampling(Choosesomeoneandaskhim

torecommendmore)


16/57


17/57

Constructvalidityreferstothedegreeto

whichinferences

can

le itimatel

be

made

fromtheoperationalizations inyourstudyto

operationalizations werebased.


18/57


19/57

Externalvalidityinvolvesgeneralizingfrom

ourstud

context

to

other

eo le,

laces

or

times,constructvalidityinvolvesgeneralizing

conceptofyour

program

or

measures.


20/57

Translationvalidity

Facevalidit

Contentvalidity

r er on

re a e

va y Predictivevalidity

Concurrentvalidity

Discriminant validity


21/57

Howaccuratelyyoutranslatedyourconstruct

FaceValidity seewhether"onitsface"it

seems

i e

a

goo

trans ation

o

t e

construct. ContentValidit checktheo erationalization

againsttherelevantcontentdomainforthe

.


22/57

re c veva y npre c veva y,weassess eoperationalization's abilitytopredictsomethingitshouldtheoreticallybeabletopredict.

Concurrentvalidity Inconcurrentvalidity,weassesstheoperationalization's abilitytodistinguishbetweengroupsthatitshouldtheoreticall beabletodistin uishbetween.

Convergent

validity

In

convergent

validity,

we

examine

thedegreetowhichtheoperationalization issimilarto

theoreticallyshouldbesimilarto.

Discriminant validity

In

discriminant validity,

we

examine

e egree ow c eopera ona za on sno s m ar o(divergesfrom)otheroperationalizations thatittheoreticallyshouldbenotbesimilarto.


23/57

Measuresofconstructsthattheoretically

shouldberelated

to

each

other

are,

in

fact,

observedtoberelatedtoeachother(thatis,

correspondenceor

convergence between


24/57

.


25/57

Measuresofconstructsthattheoretically

shouldnotbe

related

to

each

other

are,

in

fact,observedtonotberelatedtoeachother

,

betweendissimilar

constructs)


26/57

.


27/57


28/57

Cronbach andMeehl,1955


29/57

MultitraitMultimethod Matrix

(MTMM)

(Campbelland

Fiske,

1959)


30/57


(MTMM)Contd.

TheReliabilityDiagonal Estimatesofthe

reliabilit

ofeach

measure

in

the

matrix.

You

canestimatereliabilitiesanumberofdifferent

. ., , .

TheValidityDiagonals Correlationsbetweenmeasuresofthesametraitmeasuredusing

different

methods.


31/57


(MTMM)Contd.

T eHeterotra tMonomet o Tr ang es

Theseare

the

correlations

among

measures

thatsharethesamemethodofmeasurement.

TheMonomethod Blocks Theseconsistofall

ofthe

correlations

that

share

the

same

methodofmeasurement.

TheHeteromethod Blocks Theseconsistof

methods.


32/57


33/57

oe c en s n ere a y agona s ou consistentlybethehighestinthematrix. Thatis,

itselfthanwithanythingelse!Thisisuniformlytrueinourexam le.

Coefficientsin

the

validity

diagonals

should

be

significantlydifferentfromzeroandhighenoughtowarrantfurtherinvestigation. Thisisessentiallyevidenceofconvergentvalidity.Allof

criterion.


34/57

. Avaliditycoefficientshouldbehigherthan

valuesl in

in

its

column

and

row

in

the

same

heteromethod block. Inotherwords,(SE

P&P)(SD

Teacher),

(SE

P&P)

(LC

Teacher),

(SE

.

Thisistrueinallcasesinourexample.


35/57

. va ycoe c en s ou e g er ana

coefficientsintheheterotraitmonomethod

.

factorsshouldbestrongerthanmethodsfactors.Notethatthisisnottrueinallcasesinour

example.For

instance,

the

(LC

P&P)

(LC

Teacher)

correlationof.46islessthan(SETeacher)(SDeac er , eac er eac er ,an

Teacher)(LCTeacher) evidencethatthere

,Teacherobservationmethod.


36/57

. Thesamepatternoftraitinterrelationship

shouldbe

seen

in

all

trian les.

The

exam le

clearlymeetsthiscriterion.Noticethatinall

approximatelytwice

as

large

as

the

.


37/57


38/57

Reliabilityhastodowiththequalityof

measurement.In

its

ever da

sense,

reliabilit

isthe"consistency"or"repeatability"ofyour

.


39/57


40/57


41/57


42/57

PilotTesting

DoubleCheckDataBeforeEntering

Triangulation


43/57

Inresearch,thetermreliabilitymeans

"re eatabilit "or

"consistenc ".

A

measure

is

consideredreliableifitwouldgiveusthe

thatwhat

we

are

measuring

isn't

changing!).


44/57


45/57

varianceofthetruescore/thevarianceof

themeasure

Wecan'tcomputereliabilitybecausewe

'

EstimatingReliability

[covariance(X1,X2)]/sd(X1)*sd(X2)


46/57

n er a eror n er server e a yUsedtoassessthedegreetowhichdifferentraters/observersgiveconsistentestimatesofthesamephenomenon.

es e es e a yUsedtoassesstheconsistencyofameasurefromonetimetoanother.

ara e orms e a ty

Usedto

assess

the

consistency

of

the

results

of

two

tests

constructedinthesamewayfromthesamecontentdomain.

Interna Cons stencyRe a tyUsedtoassesstheconsistencyofresultsacrossitemswithinatest.

Inter Rater or Inter Observer


47/57

InterRaterorInterObserver

Reliability

Correlation


48/57

Correlation


49/57


50/57

Ininterna consistencyre ia i ityestimationweuseoursinglemeasurementinstrumenta m n s ere

o

a

group

o

peop e

on

one

occasiontoestimatereliability.

Ineffectwejudgethereliabilityoftheinstrumentby

estimating

how

well

the

items

that

reflect

the

sameconstructy e s m arresu ts.

Wearelookingathowconsistenttheresultsarefor

different

items

for

the

same

construct

within

themeasure.


51/57

AverageInteritemCorrelation

SplitHalfReliability

Cronbach's Alpha

(a)


52/57


53/57


54/57

'


55/57

'


56/57


57/57

Date post:	02-Apr-2018
Category:	Documents
Upload:	stgoxford24
View:	217 times
Download:	0 times

Validity and Reliability in Quantitative Research

Documents