Post on 19-Jun-2020
transcript
1 Lec1:Introduction...............................................................................................................................................................91.1 Levelofmeasurement........................................................................................................................................91.2 Centraltendency.................................................................................................................................................91.3 Normaldistribution............................................................................................................................................91.4 ANOVA...............................................................................................................................................................9
1.4.1 F-testvsT-test.......................................................................................................................................................101.4.2 Flexibleandpowerfultool.....................................................................................................................................10
1.5 Wheredoesknowledgecomefrom?.................................................................................................................101.6 Scientificmethod..............................................................................................................................................10
1.6.1 Inanutshell...........................................................................................................................................................101.6.2 Hypotheses,TheoriesandLaws............................................................................................................................10
1.7 Theory..............................................................................................................................................................111.7.1 Induction/InductiveReasoning..............................................................................................................................111.7.2 Deduction/DeductiveReasoning...........................................................................................................................11
1.7.2.1 Usedin............................................................................................................................................................................111.7.3 TheScientificMethodassumes.............................................................................................................................111.7.4 TheMethodischaracterisedby............................................................................................................................11
1.8 TheMethodinanutshell..................................................................................................................................111.9 Whatmakesagoodtheory...............................................................................................................................121.10 AScientificTheoryMustBeTestable.................................................................................................................121.11 AScientificTheoryshouldbeRefutable............................................................................................................12
1.11.1 ThelogicofRefutation......................................................................................................................................121.11.1.1 Introtologic....................................................................................................................................................................121.11.1.2 Anexampleofavalidlogicalinference..........................................................................................................................131.11.1.3 Anothervalidlogicalinference.......................................................................................................................................131.11.1.4 Denyingtheantecedent..................................................................................................................................................131.11.1.5 Affirmingtheconsequent...............................................................................................................................................141.11.1.6 Refuteit..........................................................................................................................................................................14
1.12 Thusitiswithscience….....................................................................................................................................142 Lec2:Experimentaldesign,VariablesandOperationalisation.............................................................................................15
2.1 ObjectivesofPsychologicalResearch................................................................................................................152.2 HowtoConductResearch.................................................................................................................................15
2.2.1 MajorMethodologicalApproaches.......................................................................................................................152.2.2 CategorizingResearchApproaches.......................................................................................................................15
2.3 Quantitative.....................................................................................................................................................152.4 TheVariable–akeyconceptinQuantitativeResearch.....................................................................................15
2.4.1 Whichofthesearevariables.................................................................................................................................162.4.2 VariablesinQuantitativeResearch........................................................................................................................162.4.3 Fundamentalquestion...........................................................................................................................................162.4.4 Othervariablesinquantitativeresearch...............................................................................................................16
2.4.4.1 ExtraneousVariables......................................................................................................................................................162.4.4.1.1 Confoundingvariable.................................................................................................................................................17
2.4.4.2 MediatingVariable/InterveningVariable......................................................................................................................172.4.4.3 ModeratingVariable.......................................................................................................................................................17
2.5 Theresearchproblem/question........................................................................................................................172.6 Workthroughanexample................................................................................................................................172.7 Aninterestingphenomena................................................................................................................................17
2.7.1 Isthereatheory?...................................................................................................................................................172.7.2 Theory/prediction/question..............................................................................................................................182.7.3 Researchquestion.................................................................................................................................................182.7.4 Howdoyouanswerthisquestion?.......................................................................................................................182.7.5 Somewaystoanswerresearchquestions.............................................................................................................182.7.6 Possiblestudyideas...............................................................................................................................................18
2.7.6.1 Gotoanightclubandasksomequestions.....................................................................................................................192.7.6.2 AKeyCharacteristicofScientificResearch.....................................................................................................................192.7.6.3 Howaboutoperationalisingattractiveness?..................................................................................................................192.7.6.4 Whatvalidinferencescanwedrawfromthis?...............................................................................................................19
2.8 Correlation........................................................................................................................................................202.8.1 TheissueofCausation...........................................................................................................................................20
2.8.2 InferringCausality..................................................................................................................................................202.9 Anexperimentshouldbe…...............................................................................................................................202.10 Someimportantethicalissues..........................................................................................................................202.11 ExperimentalApproach.....................................................................................................................................21
2.11.1 Advantages........................................................................................................................................................212.11.2 Disadvantage.....................................................................................................................................................21
2.12 ExperimentalResearchSettings........................................................................................................................212.12.1 InternetExperiments........................................................................................................................................212.12.2 Fieldexperiments..............................................................................................................................................212.12.3 Laboratoryexperiments....................................................................................................................................21
2.12.3.1 DifferentwayswecouldmanipulateIVs........................................................................................................................212.12.3.1.1 Beergogglesexperiment1.......................................................................................................................................222.12.3.1.2 Beergogglesexperiment2.......................................................................................................................................22
2.12.3.2 DifferentwayswecouldmanipulateIVs........................................................................................................................222.12.3.2.1 Beergogglesexperiment3.......................................................................................................................................22
2.12.3.3 DifferentwayswecouldmanipulateIVs........................................................................................................................222.12.3.3.1 Beergogglesexperiment4.......................................................................................................................................22
2.12.4 Potentialmanipulations....................................................................................................................................223 Lec3:Sampling,ValidityandReliability..............................................................................................................................22
3.1 TheissueofCausation......................................................................................................................................233.2 Findsomeparticipants......................................................................................................................................23
3.2.1 Somekeyterms.....................................................................................................................................................233.2.2 Aimofsampling.....................................................................................................................................................233.2.3 Representativeness...............................................................................................................................................233.2.4 Samplingbias.........................................................................................................................................................243.2.5 Samplingprocedures.............................................................................................................................................24
3.2.5.1 Probabilitysampling.......................................................................................................................................................243.2.5.2 Sub-typesofprobabilitysampling..................................................................................................................................24
3.2.5.2.1 Simplerandomsample...............................................................................................................................................243.2.5.2.2 Systematicrandomsample........................................................................................................................................243.2.5.2.3 Stratifiedsampling.....................................................................................................................................................25
3.2.5.2.3.1 SimpleRandomSamplingVersusStratifiedSampling........................................................................................253.2.5.2.4 Multi-stageClustersampling.....................................................................................................................................253.2.5.2.5 Multi-Stage/Multi-PhaseSampling............................................................................................................................263.2.5.2.6 AdvantagesofProbabilitySampling...........................................................................................................................263.2.5.2.7 Problemwithprobabilitysampling............................................................................................................................26
3.2.5.3 Non-probabilitysampling...............................................................................................................................................263.2.5.3.1 ConvenienceSamples................................................................................................................................................263.2.5.3.2 SnowballSampling.....................................................................................................................................................263.2.5.3.3 QuotaSample.............................................................................................................................................................273.2.5.3.4 Purposive/judgmentsampling...................................................................................................................................27
3.2.5.4 WhichSamplingMethod?...............................................................................................................................................273.2.6 Howmanypeopleshouldyoutest?......................................................................................................................27
3.2.6.1 DeterminingSampleSize1.............................................................................................................................................273.2.6.2 DeterminingSampleSize2.............................................................................................................................................283.2.6.3 DeterminingSampleSize3.............................................................................................................................................28
3.3 Makesomemeasurements...............................................................................................................................283.3.1 OperationalisationofIVsandDVs.........................................................................................................................283.3.2 ReliabilityandValidity...........................................................................................................................................283.3.3 ReliabilityandValidity...........................................................................................................................................28
3.3.3.1 Therelationshipbetweenreliabilityandvalidity............................................................................................................293.3.3.2 Reliability........................................................................................................................................................................293.3.3.3 TypeofReliabilitytest.....................................................................................................................................................29
3.3.3.3.1 Test-retestreliability..................................................................................................................................................293.3.3.3.1.1 ProblemwithTest-retest....................................................................................................................................29
3.3.3.3.2 Split-halfreliability:isyourmeasureinternallyconsistent........................................................................................293.3.3.3.2.1 Cronbach'sAlpha................................................................................................................................................30
3.3.3.3.3 Inter-raterorinter-observerreliability......................................................................................................................303.3.3.3.3.1 Calculationofinter-raterreliability....................................................................................................................30
3.3.3.4 Validity............................................................................................................................................................................303.3.3.4.1 TypesofValidity.........................................................................................................................................................30
3.3.3.4.1.1 FaceValidity........................................................................................................................................................313.3.3.4.1.2 Contentvalidity..................................................................................................................................................31
3.3.3.4.1.3 Criterion-relatedvalidity.....................................................................................................................................313.3.3.4.1.3.1 Criterion-relatedvalidity:Concurrentvalidity............................................................................................313.3.3.4.1.3.2 Criterion-relatedvalidity:PredictiveValidity..............................................................................................31
3.3.3.4.1.4 ConstructValidity...............................................................................................................................................323.3.3.4.1.4.1 Convergentvalidity.....................................................................................................................................323.3.3.4.1.4.2 Divergentvalidity........................................................................................................................................32
4 Lec4:ExperimentalDesignsandControl.............................................................................................................................334.1 TheExperiment.................................................................................................................................................334.2 InternalvsExternalValidity..............................................................................................................................33
4.2.1 FourStepstoInternalValidity...............................................................................................................................334.3 SomeWeakExperimentalDesigns....................................................................................................................33
4.3.1 Whatdolearnfromthis?.......................................................................................................................................344.3.2 ManipulationoftheIV...........................................................................................................................................344.3.3 IVsandStudyDesign.............................................................................................................................................34
4.3.3.1 FactorialDesign–isn’tthatabitcomplicated?..............................................................................................................344.3.3.2 FactorialDesigns.............................................................................................................................................................344.3.3.3 FactorialDesignLayoutExample....................................................................................................................................354.3.3.4 FactorialDesignNotation...............................................................................................................................................354.3.3.5 WeaknessesofFactorialDesigns....................................................................................................................................36
4.4 StrongExperimentalDesigns.............................................................................................................................364.4.1 Variability...............................................................................................................................................................36
4.5 SeparateandCompress....................................................................................................................................364.5.1 Separate.................................................................................................................................................................36
4.5.1.1 Separation:achievedbyIVOperationalisation...............................................................................................................364.5.1.2 DeterminingLevelsofIV.................................................................................................................................................36
4.5.2 Compress...............................................................................................................................................................364.5.2.1 Compression:IsachievedbycontrollingExtraneousVariables......................................................................................37
4.5.3 ExtraneousVariables:BGvsRM............................................................................................................................374.5.3.1 ThetroublewithBGdesigns...........................................................................................................................................374.5.3.2 Selection.........................................................................................................................................................................37
4.5.3.2.1 potentialforbiastoconfoundresults........................................................................................................................374.5.3.2.2 RandomAssignment/Allocation...............................................................................................................................374.5.3.2.3 Matching....................................................................................................................................................................37
4.5.3.2.3.1 IndividualMatching............................................................................................................................................374.5.3.2.3.2 DistributionMatching.........................................................................................................................................384.5.3.2.3.3 DifficultieswithMatching...................................................................................................................................38
4.5.3.2.4 AlternativelybuildtheEVintothedesign..................................................................................................................384.5.3.3 Couldusethesameparticipants.....................................................................................................................................384.5.3.4 ProblemswithRMDesigns.............................................................................................................................................38
4.5.3.4.1 Problem:OrderEffects...............................................................................................................................................384.5.3.4.1.1 Possiblesolution:Counterbalancing..................................................................................................................39
4.5.3.4.1.1.1 DifferentFlavoursofCounterbalancing......................................................................................................394.5.3.4.1.2 CarryOverEffects...............................................................................................................................................394.5.3.4.1.3 Counterbalancingisnotalwayspossible............................................................................................................394.5.3.4.1.4 AspecialCase:TimeasanIV..............................................................................................................................39
4.5.3.4.1.4.1 Maturation(internalevents)......................................................................................................................404.5.3.4.1.4.2 History(Externalevents).............................................................................................................................404.5.3.4.1.4.3 StatisticalRegression..................................................................................................................................404.5.3.4.1.4.4 Mortality.....................................................................................................................................................40
4.5.4 OtherthreatstoanExperiment’sValidity.............................................................................................................404.5.4.1 ExperimenterEffects.......................................................................................................................................................404.5.4.2 ParticipantEffects...........................................................................................................................................................41
4.5.4.2.1 ControlofParticipantEffects.....................................................................................................................................414.5.4.3 SituationalEffects...........................................................................................................................................................42
4.5.5 ControlGroups......................................................................................................................................................424.6 SoWGorBGinanutshell?................................................................................................................................42
5 Lec5:ReviewofDescriptiveStatisticsandHypothesis........................................................................................................435.1 DescriptivevsInferentialStatistics....................................................................................................................435.2 Thethingaboutequationsis…..........................................................................................................................435.3 Imaginewehaveasetofdata…........................................................................................................................43
5.3.1 Characterisingadataset.......................................................................................................................................435.3.2 TheNormalDistribution........................................................................................................................................43
5.3.3 TheShapeofDistributions-Modality...................................................................................................................435.3.4 TheShapeofDistributions:Kurtosis......................................................................................................................435.3.5 TheShapeofDistributions:skew..........................................................................................................................435.3.6 Forthemoment,wearejustgoingtoassume“normality”..................................................................................445.3.7 Sigmaå&Mean....................................................................................................................................................455.3.8 Equation................................................................................................................................................................465.3.9 df............................................................................................................................................................................46
5.4 ThePurposeofMeansandSD’s........................................................................................................................465.5 ThePurposeofMeansandSD’s........................................................................................................................465.6 DescriptivevsInferentialStatistics....................................................................................................................475.7 Howwedoinferentialstatistics........................................................................................................................475.8 Outlier..............................................................................................................................................................475.9 NormalDistribution..........................................................................................................................................475.10 Z:theStandardisedNormalDistribution...........................................................................................................47
5.10.1 TheZ-score........................................................................................................................................................475.10.2 TheZ-test..........................................................................................................................................................485.10.3 Whatwearedoinghereisaveryspecialcaseofhypothesistesting...............................................................48
5.11 T-tests...............................................................................................................................................................495.11.1 Whatist?..........................................................................................................................................................495.11.2 Thetdistributionvs.normaldistribution.........................................................................................................505.11.3 TypesofT-tests.................................................................................................................................................50
5.11.3.1 Howaboutcomparingdifferentgroups?........................................................................................................................505.11.3.1.1 IndependentGroups................................................................................................................................................50
5.12 Summaryofformulas........................................................................................................................................515.13 Error&StatisticalSignificance...........................................................................................................................515.14 Makingandcheckingassumptions....................................................................................................................515.15 AfinalnoteonStatisticalSignificance...............................................................................................................51
6 Lec6:MultipleGroupsDesigns&AnalysisofVariance(ANOVA).........................................................................................536.1 Whymorethan2groups...................................................................................................................................53
6.1.1 Morethantwogroupsofinterest.........................................................................................................................536.1.2 Examiningmultipletreatments.............................................................................................................................536.1.3 De-confoundingastudy.........................................................................................................................................536.1.4 Refiningourunderstanding...................................................................................................................................536.1.5 Lookingfornatureofrelationships.......................................................................................................................546.1.6 IVandDVRelationships.........................................................................................................................................546.1.7 SettingyourIVupwithyourDV............................................................................................................................54
6.2 t-tests...............................................................................................................................................................546.3 Analysingmultiplegroupdesigns......................................................................................................................546.4 t-testtoANOVA................................................................................................................................................556.5 TheFratio.........................................................................................................................................................556.6 VariabilityandANOVA......................................................................................................................................56
6.6.1 BGVariability(BG).................................................................................................................................................566.6.2 WGVariability(WG)..............................................................................................................................................566.6.3 VariabilityandANOVA...........................................................................................................................................56
6.7 Hypothesistesting&Fratio..............................................................................................................................566.7.1 ANOVAanalysesvariance,butittellsusaboutmeans.........................................................................................576.7.2 ANOVATable.........................................................................................................................................................576.7.3 ComputingBG/WGvariability...............................................................................................................................576.7.4 Example.................................................................................................................................................................57
6.7.4.1 ComputingWGvariability...............................................................................................................................................576.7.4.2 ComputingBGvariability................................................................................................................................................576.7.4.3 Sumofsquares(SS).........................................................................................................................................................586.7.4.4 SStotal................................................................................................................................................................................586.7.4.5 Computingtotalvariability.............................................................................................................................................586.7.4.6 Linearmodel...................................................................................................................................................................58
6.7.5 WhattodowithSSs...............................................................................................................................................586.7.6 Df...........................................................................................................................................................................59
6.7.6.1 DeterminingDf................................................................................................................................................................596.7.6.2 dfarealsopartitioned....................................................................................................................................................60
6.7.7 CalculatingMS&Fratio........................................................................................................................................60
6.7.8 ANOVASummaryTable.........................................................................................................................................606.7.9 InterpretingF.........................................................................................................................................................606.7.10 ReportingANOVA..............................................................................................................................................616.7.11 Summaryofanalysis..........................................................................................................................................61
6.8 Takehomepoints.............................................................................................................................................616.9 QuickNotes......................................................................................................................................................616.10 Next2weeks....................................................................................................................................................61
7 Lec7:FdistributionAssumptionsofANOVA.......................................................................................................................627.1 StatisticalHypothesisforIGANOVA..................................................................................................................627.2 SamplingError&Fdistribution.........................................................................................................................627.3 FDistribution....................................................................................................................................................62
7.3.1 F-distributionshape...............................................................................................................................................627.3.2 ProbabilityandtheF-distribution..........................................................................................................................637.3.3 ErrorsinHypothesisTesting..................................................................................................................................637.3.4 Errorrates..............................................................................................................................................................63
7.4 IGANOVAAssumptions....................................................................................................................................637.4.1 TheIndependenceAssumption.............................................................................................................................647.4.2 TheNormalityAssumption....................................................................................................................................64
7.4.2.1 Outliers...........................................................................................................................................................................657.4.2.1.1 Checkingfor“Outliers”..............................................................................................................................................657.4.2.1.2 Outliers:UseaZscore................................................................................................................................................65
7.4.2.2 Whattodowithoutliers.................................................................................................................................................657.4.2.2.1 Transformdatatoremoveoutlier..............................................................................................................................65
7.4.3 TheHomogeneityofVarianceAssumption...........................................................................................................667.4.3.1 Levenestatistics..............................................................................................................................................................667.4.3.2 Dealingwithbreaches.....................................................................................................................................................66
7.4.3.2.1 LoweringtheαLevel..................................................................................................................................................667.4.3.2.2 Distribution-FreeTests...............................................................................................................................................66
7.4.3.2.2.1 MajorRank-Ordertestscorrespondingtomajorparametrictests....................................................................677.4.3.2.2.1.1 Kruskal-WallisOne-WayANOVA.................................................................................................................67
7.4.4 DataTransformations............................................................................................................................................677.4.4.1 Logarithmictransformation............................................................................................................................................67
7.4.5 Stepsindoingatransform.....................................................................................................................................687.4.6 TotransformdatawithSPSS.................................................................................................................................687.4.7 ComputerBootstrapTechniques...........................................................................................................................687.4.8 ComparisonofMethods........................................................................................................................................687.4.9 NormalityandHomogeneitySummary.................................................................................................................697.4.10 Reporting...........................................................................................................................................................69
8 Lec8:PlannedComparisonsandPostHocTests.PowerandEffect.....................................................................................708.1.1 ANOVASummaryTable.........................................................................................................................................708.1.2 Fratiodoesnotpaintthewholepicture...............................................................................................................708.1.3 ApproachestoComparisons..................................................................................................................................70
8.2 Plannedcomparisons........................................................................................................................................718.2.1 AssigningWeightsorCoefficients.........................................................................................................................71
8.2.1.1 Plannedcontrasts...........................................................................................................................................................718.2.1.2 ComplexPlannedComparisons......................................................................................................................................72
8.2.2 TestingtheSignificanceofContrasts.....................................................................................................................728.3 PlannedComparisons.......................................................................................................................................72
8.3.1 AssumptionsofPlannedComparisons..................................................................................................................738.3.2 SPSS&PlannedComparisons................................................................................................................................738.3.3 WriteUpPlannedcontrasts...................................................................................................................................748.3.4 TypeIErrorRates..................................................................................................................................................74
8.4 PostHocComparisons.......................................................................................................................................758.4.1 SPSS&PostHocTests............................................................................................................................................768.4.2 SPSS&PostHocTests(table1).............................................................................................................................768.4.3 SPSS&PostHocTests(table2).............................................................................................................................778.4.4 WriteUpofpost-hocs............................................................................................................................................77
8.5 Summary–whichcomparisontouse................................................................................................................778.6 EffectSize.........................................................................................................................................................77
8.6.1 Etasquared(h2).....................................................................................................................................................78
8.6.1.1 EffectSizeforANOVA.....................................................................................................................................................788.6.1.2 ANOVASummaryTable..................................................................................................................................................788.6.1.3 Etasquared(h2)forourIGANOVA.................................................................................................................................788.6.1.4 Criteriaforassessingh2..................................................................................................................................................78
8.6.2 Cohen’sd...............................................................................................................................................................788.6.2.1 Plannedcontrastsexample.............................................................................................................................................798.6.2.2 Cohen’sdWorkedExample............................................................................................................................................79
8.6.3 InterpretingEffectSize..........................................................................................................................................798.6.4 ReportingEffectSize..............................................................................................................................................798.6.5 Examples................................................................................................................................................................79
8.7 ConsideringErrorsinStatisticalDecisionMaking..............................................................................................808.7.1 MinimisingError....................................................................................................................................................808.7.2 Power.....................................................................................................................................................................808.7.3 Designissues..........................................................................................................................................................818.7.4 PowerandSampleSize..........................................................................................................................................818.7.5 Power,Effect,SampleSize....................................................................................................................................818.7.6 AStrategyforUsingPowerandEffectSize...........................................................................................................81
9 Lec9:RMDesigns...............................................................................................................................................................839.1 Example............................................................................................................................................................839.2 Gettingstarted..................................................................................................................................................839.3 BrainstormingDesign........................................................................................................................................839.4 IVmanipulationwithRMDesign.......................................................................................................................849.5 RMDesigns.......................................................................................................................................................84
9.5.1 IssueswithRMdesigns..........................................................................................................................................859.5.1.1 Remediestoordereffects...............................................................................................................................................85
9.5.1.1.1 Counterbalancing.......................................................................................................................................................859.6 RManalysis.......................................................................................................................................................85
9.6.1 PartitioningSSinRMANOVA................................................................................................................................869.6.2 PartitioningDFinRMANOVA................................................................................................................................869.6.3 ConceptualSSformulaeRMANOVA.....................................................................................................................869.6.4 SSparticipants/subjects.......................................................................................................................................869.6.5 Anapproachtocalculation....................................................................................................................................869.6.6 SSerrororresidual................................................................................................................................................879.6.7 ObtainingourFratio.............................................................................................................................................879.6.8 RMDesignMoreSensitive.....................................................................................................................................879.6.9 TestingSignificance...............................................................................................................................................879.6.10 Assumptions......................................................................................................................................................88
9.6.10.1 BreachesofSphericity.....................................................................................................................................................889.6.10.1.1 TraditionalModel.....................................................................................................................................................88
9.6.10.1.1.1 EpsilonAdjustments.........................................................................................................................................889.6.10.1.1.2 SPSSoutputprovidesEpsilonadjustFtests.....................................................................................................89
9.6.10.1.2 MultivariateModelApproach..................................................................................................................................909.6.10.1.2.1 RMANOVA:Power&EffectSize&Comparisons.............................................................................................909.6.10.1.2.2 FollowUpTestsorPlannedComparisons........................................................................................................919.6.10.1.2.3 RunningaRMANOVAinSPSS..........................................................................................................................919.6.10.1.2.4 TraditionalModelOutput.................................................................................................................................929.6.10.1.2.5 MultivariateApproachOutput.........................................................................................................................929.6.10.1.2.6 ContrastsandPostHocs...................................................................................................................................92
9.6.10.1.2.6.1 OutputforLinearContrasts......................................................................................................................929.6.10.1.2.6.2 OutputforPost-hocTests.........................................................................................................................93
9.6.10.1.2.7 PlannedComparisonsusingSPSSPairedSamplest-tests.................................................................................939.6.10.2 Examplewriteup............................................................................................................................................................93
9.6.11 Summary...........................................................................................................................................................949.6.12 Whichapproachtouse?....................................................................................................................................94
10 Lec10:CorrelationandRegression.....................................................................................................................................9510.1 Correlation........................................................................................................................................................95
10.1.1 CorrelationRevisited.........................................................................................................................................9510.2 examplefromPYB110…....................................................................................................................................95
10.2.1 Calculatingthecorrelationcoefficientr............................................................................................................9610.2.1.1 ZscoresandPearson’sr:themissinglink.......................................................................................................................9610.2.1.2 Sowhatdoesthatmean?...............................................................................................................................................96
10.2.1.3 Correlationonlytellshalfofthestory...........................................................................................................................9610.3 Regression........................................................................................................................................................96
10.3.1 Aninvisiblelineofbestfit…..............................................................................................................................9710.3.2 Whydowecallitthe“lineofbestfit”?............................................................................................................9710.3.3 Howcanwedrawthislineofbestfit?..............................................................................................................9710.3.4 Slope..................................................................................................................................................................9710.3.5 Allhailthemightyregressionequation............................................................................................................97
10.3.5.1 HowdoIcalculatetheslope?.........................................................................................................................................9710.3.5.2 WhatistheY-axisintercept?..........................................................................................................................................98
10.3.5.2.1 Exampleofapositiveintercept................................................................................................................................9810.3.6 HowdoIcalculatetheintercept?.....................................................................................................................98
10.3.6.1 So...Usingthisinformationwecan................................................................................................................................9810.3.6.2 Calculationtable.............................................................................................................................................................9810.3.6.3 Plottingtheregressionline.............................................................................................................................................9910.3.6.4 Ohthepowerwewield...................................................................................................................................................99
10.3.7 Howaccurateismyprediction?........................................................................................................................9910.3.8 Residuals...........................................................................................................................................................99
10.3.8.1 Let’slookatsomeresiduals............................................................................................................................................9910.3.8.2 Towardsameasureofaccuracy....................................................................................................................................10010.3.8.3 StandardErroroftheEstimateorSEE..........................................................................................................................10010.3.8.4 TheEstimatedPopulationStandardErroroftheEstimate...........................................................................................100
10.3.9 BackintoSSland.............................................................................................................................................10110.3.9.1 Alittlesidenoteaboutr2andSEE.................................................................................................................................10110.3.9.2 TheAccuracyofPrediction...........................................................................................................................................10210.3.9.3 Whyisn’titsignificant?.................................................................................................................................................10210.3.9.4 Pullingitalltogether.....................................................................................................................................................10210.3.9.5 OurexampleinSPSS.....................................................................................................................................................102
10.3.9.5.1 SPSSRegressionOutput.........................................................................................................................................10310.3.10 AssumptionsforCorrelationandRegression..................................................................................................103
10.3.10.1 Possiblerelationships...............................................................................................................................................10310.3.10.2 APotentialProblemforCorrelationandRegression...............................................................................................10410.3.10.3 CorrelationDOESNOTimplyCausation...................................................................................................................104
10.4 Review.............................................................................................................................................................10411 Lec11:QualityofQualitativeResearch.............................................................................................................................106
11.1 ParadigmsinSocialResearch...........................................................................................................................10611.2 Quantitativevs.QualitativeResearch..............................................................................................................10611.3 QualitativeResearch–When?..........................................................................................................................10611.4 CriteriaforEvaluatingQualityofQuantitativeResearch..................................................................................10611.5 ElementsofRigourinQualitativeResearch......................................................................................................107
11.5.1 MethodologicalRigour....................................................................................................................................10711.5.2 InterpretiveRigour..........................................................................................................................................10711.5.3 DemonstratingRigour:DesignandMethods..................................................................................................10811.5.4 DemonstratingRigour:CodingandAnalysis...................................................................................................10811.5.5 DemonstratingRigour:Reflexivity...................................................................................................................109
11.5.5.1 PerspectiveoftheResearcher......................................................................................................................................10911.5.5.2 Reflexivity......................................................................................................................................................................10911.5.5.3 ReportingInformationabouttheResearcher...............................................................................................................10911.5.5.4 PreconceptionandBias–What’stheDifference?........................................................................................................110
11.5.5.4.1 Beingreflective.......................................................................................................................................................11011.5.5.4.2 WhoseStoryisitAnyway?.....................................................................................................................................110
12 Lec12:DoingQualitativeResearch...................................................................................................................................11112.1 ParadigmsinSocialResearch...........................................................................................................................11112.2 ApproachingQualitativeResearch...................................................................................................................11112.3 TheoreticalFrameworks...................................................................................................................................11112.4 Methodologies.................................................................................................................................................111
12.4.1 PeopleasResearchSubjects...........................................................................................................................11112.4.2 PeopleasResearchInformants.......................................................................................................................11212.4.3 PeopleasResearchPartners...........................................................................................................................11212.4.4 Examples.........................................................................................................................................................112
12.5 Method............................................................................................................................................................11212.5.1 SelectingParticipants......................................................................................................................................112
12.5.2 CollectingData................................................................................................................................................11312.5.2.1 Interacting(Mostcommon)..........................................................................................................................................11312.5.2.2 Observing......................................................................................................................................................................11412.5.2.3 Gathering......................................................................................................................................................................114
12.6 AnalysingData.................................................................................................................................................11412.7 Summary.........................................................................................................................................................115
13 CourseOverview..............................................................................................................................................................11613.1 TheMethodinanutshell.................................................................................................................................11613.2 Induction/InductiveReasoning........................................................................................................................11613.3 Deduction/DeductiveReasoning......................................................................................................................11613.4 AScientificTheoryMustBeTestable................................................................................................................11613.5 AScientificTheoryshouldbeRefutable...........................................................................................................11613.6 ObjectivesofPsychologicalResearch...............................................................................................................11613.7 CategorizingResearchApproaches...................................................................................................................11613.8 InferringCausality............................................................................................................................................11713.9 Operationalisation...........................................................................................................................................11713.10 ExtraneousVariables101.............................................................................................................................11713.11 Sampling......................................................................................................................................................11713.12 Representativeness......................................................................................................................................11713.13 Reliability&Validity....................................................................................................................................118
13.13.1 TypesofReliability..........................................................................................................................................11813.13.2 TypesofValidity..............................................................................................................................................118
13.14 ManipulationoftheIV.................................................................................................................................11813.14.1 IVsandDesign.................................................................................................................................................118
13.14.1.1 ThetroublewithBGdesigns....................................................................................................................................11813.14.1.2 ProblemswithRMDesigns.......................................................................................................................................118
13.15 AspecialCase:TimeasanIV........................................................................................................................11913.16 OtherthreatstoanExperiment’sValidity....................................................................................................11913.17 SoWithinorBGinanutshell?......................................................................................................................11913.18 Howwedoinferentialstatistics...................................................................................................................11913.19 ANOVASummary.........................................................................................................................................119
13.19.1 PartitioningSSinBGANOVA...........................................................................................................................12013.19.2 TheFdistribution............................................................................................................................................12013.19.3 IGANOVAAssumptions..................................................................................................................................12013.19.4 AprioriandPostHocComparisons..................................................................................................................12013.19.5 PowerandEffect.............................................................................................................................................12013.19.6 RMorDependentGroupsANOVA.................................................................................................................120
13.19.6.1 PartitioningSSinRMANOVA...................................................................................................................................12113.19.6.2 Assumptions.............................................................................................................................................................121
13.19.7 KeysforstudyingANOVA................................................................................................................................12113.20 CorrelationandRegression..........................................................................................................................121
13.20.1 Correlationonlytellshalfofthestory............................................................................................................12113.20.2 CorrelationandRegression.............................................................................................................................12113.20.3 Breakingdowntheequation...........................................................................................................................12113.20.4 PortionsoftheSumofSquares.......................................................................................................................12113.20.5 SPSSRegressionOutput..................................................................................................................................122
13.21 QualitativeResearch....................................................................................................................................122
KeyforthisstudynotesSS:SumofSquareBG:Between-Group,Between-Subject,Independent-GroupWG:Within-Group,Within-Subject1 Lec1:Introduction1.1 Levelofmeasurement Nominal
• Variablewithvaluesthatarenamesorcategories(thatis,theyarenamesratherthannumbers)- Nominalcomesfromtheideathatitsvaluesarenames- Variableinnameonly.category,numberdon’tnecessarymeananything,justacategory,
e.g.religion,gender(1=male,2=female)- Doesn’tdenoteanythingabouttherelativemagnitude
Ordinal/Rank-ordervariables(inorderonly)• numericvariableinwhichthevaluesareranked,suchasclassstandingorplacefinishedina
race.• numericvariableinwhichvaluescorrespondtotherelativepositionofthingsmeasured• differenceinmagnitudeimplied,Nosetmagnitudebetweenthe2• notequalintervalsbetweenranks• grouphasorder,e.g.race,1st2nd3rd,stillacategory1st(10seconds)2nd(11secs)3rd(14
secs),magnitude• ranks:e.g.,placeinclass,orderinahorserace• e.g.GPAbetweenbeing2ndand3rdintheclasscouldbedifferentto8thand9thInterval• variableinwhichthenumbersstandforapproximatelyequalamountsofwhatisbeing
measured• numericvariableinwhichdifferencesbetweenvaluescorrespondtodifferencesinthe
underlyingthingbeingmeasured• hasmagnitude• differenceinmagnitudeimplied• equalintervalsareassumed• e.g.,timeelapsed,temperature,ages,GPA,weight,stresslevel• e.g.GPA2.5and2.8meansaboutasmuchasthedifferencebetweenaGPAof3and3.3Ratio
1.2 Centraltendency Mean:arithmetic“average”Median–mid-pointMode–mostcommonvalue
1.3 Normaldistribution Weknowwhatthepopulationaverageis,andwhatpeoplevaryaroundtheaverage
1.4 ANOVA ANalysisOfVariance• akaF-test• Variance
o Comparing2groupsofpeople=comparing2differentprobabilityofdistribution
o ANOVAistesting,istherealsovariancebetweenthegroupsintermsoftheirmeanasisscaledbythevariancewithinthegroup.Doesitexceedcertainamount?
• Forsimple(one-way)ANOVA:whatisratioofthevariabilitybetween2groupmeansdividedbythevariabilityoftheWGvariation
• Simplyaratiobetween2differenttypesofvariability• Totestiftheyaredifferenttoeachother
1.4.1 F-testvsT-test • T-test:testthenullhypothesis.Inotherwords–istherea‘significanteffect’inmydata?• So,–infact,F=t2• buttheANOVAismoreflexible,differencemorethantwogroups,twodimensions• BoththeT-testandtheANOVAarespecificcasesoftheGeneralLinearModel(apowerful
analytictool)
1.4.2 Flexibleandpowerfultool
• Howyouuseitdependsonthetypeofquestionyouwanttoask,andultimatelythisisintimatelyrelatedtoyourresearchdesign
• Inordertobesuccessful,youhavetomakedecisionabouthowyouaregoingtodoyouranalysisaspartoftheprocessofdesigningthewholeresearchmethodology
“Thegeneralwhowinsabattlemakesmanycalculationsinhistemplebeforethebattleisfought.Thegeneralwholosesabattlemakesbutfewcalculations.”―SunTzu,TheArtofWar
1.5 Wheredoesknowledgecomefrom?
Thesearesometraditionalideas(priortodevelopmentscientific/hypotheticaldeductivemethod)aboutwhereknowledgemightcomefrom.Butcanwerelyuponthem?• Authority:someonewithauthoritythatyoucantrust.Whatisareliablesource?
o Law,Professor,Newspaper??,Hitler• Intuition:justcometoyou,internallygeneratedanditseemsgood
o “gutfeelings”-theabilitytoacquireknowledgewithoutinferenceortheuseofreason• Rationality:Youdeduceitfromtheapplicationoflogicalprinciple
o Purereason–thetruthcanbederivedfromfirstprincipalsusinglogico Whataboutdistortedlogic?Witchesburnthereforetheyaremadeoutofwood,Wood
floats,Ducksfloat,Thereforeifthewomanweighsthesameasaduckshemustbeawitch.
• Empiricism:yousawitandmeasureito Seeingisbelievingo Amesroomisadistortedroomthatisusedtocreateanopticalillusion.
1.6 Scientificmethod Sometimesalsoreferredtoasthehypothetico-deductivemethod.Itischaracterisedbythe
developmentandsystematictestingoftheories.TheMethodinvolvesaspectsof
• Authority:trustscholarlyjournalsasareliablesourceofinformation• Intuition• Rationality:logicalrationalthinkingintermsofgenerationofhypotheses,andthe
structureoftestinghypotheses• Empiricism:actualgatheringofevidence
Scientificmethoduseswhatisgoodabouttheaboveaspects,butmaintainadegreeofscepticismtoo.Itischaracterisedbythedevelopmentoftheorieswhichhaveexplanatoryandpredictivecapacityandwhichmustbetestableandrefutable
1.6.1 Inanutshell • Youhaveatheorythatattemptstoexplainaparticularphenomenonofinterest• Thattheoryisusedtogeneratehypotheses–iftheoryXistrue,itfollowslogicallythatYshould
occur• Youtestthehypotheses
1.6.2 Hypotheses,Theories
andLaws• Ahypothesisisastatementthatcanbetested.Sothestatement,"Awatchedpotneverboils,"isa
validscientifichypothesisbecausewecantestit(andfindthatinthiscaseitisNOTsupportedbytheevidence).
• Atheoryisageneralprincipleorbodyofprinciplesthathasbeendevelopedtoexplainawidevarietyofphenomena.Itmustbeconsistentwithknownobservationsanditmusthavepredictive
power.Asnewknowledgeisgained,theoriesarerefinedtobetterexplainthedata.o Mustallowyoutomakenewprediction/hypothesis,whichyoucantest.Thenmodify
yourtheorybasedonthetest• Alawisamathematicalrelationshipthatisconsistentlyfoundtobetrue.E.g.,oneofthemost
famouslawsinphysicsisEinstein'se=mc^2.o Thereisnolawinpsychology,ithastheoryandhypothesis.So,nottooworry
1.7 Theory InductionandDeduction1.7.1 Induction/Inductive
Reasoning• Reasoningfromthespecifictothegeneral• Takingsomespecificsexamplasofcategoryofthings,thanassumeallthesecharacteristics
holdtrueacrossalltheseexamplas• E.g.Rainbowlorakeets,penguinsandeaglesallhavefeathersandbeaks.Therefore,toinduce
fromthis,allbirdshavefeathersandbeaks.• Makinginductiononthingsthatsomebirdshaveincommon,areallthingsbirdshavein
common• AlltheswanswehaveseensofarareblackinAustralia,thereforeallswansareblack(not
true)• Inductionreasoning:youarehopingtogeneralizewhatyouhaveobserveThelogicofdiscovery:TheoryDevelopment• Inductioncanbeuseful.Kindoflogicweusewhendevelopingtheory.Youcanhaveatheory
basedoninductiveprinciplefromtheobservations.Itcangowrong,butitisanimportantpartofscientificprocessfordevelopingtheory.
1.7.2 Deduction/Deductive
Reasoning• Oppositedirectiontodeduction• Reasoningfromthegeneraltothespecific,usinglogicalchainsofreasoning–syllogisms• E.g.allbirdshavefeathersandbeaks,rainbowlorakeetsarebirds,thereforerainbow
lorakeetshavefeathersandbeaks.Thelogicofjustification-Theorytesting• totestthehypothesis
1.7.2.1 Usedin • Developinghypotheses• IfXistrue,thenYshouldoccur• Hypothesesshouldbelogicalconsequencesofthetheory
o Ifyouhaveatheorythat“theheartistheseatoflove”ahypothesisthatfollowslogicallyfromthismightbethatifyouremovesomeone’shearttheywillno-longerbeabletolove,butthisalsomeansthepersonisdead.Needtobecarefulhowwetesthypotheses
• TestingHypotheses
1.7.3 TheScientificMethodassumes
• Thattheuniverseisordered,thereisastructureintheuniverse,thereisanunderliningprinciple
• Thatorder/structurewhichexistsisdiscoverable
1.7.4 TheMethodischaracterisedby
• Control• Operationism• Replication(thingthatyoucanshowtobetrueandcontinuouslyshowingthemtobetrue.It
isonlybydoingthingsanumberoftimes,andbereplicabledemonstrateable,thenwecanbegintobelieveinthem)
1.8 TheMethodina
nutshell• Youhaveatheorythatattemptstoexplaina
particularphenomenonofinterest• Thattheoryisusedtogeneratehypotheses–
iftheoryXistrue,itfollowslogicallythatYshouldoccur
• Youtestthehypotheses,thentrytorefutethetheory
• Ifnecessaryyouupdateyourtheorytoaccommodatethenewempiricalfindings
Thediagramiscyclical.Hopefullybyhavingthis
cyclicalprocess,weconvergeonknowingmoreandmorethetruth
1.9 Whatmakesagoodtheory
• Theoryiswherepredictioncomefrom,andexplainsphenomena• FromtheperspectiveoftheScientificMethod• Arethesegoodtheories?
- Aristotle’sTheoryofGeocentrism(Theoryoftheworld/earthisthecentreofeverything(theuniverse),thesun/star/planetallrevolvearoundtheearth)o Goodtheorybutincorrecto Testable,thereforerefutable.Itgeneratesthetheoryyoushouldbeabletomake
observationsoftheorbitofplanetsaroundtheearth- SigmundFreud’sPsychoanalyticTheory?
o Notagoodtheoryo Itdoesn’tgeneratepredictions,thereforecan’tberefutable.Onthebasisonhaving
somewillythoughtsaboutsuperego.Itmightbeagoodculturaltheory,itisnotastrongtestablescientifictheory.
- CharlesDarwin’sTheoryofEvolution?o Goodtheoryandcorrect,havereceivedoverwhelmingsupporto Itgeneratespredictionlikeevidenceofcommonancestorsinthefossilrecords,
testableandrefutable.Whethersomethingisagoodtheoryornot,isn’tthesameasaskingwhetheryoulikeit,isn’tthesameasaskingwhetheriscorrectornot.
1.10 AScientificTheoryMustBeTestable
• Scienceproceedsbymakingobservationsofnature(byperformingexperiments).Ifatheorydoesnotgenerateanyobservationaltests(orpredictions),thereisnothingthatascientistcandowithit.Ithastogeneratehypotheses
Considerthistheory:"Ouruniverseissurroundedbyanother,largeruniverse,withwhichwecanhaveabsolutelynocontact."• Isnottestable,thereforeisNOTagoodtheory(couldstillbetrue….)
1.11 AScientificTheoryshouldbeRefutable
Considerthistheory:"Thereareotherinhabitedplanetsintheuniverse."• ThisTheoryistestable(wecangotoanotherplanetandsee),butitisnota“good”scientific
theory(notrefutable).Here'swhy.Itmaybeeithercorrectorwrong.Ifitiscorrect,thereareseveralwaysthatitscorrectnesscouldbedemonstratedincluding:
1. wevisitanotherplanetandfindMorbolivingthere.2. radiotelescopesonearthbegintoreceivesignalsfromsomewhereintheAndromeda
Galaxythatappeartobererunsofthe"ILoveMorbo"show.3. Morbolandsinyourbackyardandsays,“IwilldestroyyoupunyEarthlings!”
But,sofarthishasnothappened
1.11.1 ThelogicofRefutation
TheWASONCardSelectionTaskTheruleis:ifthecardhasanevennumberononeside,theothersidemustbered.Whichcard(s)mustyouturnovertotestifthisruleisTRUE?• 3/8• red/8• 3/red• 8/brown(nottheanswerpeopleintuitivelythink,hencewhyoneneedstobecarefulabout
theapplicationoflogic)
1.11.1.1 Introtologic Asyllogismisalogicalchainofargument,isgenerallystructuredlikethis:1. Astatementthatdeclaresarule2. Astatementthatdescribesanobservationthatrelatestothatrule3. Aconclusionthatfollowsfromthatobservationinthecontextofthatrule
e.g.
1. IfitisThursdayatbetween10-12thereisaResearchMethodsLecture2. Itisaround11:30onThursday3. ThereforethereisaResearchMethodsLecture
• Inwhatfollows,don’tgethunguponwhetheryouthinkthe“rules”aretrueornot.
• Whatisimportantistounderstandthatthestructureofsometypesofargumentarelogicallyvalid
• Thatmeans,iftherulewereTRUEthentheconclusionmustalsobeTRUE• OtherstructuresorNOTlogicallyvalid.• Thatmeans,eveniftheruleistruewedocannottrusttheproposedconclusiontobetrue
1.11.1.2 Anexampleofavalidlogicalinference
Hereisanexampleofavalidlogicalstructure(modusponens)IfPthenQPThereforeQ(validstructure)IfMANEDthenMALEMANEDThereforeMALE(validstructure)Animportantthingtonotehereis:IfMANEDthenMALE¹IFMALEthenMANEDMANEDthenMALE:Manedlionisthesubsetofmalelion(malechildliondoesnothavemaned)MALEthenMANED:MalelionisthesubsetofmanedlionAbovetwoisnotthesamething.Male=Manedisanotherpossibleworld
1.11.1.3 Anothervalidlogicalinference
Hereisanexampleofavalidlogicalstructure(modusponens)IfPthenQNotQThereforeNotPIfMANEDthenMALENotMALEThereforenotMANEDMANEDthenMALE:Manedlionisthesubsetofmalelion(malechildliondoesnothavemaned)Maleandfemaledon’tintersect,then,ifitisnotmale,thencan’tbemaned.Anothervalidargument
1.11.1.4 Denyingtheantecedent
Denyingtheantecedent,isaformalfallacyofinferringtheinversefromtheoriginalstatement.Itiscommittedbyreasoningintheform:IfP,thenQ.NotP.Therefore,notQ.IfEVENthenREDNotEVENThereforenotRED
IfMANEDthenMALENotMANEDTherefore,notMALEInotherwords–don’tturnovertheTHREE–itdoesn’thelpTheruleis:ifthecardhasanevennumberononeside,theothersidemustbered.Whichcard(s)mustyouturnovertotestifthisruleisTRUE?IfEVENthenRED,NotEVEN,ThereforenotRED
1.11.1.5 Affirmingtheconsequent
Affirmingtheconsequent,isaformalfallacyofinferringtheconversefromtheoriginalstatement.Thecorrespondingargumenthasthegeneralform:
• Backwards• Fallacyofthinking
IfP,thenQ.Q.Therefore,P.IfEVENthenREDREDThereforeEVENIfMANEDthenMALEMALETherefore,MANEDInotherwords–don’tturnovertheRED–itdoesn’thelpTheruleis:ifthecardhasanevennumberononeside,theothersidemustbered.Whichcard(s)mustyouturnovertotestifthisruleisTRUE?IfEVENthenRED,RED,ThereforeEVENTurningthe3overcan’tdisproveit,itcan’tevenproveit.Turningtheredover,whetherit’sevenoroddnumber,itdoesn’thelpastheruledoesn’ttellyouanythingaboutthat.Thelogicalstructureisincorrect.Sothecorrectansweristhatweshouldturnoverthe:TheEIGHT- SinceifittheothersideisBrownthiswouldrefutethe“rule”TheBROWN
- SinceifittheothersideisEVENthiswouldrefutethe“rule”
1.11.1.6 Refuteit ThefundamentalpointisthatthewaytoTESTtheruleisbytryingtorefuteitratherthantryingtoproveitKarlPopper–allthewhiteswansintheworldcannotprovethetheory“allswansarewhite”–butasingleblackswancandisproveit
1.12 Thusitiswithscience…
Scienceprogressesbysystematicallyeliminatingfalsehoodsratherthandemonstratingtruths!Andonthatbombshell…
2 Lec2:Experimentaldesign,VariablesandOperationalisation2.1 Objectivesof
PsychologicalResearch
TodeveloptheoriesthatDescribe
– portrayingthephenomenonaccurately• e.g.,Piaget’stheoryofchilddevelopmentarosefromdetailedobservationsofhisown
children• describeaccuratelythephenomenathatareofinterest
Explain– identifyingthecause(s)ofthephenomenon– positexplanatorymechanism– causalrelationshipbetweenthings
• e.g.,socialconnectionanddepressionPredict
– identifyingriskfactorsofaphenomenoncanhelpyoutopredictwhenitmighthappen– generatenewprediction
• e.g.,whatfactorsbestpredictacademicsuccess
2.2 HowtoConductResearch
• Identifyphenomenaofinterest(thatinterestus),thatdescribing,explainingandmakingpredictionabout
• Readthescientificliterature,hasanyoneelsehadanythingsensibletosaysomethingabout▫ Isthereanestablishedtheorythatgeneratespredictionsaboutthephenomena,that
aretestable?▫ Ifnot,whatevidenceisneededtoallowatheorytobedeveloped.▫ Iftherearecompetingtheoreticalperspectives,askwhatevidenceisneededto
establishwhichtheoryiscorrect/thebest?• Formulatearesearchquestion• IdentifybestmethodtoaddresstheResearchQuestion
2.2.1 MajorMethodologicalApproaches
(DanainterpretedPatassayingthatwedon'tneedtoknowthetermPositivist,eticetcjustneedtoknowthetermsQuantitative&Qualitative,stillneedtoknowwhattheymean)
Quantitative• PositivistorEtic:Concernedwithuncoveringgeneralizablepatternandlawsbasedon
objectiveempiricaldata(tendstobedeductiveinnature)Qualitative• InterpretivistorEmic:Concernedwithsubjectiveinterpretation,personal/culturalmeaning,
contextspecific,notconcernedwithgeneralisabilitybutwithdeepunderstandinginlinewithinductiveapproaches.
*tiptoremember:
- Etic,tfortheory,Emic,mforme(interpreting)- Deductivesoundslikereductive,reducingfromgeneraltospecific- Inductiveistoincrease,specifictobroadertheory
2.2.2 CategorizingResearch
ApproachesQuantitativeversusQualitativeResearchQuantitativeStudies–collectnumericaldata,ordatathatcanbeconsideredinnumericaldata
– e.g.,ratingsofattractiveness,numberoftimesaratpressesabarinordertoberewarded,reactiontimes,peopleresponsestosurveys
QualitativeStudies–collectnon-numericaldatatoanswerresearchquestions,relatemoretopeople’sexperience,understandingandpersonalmeanings
– e.g.,pictures,clothingworn,interviewstatements,documentsMixedMethods
– quantitativedataprovidesanincompleteanalysisofwhatisbeinginvestigated,numerationofphenomena
– qualitativedataaddsadditionallevelofunderstanding,layerofmeanings
2.3 Quantitative Theygenerallyworklikethis• Youhaveahypothesis• Youcollectsomekindofnumericaldatatotestthathypothesis
2.4 TheVariable–akey • Variable
conceptinQuantitativeResearch
– somethingthatvaries– takesondifferentvaluesorcategories– e.g.,gender,anxietylevels,IQscores,on/off,heights,weights,theseareallthingsthat
vary.Wecannumerateorcategorisetheirlevelofvariability• CategoricalversusContinuousVariables
– CategoricalVariables• variesbytypeorkinde.g.,gender,religion,universitycourse,typeoftherapy• e.g.75%enrolledinpsychologyand25%inlaw,it’scategorical,onethingortheother.• NOMINALMEASUREMENT
– ContinuousVariables• variesbydegreeoramount• Continuesgradedspectrumofvaluesofaparticularvariable• e.g.,reactiontime,height,age,anxietylevel• INTERVAL/RATIOMEASUREMENT
2.4.1 Whichoftheseare
variablesInterval–IQ(ratioisunknown)Ratio–0(meaningful0,meansnonegativenumber)
Variable? Correct Type ScaleMale Gender categorical nominalWeight continuous ratio(sizeofintervalsisequaltoeachother,andit
hasmeaningful0(nonegativenumber),thatmeansifsomethingisweight20kg,itisexactlytwiceasheavyas10kg.Sothedifferentpointsofthescalehasameaningfulrelationshiptoeachother)
Reactiontime continuous ratio6foot2 Height continuous ratioblue Colour categorical nominalIQ continuous interval,thepointsbetweenpointsonthescaleare
assumedtobeequalandmeaningfullysobutthereisnoabsolute0fromwhichcanbecalibrated.Sowhilstthepointsonthescaleareassumetobeequalsizeinterval,theratiobetweenthemareunknown.
2.4.2 VariablesinQuantitativeResearch
• IndependentVariable(IV)– presumedtocausechangesinanothervariable– thevaryingofIVleadstochangesinDV– oftenmanipulatedbytheresearcher
o therapyvs.notherapyo alcoholdose(1unitversus2units)o locationoflearningwordlist(underwaterversusabovewater)
– needtoseeifthesechangesaffecttheoutcome• DependentVariable(DV)
– thepresumedeffectoroutcomeofthestudy– variablethatismeasuredbytheresearcherandinfluencedbytheIV– isthethingwemeasure,wehopehasbeeninfluencedbythemanipulationoftheIV– essentiallyisanythingthatyoumeasureintheexperiment
o behaviours,attitudes,feelingsmeasuredthroughtests,monitoring,questionnaires,numberofitemsrecalledonmemorytask,reactiontime,EEGdata
2.4.3 Fundamental
questionSothequestionthatisgenerallyaskedinaquantitativeresearchstudyis:• arechangesintheIVassociatedwithchangesintheDV?• OrdoeschangingtheIVcausechangesintheDV?
2.4.4 Othervariablesinquantitativeresearch
2.4.4.1 ExtraneousVariables
• variable/sthatcompeteswiththeIVinexplainingtheoutcomeorDV• allofthethings(youcan/can’timagine)thatmightimpactuponaperson’sabilitytoperform
atask• itisimportanttotrytocontrolforextraneousvariables,tonotallowittobesystematic
variabilityasafunctionofextraneousvariable
Isice-skatingfasterthanroller-skating?• Thethingweareinterestediniswhattypeofskatesarebeingusedinthisspeedtest.Sowhat
arethethingsthatmightimpingetheoutcome?• Whatkindofextraneousvariablesmightbeimportanttoconsiderhere?
o Theexperienceoftheskater(uncontrolledextraneousvariable)o Environmento Timeofthedayo Weightsdifferenceo Everythingthatcanvaryandhasanimpacttotheoutcomeofthestudy
2.4.4.1.1 Confounding
variable
Anextraneousvariablethatisallowedtoco-vary(tovarytogetherwithanothervariable)alongwiththelevelsoftheIVIsice-skatingfasterthanroller-skating?• Foundindividualswhoareconfidentandequallyexperienceintheuseofbothtypeofskate,
wehaveequippedthemwiththebestpossibleskate.Theyaretrainedtopeakleveloffitness• However,theconditionhasasystematicconfoundbecauseboththeskates(IV)andthe
coursedifferacrossthetests,inawaythatistotallycorrelated.HavingaconfoundisprettyseriousbecauseitmeansthatyoureallycannottellwhetheritistheIVortheconfoundthatisaffectingperformance.
- Uncontrolled3rdvariableisoperating.If2variablesareconfounded,theyareintertwinedsoyoucannotdeterminewhichofthevariableisoperatinginagivensituation
2.4.4.2 MediatingVariable
/InterveningVariable
• occursbetweentwoothervariablesinacausalchain- e.g.,anxietycausesdistraction(mediatingvariable)whichaffectsmemory- distractionhastheproximaleffectonmemoryperformance,notanxiety.- somethingthatintervenesbetweenonethingandanotherthing
2.4.4.3 Moderating
Variable• qualifyacausalrelationshipasdependentonanothervariable• qualifyacausalrelationshipbetweenIVandDV
- e.g.,theimpactofanxietyonmemoryisdifferentformenandwomen(sexisamoderatingvariable)
- genderismoderatingtheeffectofarelationshipbetweenanxietyandperformance
2.5 Theresearchproblem/question
Agoodtheorygenerateshypotheses–thesepredictionsgiverisetotheresearchproblem,orresearchquestion:• aninterrogativesentencethatstatestherelationshipbetweentwoormorevariablesorthe
keyresearchquestion• criteriaforgoodresearchproblems
- variablesshouldexpressaclearrelationship- statedinquestionform- capableofempiricaltesting
Soaresearchquestionshouldbe,specifiedinawaythatmakesclearwhatcausalrelationshipisbeingtested.• isnumberofhoursofCBTassociatedwithreducedanxietyscalescores?• arechangesintheIVassociatedwithchangesintheDV?
2.6 Workthroughanexample
• Identifyinganinterestingphenomenon• Relatingittotheory• Generatingahypothesis• Framingaresearchquestion• Identifyingvariables• Andconsideringdifferentmethodsforaddressingtheresearchquestion
2.7 Aninteresting
phenomenaShaneMacgowan¹JonnyDepp,but + =
2.7.1 Isthereatheory? The“InverseCinderella”theory
• Cinderellatheorysaysthingsturnsintopumpkinatmidnight• InverseCinderellatheory:everyonegetsmoreattractivewhentheclockstrikesmidnightWhat’swrongwiththistheory?• Wejustneedtolookatpeoplebeforeandaftermidnighttoknowthisisnottrue–easyto
disprove.The“BeerGoggles”Theory• Theingestionofalcoholhasanumberofeffectsonthehumanbrainincludingsimultaneously
increasinglevelsofsexualdesireanddecreasingaestheticjudgementwithrespecttothesuitabilityofpotentialsexualpartners
• Shane+alcohol=JohnnyDeppIsthisagoodtheory?Doesitgeneratepredictions?Isitrefutable?YES• Consistentwithexistingobservations• Generatespredictions(ifyougivesomeonealcohol,theirjudgementmightchange)
o testableo refutable
2.7.2 Theory/prediction/
question• Theory
– “Theingestionofalcoholhasanumberofeffectsonthehumanbrainincludingsimultaneouslyincreasinglevelsofsexualdesireanddecreasingaestheticjudgementwithrespecttothesuitabilityofpotentialsexualpartners”
• Prediction– “drinkingalcoholwillmakepeoplemoreattractedtopeoplewhomtheywould
normallyconsiderunattractive”• ResearchQuestion
– “doesalcoholconsumptionaffectattractivenessjudgments?”– dochangesinIVaffectsDV?
2.7.3 Researchquestion IV:AlcoholIngestion-wecanoperationaliseintothefollowing
• couldbecategorical-YES/NO• couldbecontinuous–numberofdrinksDVAttractivenessJudgements• couldbecategorical• couldbecontinuous
2.7.4 Howdoyouanswerthisquestion?
• Designastudy• Findsomeparticipants• Makesomemeasurements• Analysethedata• Writeapaperexplainingwhatyouhavedone
2.7.5 Somewaystoanswer
researchquestions1. Naturalisticobservation:simplyobservethebehaviour,nomanipulation,e.g.animalsin
nature2. Correlationalstudy:makingmeasurementandaskingthereisarelationshipbetweendifferent
measurement3. Internetstudy:online4. Fieldexperiment:innaturalenvironmentbutwithmanipulation(differenttoNaturalistic)5. Laboratorybasedexperiment
2.7.6 Possiblestudyideas InchoosinghowbesttoaddressourresearchquestionweneedtoaskPossiblestudyideas Isitpossibletodothe
thingthatwewanttodo?(logistics)
IsitOKtodothethingwewanttodo?(ethics)
Willdoingwhatwewanttodotellusanythinguseful?(validity)
1. Gotoanightclubandwatchwhathappens(NaturalisticObservation) Yes Yes No
2. Gotoanightclubandasksomequestions Yes Yes No3. Getonfacebook,encourageyourfriendstoget
drunk,thenpostapictureofShaneMacGowan Yes Maybe No
andseehowmanylikesitgets(Internetexperiment)
4. TakeShaneMacGowantoanightclub,spikesomeone’sdrink,andseewhathappens(fieldexperiment)
No(wedon’tknowhim) No No
5. Gotoyourlabandperformanexperiment Yes Yes Yes
2.7.6.1 Gotoanightclubandasksomequestions
Name Drank Rating
Sandy 1bacardibreezer “yuk!”
Leslie 12schooners “phwoar!”
Gabby 2lemonruskis “meh”
Ashley 8bacardibreezers “bringiton!”
Pat 3whitewines “nochance!”
Tyler 6schooners “notbad!”
Drew 4mineralwaters “areyounuts?”
Morgan 3schooners “probablynot”
Wynn 5doublevodkas “maybe”
Sydney 7vodkaandredbull “definitely”
2.7.6.2 AKeyCharacteristicofScientificResearch
• Operationism– representingconstructsbyaspecificsetofdefinitionsoroperations– operationaldefinition
• definingaconceptbytheoperationsusedtorepresentormeasureit
Name Standarddrinks Rating
Sandy 1.5 “yuk!”
Leslie 18 “phwoar!”
Gabby 3 “meh”
Ashley 12 “bringiton!”
Pat 5.4 “nochance!”
Tyler 9 “notbad!”
Drew 0 “areyounuts?”
Morgan 4.5 “probablynot”
Wynn 10 “maybe”
Sydney 7.2 “definitely”
2.7.6.3 Howaboutoperationalisingattractiveness?
Outof10score/Attractivenessratings
Name Standarddrinks Rating
Sandy 1.5 1
Leslie 18 10
Gabby 3 4
Ashley 12 8
Pat 5.4 3
Tyler 9 6
Drew 0 1
Morgan 4.5 4
Wynn 10 7
Sydney 7.2 9
2.7.6.4 Whatvalidinferencescanwe
• Alcoholimpairsjudgement?o Wehaven’ttestedjudgementmoregenerally,wehaveonlytestedaspecific
Thisstudyhasn’toperationalisedthemeasuresverywell.Theyaren’tstatedoperationalizewithadegreeofrigorandspecificityandaccuracywithwhichwecouldreallygetmeaningfulthingsfrom
Howmightweoperationalisetheconceptof“alcoholintake”?standarddrinks,itisanoperationalizationofconceptoflevelofalcoholintake.Byasimpleformula,wecantransformoutmeasurementintosomethingoperationallyuseful
NowwehaveoperationalizeIVandDV,wewanttoseeifthereisarelationshipbetweenthem.OurdatashowthatalcoholintakeinCORRELATEDwithattractivenessratings
drawfromthis? judgement• Alcoholcausespeopletolowertheirthresholdfor“sufficientlyattractive”?
o Wecan’tsaythis.Whatwecansayis,o PeoplewhohaddrunkmorealcoholratedShaneMacGowanasbeingmore
attractive?
2.8 Correlation • PrecipitationinNewYorkcorrelateswithprecipitationinVermont• Isthisonethingcausing
theother?• Thesetwothingsare
highlycorrelated• Becausetheyare
geographicallyclosetoeachotherandsubjecttosameweathersystem
• Theweathersystemcausestherainfall:thegeographicproximallocationoftheregionsmoderatesthevariables/relationship
BUT,becarefulofcorrelationTheremightbeapossiblemediatingvariable,e.g.eatingcheesemaycausesbaddreamwhichcausedthebedsheetstangled,butunlikely.
IfonethingcausesanotherthingtheyMUSTbecorrelated<doesnotequal>IftwothingsarecorrelatedthereMUSTbeacausalrelationship(randomchance)
2.8.1 TheissueofCausation
Causation– aconditioninwhichoneevent(thecause)generatesanotherevent(theeffect)
Criteriaforidentifyingacausalrelation– cause(IV)mustberelatedtotheeffect(DV)(relationshipcondition)– changesinIVmustprecedechangesinDV(temporalordercondition,causemusthappen
beforeeffect)– nootherplausibleexplanationmustexistfortheeffect– weneedthesethingstobetruetoinferacause
therelationshipbetweenalcoholandShane,wehaven’testablishedcausality,becauseotherexplanationdoexist.Therearepeoplereallylikealcoholand/orShane.
2.8.2 InferringCausality Awelldesignedandappropriatelycontrolledandconductedexperimentcanallowinferencesaboutcausality
– Performanaction(manipulateIV)– Measuretheconsequences(changesinDV)– CONTROLforotherpossibleexplanations
2.9 Anexperimentshould
be…• Carefullydesigned• RigorouslyControlled(trytocontrolasmanyextraneousvariablesaspossible,and
avoidingconfound,ifwedon’t,wecan’tdrawcausalinfluences)• Replicable(othersshouldgetthesameresultsifcopiedthemethodandgetthesame
results)• Ethical
2.10 Someimportant
ethicalissues• Informedconsent(peopleshouldbe
asked,andconsenttotheparticipationofresearch)
• Righttoconfidentiality• Righttowithdraw• Donotcausephysicalormentalanguish,
harm/distress• Exampleofunethicalexperiment:Milgram(induceanxiety/stresstotheparticipant,
resultscan’tbetrusted)2.11 Experimental
Approach
2.11.1 Advantages • Causalinference–experimentalapproachisbestmethodforinferringcausation- causaldescriptionreferstoidentifyingtheconsequencesofmanipulatinganIV- causalexplanationreferstoexplainingthemechanismsthroughwhichthe
relationshipexists• Abilitytomanipulatevariables
- onlyscientificmethodologyinwhichvariablesaremanipulated• Control
- extraneousvariablesarecontrolledby:o holdingthemconstant,e.g.sameIQo usingrandomassignmento matching(methodthatareavailabletousthatfacilitatesexperimentalcontrol
withwhichwecanmakecausalinference,bybeingabletosaythisextraneousvariableisn’ttheexplanationaswehavetakenthiscontrolmeasuretocounteractpossibleeffectsofthisextraneousvariable)
2.11.2 Disadvantage • Doesnottesttheeffectsofnon-manipulatedvariables
– manypotentialIVscannotbedirectlymanipulated• e.g.,people’sages,gender
• ArtificialityorGeneralisability– referstopotentialproblemsingeneralisingfindingsfromlaboratorysettingsto
the“realworld”– peoplemaybehavedifferentlyinlabsettingvsnaturalenvironment
2.12 Experimental
ResearchSettings
2.12.1 InternetExperiments • advantages– accesstodiversepopulation– bringexperimenttoparticipant– largesampleandthusgreaterpower– costsavings
• disadvantages
– multiplesubmissions(fromsameperson)– lackofcontrol– self-selection– dropout
2.12.2 Fieldexperiments • anexperimentalresearchstudythatisconductedinareal-lifesetting
– advantage–maybeeasiertogeneralizefindings,cutouttheartificialityoflaboratorysetting,thereforegettingmorerealdataonhowpeoplebehaves
– disadvantage–lesscontrolofextraneousvariables,canbetimeconsuming• confederate
– useofdeception,apersonwhoisinleaguewiththeexperimenter,unbeknownsttotheparticipant
– e.g.peoplearemoregenerousandwillingtogivemoremoneyinthelabsetting.Peoplearelessgenerousinreallife,e.g.sellingbaseballcardsataconvention,reallifesetting–lessgenerous.
– Thisisbecauseinthelab,theyfeelthepressureofsocialjudgement.Theyaltertheirbehaviourtoconformtowhattheythinkisthenicewaytobehave
2.12.3 Laboratory
experimentsanexperimentalresearchstudythatisconductedinacontrolledlaboratorysetting
• advantage–morecontroloverextraneousvariables,e.g.sametimeoftheday,temperatureetc
• disadvantage–lessgeneralizationrelatedtoartificiality(lab)
2.12.3.1 Differentwayswecouldmanipulate
ExperimentalmanipulationExperimenterdetermineswhichleveloftheIVaparticipantistestedat;
3 Lec3:Sampling,ValidityandReliabilityHowdoyouansweraresearchquestion?
• Designastudy
IVs • eventmanipulation(e.g.presenceofalcoholvabsenceofalcohol),completecontrol• instructionalmanipulation(e.g.drinkalcoholquickly/slowly)
2.12.3.1.1 Beergoggles
experiment1• IV:DrinkType:alcohol,water(alcoholvsnon-alcohol)• DV:attractivenessofthepictureofShane
2.12.3.1.2 Beergoggles
experiment2
• IV:varythestandardofdrinks:e.g.nodrinks,onedrink,5drinks• DV:attractivenessofthepictureofShane
2.12.3.2 DifferentwayswecouldmanipulateIVs
Individualdifferencemanipulation• Althoughwecan’tallocatepeopletobemale/female,high/lowIQ• Quasiexperimentalmanipulationratherthantrueexperimentalmanipulation
§ Quasi-experimentsaresubjecttoconcernsregardinginternalvalidity,becausethetreatmentandcontrolgroupsmaynotbecomparableatbaseline.Withrandomassignment,studyparticipantshavethesamechanceofbeingassignedtotheinterventiongrouporthecomparisongroup.(Wikipedia)
• Wecouldtryandlookattheeffectsofindividualdifferencesacrossparticipants• Trytolookattheeffectsofvariablesrelatedtoindividualdifferences• AcharacteristicoftheparticipantdeterminestheleveloftheIVatwhichtheyaretested;
– Computeranxiousvs.non-computeranxious– Malevs.female– Levelofsocialsupportreceived(highvlow)
2.12.3.2.1 Beergoggles
experiment3
Isthereaneffectforalcoholvsnoalcoholbasedonindividual’ssexualpreference?Whethertheeffectsofalcoholontheattractivenessjudgementaregeneralthatyouwillsayeveryoneismoreattractivewhetheryouwouldconsiderthemasasexualpartnerornotvswhetheritismoderatedbywhethertheyarethekindofgenderpeoplewithwhothemwantstoengageinsexualactivity.Thesearesortofthingswecanstarttomakecasualinference.
2.12.3.3 DifferentwayswecouldmanipulateIVs
RepeatedMeasure(WithinGroup):eachparticipanttestedateachleveloftheIV;• SameparticipantiscontributingtomorethanoneIV• Moresensitivedesign(easiertodetecttheeffectofinterest)• Can’talwaysusethisdesign• Whenusedappropriately,itisareallygoodmethod
BetweenGroup:eachparticipanttestedatonlyoneleveloftheIV;
• Lesssensitivedesign• Oftenforcedtousethisdesign
MixedDesign:
• morethanoneIVwithatleastoneIVmanipulatedBG• andatleastoneWG.
2.12.3.3.1 Beergoggles
experiment4
MultifactorialBeerGogglesExperiment
2.12.4 Potentialmanipulations
• Alcoholvsnoalcohol• Differentdosesofalcohol• Malevsfemale• Malevsfemalestimuluspictures• Alloftheabove
• Findsomeparticipants• Makesomemeasurements• Analysethedata• Writeapaperexplainingwhatyouhavedone
3.1 TheissueofCausation Criteriaforidentifyingacausalrelationship
– cause(IV)mustberelatedtotheeffect(DV)(relationshipcondition)– changesinIVmustprecedechangesinDV(temporalordercondition)
3.2 Findsomeparticipants Thisisknownassampling
Ifwewouldliketobeabletosaythatourdataallowustomakegeneralisableinferencesitisveryimportanttogetthisright!
3.2.1 Somekeyterms Population– Agroupofpeopleaboutwhomonewouldliketodrawsomemeaningfulconclusions,e.g.
• Adolescents• Peoplewithschizophrenia• QUTPsychologyundergraduates
Sample– Asubsetofthatpopulationthatisactuallyincludedinyourresearchstudyi.e.participants
• 150Year10students• 30outpatients• Everyonewhoattendswk3lecture
Samplingframe– Alistofmembers/elementsofapopulationfromwhichonemightobtainasample
• Electoralrole• Telephonedirectory• Studentenrolmentlist
Census– Alistofallthepeoplecomprisingaparticularpopulation.
• E.g.allthememberoftheAFLclubs
3.2.2 Aimofsampling Tomakegeneralisableinferencesaboutthepopulationonthebasisofmeasurementsfromyoursample.Itiscrucialthatyouhavearepresentativesample-asamplethatislikethepopulation.Thissimplymeansthatyoushouldselectasamplewhosetypicalcharacteristicsareapproximatelythesameasthetypicalcharacteristicsofthepopulation.Ifyoucan’tguaranteethatthisisso,youcan’tguaranteethatyourinferencesgeneralise.
3.2.3 Representativeness • SampleStatistic– Anumericcharacteristicofasample-(measured)– Somethingthatwemeasureinthesample
• PopulationParameter– Anumericcharacteristicofthepopulation-(oftennotknown)– Ifwehavearepresentativesample,thenthissamplestatisticswillbecloselyrelated
tothepopulationparameterwhatthatvaluewillbefortheentirepopulation• Responserate
– Whatproportionofpeopleresponded?• Samplingerror
– Thedifferenceinvaluebetweenthesamplestatisticandthepopulationparameter(dependsonsamplesize)
Thesmallerthesample,thelargerthesamplingerror.Ifthesampleistoosmall,itisnotlikelytoreflectthecharacteristicofthepopulationingeneral
3.2.4 Samplingbias Population:PeopleenrolledonPYB210Sample:peoplewhoattendthislectureHowwasthissampleselected?
- Ifitwasrandomsampling,thenstudentswouldtossedthecoinwhentheygotoutofthebed,heads:gotouni,tails:backtobed
- Inthiscase,peopleselectedthemselvestobepartofthesample
- Thisisnotarepresentativesample- Self-selection:thereisalwaysadanger
onpeoplewhoselectandwhodon’t.Datacan’ttrusted.Mighttherebesystematicdifferencesbetweenpeoplewhodoversusdon’t.
- Theonesthatarenotinthelecturemayhaveafulltimejob,childcareresponsibilityetc- Don’ttrustaself-selectingsample-anexampleofsamplingbias
3.2.5 Samplingprocedures
3.2.5.1 Probabilitysampling
• E.g.Tossingthecoin• Awaytoensurethatyoursampleisrepresentativeofthepopulation(onthe
characteristicsdeemedimportantforthestudy)• Basicprinciple:
– Asamplewillberepresentativeofthepopulationifallmembersofthepopulationhaveanequalchanceofbeingselectedinthesample
– Allowstheresearchertocalculatetherelationshipbetweenthesamplestatisticandthepopulationparameter
– Everyonehasanequalchanceofbeingselected=>representativeofthepopulation,providingyouhavelargeenoughsamplesize
3.2.5.2 Sub-typesof
probabilitysamplingo Simplerandomsampleo Systematicrandomsampleo Stratifiedrandomsamplingo Multistageclustersampling
3.2.5.2.1 Simplerandom
sample
• Eachmemberhasanequalandindependentchanceofbeingselected• Definethepopulation,listallmembers,assignnumbers
– Useatableofrandomnumberstoselect,e.g.alloddnumbers– Usea“lottery”method,pullnamesoutofahat– Useacomputerprogramtorandomlyselect
• WorkswellprovidingsamplesizeisnottoosmallExample:FirstisahistogramshowingtheIQscoresofapopulationof1,000,000people.ThepopulationmeanisanIQof100andtheSDis10IQpoints.Let’stakesomesamples.- Smallersamples=>movesawayfromthepopulation
meanandSD- Simplerandomsamplingworksreallywellprovidedthatyoursamplesizedon’tgettoo
small.3.2.5.2.2 Systematicrandom
sampleEveryKthperson
• Systematicismorehistoricwhencomputerwasn’taccessibletouseforrandomization.
• Randomlyselectthefirstpersonthendividethesizeofthepopulationbythesizeofthedesiredsample,andusethistodeterminetheintervalatwhichsampleisselected.– e.g.,toselectasampleof1000peoplefromalistof10,000,randomlyselectthefirst
personandstartthelistwiththem-thenselectevery10thpersonfromthelist• Needtoensurethelistofelementsisnotarrangedinawaythatmeanssystematic
samplingcouldleadtoabiasedsample(e.g.,studentlistinGPAorder!).– e.g.,differentresultsifyoustartwiththe2ndpersonandsampleevery10thperson
beyondthatthanifyoustartwiththe8thpersonandsampleevery10thperson• Wheneverpeopledon’thavetheequalandindependentchanceofbeingpicked,you
areintroducingpossiblefactorsofthingsgoingwrong.Whichshouldweprefer?Simple,orsystematicrandomsampling?
- Simple,lesschanceofanythingsystematicgoingon.
3.2.5.2.3 Stratifiedsampling - Ifyouwanttomakesuretheprofileofthesamplematchestheprofileofthepopulationonsomeimportantcharacteristicse.g.ethnicmix,gender.
- Dividepopulationintosubpopulations(strata)andrandomlysamplesfromthestrataWhyusestratifiedsampling?
- Whenthereisheterogeneitywithinthepopulation,andyouwanttoendupwithasamplewhosecharacteristicsreflecttheproportionalheterogeneityofthepopulation
- Canreducesamplingerrorbyensuringratiosreflectactualpopulation(e.g.,ratioofmalestofemales)
- ToensurethatsmallsubpopulationsareincludedinthesampleNB:
- canhaveproportionalrepresentationordisproportionaterepresentation- butdisproportionatesamplewouldnotbeusedtogeneralisetoentirepopulation,only
thesubgroups
3.2.5.2.3.1 SimpleRandomSamplingVersusStratifiedSampling
Ourpopulationis“AnimalsofWestQueenslandSavannah”–acensusrevealsthattheentirepopulationconsistsof60lions,30tortoisesand10rabbits.SimpleRandomSampling
- Notagoodinferenceofpopulationlevel- Becausethestratificationofthepopulationhasn’t
beenreflectedinthesampleMoreexampleofSimplerandomsampling,sometimeswegetitright
StratifiedRandomSamplingRegardlesshowmanytimeswedothestratifiedrandomsampling,wearealwaysgoingtoendupwiththisfigure,reflectsproportionsinthepopulation.
3.2.5.2.4 Multi-stageClustersampling
Beginwithasampleofgroupingandthensampleofindividualse.g.Ruralsample
- Defineruraltownshipsasthosewithpopulation<X- Getlistingofallrelevanttownships- Takearandomsampleoftownships- Randomlysamplepeoplefromwithintherandomlysampledtownships
- Ifallthesamplearefromthesametown,theremightbesomethingsystematically
differentaboutthattown,e.g.highwithunemployment- Bettertoselectsamplefrombunchofdifferenttown,randomlyselectsthetown,then
randomlyselectsthepeople(multi-stageofprocessinggoingon)
Whenmightyouusethis?- Whenyouhavedifferentregion,differentcharacteristics- e.g.Hungergame
3.2.5.2.5 Multi-Stage/Multi-
PhaseSampling
• Typeofrandomsamplingwhereby• Largersampleobtainedfirst• inordertoidentifymembersofasub-sample• Sub-samplethenrandomlychosenfromforstudy• Good(butcostly)waytoidentifynotreadilyidentifiablesubgroups
E.g.usingAustralianMentalHealthandWellbeingSurveytoidentifypeoplewithpsychoticillness
• largescalesurvey• Needpeoplewithpsychoticillness,“Lowprevalence”(1%ofpopulation)disordersstudy• fromAUSMentalHealthandWellbeingSurveytoidentifylowprevalentdisordersstudy• Thisishow(psychoticillness)peoplearerandomlyselectedbasedontheirprevious
involvement(AustralianMentalHealthandWellbeingSurvey)
3.2.5.2.6 AdvantagesofProbabilitySampling
- Nosystematicbias- Helpsovercomesamplingbias- Ensuresrepresentativeness!
3.2.5.2.7 Problemwith
probabilitysampling
- accesstolistofpeople- costly- difficult- youcanrandomlyselectsomeonebutthereisnoguaranteetheywillagreeto
participateinyourstudy.- Isthereasystematicdifferencebetweenpeoplewhoagreetoparticipateandthosethat
don'tagreetoparticipate?- Self-selection:askingpeople’sagreementtoparticipateinresearchisaformofself-
selection,andthiscancausebias 3.2.5.3 Non-probability
samplingNoteverymemberofthepopulationhasanequalchanceofbeingpartofthesampleWhyusethen?
- Therearenolistsforsomepopulationsunderstudy,- Logisticalorcostrelatedproblem- e.g.
o Thehomelesso Certainoccupations(e.g.,farmers)o Hiddenpopulations(e.g.,peopleinvolvedin“clandestine”activities)o Convenience/resourcerestrictions
3.2.5.3.1 Convenience
Samples- Mostusedinpsychology- Peoplehappentobeavailable• Asampleofavailableparticipants,e.g.,
– studentsenrolledinaparticularcourse– Peoplepassingaparticularlocation
• Self-selecting,non-random• systematicdifferenceonwhoyoumightbeexposedto,e.g.standing
outsidecentrelinkvscasino• Advantages:
– Easy,inexpensive• Disadvantages:
– Nocontroloverrepresentativeness
3.2.5.3.2 SnowballSampling • Likeasnowballrunningdownthehillandgathermoreasitgoes• Usedmainlyforhardtostudysub-populations• Identifyonememberforthestudy,thenaskingfortheirfriendstoparticipate
– e.g.,Gaymen,Homelessyoungpeople,Illegalimmigrants• Involvescollectingdatawithmembersofthepopulationthatcanbelocatedandthen
asksthosememberstoprovideinformation/contactsforothermembersofthepopulation
Problems
- peopletendtoassociatewithpeoplesimilartothemselveso e.g.hipsterbeardedguys,arebeardedmenmoredesirablethanothermen?
Dependingonthestudy,mayormaynotbeaproblem.- Peoplehavethesamenetworkofpeoplewhomayjustbelikethem,beardedmen
havingmorebeardfriends.
3.2.5.3.3 QuotaSample - Non-probabilitysamplingequivalentofastratifiedrandomsampleo Uknowthereisstratawithinyourpopulation,andyouwanttoreflectrelative
proportionofthosedifferentstratapopulationinthesample- Butyoudon’t/aren’tabletosamplerandomlyfromeachstrataasyoudoinstratified
randomsamples- Soyouusenon-probabilitysampling
Problem
- can'tguaranteerepresentativeness
3.2.5.3.4 Purposive/judgmentsampling
- Clearpurposetothesamplingstrategy:selectkeyinformants,atypicalcases,deviantcasesoradiversityofcases.
- Samplinginawaytryingtofindparticularcharacteristic,togetparticularinformation- Oftenusedto:
– Selectcasesthatmightbeespeciallyinformative– Selectcasesinadifficult-to-reachpopulation– Selectcasesforin-depthinvestigation
Examples:
– Studyingtheproblemsexperiencedbynewimmigrants– Interviewkeypeopleinvolvedinagenciesthathelpimmigrantssuchasethnic
welfaregroups,communityimmigrationlegalaidgroups– Interviewingpeoplewithextensiveexperiencewithimmigrantslikelytoprovide
richdata– Comparisonofleft-wingandright-wingstudents
– Maynotbepossibletosampleallleft-wingandright-wingstudents– Instead,youcouldsamplethemembershipofleft(e.g.,SocialistAlliance)and
right-winggroupsoncampus(e.g.,youngliberals)
3.2.5.4 WhichSamplingMethod?
- Asamajoraimofquantitativeresearchistheabilitytogeneraliseresultstheultimatemethodisaprobabilitysamplingone.Representative
- Howeverthisisoftennotworkableorfeasiblegivenresources,time,thespecifictargetpopulation.
- Samplingmethodusedshouldbefullyexplainedtoparticipants- andcaveatsaboutthelikelygeneralisabilityofresultsmadeaccordinglysothatthe
readercanreviewyourresultsinaninformedway.- Wewillalwayshavenon-optimalsamplingmethodaswecan’tjusthavethecensusof
thewholepopulationandselectthesub-populationfromit.Thereforearesearchpaperneedstostateclearlywhathasbeendoneandtheproblemassociatedwithit.
3.2.6 Howmanypeople
shouldyoutest?SampleSize-aswehavealreadyseenthesizeofyoursamplecaninfluencehowrepresentativeitisofthepopulationItisthereforeimportanttoensurethatyoursamplesizethatisappropriate
3.2.6.1 DeterminingSampleSize1
HowmanyparticipantsdoIneedformystudy?- Largelydeterminedbytheanalysisyouplantoconductwiththedataderived.Howare
yougoingtotreatthedata?- Generallythemorecomplextheanalysisthelargerthesampleyourequire- Increasesinsamplesizebringwiththemincreasesinaccuracy/precision/reduces
samplingerror.- Greaterheterogeneityofthepopulation,greatervariationinthepopulation,thelarger
thesampleshouldbetocaptureandreflecttheheterogeneityinthesamplesize.- Therearemanytextswhichwillprovideyouwithsamplesizerequirementsforany
givenstatisticaltestaswellascalculationtoolswhichwillprovideyouwithasamplesizegivenanumberofparameters.
*Heterogeneity–beingdiverseincontent.
3.2.6.2 DeterminingSampleSize2
Largersamplesizesareneededifpopulationis:- Heterogeneous- youwanttobreakdownthesampleintomultiplesubcategories
o e.g.,lookatmalesandfemalesseparately- whenyouexpectasmalleffectorweakrelationship- whenyouuselessefficientmethodsofsampling
o e.g.,clustersampling- forsomestatisticaltechniques- ifyouexpectalowresponserate
ifyouareusingnotrepresentativesamplingmethod,thenerronthesideofhavinglargersample
3.2.6.3 DeterminingSampleSize3
Fivesimplerulesfordeterminingsamplesize1. ifpopulationislessthan100,useentirepopulation2. largersamplesizesmakeiteasiertodetectaneffectorrelationshipinthepopulation3. comparetootherresearchstudiesinareabydoingaliteraturereview4. useapowerTableforaroughestimate5. useasamplesizecalculator(e.g.,G-Power)- whatsamplesizeisneededforaneffectofaparticularsize
3.3 Makesome
measurements
3.3.1 OperationalisationofIVsandDVs
OperationalisationofIVs– Howareyougoingtomanipulateit?Howisitmanipulated(ifyoucan’t)?
OperationalisationofDVs– Howareyougoingtomeasureit?– Whatmeasurementsareyoutaking?
Howmightwemeasureintoxication?Example,DV:usingalcoholconsumption- Theirlooks,ortheirabilitytowalkstraightlineisnotgoodenough- Breathalyzerandbloodtestmayalsonotbegoodmeasurementforalcoholic,astheir
bodyisusedtotoxication.
3.3.2 ReliabilityandValidity Reliability- Doesourmeasurementinstrumentbehavesensibly?- Doesitalwaysmeasurethesamethinginthesameway?
Validity- Arewemeasuringwhatwethinkwearemeasuring?
o Arewemeasuringintoxicationwhenwemeasurebloodalcohollevel?o Doesthebloodmeasuregivesthesameresulteverytimeweuseit?
Ifwelookatsomeone,ismyviewofhisintoxicationthesameasyours?Theseareallquestionsweneedtoaskforreliabilityandvalidity.
3.3.3 ReliabilityandValidity Appliesmostlytoindexes/scales– Inpsychology,wetrytonumeratethereliabilityandvalidityofthingslikequestionnaire,
surveyHowdoweassesswhetherourmeasures/operationalisationsaregood?
– Aretheyvalid?– Aretheyreliable?
Snagisthatyoucan’tassesstheseuntilAFTERyouhavedevelopedyourquestionnairesandusedthem
– Thisiswhyapilottestcanbesobeneficial
– Thisiswhymanypeoplechosetouseestablishedmeasuresratherthandeveloptheirownandtaketherisk
– E.g.useIQtestExamples- Notreliable/notvalid:birthdayandstarsigns- Reliable/notvalid,judgepeople’sintelligencebasedontheirlooks(wethinkpeople
wearglassesaresmarter)- Measuringwitharuler(reliable,valid)- Notreliable/valid:notpossiblescenario,ifyoucan’trelyonyourmeasurement,you
can’ttrustitsvalidity
3.3.3.1 Therelationshipbetweenreliabilityandvalidity
Canameasurebereliablebutnotvalid?– Yes!Youcouldhaveaconsistentmeasurethatdoesnotactuallymeasuretheconstruct
Canameasurebevalidbutnotreliable?– No!Ifyourmeasuredoesn’tconsistentlyanddependablymeasuretheconstructit
cannotpossiblybemeasuringwhatitsaysit’smeasuringPhysicalmeasurementsclearandeasytoseethattheyarereliablyandvalidi.e.wecansee.Psychologicalmeasurementsarealittlebitmoretricky.
3.3.3.2 Reliability • Theconsistencyorrepeatabilityofthemeasuremento SayIweightmyselfonsomescalesatonepointintimeandthenweighmyself5
minslateranditsaysI’m5kilosheavier.o Conclusion:dodgyscale,don’tuseit.o Scientificconclusion:thescalesareanunreliablemeasurementinstrument
3.3.3.3 TypeofReliability
test• Stabilityofthemeasure(Test-retest)• Internalconsistencyofthemeasure(Split-half,Cronbach’salpha)• Agreementorconsistencyacrossraters(Inter-raterreliability)
• Acrossdifferentpeoplemakingthejudgementormends
3.3.3.3.1 Test-retestreliability
Doesyourtestmeasurethesamethingeverytimeyouuseit?• Addressesthestabilityofyourmeasure
• Sameanswerseverytime• Youadministerthemeasureatonepointintime(Time1)• thengivethesamemeasuretothesameparticipantsatalaterpointintime(Time2)• Hopingtherewillbeacorrelationbetweenthetwotimes• Youcorrelatethescoresonthetwomeasures• Ifitishighcorrelation,thenithashightest-retestreliability• Ifitistoolow,thenitisnotworthusingitasthetest-retestliabilityislow
3.3.3.3.1.1 Problemwith
Test-retestImaginethatyouwanttotestwhethergivingpeoplevitaminsupplementscanimprovesapersonsIQ.Twomainproblems:1. Memoryeffect
– youmightrememberthequestionsandlookuptheonesyoudidn’tknow2. Practiceeffect
– Performanceimprovesbecauseofpracticeintesttaking
• Iftooshortthere’sagreaterriskofmemoryeffects• Iftoolongthere’sariskofothervariables(e.g.,additionallearning)influencingresults
3.3.3.3.2 Split-halfreliability:
isyourmeasureinternallyconsistent
Psychologytestisnotsimplyaonequestionsurvey,e.g.areyouanextrovert?Testwillhaveasetofitems.byendorsingasetofitems,itwillleadyouhighin,e.g.extrovertcategory.Sointhetest,therearesetofsub-itemsthatrelatestotheconstructofextrovert,introvert.Inordertodefinethepersonality.Eachofthesepersonalitytraitsisassessedbydifferentsetofitems.Split-halfreliability:Arethedifferentitemsconstanttowhattheyaremeasuring?
• Youadministerasinglemeasureatonetimetoagroupofparticipants• But,foryourpurposes(ofunderstandofpsychometricqualityofyourexperiment)you
splitthemeasureintotwohalves.OdditemisgoingtopoolA,evenitemsisgoingtopoolB.
• andyoucorrelatethescoresonthetwohalvesofthemeasure(highercorrelationmeansgreaterreliability)
• e.g.IQtest,2setofquestionsinonetest.Ifbothsethavehighcorrelation.Thissuggests2halvesaremeasuringthesameconstruct.
• Thisway,youdon’tneedtodotest-retestreliability,rather,youaskinternalconsistencyofthetest
• Strength:eliminatesmemory&practiceeffects• Limitation:Arethetwohalvesreallyequivalent?
• UseCronbach’sAlpha(measureofinternalconsistency.Itisconsideredtobeameasureofscalereliability.)
3.3.3.3.2.1 Cronbach'sAlpha • Assessesthe‘internalconsistency’ofyourmeasure
– i.e.,tellsyouhowwelltheitemsorquestionsinyourmeasureappeartoreflectthesameunderlyingconstruct
• Youwouldgetgoodinternalconsistencyifindividualsrespondinapproximatelythesamewaytoquestionsonyoursurvey
– Differentitemsofthesametestmeasuringthesameconstruct• Mathematicallyit’stheequivalentoftheaverageofallpossiblesplit-halfreliabilities• Coefficientalphacanrangefrom0to1.00
– Thecloserthealphaisto1.00,thebetterthereliabilityofthemeasure
3.3.3.3.3 Inter-raterorinter-observerreliability
• Dodifferentratersmeasurethesamething?• Relythejudgementoftheobservers.• Checkingthematchbetweentwoormoreratersorjudges
o E.g.peopleobservethebehaviourofyoungbabieso codingvideosforinfant“lookingtime”–needtochecktheagreementamongst
thecoderso Codingthelengthoftimeaninfantislookingatoneparticularobjectvsanother
• Thereisadegreeofsubjectivityofinterpretationinthesekindsofmeasures.o Wastheinfantdirectlylookingattheobjectorclosetotheobject?
• Wherethereisapossibilityforsubjectivity,whatpeopleareinterestedinistheinter-raterreliability
o Ifpeoplearetrainedproperly,differentpeopleshouldbehighlycorrelatedwitheachotherwithrespecttothesubjectjudgementtheymake
• Arethedifferentpeoplemakingthejudgementbehavingsimilarlyinthesetofjudgementtheyaremaking?
o Highcorrelation=Highdegreeofreliability
3.3.3.3.3.1 Calculationofinter-raterreliability
– nominalorordinalscale– thepercentageoftimesdifferentratersagree
– intervalorratioscale– correlationcoefficient
3.3.3.4 Validity • Arewemeasuringwhatwethinkweare?
– Isourmeasurecredible,isitbelievable?• Whyisvalidityanissue?
– Forreliability,wecancomewiththeseclearmeasuresofthedegreetowhichmeasurementisreliable
– Forvalidity,many(ifnotmost)variablesinsocialresearchcannotbedirectlyobserved.Youhavetoinferonthebasisofsomething
• e.g.,motivation,satisfaction,helplessness• needtouseinstrumentsuchasquestionnaire
• Thechallenge:– Wecanquantifyreliability– Wecan’tquantifyvalidity– tomakeajudgmentcallaboutwhetherwearemeasuringwhatwethinkwe’re
measuring
3.3.3.4.1 TypesofValidity • Facevalidity