PISA 2012 FAQ

PISA 2012 FAQ

•WhatisPISA?•WhatisPISA2012?•WhydoesCanadaparticipateinPISA?•WhoaretheCanadianpartnersinvolvedinPISA?•HowmuchdoesPISAcost?•HowmanystudentsinCanadatookpartinPISA2012?•WhyisCanada’ssamplesizesomuchlargerthanthatofmanyothercountries?

•WhatisthedifferencebetweentheOECDreportandtheCanadianreport?

•WhatisthePISAassessmentcycle?•Howcanstudentperformanceacrosscountries,economies,andCanadianprovincesbecompared?

•Howcanwecompareaveragescores?•HowwasmathematicalliteracydefinedinPISA2012?•WhatwerethelevelsofmathematicsachievementinPISA2012?•HowwerereadingliteracyandscientificliteracydefinedinPISA2012?

•IstheassessmentfairtoCanadianstudents?•HowwilltheresultsfromPISAbeused?

WhatisPISA?Launchedin1997bytheOrganisationforEconomicCo-operationandDevelopment(OECD)andfirstadministeredin2000,theProgrammeforInternationalStudentAssessment(PISA)isatriennialsurveyoftheknowledgeandskillsof15-year-oldsneartheendoftheircompulsoryschooling.PISAdrawsonleadinginternationalexpertisetodevelopvalidcomparisonsacrosscountriesandculturesandprovideseducationresearchers,policy-makers,andthepublicwithcomprehensiveinternationaldatainthreecorelearningareas:mathematics,reading,andscience.

Todate,studentsrepresentingmorethan70countriesandeconomieshaveparticipatedintheassessment.

PISA2012isthefifthiterationoftheOECDassessment.In2012,mathematicswasthefocus,buttheassessmentalsoincludedtestingofreading,science,problemsolving,andfinancialliteracy.PISA2012evaluatedapproximately510,000studentsfrom65countriesandeconomies.Theassessmentcameintwoformats:paper-and-pencilandcomputer-based.Inaddition,contextualquestionnaireswereadministeredtostudentsandschoolprincipalstocollectdataonstudent,family,andschoolfactorsthatcanhelpexplaindifferencesinperformance.

WhatisPISA2012?

WhydoesCanadaparticipateinPISA?Canada’sparticipationinPISAensuresthattheperformanceofCanadianstudentsinthecorelearningareasofmathematics,reading,andsciencecanbemeasuredagainstthatofstudentsin65othercountriesaroundtheworld.

ThedatacollectedbyPISAnotonlyprovideacomparablemeasureofsuccessbutcanalsobefurtheranalyzedbypolicy-makersandresearcherstohelptoidentifythecharacteristicsofhigh-performingeducationsystems.

WhoaretheCanadianpartnersinvolvedinPISA?EmploymentandSocialDevelopmentCanada(ESDC),StatisticsCanada,andthe10provincialministriesanddepartmentsofeducation,throughtheCouncilofMinistersofEducation,Canada,arepartnersinadministeringPISA.TheCanadianreportwasdevelopedbyCMECandESDC.

HowmuchdoesPISAcost?Attheinternationallevel,PISAisfundedbycontributionstoOECDfromparticipatingcountries.InCanada,mostofthedirectcostsforPISAareassumedbyESDC,whileindirectcostsarecoveredbytheprovincesthroughCMEC.ForthethreeyearsofthelatestcycleofPISA,thedirectcostwasapproximately$2.4millionperyear.

HowmanystudentsinCanadatookpartinPISA2012?

InCanada,approximately21,000studentsfromabout900differentschoolsinthe10provincestookpartinPISA2012.TheassessmentwasadministeredinEnglishorinFrenchaccordingtotheprovince’sschoolsystem.ThetotalnumberofstudentswhoparticipatedineachprovinceisoutlinedintheappendixtothePISA2012Canadianreport.Currently,Yukon,NorthwestTerritories,andNunavutdonotparticipateinPISA,nordoAboriginalstudentsfromband-operatedschools.

WhyisCanada’ssamplesizesomuchlargerthanthatofmany

othercountries?Thesmallersamplesizesusedinmanycountriesprovideresultsthatarevalidatthenationallevelonly.InCanada,educationpolicyisdevelopedandimplementedattheprovincialandterritoriallevel,soalargersampleisrequiredtoobtainstatisticallyreliableresultsineachjurisdiction.Inaddition,Canadaoversampledofficial-languageminoritiesinsevenprovincestolearnmoreabouttheskillsofthatpopulation.

WhatisthedifferencebetweentheOECDreportandtheCanadianreport?TheOECDPISAreportfocusesontheresultsforeachcountryandeconomicregionthathaschosentoparticipateintheassessment.Italsooffersdetailedquantitativeandqualitativeinformationonpersonalandschoolfactorsthatinfluenceastudent’sperformance.

CanadareleasesitsowncompanionreportatthesametimeastheOECD

reporttoprovidefurtherinformationonstudentperformanceattheprovinciallevelandshowcomparativeresultsacrossandbetweenCanadianprovincesaswellasothercountriesandeconomies.TheCanadianreportalsoprovidesperformanceresultsforEnglish-andFrench-languagesystemsandbygender.

WhatisthePISAassessmentcycle?PISAoperatesonacyclethatallowsforcomparisonofresultsovertimeinallthreecorelearningareas―reading,mathematics,andscience.Thesedatahelpparticipatingcountriesandeconomies―andCanadianprovinces―understandhowtheperformanceoftheireducationsystemsmayhave

changedovertime.Since2000,eachPISAassessmenthashadamajordomainandtwominordomains.Themajordomain(showningreeninthetablebelow)haschangedeverythreeyears.

2000 Reading Math Science

2003 Reading Math Science

2006 Reading Math Science2009 Reading Math Science

2012 Reading Math Science

2015 Reading Math Science

Asthetableshows,in2012,mathematicswasthemajordomainforasecondtime,withreadingandscienceasminordomains.TwodevelopmentaldomainswereaddedtoPISA2012:problemsolvingandfinancialliteracy.Canadaparticipatedintheproblem-solvingcomponent,butnotfinancialliteracy.

PISA2015willbethesixthroundtobecompleted,andsciencewillbeitsmajordomain.The2015assessmentwilllikelybefullycomputer-based,andCanadianstudentsareexpectedtobeassessedintwoadditionaldevelopmentaldomains:collaborativeproblemsolvingandfinancialliteracy.

Howcanstudentperformanceacrosscountries,economies,andCanadianprovincesbecompared?Educationsystemsandschoolprogramsdifferfromonejurisdictiontoanother,socomparingresultscanbeacomplextask.PISAallowsavarietyofeducationsystemstobecomparedaccordingtoasetofcommonbenchmarksinreading,mathematics,andscience.ThesebenchmarkshavebeenestablishedthroughextensiveconsultationswithparticipatingOECDcountriesandwiththeguidanceofstatisticians,psychometricians,andotherpedagogicalexpertsfromaroundtheworld.They

reflectingeneraltermswhat15-year-oldstudentsareexpectedtoknowandbeabletodointhethreecorelearningareasinordertomeetreal-lifechallenges.

ByagreeingtothecommonbenchmarksdevelopedbyPISAtoevaluatestudentachievement,Canadianprovinces—aswellasOECDcountriesandeconomies—areabletodeterminetheirperformanceinrelationtootherPISAparticipants,eveniftheirapproachestoeducationdiffer.

Howcanwecompareaveragescores?Becausescoresarebasedonsamplesofstudentsfromeachcountryandprovince,wecannotsaywithcertaintythatthesescoresarethesameasthosethatwouldhavebeenobtainedhadall15-year-oldsbeentested.A“confidenceinterval”isarangeofscoreswithinwhichthescoreforthepopulationislikelytofall95percentofthetime,or19timesoutof20.

Whencomparingtwocountriesortwoprovinces,thetwoaveragescorescannotbesaidtodifferiftheconfidenceintervalforthetwoaveragescoresoverlaps.

Forexample,countriesperformingataboutthesamelevelasCanadahaveaconfidenceintervalfortheaveragescorethatoverlapswithCanada’sconfidenceinterval.

Incaseswheretheconfidenceintervalsoverlaponlyslightly,anadditionaltestisusuallyconductedtodeterminewhetherthereisasignificantdifferencebetweenthetwopopulations.

HowwasmathematicalliteracydefinedinPISA2012?OECDdefinedmathematicalliteracyas:“…anindividual’scapacitytoformulate,employ,andinterpretmathematicsinavarietyofcontexts.Itincludesreasoningmathematicallyandusingmathematicalconcepts,procedures,facts,andtoolstodescribe,explain,andpredictphenomena.Itassistsindividualstorecognizetherolethatmathematicsplaysintheworldandtomakethewell-foundedjudgmentsanddecisionsneededbyconstructive,engaged,andreflectivecitizens.”

PISA’sassessmentofmathematicalliteracyfocusednotonlyonoverallskillsinmathematicsbutalsoonspecificmathematicssubdomainssuchasspaceandshape,changeandrelationships,quantity,anduncertainty.

Whatwerethelevelsofmathematicsachievementin

PISA2012?Mathematicsachievementwasdividedintosixlevels,eachrepresentingagroupoftasksofincreasingdifficulty,andwithLevel6beingthehighestandLevel1thelowest.OECDconsidersLevel2tobethebaselinelevelofproficiencythatisrequiredtoparticipatefullyinsociety.Adifferenceofonelevelisconsideredanimportantdifferenceinstudentperformance.

HowwerereadingliteracyandscientificliteracydefinedinPISA2012?InPISA2012,readingliteracywasdefinedasanindividual’scapacitytounderstand,use,reflecton,andengagewithwrittentextsinordertoachieveone’sgoals,developone’sknowledgeandpotential,andparticipateinsociety.Scientificliteracywasdefinedasanindividual’scapacitytousescientificknowledge,identifyquestions,anddrawevidence-basedconclusionsinordertounderstandandmakedecisionsaboutthenaturalworldandthechangesmadetoitthroughhumanactivity.

IstheassessmentfairtoCanadianstudents?CanadahasactivelyparticipatedinPISAtoensurethattheuniquenessofourcountry’seducationsystemsistakenintoaccount.Factorssuchaslinguisticdifferences,ruralandurbanschoollocations,andculturalinfluenceswereallconsideredwhendevelopingtheassessment.Inaddition,theuniversalframeworkforeachsubjectincorporatesanagreed-uponphilosophyforallcountriesthatisbaseduponthelatestpedagogicalresearch.

InthesensethatCanadianstudentsanswerthesamequestionsasstudentsfromeveryothercountry,itisveryfair.Theassessmentisalsouniqueinthatitisnottiedtothecurriculumofanyprovinceorparticipatingcountryoreconomybutisinsteadafairmeasurementofstudents’abilitiestousetheirlearningskillstosolvereal-lifesituations.

HowwilltheresultsfromPISAbeused?Theassessmentresultswillbeusedbyprovincialeducationministries/departmentsandresearcherstoevaluatethequalityofoureducationsystemsandinformpolicydevelopment.

OECDplanstoproducefurtherthematicreportsbasedonthePISA2012performancedataandtheresultsofthestudentandschoolquestionnaires.CMECanditsfederalpartnerswillalsoconductanalysesofPISAdata,andprovinceswillundertakeanalysespertinenttotheiruniqueeducationalcircumstances.

www.cmec.ca