+ All Categories
Home > Documents > CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs ›...

CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs ›...

Date post: 07-Jun-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
123
BLENDED LEARNING PROGRAMS CRITERIA FOR EXCELLENCE IN Quality Scorecard ONLINE LEARNING CONSORTIUM
Transcript
Page 1: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

B L E N D E D L E A R N I N G P R O G R A M SC R I T E R I A F O R E X C E L L E N C E I N

Quality ScorecardO N L I N E L E A R N I N G C O N S O R T I U M

Page 2: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Quality Scorecard Handbook:  

Criteria for Excellence in  

Blended Learning Programs  

Editors:

Jennifer Mathes

Karen Pedersen

The Online Learning Consortium 

Page 3: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ThepurposeoftheOnlineLearningConsortiumistohelplearningorganizationscontinuallyimprovethequality,scale,andbreadthoftheironlineprogramsaccordingtotheirowndistinctivemissions,sothateducationwillbecomeapartofeverydaylife,accessibleandaffordableforanyone,anywhere,atanytime,inawidevarietyofdisciplines.

Thispublicationcontainsinformationobtainedfromauthenticandhighlyregardedsources.Reprintedmaterialisquotedwithpermission,andsourcesareindicated.Awidevarietyofreferencesarelisted.Reasonableeffortshavebeenmadetopublishreliabledataandinformation,buttheauthorsandthepublishercannotassumeresponsibilityforthevalidityofallmaterialsorfortheconsequencesoftheiruse.

Neitherthispublicationnoranypartmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,includingphotocopying,microfilming,andrecording,orbyanyinformationstorageorretrievalsystem,withoutpriorpermissioninwritingfromthepublisher.

TheconsentoftheOnlineLearningConsortiumdoesnotextendtocopyingforgeneraldistribution,forpromotion,forcreatingnewworks,orforresale.SpecificpermissionmustbeobtainedinwritingfromtheOnlineLearningConsortiumforsuchcopyingoruseinotherworks.Directallinquiriestopublisher@onlinelearning‐c.org.

Coverdesigncopyright©2016byOnlineLearningConsortium™Allrightsreserved.Published2016

PrintedintheUnitedStatesofAmerica

InternationalStandardBookNumber978‐1‐934505‐06‐9(digitalebook)

Page 4: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Permissions

TheQualityScorecardforBlendedLearningProgramshasmanyindicatorsthatareverysimilartotheQualityScorecardfortheAdministrationofOnlineLearning.Assuch,thereaderwillnoticethatdescriptionsformanyindicatorsinthisHandbookandaccompanyingrubricareidenticalorhighlysimilartothosefoundintheQualityScorecard2014Handbook:CriteriaforExcellenceintheAdministrationofOnlinePrograms.AsbotharepublishedbytheOnlineLearningConsortium,theeditorshavebeengrantedpermissiontoreuseappropriatecontentintheQualityScorecardHandbook:CriteriaforExcellenceinBlendedLearningProgramsrevisingasneededtomaintainrelevancetoblendedlearning.

Page 5: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs
Page 6: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Table of Contents

ExpertPanelists........................................................................................................................................................................vi 

AbouttheEditors.....................................................................................................................................................................ix 

Foreword.....................................................................................................................................................................................x 

IntroductiontotheQualityScorecardforBlendedLearningPrograms...........................................................1 

InstitutionalSupport...............................................................................................................................................................3 

TechnologySupport.............................................................................................................................................................16 

CourseDevelopment&InstructionalDesign.............................................................................................................25 

CourseStructure....................................................................................................................................................................43 

TeachingandLearning........................................................................................................................................................53 

FacultySupport......................................................................................................................................................................60 

StudentSupport.....................................................................................................................................................................68 

EvaluationandAssessment...............................................................................................................................................81 

BlendedQualityScorecardRubric..................................................................................................................................95 

Page 7: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Expert Panelists OursincerestthanksgoestothefollowingblendedlearningprofessionalswhocontributedtothefirsteditionofQualityScorecardHandbook:CriteriaforExcellenceinBlendedLearningPrograms.Throughyourcontributionthisfirsteditionwillprovideagoodfoundationtosupportqualityinblendedlearningthroughouthighereducation.

PhyliseBannerClarksonUniversity

JenBradleyNorthamptonCommunityCollege

StevenCrawfordArizonaStateUniversity

DebMerriamDaemenCollege

MelanieShawNorthcentralUniversity

ThomasSlomkaUniversityofBuffalo

CourtneyStewartUtahStateUniversity

vi Quality Scorecard for Blended Learning Programs

Page 8: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

TheOnlineLearningConsortiumQualityScorecardfortheAdministrationofBlendedLearningProgramsistheresultofaDelphiresearchstudyconductedbyDr.KayeSheltonin2015.Thankyoutothefollowingblendedlearningeducatorswhowillinglyparticipatedinthestudyandcontributedtheirexpertise:

VictoriaAbramenka

FortHaysStateUniversity

ReneeAitken

WrightStateUniversity

BarbaraAltman

TexasA&MUniversityCentralTexas

ElizabethAnderson

UniversityofDenver

PhyliseBanner

PhyliseBannerConsulting

RachelBehling

GrandCanyonUniversity

JenBradley

NorthamptonCommunityCollege

BrookeBurkle

RelayGraduateSchoolofEducation

LisaBunkowski

TexasA&MUniversityCentralTexas

TerryBuxton

RegisUniversity

MauricioCadavid

CaliforniaStateUniversity,SanBernardino

MargaretChristmas

ConcordiaUniversityIrvine

MarieCini

UniversityofMarylandUniversityCollege

RosemaryCleveland

GrandValleyStateUniversity

KristyConger

UniversityofTennessee

StevenCrawford

ArizonaStateUniversity

DarrylDraper

OldDominionUniversity

TimmoDugdale

UniversityofWisconsin‐Madison

LindaFutch

UniversityofCentralFlorida

JulieGolden

FloridaAtlanticUniversity

CharlesGraham

BrighamYoungUniversity

DougGrove

ConcordiaUniversityIrvine

NancyHale

PaceUniversity

LisaHalverson

BrighamYoungUniversity

CindyHayden

EasternKentuckyUniversity

JulieHinkle

UniversityofCentralFlorida

LynetteHudiburgh

MiamiUniversity

SandraJohnson

NewMexicoStateUniversity

Expert Panelists vii

Page 9: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

KimberlyKenward

GrandValleyStateUniversity

ShannonMcCarty

RioSaladoCollege

PaigeMcDonald

TheGeorgeWashingtonUniversity

TawnyaMeans

UniversityofFlorida

DeborahMerriam

DaemenCollege

PatMielke

WesternTechnicalCollege

NoreenBarajas‐Murphy

UniversityofLaVerne

JasonNeben

ConcordiaUniversityIrvine

MaryNiemiec

UniversityofNebraska

SunayPalsole

TheUniversityofTexasatSanAntonio

RobertPaz

NewMexicoStateUniversity

BrendaPerea

ColoradoCommunityCollegeSystem

KarenPollack

ThePennsylvaniaStateUniversity

LauriePosey

TheGeorgeWashingtonUniversity

DonnaRedman

UniversityofLaVerne

LindaRuest

SouthernNewHampshireUniversity

MelanieShaw

NorthcentralUniversity

ChadShorter

UniversityofWisconsin‐Madison

ThomasSlomka

UniversityofBuffalo

CourtneyStewart

UtahStateUniversity

KarenSwan

UniversityofIllinoisSpringfield

KristenTaubman

NorthernArizonaUniversity

AlexisTerrell

OregonStateUniversity

JanetThompson

TheCollegeofWestchester

KelvinThompson

UniversityofCentralFlorida

BrianUdermann

UniversityofWisconsin–LaCrosse

viii Quality Scorecard for Blended Learning Programs

Page 10: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

About the Editors Jennifer Mathes ‐ Dr.JenniferMathesistheDirectorofStrategicPartnershipsattheOnlineLearningConsortium(OLC).Inthisrole,sheprovidesleadershipinresearching,scoping,managing,andevaluatingkeyprojects,programs,andcustomizedsolutionsthatarealignedwithOLC’sAdvisoryServicesdomesticallyandinternationally.Dr.Matheshas20yearsofexperienceinbothpublicandprivatehighereducationwhereshehassupportedonlinelearninginitiativessinceshetaughtherfirstonlinecoursein1997.Shehasbeeninstrumentalinworkingwithstart‐uponlineinitiativesaswellasleadinggrowthininstitutionswithanexistingonlineprogram.SheholdsadoctorofphilosophydegreeineducationfromtheUniversityofIllinoisatUrbana‐Champaignwhereshewroteherdissertationon“PredictorsforStudentSuccessinOnlineEducation.”ShealsohasearnedamasterofsciencedegreeinbusinesseducationandabachelorofsciencedegreeinmasscommunicationsfromIllinoisStateUniversity.

Karen Pedersen ‐Dr.KarenPederseniscurrentlytheChiefKnowledgeOfficerfortheOnlineLearningConsortium(OLC)wheresheservesasathoughtleaderandpresenteronthetopicsofquality,learnersuccess,militaryeducationprograms,institutionaltransformation,andexcellence.PriortojoiningOLC,Pedersenservedasaseniorleaderinvariousadministrativepositionsforbothpublicandprivatehighereducationinstitutions.Sheledaward‐winningacademicoperations,marketingunits,technologyteams,andstudentsupportservicesdivisions.Duringhercareer,shehasenvisionedandbuiltasuccessfulonlineoperationfromthegroundupandlaunchedover25innovativeonlinedegreeprograms.Otherexperiencesincludeleadingasystem‐wideenrollmentmanagementtransformation,servingonthelaunchteamforacompetency‐basededucationinitiative,andbuildingacademicpartnershipsdomesticallyandinternationallyincludinginBrazil,HongKong,Indonesia,Malaysia,andSingapore.EvenbeforecomingtoOLC,herinterestinacademicqualityledhertodeploytheOLCQualityScorecardfortheAdministrationofOnlineProgramsattwoinstitutionswhereshealsoservedasa“Leanchampion”torealizebusinessprocessimprovementsacrosskeyoperationalareassuchasrecruitment,advising,andretention.Priortostartingheradministrativecareer,Pedersenservedasafull‐timefacultymemberandcurrentlyservesasafacultymemberfortheOLC/PennStateInstituteforEmergingLeadershipinOnlineLearning(IELOL).SheholdsbachelorofscienceandmasterofsciencedegreesfromtheUniversityofNebraska‐LincolnandadoctorofphilosophydegreefromOklahomaStateUniversity.

About the Editors ix

Page 11: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Foreword

In2011,theOnlineLearningConsortium(OLC)begansharingourfirstcomprehensiveframeworkusedtoevaluateonlineprograms.DevelopedbyDr.KayeSheltonaspartofherdoctoraldissertation,theQualityScorecardfortheAdministrationofOnlineProgramshasbeenusedbymanyhighereducationinstitutionstovalidatethequalityoftheirprogramforavarietyofstakeholders.Recognizingthatthetimehadcometomeettheneedforasimilartooltoevaluateblendedlearningprograms,theOLCintroducedtheQualityScorecardforBlendedLearningProgramsin2015.

ThishandbookwasdevelopedtoaccompanytheQualityScorecardforBlendedLearningProgramsandprovideblendedlearningprogramadministratorswitharesourcebasedonthebestpracticesidentifiedbyourpanelofexpertsinblendedlearning.Inthishandbook,descriptionshavebeenprovidedforall70indicatorsalongwithrecommendationsformeetingeachstandard.AspecialthankstoDr.SheltonwholedtheresearchtodeveloptheQualityScorecardforBlendedLearningProgramsandcontinuestoidentifynewwaystosupportfaculty,staffandadministratorsintheirquesttocreatequalitylearningexperiencesinonline,blendedanddigitallearningasawhole.

Wewouldalsoliketothankthe54experiencedblendedprogramadministratorsthatservedasourexpertpanelfortheDelphistudy.TheQualityScorecardforBlendedLearningPrograms,andthishandbook,wouldnotbepossiblewithouttheirwillingnesstosharetheirknowledgeofblendedlearningprogramadministrationandparticipateinthestudy.

Aswefindnewwaystomeetstudentneedsandtechnologyisincreasinglyreliedontodelivercoursecontent,weanticipatecontinuedgrowthinblendedlearning.Withthisgrowth,itishighlyimperativethatinstitutionscreateaneffectivelearningenvironmentthatsupportsstudentsuccess.WiththeQualityScorecardforBlendedLearningPrograms,institutionsnowhaveawaytovalidatetheirblendedlearningprogramwhichincludestheoptionforaformalreviewprovidedbytheOLCwhichcouldleadtoanofficialOLCendorsement.

JenniferMathes,Ph.D.DirectorofStrategicPartnershipsOnlineLearningConsortiumKarenPedersen,Ph.D.ChiefKnowledgeOfficerOnlineLearningConsortium

Page 12: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs
Page 13: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

 

 The  Online  Learning  Consortium’s  Quality  Scorecard  for  Blended  Learning  Programs  (QSC-­‐‑BL)  is  an  easy  to  use  process  for  measuring  and  quantifying  elements  of  quality  within  blended  learning  programs  in  higher  education.  By  evaluating  each  of  the  respective  quality  indicators  within  the  established  eight  categories,  an  administrator  of  blended  learning  programs  can  determine  strengths  and  areas  of  improvement  within  their  program.  The  results  of  this  tool  can  be  used  to  support  program  improvement  and  strategic  planning  initiatives.  The  Quality  Scorecard  for  Blended  Learning  Programs  could  also  be  used  to  demonstrate  to  accrediting  bodies,  elements  of  quality  within  the  program  as  well  as  an  overall  level  of  quality.  

The  Quality  Scorecard  for  Blended  Learning  Programs  contains  70  unique  quality  indicators  each  indicator  is  worth  up  to  three  points.  The  70  quality  indicators  are  within  eight  different  categories:  Institutional  Support;  Technology  Support;  Course  Development  and  Instructional  Design;  Course  Structure;  Teaching  and  Learning;  Faculty  Support;  Student  Support;  and  Evaluation  and  Assessment.  

The  administrator  will  first  determine  at  what  level  their  program  meets  the  intent  of  each  of  the  quality  indicators  after  examining  all  procedures  and  processes.  The  next  step  is  to  determine  how  that  score  can  be  substantiated  or  demonstrated  to  others.    Each  point  value  is  described  below.  

0  points  =  Deficient.  The  administrator  does  not  observe  any  indications  of  the  quality  standard  in  place.  

1  point  =  Developing.  The  administrator  has  found  a  slight  existence  of  the  quality  standard  but  difficult  to  substantiate.  Much  improvement  is  still  needed  in  this  area.  

2  points  =  Accomplished.  The  administrator  has  found  there  to  be  moderate  use  and  can  substantiate  the  quality  standard.  Some  improvement  is  still  needed  in  this  area.  

3  points  =  Exemplary.  The  administrator  has  found  that  the  quality  standard  is  being  fully  implemented,  can  be  fully  substantiated,  and  there  is  little  to  no  need  for  improvement  in  this  area.  

The  Quality  Scorecard  for  Blended  Learning  Programs  is  versatile  enough  to  be  used  to  demonstrate  the  overall  quality  of  blended  learning  programs,  no  matter  what  size  or  type  of  institution.    Using  the  Delphi  research  methodology,  a  panel  of  54  experts  came  to  consensus  concerning  which  of  the  possible  indicators  should  be  included  in  the  scorecard.  The  indicators  were  derived  from  the  research  literature  on  blended  in  2015.  The  following  range  of  scores  should  be  applied  after  scoring,  developing  justifications,  and  providing  artifacts  for      substantiation:  

90-­‐‑100%  (189—210  points)  =  Exemplary    

80-­‐‑89%  (168—188  points)  =  Acceptable    

70-­‐‑79%  (147—167  points)  =  Marginal    

60-­‐‑69%  (126—146  points)  =  Inadequate    

<  59%  (<  125  points)  =  Unacceptable  

 Introduction  to  the  Quality  Scorecard  

for  Blended  Learning  Programs  

Page 14: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ContacttheOnlineLearningConsortiumat(617)716‐1414ifinterestedinaformalprogramreviewofblendedlearningprograms.

Note:TheQualityScorecardforBlendedLearningProgramscontainsadaptationsofthe24qualitystandardsidentifiedbytheInstituteforHigherEducationPolicyreport,QualityontheLine:BenchmarksforSuccessinInternetbasedDistanceEducation(2000).

Page 15: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Theinstitutionhasagovernancestructuretoenablesystematicandcontinuousimprovementrelatedtotheadministrationofblendededucation.

2. Theinstitutionhasagovernancestructuretoenableclear,timely,effective,andcomprehensivedecisionmakingrelatedtoblendedlearningcourses/programs.

3. Theblendedlearningprogram’sstrategicplanisreviewedforitscontinuingrelevance,compliancewithaccreditationobjectives,andisperiodicallyimprovedandupdated.

4. Theinstitutionhasdefinedthestrategicvalueofblendedlearningtoitsenterpriseandstakeholders(students,faculty,parents,etc.).

5. Theorganizationalstructureoftheblendedlearningprogramsupportstheinstitution'smission,values,andstrategicplan.

6. Theinstitutionhasaprocessforplanningandresourceallocationfortheblendedlearningprogram,includingfinancialresources,inaccordancewithstrategicplanning.

7. Theinstitutiondemonstratessufficientresourceallocation,includingtechnologyandfinancialresources,inordertoeffectivelysupportthemissionofblendededucation.

8. Theinstitutionhaspolicyandguidelinesthatconfirmastudentwhoregistersinablendedcourseorblendedlearningprogramisthesamestudentwhoparticipatesinandcompletesthecourseorprogramandreceivesacademiccredit.Thisisdonebyverifyingtheidentityofastudentbyusingmethodssuchas(a)asecureloginandpasscode,(b)proctoredexaminations,or(c)othertechnologiesandpracticesthatareeffectiveinverifyingstudentidentification.

9. Aprocessisfollowedthatensuresthatpermissions(CreativeCommons,Copyright,FairUse,PublicDomain,etc.)areinplaceforappropriateuseofallcoursematerials.

Institutional Support

Page 16: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. The institution has a governance structure to enable clear systematic and continuousimprovement related to the administration of blended education.

Thisqualityindicatorexaminestheassessmentofblendededucationwithinthegovernancestructuretoensurecontinuousimprovement.Qualityoftheblendededucationprogramshouldbeevaluatedannually,andnotwithjustaone‐timeexamination.Workingtowardscontinuousimprovementshouldbeafocusthroughoutallareasoftheblendededucationprogram.Withinthisconceptualframework,itisunderstoodthattheleadershipofanyinstitutionisbasedonapplyingmanagementtechniquestoimplementitsactivitiesandachieveitsobjectives.Programadministrationrequiresastrongcommitmentbyallthoseinvolved(faculty,staff,technicians,andstudents)guaranteeingtheefficacyandefficiencyofactionsimplementedforprogramdevelopmentareeffective.Becausegoalsandobjectivesarepartoftheoverallinstitutionalstrategicplan,theseactivitiesandstrategiesforachievingthosegoalsshouldbemeasuredwithintheinstitutionalgovernancestructureonaperiodicbasis.TheOnlineLearningConsortium(formerlySloan‐C)Pillarsoflearningeffectiveness,access,costeffectiveness,studentsatisfactionandfacultysatisfactioncanbeusedtodevelopanevaluationframeworkastheycloselyalignwiththreeprimarymotivationsforadoptingblendedlearningof:enhancedlearningeffectiveness,enhancesaccessandflexibilitytolearningandenhancedcosteffectiveness(Laumakis,Graham,&Dziuban,2009).

Useofbothfacetofaceandonlinelearningnecessitatesmorecarefulconsiderationofappropriatemethodsofassessment.Whenusingblendedlearning,theuseoffacetofaceclassroomobservationscanbecombinedwithlearnertrackingtoolswithincoursemanagementsystemstoprovideacorecomplexassessmentview(Laumakisetal.,2009).Datacapturepossibilitieswithinonlinelearningfacilitatestheabilitytonotonlyassessstudentlearningthroughassignments,test,etc.,butalsostudentengagementthroughanalysisofstudenttransactions(Picciano,2015).Thosesamedatacapturepossibilitieswouldalsoapplytothoseonlinecomponentsofablendedcourse.Forgoalsnotbeingmet,animprovementplanshouldbedevelopedinordertomeetthegoalthenexttimemeasurementwilloccur.Thesystematicevaluationplanshouldprovidevaluablefeedbackthatwillinformandsupportblendedlearning,aswellasidentifyareastoimprove(Laumakisetal.,2009).

Recommendations

Establishseniorpositionstoguidestrategicandoperationalobjectivesandcreateaccountabilitymeasuresforalldesignatedstafftoensuretheorganization’sabilitytoachieveobjectivesforblendedprograms.

Committodevelopingandofferingnecessaryservicesandresourcesinsupportofthemyriadoftechnical,social,andlearningissues(bothinternalandexternal)encounteredbystudents,faculty,andstaffengagedinblendededucation.

Distinguishhowprogramgoalsandobjectivesarepartoftheoverallstrategicplananddevelopstrategiesformeasuringthosegoalswithintheinstitutionalgovernancestructure.Keepaccuraterecordsofwhengoalsaremet;iftheyarenotachieved,developanimprovementstrategyinordertoreachthegoalsforthenextmeasurementperiod.

Datafromsurveysshouldbemadeavailabletofacultyaswellasinstructionaldesigners. Developareportingcyclethatidentifiesmeasuredgoalsandimprovementstrategiesfor

meetingthosegoalsannually. Chooseoneareaoftheprogramandfocusonitsimprovementforaone‐yearperiod.Once

thegoalsareachieved,selectanotherareatoimprovethenextyear.

4 Quality Scorecard for Blended Learning Programs

Page 17: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. The institution has a governance structure to enable clear, timely, effective, and comprehensive decision making related to blended learning courses/programs. 

Academicgovernanceisaregulatorytermthatclarifieshowinstitutionsareorganizedandhowresponsibilitiesaredividedandassessed.Commonacrossbothpublicandprivateinstitutions,governancestructuresensureorderlyandcontinuousoperation.Thisqualityindicatorexaminesthegovernanceofblendedlearningeducationtoensureitsorderlyandcontinuousoperationandcleardecisionmakingprocess.

Duringtheemergenceofblendedlearningeducationinhighereducation,institutionsoftenstructuredblendedlearningeducationasanauxiliaryserviceortemporaryentity.Theseearlyprogramswereoftenseenastangential,ratherthanasintegralpartsoftheinstitution’smissionandstrategicplan.Asaresult,governancestructuresforblendedlearningeducationwerehaphazardandinstitutionaldecisionmakingregardingblendedlearningeducationwasneithereffectivenorcomprehensive.However,withrapidgrowthandacceptance,blendedlearningeducationquicklybecameacoreeducationalservice.Continuedgrowthoverthepasttwodecadeshasdemonstratedthatblendedlearningeducationhasenteredthemainstream(Graham,Woodfield,&Harrison,2013).Consequently,blendedlearningprogramsrequireattentiveplanningandstructuringthatreflectsthecoreeducationalroleonlineeducationnowplaysinhighereducation.

Thisqualityindicatormakesnoprescriptionaboutthenatureofthegovernancestructure.Institutionsenjoythelibertytoorganizeblendededucationgovernancestructuresdifferentlybasedonsize,mission,androle;however,aninstitutionmustdemonstratethatstrategicdecisionmakingregardingblendededucationisappropriatelymadeattheinstitutionlevel.

Recommendations

Adoptaninstitutionalapproachtowardthegovernanceandorganizationofonlineandblendededucationprograms.

Includeallinstitutionaldivisionsthatarelikelytobeinvolvedinand/oraffectedbythedecisionmakingprocessforblendededucationinthegovernanceframework.

Clarifyresponsibilitiesforallauthoritiesoverblendededucationprogramsandcommunicatethatclearlytostakeholders.

Developpoliciesandpracticesforgovernanceviaasteeringcommitteewithrepresentativesfromalldivisionsimpactedbythedeliveryofblendededucation.

Institutional Support 5

Page 18: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. The blended learning program’s strategic plan is reviewed for its continuing relevance, compliance with accreditation objectives, and is periodically improved and updated. 

Toprovidearigorousblendedlearningprogram,aninstitutionmustbeginwithastrategicplanthatdemonstratesthatvaluesthatadministrationandfacultyplaceonblendedlearning.Astrategicplandirectshowoperationsfunctionandresourcesareallocatedinaccordancewiththegoalsandobjectivesestablished(Moskal,Dziuban,&Hartman,2013).Infact,thestrategicplanprovidesinformationontheresultsthattheorganizationistryingtoachieve.Becauseofthis,itisimportantforthestrategicplantoremaincurrentandrelevant.Someinstitutionswilldevelopafiveorten‐yearstrategicplan;however,thatdoesnotmeanthatitisleftunchangedforthattimeperiod.Eachyear,itshouldbereviewedforrelevancyandcurrentapplication.

Whendevelopingastrategicplanforablendedlearningprogram,itisnecessarytoconsiderboththepresentandthepastactivitiesoftheorganizationinordertoreachitsgoals;inotherwords,itisvitaltoknowthelocationofthedestinationinordertogetthere.Becausethestrategicplanshouldoutlineprincipalactivities,implementation,andtimeframes,theseshouldbeidentifiedtoreviewforcompletionandcontinuedrelevance.Performancefunctionanddutiesidentifiedfortheinstitutionalmembersshouldbeperiodicallyreviewedinordertoidentifypotentialweaknessesandbarrierforcompletionofthefunctionsandactivities

Thestrategicplanshouldincludespecificactivitiesthatsupporttheidentifiedgoalsandobjectives.Itishelpfultotakethestartingandendingdatesofactivitiesandtoevaluatethemperiodicallytoseeiftheyarebeingfullyimplemented,orwhetheritisnecessarytomakechangestoachievewhatwasplannedand/ortoupdatethestrategicplaninordertoachievecontinuousimprovement.

Recommendations 

Includespecificactivitieswithinthestrategicplanthatshouldbeevaluatedperiodically.Theseactivitiesshouldbescheduledonatimelineforevaluationandupdates.

Evaluatethestrategicplanatleastonceayearandproposenecessaryimprovementstomaintainrelevancy.

Withintheconceptualframework,itisunderstoodthattheleadershipofanyinstitutionisbasedonapplyingmanagementtechniquestoimplementitsactivitiesandachieveitsobjectives.Developareportingcyclethatreportsmeasuredgoalsandimprovementstrategiesformeetingthosegoalsannually.

Chooseoneareaoftheprogramandfocusonitsimprovementforaone‐yearperiod.Oncethegoalsareachieved,selectanotherareatoimprove.

6 Quality Scorecard for Blended Learning Programs

Page 19: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. The institution has defined the strategic value of blended learning to its enterprise and stakeholders (students, faculty, parents, etc.). 

Theextenttowhichqualitymaybeachievedinblendedlearningprogramsdependsonmanyfactors;chiefamongthemisthedegreetowhichtheorganization’sstrategicplan,mission,visiondefineandsuccessfullycommunicatetothestakeholders.Systematicplanningfocusesonexpressingtheblendededucationstrategicobjectiveswithintheinstitutionalvision,developingguidelinesandoperationalproceduresforprogrammaticoversight,committingtothequalitydeliveryofservices,andestablishingbudgetaryandfinancialprotocolsthataddresstheuniqueaspectsofblendededucation.Theseactionsprovideaframeworkfortherelationshipsamongprogramdevelopment,strategyandgrowth,andoperationalobjectivesaswellashelptoestablishtheinstitutionorprogram’scommitmenttoquality.

Thereisatendencyamongcollegesanduniversitiestofocussolelyonthemodeofdeliveryforblendedlearning.Suchalimitedapproachdoesnotconveythevalueofblendededucationtotheinstitutionasawhole.Theextenttowhichtheinstitution’svisionforandoperationalapproachtoblendededucationisarticulatedtoallstakeholdersiscriticalforthequalityofblendededucationprograms.Regardlessoftheoperationalmodel,thesystemcanmirrorthecampusvirtually.Blendedlearningshouldfitwithintheinstitutionalgoalsandbealignedwithspecificstrategies(Niemiec&Otte,2009).

Strategicdevelopmentofblendedprogramsthatleveragetheexpertiseofmultiplestakeholdersacrossthecollegeoruniversityprovidesthenecessarycollectiveeffortsthatcontributetoqualityinblendededucation.Clarityofvision,buy‐in,oversight,services,andcostarekeyareasforconsiderationwithrespecttodefiningtheinstitution‐widestrategicvalueofblendedprograms.

Recommendations 

Affirmthevalueofblendededucationwithintheinstitutionalvisionbylinkingoperationalactivitiestostrategicobjectives.Alsoemphasizetheinstitution’scommitmenttothevalueofblendededucationbyannouncingitinprominentplacessuchastheinstitution’sandprogram’shomepagesonthewebsite,thepresident’swelcomepage,andstudentandfacultydevelopmentcentersoncampus.

Establishseniorpositionstoguidestrategicandoperationalobjectivesandcreateaccountabilitymeasuresforalldesignatedstafftoensuretheorganization’sabilitytoachieveobjectivesforblendedprograms.

Institutional Support 7

Page 20: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. The organizational structure of the blended learning program supports the institution's mission, values, and strategic plan. 

Inanyorganization,itisimportanttomaintainanopenorganizationalculture,andforitsmemberstoshowcommitmentandactresponsibly,therebyenabletheorganizationtoachieveexcellenceandsuccess.Typically,whenwespeakoforganizationalculture,wearereferringtospecificpatternsofbehaviorthatmayberecognized,transmitted,andlearned(Schein,1984).Likewise,Mintzberg(1979)definedtheorganizationalstructureasthesumofallthewaysinwhichworkisdividedintodifferentactivitiesandcoordinated.Itisthesumofthevaluesthatdefinetherelationshipsbetweenitsmembers.

Itisessentialforeveryorganizationtodevelopastrategicplanthatoutlinestheprincipleactivities,implementation,andtimeframes,includingblendededucationprograms.Inaddition,itisimportantforthefunctionscarriedoutbythemembersofaninstitutiontobeclearlydefined,sothattogether,theycanmeettheinstitutionalgoals;i.e.eachpersonshouldknowhisorherrights,dutiesandactivities,sothatoptimalresultsmaybeachieved.Withoutastrategicplan,everymemberoftheorganizationwouldbeworkinginisolation,andunawareoftheresultsthattheorganizationisaimingtoachieve.

Thepoliciesoftheinstitutionshouldclearlyreflecttheinstitution’svaluesandthepartplayedbyitspersonnel.Iftheinstitutionalpoliciesareclearandcoherent,theywillbekeytoachievingeducationalqualityand,moreover,enableallprocessestobecarriedoutinaneffectiveandefficientmanner.Forallthesereasons,itisessentialfortheblendedprogramstructuretosupportthepolicies,thestrategicplan,thevaluesandthecultureoftheorganization.

Recommendations 

Provideinformativeopportunitiestoensureallmembersoftheorganizationbecomefamiliarwiththestrategicplan,supportingpolicies,andthevaluesandcultureoftheorganizationinwhichtheywork.

Surveymembersoftheorganizationtofindouthowclearlythecommunicationregardingthestrategicplanhasbeenandhowitsupportsblendededucation.

Createanorganizationalstructurechartthatisvisuallysupportedbythemissionandvisionoftheinstitution.Developjustificationsastowhytheorganizationalstructureisappropriatelyaligned.

 

8 Quality Scorecard for Blended Learning Programs

Page 21: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. The institution has a process for planning and resource allocation for the blended learning program, including financial resources, in accordance with strategic planning. 

Planninginvolvesaprocessfordeterminingthegoalsofanorganization,resourcesthataretobeutilized,andgeneralpoliciesthatwillguidetheacquisitionandadministrationofsuchresources,supportingtheorganizationasawhole.Withinthisprocess,variousactionsaretaken,oneofwhichisthattheorganizationasawholeanalyzesandevaluatesthefinancialresourcesneededtodevelopitsactivitiesandsatisfytheneedsofitsstakeholders,usingdifferentstrategiesaccordingtotheparticularstructureoftheprogram.Thestrategicplanfortheblendedprogramshouldsupporttheneedforresourceallocation.

Eachblendededucationprogramshouldhaveitsownbudget,sothattheoriginandapplicationofeconomic,material,andendowmentresourcesmaybeidentified.Theblendedprogramcannotbesuccessfulwithoutanadequatebudgetandothernecessaryresourcessuchaspersonnel.Intheearlydaysofblendededucation,manyprogramstriedtooperatewithwhateverfundscouldbegeneratedfromgrantsorextrafunding.Whileseekingoutsidefundingisstillanoption,wenowunderstandthatablendededucationprogramwillnothavelong‐termviabilitywithoutitsownlongtermfundingsources.

Recommendations 

Engageallmembersinvolvedintheblendededucationprogramintheprocess,sothatafinancialplanisdevelopedbyindividualsthroughouttheorganizationatdifferentlevelsaffectedbytheblendedprogramandindifferentsupportunitssotheirspecificneedsmaybebudgeted.

Developclearbudgetaryguidelinesforunits,takingintoaccountcostandtheextenttowhichbudgetingandaccountingpracticescanclearlydelineateatruereturnoninvestment.Distinguishbetweenbenefitsofblendededucationandhoweachmayormaynotimpacttheoverallunitcostintermsofdeliveringservices,infrastructure,andstaffing.

Estimatetheresourcesassignedtoeachofthemainactivitiesdescribedintheblendedprogram’sstrategicplan.Checkhalfwaythroughthebudgetyeartobesureadequateresourcesstillexistforallactivitiesidentifiedinthestrategicplanforthatyear.

 

Institutional Support 9

Page 22: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

7. The institution demonstrates sufficient resource allocation, including technology and financial resources, in order to effectively support the mission of blended education. 

Theinstitutiondemonstratessufficientresourceallocation,includingtechnologyandfinancialresources,inordertoeffectivelysupportthemissionofblendededucation.

Planninginvolvesaprocessfordeterminingthegoalsofanorganization,resourcesthataretobeutilized,andgeneralpoliciesthatwillguidetheacquisitionandadministrationofsuchresources,supportingtheorganizationasawhole.Withinthisprocess,variousactionsaretaken,oneofwhichisthattheorganizationasawholeanalyzesandevaluatesthefinancialresourcesneededtodevelopitsactivitiesandsatisfytheneedsofitsstakeholder,usingdifferentstrategiesaccordingtotheparticularstructureofeachprogram.Thestrategicplanningfortheblendedprogramshouldsupporttheneedforresourceallocationasacomprehensiveframeworkofsupportisoneoftheessentialsuccessfactorsinblendedlearning(Niemiec&Otte,2009).

Eachblendedprogramshouldhaveitsownbudget,sothattheoriginandapplicationofeconomic,material,andendowmentresourcesmaybeidentified.Theblendedprogramcannotbesuccessfulwithoutanadequatebudgetandothernecessaryresourcessuchaspersonnel.Blendedlearning,supportedbypedagogy,necessitatesthatoneoftheessentialresourceswillincludepeopleaswellastechnology.Instructorsmustacquiretheskillsandabilitiestoleveragetechnologytofacilitateeffectivestudentlearning(Poon,2013).Asblendedlearningtargetscorestudentsandhastheabilitytopositivelyimpactallstudents,itmaynotnecessitateaspecialstreamoffundingifitisseenasacoreexpense(Niemec&Otte,2009).Seekingadditionaloutsidefundingisalwaysencouraged.

Recommendations 

Engageallmembersinvolvedintheblendedprogramintheprocess,sothatafinancialplanisdevelopedbyindividualsthroughouttheorganizationatdifferentlevelsaffectedbytheblendedprogramandindifferentsupportunitssotheirspecificneedsmaybebudgeted.

Developclearbudgetaryguidelinesforunits,takingintoaccountcostandtheextenttowhichbudgetingandaccountingpracticescanclearlydelineateatruereturnoninvestment.Distinguishbetweenbenefitsofblendededucationandhoweachmayormaynotimpacttheoverallunitcostintermsofdeliveringservices,infrastructure,andstaffing.

Estimatetheresourcesassignedtoeachtothemainactivitiesdescribedintheblendedprogram’sstrategicplan.Checkhalfwaythroughthebudgetyeartobesureadequateresourcesstillexistforallactivitiesidentifiedinthestrategicplanforthatyear.

10 Quality Scorecard for Blended Learning Programs

Page 23: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

8. The institution has policy and guidelines that confirm a student who registers in a blended course or blended learning program is the same student who participates in and completes the course or program and receives academic credit. This is done by verifying the identity of a student by using methods such as (a) a secure login and pass code, (b) proctored examinations, or (c) other technologies and practices that are effective in verifying student identification. 

Regardingstudentauthenticationandtheonlinecomponentofblendedcoursesandprograms,theUSHigherEducationandOpportunityActof2208(HEOA)mandatedthataccreditingagenciesmustrequire:

Aninstitutionthatoffersdistanceeducationorcorrespondenceeducationtohaveprocessesthroughwhichtheinstitutionestablishesthatthestudentwhoregistersinadistanceeducationorcorrespondenceeducationcourseorprogramisthesamestudentwhoparticipatesinandcompletestheprogramandreceivestheacademiccredit.(H.R.4137‐‐110thCongress,2008)

InMay2009,theNegotiatedRuleMakingCommitteeonaccreditationclarifiedtherequirementbyinstructingregionalaccreditorstorequirethatinstitutionswhichrequiredistanceeducationprogramshaveprocessesinplacefordistanceeducation.Theaccreditingagencyisincomplianceifit:

1.Requiresinstitutionstoverifytheidentityofastudentwhoparticipatesinclassorcourseworkbyusing,attheoptionoftheinstitution,methodssuchas‐

a.Asecureloginandpasscode;

b.Proctoredexaminations;and

c.Identificationtechnologiesthatareeffectiveinverifyingstudentidentification;

2.Makesclearinwritingthatinstitutionsmustuseprocessesthatprotectstudentprivacyandnotifystudentsofanyprojectedadditionalchargesassociatedwiththeverificationofstudentidentityatthetimeofregistrationorenrollment(20U.S.C.1099b).

Authenticationisanalogoustotakingattendanceinaface‐to‐facecourse.Thesophisticationwithwhichstudentsareauthenticatediflargelyafunctionoftheeffortoftheinstructorandinstitution(Bailie&Jortberg,2009).Forexample,forallclasses,institutionsnormallyproduceclassrosterslistingstudentswhohaveregisteredandpaidforcourses.Thisrosterisprovidedtotheinstructor,whointurn,hastheresponsibilityofmakingsurestudentsontherosterarewhotheysaytheyare,andthattheyaretheonesperformingtheacademicworkthroughoutthecourse.Thesesameprinciplesapplytotheonlinecomponentofblendededucation;however,themethodsofauthenticationaredifferentaccordingtowhetherstudentsarephysicallyorvirtuallypresent.

Technologyplaysamoreimportantroleintheauthenticationofstudentsintheonlineenvironment.Asidentificationtechnologyevolvesandbecomesmoresophisticated,theabilitytocheckstudentidentitywillbecomeeasierinbothvirtualandphysicalenvironments.Ontheotherhand,thetechnologicalsophisticationofstudentsdeterminedtocommitacademicfraudwillalsoincrease(Bedford,Gregg,&Clinton,2009).Consequently,allinstitutionsshouldregularlyexaminetheirstudentauthenticationprocessandbecognizantoftechnologicaladvancesthatmakestudentauthenticationmoreaccurate,affordable,andeasytouse.

Institutional Support 11

Page 24: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Recommendations 

Employasecurestudentloginandpasswordprocessforstudentstoaccesstheonlinecoursematerials,studentinformation,financialaidprocessing,andsoon.

Adoptaproctoringsystemforstudentexaminationscompletedonline.Thismayincludebothofflineand/oronlineauthentication.Onlinemethodsmayincludewebcams,typingheuristics,orotherbiometricidentification.

Authenticatestudentsuponloginwithsecondaryauthentication,uptoandincludingbiometricauthentication.

12 Quality Scorecard for Blended Learning Programs

Page 25: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

9. A process is followed that ensures that permissions (Creative Commons, Copyright, Fair Use,Public Domain, etc.) are in place for appropriate use of all course materials.

Thegrowthofopenlysharedcontentcreatedbyauthorswhoencouragethere‐mixingoftheirworkhasforgednewfrontiersintotheavailabilityofqualityeducationalresourcesavailableatlowornocost.Thisnewsharingeconomybringswithitawholenewmixofquestionsaboutwhatconstitutesafairuse.Facultyareincreasinglyunsureofwhatroutetotakewhenconsideringauthoringandsourcingcontentforblendedcourses.Facultyfinditdifficulttonavigatethepermissionsforcontentcollectedanddeterminingifthosepermissionspermitthefaculty’sintendeduseofthecontent.

Theinstitutionhasastandardfordistributionofcoursecontent,throughtheteachingandlearningprocess,withspecificstandardsforblendedcoursecontent:

● Courseintellectualpropertyownershipisdefinedfortheinstitution,department,andfaculty;

● Apolicyforcitingpermissionsforcontentintendedfordistributioninablendedcourse.

Institutionalsupportisavailabletostaffandfacultywhowishtoauthorneworderivativeworkfordistributionorcreateacompilationofresourcedcontentinablendedcourse:

● Aspecialistofcopyrightandfairuse.● Aspecialistofinstructionaldesign.● Aspecialistofcourseandprogramassessment.

Recommendations 

Provideinstructionaldesignsupporttofacultyoneffectiveuseofeducationalresourcesfordistributedlearningactivitiesandpairingappropriatecontentwithappropriatelearningactivities.

Provideeducation(guides,tools,training,policy)onauthorshipofnewandderivativeworks,thedifferencesbetweenopen/sharedworksandworksprotectedbycopyrightorpatent,andthebenefitsopeneducationresourcesmayprovideincontrasttouseofresourcesprotectedbycopyright.

Provideeducation[guides,tools,training]onopeneducationresourceclearinghouses,howtoassessthequalityofopeneducationresources,anddiscerningthetypesresourcesandtheirbestuse.

Provideeducation[guides,tools,training]onthenatureoffairuseandunderstandingthelegalscopeofdifferentusemethodologies:freelyshared,limitedbyanopencopyrightsuchasCreativeCommons,orprotectedbyanationalcopyrightorpatent.

Provideeducation(guides,tools,training,policy)onproperlycitingopenandprotectedcontentwithinblendedcourses.

Institutional Support 13

Page 26: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

References for Institutional Support 

Bailie,J.L.,&Jortberg,M.A.(2009).Onlinelearnerauthentication:Verifyingtheidentityofonlineusers.MERLOTJournalofOnlineLearningandTeaching,5(2).Retrievedfromhttp://jolt.merlot.org/vol5no2/gregg_0609.htm

Bedford,W.,Gregg,J.,&Clinton,S.(2009).Implementingtechnologytopreventonlinecheating:AcasestudyataSmallSouthernRegionalUniversity(SSRU).MERLOTJournalofOnlineLearningandTeaching,5(2),230–238.Retrievedfromhttp://jolt.merlot.org/vol5no2/gregg_0609.htm

CornellLegalInformationInstitute.(2016).Copyright:Anoverview.Retrievedfromhttp://www.law.cornell.edu/topics/copyright.html

CreativeCommons.(2016).ExploretheCreativeCommonslicenses.Retrievedfromhttps://creativecommons.org/choose/

CreativeCommons.(2016).WhatisOER?Retrievedfromhttps://wiki.creativecommons.org/wiki/What_is_OER%3F

Graham,C.R.,Woodfield,W.,&Harrison,J.B.(2013).Aframeworkforinstitutionaladoptionandimplementationofblendedlearninginhighereducation.InternetandHigherEducation,18,4‐14.

H.R.4137—110thCongress.(2008).HigherEducationOpportunityAct.Retrievedfromwww.GovTrack.usRetrievedfromhttp://www.govtrack.us/congress/bills/110/hr4137

Laumakis,M.,Graham,C.,Dziuban,C.(2009).TheSloan‐Cpillarsandboundaryobjectsasaframeworkforevaluatingblendedlearning.JournalofAsynchronousLearningNetworks,13(1),75‐87.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ837582.pdf

LibraryofCongress.(2011).CopyrightLawoftheUnitedStates.Retrievedfromhttp://www.copyright.gov/title17/

Mintzberg,H.(1979).Thestructuringoforganizations:Asynthesisoftheresearch.EnglewoodCliffs,NJ:PrenticeHall.

Moskal,P.,Dziuban,C.,&Hartman,J.(2013).Blendedlearning:Adangerousidea?InternetandHigherEducation,18,15‐23.

Niemec,M.,&Otte,G.(2009).Anadministrator’sguidetothewhy’sandhow’sofblendedlearning.JournalofAsynchronousLearningNetworks,13(1),19‐30.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ909846.pdf

Picciano,A.(2015).Planningforonlineeducation:Asystemsmodel.OnlineLearning,19(5),Retrievedfromhttp://onlinelearningconsortium.org/read/journal‐issues/

Poon,J.(2013).Blendedlearning:Aninstitutionalapproachforenhancingstudents’learningexperience.MERLOTJournalofOnlineLearningandTeaching,9(2).Retrievedfromhttp://jolt.merlot.org/vol9no2/poon_0613.htm

Schein,E.H.(1984,Winter).Comingtoanewawarenessoforganizationalculture.SloanManagementReview,25(2),3‐15.

Shelton,K.,&Saltsman,G.(2005).Anadministrator'sguidetoonlineeducation.Greenwich,CT:InformationAge.

14 Quality Scorecard for Blended Learning Programs

Page 27: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

StateUniversityofNewYorkAcademicAffairs.(2016).CopyrightandFacultyOwnershipofIntellectualProperty.Retrievedfromhttp://system.suny.edu/academic‐affairs/faculty/faculty‐ownership/UnitedStatesCopyRightOfficeWebSite.LibraryofCongress.(April,2016).Retrievedfromhttp://www.copyright.gov/

UniversityatBuffalo.(2016).FairUseinHigherEducation.Retrievedfromhttp://www.buffalo.edu/ubit/service‐guides/safe‐computing/dmca/fair‐use‐higher‐education.html

Institutional Support 15

Page 28: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Thecoursedeliverytechnologyisconsideredamissioncriticalenterprisesystemandsupportedassuch.

2. Whethertheinstitutionmaintainslocaldatacenters(servers),and/orcontractsforoutsourced,hostedservicesorcloudservices,thosesystemsareadministeredincompliancewithestablisheddatamanagementpracticessuchastheInformationTechnologyServiceManagement(ITSM)standards,whichincludeappropriatepowerprotection,backupsolutions,disasterrecoveryplans,etc.

3. Thetechnologysystemsrelatedtothedeliveryofblendedlearningprogramsarehighlyreliableandoperablewithmeasurablestandardsbeingutilizedsuchassystemdowntimetrackingandtaskbenchmarking.*

4. Adocumentedtechnologyplanthatincludeselectronicsecuritymeasures(e.g.,passwordprotection,encryption,secureonlineorproctoredexamsifapplicable,etc.)isinplaceandoperationaltoensurequality,inaccordancewithestablishedstandardsandregulatoryrequirements.*

5. Theinstitutionhasanestablished(updatedandcontinuouslyreviewed)contingencyplanforthecontinuanceofdatacentersandsupportservicesintheeventofprolongedservicedisruption.

6. Acentralizedsystemprovidessupportforbuildingandmaintainingtheblendededucationinfrastructure.*

7. Faculty,staff,andstudentsaresupportedinthedevelopmentanduseofnewtechnologiesandskillsapplicabletoblendedlearning.

Technology Support

Page 29: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. The course delivery technology is considered a mission critical enterprise system and supportedas such.

Theinstitutionmustconsidertheblendedprograms’technologicalinfrastructureasmissioncriticalevidencedbyacommitmentofresources,continuousimprovementprocesses,andacommitmenttonear100%uptime.Studentsandfacultymusthaveaccesstoblendedcoursematerialswhenneeded.Learningmanagementsystemsforcoursedelivery,inwhateverformat,mustbeaninstitutionalpriority.Coordinatingriskmanagementandtechnologyupgradestrategieswithonlineprogramsisawaytoleverageexistinginstitutionalizedcultureandinfrastructure.Theremustbededicatedtechnologysupportpersonnelavailable24x7x365forsolvingissues,asonlineandblendedstudentsaccesscoursematerialsatvarioustimesoftheday,typicallyoutsideofthedaytimeMondaythroughFridayhours.Bothonlineandblendedclassstudentsusecoursetechnologyduringtheeveningsandonweekends(Moskal,Dzuiban,&Hartman,2013).

Recommendations 

Seekcollaborationbetweenblendedprogramadministratorsandinstitutionaltechnologyservicesinordertobeinvolvedwiththedevelopmentandupdatingofthetechnologysupportplan.

Establishandclearlycommunicatetoallpersonnelinvolvedemergencyresponseproceduresforunexpecteddowntime.

Budgetandplaninordertomaintainthetechnologicalinfrastructureforcoursedelivery. Utilizemetricsandbenchmarkingforupgradingandimprovingtechnologiesusedto

supporttheblendededucationprogram.

Technology Support 17

Page 30: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. Whether the institution maintains local data centers (servers), and/or contracts for outsourced,hosted services or cloud services, those systems are administered in compliance withestablished data management practices such as the Information Technology ServiceManagement (ITSM) standards, which include appropriate power protection, backup solutions,disaster recovery plans, etc.

Maintainingthemyriadoftechnologicalinfrastructuresystemsrequiredforthedeliveryandsupportofblendededucationprogramsrequiresenterprise‐levelcommitmenttoensuresystemavailability,responsetime,dataintegrity,studentprivacy,andsystemsecurity.Suchpracticeschangequicklyastechnologyadvancesandservicesevolve.InstitutionsshouldcreateandadheretoInformationTechnology(IT)anddatamanagement.TheInformationTechnologyServiceManagement(ITSM)standards(orsimilar),maintainestablishedandevolvingbestpracticesforthemanagementofqualityinformationtechnologyservices.TheITSMstandardsshouldbefollowedaccordingtotheinstitution’sacademicpolicies.Thisqualityindicatorrecognizesthatblendedprogramtechnologicalsupportisbestprovidedbytechnologistswhosubscribetoandadheretoestablishedtechnologyindustryqualitystandards.

Recommendations 

Basecurrenttechnologicalsupportoperationsonestablishedinformationtechnologyindustrystandardsthatensurequalityinformationtechnologyservices.

Providesufficientbudgetsupportinordertoensuretechnologysupportstaffpersonnelareabletoparticipateinongoingprofessionaldevelopment.

Monitorregularlyandadheretobestpracticesininformationtechnologyinordertoupdateandexpandservicesasneeded.

18 Quality Scorecard for Blended Learning Programs

Page 31: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. The technology systems related to the delivery of blended learning programs are highly reliable and operable with measurable standards being utilized such as system downtime tracking and task benchmarking.* 

Ifthelightsgooutinatraditionalclassroom,itmaybeaninconvenience.Ifthesystemcrashes,it’sacalamity!(Phipps&Merisotis,2000,p.15).

Whenstudentsareenrolledinblendedcourses,theimportanceofreliabletechnicaldeliverysystemsisascriticaltothequalityofeducationasphysicalclassroomshavebeentotraditionaleducation.Infact,foranInternet‐deliveredclassroomtobeoffline,itisanalogoustolockingtheclassroomdoortostudentsbeforetheyarriveforclass.Technologydeliverysystemsshouldberesponsiveandoperationalduringpeakloadtimessothatstudentsareabletofocusonacademiccontentandbeminimallydisruptedbyunresponsivetechnology.Providingastableandreliabletechnicalinfrastructureisessentialasitinfluenceslearning(Gibbons&Wentworth,2002),studentsatisfaction(O'Brien&Renner,2002),andstudentretention(Fetzner,2003).Blendededucationprogramsrequireastrongtechnicalinfrastructuretoprovidethefoundationalstructureswithintheonlinecomponentforhowstudentsandfacultyinteractwitheachotherandwithcoursecontent.

Intheearlydaysofonlineeducation,itwascommonforinstitutionstohostthelearningmanagementsoftwareandothercriticalinfrastructureonsiteusinginstitutionallyownedhardwareandstaff.Itisnowmorecommontoencounteronline,andtherebyblended,programsthathavecontractedwithexternalserviceprovidersthatofferhostedorcloud‐basedservicesexternaloftheinstitutionalnetwork.Regardlessoftheservicemethodselected,blendedprogramsshouldbeawareofthequalityofserviceandworktoensuretheleastdisruptivelearningenvironmentisprovided.

Notechnologycanbeexpectedtoprovide100%uptime.Allsystemswillneedtobetakendownforupdates,upgrades,andrepair.Largerprogramsmaybeabletoestablishredundantserversystemsthatallowforcontinuousuptime,orprogramsofanysizemaycontractforsuchwithanexternaldatacenter.Ataminimum,expecteddowntimeshouldbescheduledwellinadvance,completedduringtimesofleastdisruption,andcommunicatedtostudents,faculty,andblendedprogramstaffwithsufficientpriornotice.

Recommendations 

Providestudentsandfaculty24x7x365access(asidefrompre‐scheduledmaintenanceperiods)forallnecessarytechnicalsystemssuchasthelearningmanagementsystem,mediaservers,etc.thatsupporttheblendededucationprogram.

Carefullymonitorsystemdowntimeanddevelopdatameasurestoensurefuturereliabilitywithagoalof100%uptime.

Establishstandardmetricsforsystemperformancesuchaswebpageloadtimesandincludeconsistentmeasurementsformonitoringgoalachievement.

Monitorsystemutilizationduringpeakusagetimesinordertoensurethatsystemstabilityismaintained.

Trackanddocumentprogramgrowthandtechnologyusageinordertoprojectandscalefuturetechnologyneeds.

Determinelowestuseperiodsandusethemfornecessarymaintenanceperiodsifredundantsystemsarenotfinanciallypossible.PublishthedowntimescheduleontheLMS,institutionalwebsite,viaemail,tweets,andthroughotherapplicablecommunicationmethodssothatstudentsandfacultyarefullyawareandcanplanaccordingly.

Technology Support 19

Page 32: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. A documented technology plan that includes electronic security measures (e.g., passwordprotection, encryption, secure online or proctored exams if applicable, etc.) is in place andoperational to ensure quality, in accordance with established standards and regulatoryrequirements.*

Atechnologyplanshouldensurethattheblendedprogram’stechnologicalinfrastructureadequatelysupportstheacademicenvironment.Thetechnologyplanshouldencompassalltechnologicalinfrastructurerequiredbytheprogram,includingsystemshostedlocallyandoffsite,orthroughcontractedservices.Ataminimum,suchsystemsincludethelearningmanagementsystem,userauthenticationsystems,collaborationsoftware,andtestingsoftware.Asconsumerexpectationscoincidewithcontinuouslyemergingtechnologyapplications,technologyplansshouldreflectvirtuallyeverybusinessandacademicprocess(NationalAssociationofCollegeandUniversityBusinessOfficers,2011)andthusmustbeflexibletoadapttofiscalandenvironmentalcontexts.Therefore,thehallmarksofeffectivetechnologyplansincludeexplicitlylinkingtheplantotheinstitution’smissionandvision;engagingstakeholdersinidentifyingcurrentconditionsandassumptions;prioritizingstakeholderneedsanddesires;andpublicizingtimelinesandbenchmarksviaclearcommunicationschannels(Geer,2006;November,Staudt,Costello,&Huske,1996;White&Weight,2000).

WICHECooperativeforEducationalTechnologies(WCET)(2009)recommendedthatinstitutionsprovide“securestudentloginsandpasswordstoaccessonlinecoursesandrelatedresources,discussions,assignmentsandassessments”(p.1).Currently,withtheubiquityofcommerciallearningmanagementsystems,muchoftheresponsibilityforestablishinglogincredentialsandencryptioncanbeappropriatelymanagedbythesevendors.However,accordingtotheHEOActof2008,asthetechnologyforauthenticationbecomesmoreprevalent,institutionswillberequiredtodomorethanjustprovideasecureloginprocess(H.R.B4137‐110thCongress,2008).

TheHEOActof2008requirementspurrednumeroushardwareandsoftwareoptionstoauthenticatestudentsandtheelectronicproctoringofexams.Theseincludeoptionsforusingbiometricdata,secondaryauthenticationwithchallengequestions,keystrokeheuristics,useofwebcams,andbrowserlockdownsoftware(Foster,2008).

Recommendations 

DemonstratefullprogramcompliancewithrequirementsoutlinedbytheHEOActof2008fortheonlineportionofthecourse.

Providestudentssecurelogincredentialstoonlinecoursematerialsusingcurrentauthenticationstandards.

DevelopanddisseminatetoallstudentsandfacultypoliciesthatclearlyindicateexpectationsforAcceptableUse,includingaccesstocourses,assignments,andrelatedmaterials.

EstablishandenforcepenaltiesforstudentsandfacultyviolatingAcceptableUsepolicy. Providefacultytraininginhowtousetechnologyandinstructionalmethodsthatsupport

academicintegrity. Monitorinnovationsinauthenticationtechnologiesandproctoringandevaluatetheirfitfor

theprogram.

20 Quality Scorecard for Blended Learning Programs

Page 33: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. The institution has an established (updated and continuously reviewed) contingency plan for the continuance of data centers and support services in the event of prolonged service disruption. 

Becausethetechnologyinfrastructureismissioncriticaltotheblendededucationprogram,theinstitutionshouldhaveinplaceatechnologysystemthatensurestheintegrityofprogramdataandprotectsagainstcatastrophicloss.Similarly,theinstitutionshouldhaveacontingencyplaninplaceintheeventofmajorsystemsdisruption.Institutionalservicesthatprovideforconstantavailability,backup,archival,datasecurity,andbusinesscontinuanceintheeventofcatastrophicfailuresdemonstrateinstitutionalcommitmenttotheprogram.

Recommendations 

Providebudgetsupportforasystembackupsolutionwithoff‐sitevaulting(physical,electronic,orboth).

Develop,maintain,andperiodicallyreviewandupdateadisasterrecoveryandbusinesscontinuityplan.

Conductdisasterrecoveryormockdisasterdrillsperiodicallytoverifysystemdatabackupandthatrestorationprocessisviableandmaintainsdataintegrity.

Establishgoalsforhowquicklysystemdataisrecoveredincaseofsystemfailure.

Technology Support 21

Page 34: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. A centralized system provides support for building and maintaining the blended education infrastructure.* 

Commitmentfromtheinstitutionalchiefinformationofficer(CIO)ensurestheblendedprogramisalignedwithoverallcollegeoruniversityprioritiesanddemonstratestechnologycommitmentatthehighestlevelregardlesswhethertheservicesareultimatelyprovidedin‐house,viaaconsortium,orbyout‐sourcing.Forthisqualityindicator,acentralizedsystemmeansthatthetechnologythatsupportstheblendedprogramisoverseenbyaninstitutionaltechnologydecision‐makingandsupportstructure.Havingacentralizedsystemfortheinfrastructurecanbeeffectiveinseveralways:itreducescomplexityandmisunderstandingbetweentechnicalstaff,itstandardizessupport(singlehelpdesksupportanduserexperience).Centralizedsupportenablesa"focusonfaculty,student,andlocalprocessissues"(Fetzner,2003,p.240).Thecentralizedsystemneedstobecoordinated,orclearcommunicationneedstobeinplace,betweentheinstitution’sCIO/ITinfrastructureandthechiefacademicofficers(CAO)settingclearexpectationsofdesiredsupportfornewandongoingacademicbusiness,suchasblendedlearning.

Thisdoesnotmeanthatalltechnologyservicesneedtoresideinonespaceorthattheybedirectlymanagedbyinstitutionalstaff.Infact,theremaybeamixtureofoutsourcedandinhousetechnologysolutionsthatsupportablendedprogram.Thequalityindicatorexpectsthattheinstitutionaltechnologysupporthasknowledgeofthetechnologyandnecessarysupporttobuildandsustaintheprogramandactivelyadvisestoensurecontinuedreliability.

Recommendations 

Designthecentralizedsystemtoparallelthebestaspectsofonlinelearning;makeit“team‐based,collaborative,comprehensive,action‐oriented,andnon‐hierarchical”(Fetzner,2003,p.232).

Includethefollowinginthecentralizedsystem:“academicsupport,trainingcoordination,instructionaldesignandsupport,librarysupport,technicalsupport,andstudentservices”(Fetzner,2003,p.234).

Collectandreviewtheperspectivesofprogramstakeholdersinordertocontinuouslyimprovethetechnologysystems.

Utilizethecentralizedsystemasaninformationexchangeandforbenchmarkingservicecalls,responsetimes,andproblemresolution.

22 Quality Scorecard for Blended Learning Programs

Page 35: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

7. Faculty, staff, and students are supported in the development and use of new technologies andskills applicable to blended learning.

Theincreasingchangesinworkforcerequirementsandstudents’needforflexiblelearningoptionswillcontinuetodrivehighereducationintheuseofnewtechnologiestobetterserveitsstudents.Infact,students“expecttheireducationtobepersonalizedtotheirindividualwantsandneeds;theyexpecttheireducationtobeeasilyachievedandfocusedontheskillstheywillrequireintheirfutureprofession.Inotherwords,moststudentsarenotlearningforlearning'ssakeorforgettingadegree;instead,theywanttheskillsneededtoentertheworkforce”(Tamarkin,2010,para.11).Becauseofthisneedforstudentpreparation,aninstitutionshouldsupportnewtechnologyinitiativesthatwillbettersupportlearningandpreparestudentsforthecurrentworkforce.

Whennewtechnologiesareadoptedandimplemented,allthoseinvolved(students,faculty,andinstitutionalstaffmemberswhowillbesupportingallthoseinvolved)willmostlikelyneedsomeformoftraining.Dalziel(2003)pointedoutthat“facultyneedtoknowhowtousetechnologytoincorporatedifferenttypesoflearningmediatocreateacompleteeducationalpackagethatmakessensetotheirstudents”(p.669).Studentswillbenefitfromhelpdesksupport,tutorials,anddemonstrations,andaccesstoaknowledgebaseforself‐‐‐help.Staffwillneedtraininginhowbesttosupportboththefacultyandstudentswiththenewtechnology.

Recommendations 

Provideongoingtrainingforfacultyandotherrelevantpersonnel.Thetimesanddaysshouldbevariedsothatallstakeholderscanbereached.

Encouragefacultytosharehowtheyembracenewtechnologiesforteachingandlearningandhowitbetterservesorbetterengagestheirstudents.

Providefacultyandstudentswithopportunitiestoexplorenewandexistingtechnologiesforcreatingandcuratingblendedcoursecomponents.

Providestudentswithtechnicalsupport,opportunitiesforself‐‐‐trainingmodules,andtutorialsonnewtechnologiesthatareimplementedforteachingandlearning.

Technology Support 23

Page 36: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforTechnologySupport

Dalziel,C.(2003).Communitycollegesanddistanceeducation.InM.G.Moore,&W.G.Anderson(Eds.),Handbookofdistanceeducation(pp.663‐671).Mahwah,NJ:LawrenceErlbaum.

Fetzner,M.(2003).Institutionalsupportforonlinefaculty.Expandingthemodel.InJ.Bourne&J.Moore(Eds.),Elementsofqualityonlineeducation:Practiceanddirection:Volume4intheSloan‐CQualitySeries(pp.229‐241).Needham,MA:TheSloanConsortium.

Foster,A.(2008,July21).Keepaneyeononlinestudents.TheChronicleofHigherEducation.Retrievedfromhttp://chronicle.com/blogs/wiredcampus/keeping‐an‐eye‐on‐online‐students/4109

Geer,W.T.(2006,November).Craftingtheperfecttechnologyplan.EdTechFocusonHigherEducation.Retrievedfromhttp://www.edtechmag.com/higher/october‐november‐2006/crafting‐the‐perfect‐game‐plan.html

Gibbons,H.,&Wentworth,G.(2002,August).Processesformotivatingonlinelearnersfromrecruitmentthroughdegreecompletion.VirtualUniversityGazette:MotivatingandRetainingAdultLearnersOnline,127‐135.

H.R.4137‐‐110thCongress.(2008).HigherEducationOpportunityAct.Retrievedfromhttp://www.govtrack.us/congress/bills/110/hr4137

MiddleStatesCommissiononHigherEducation.(2011).Distanceeducationprograms:Interregionalguidelinesfortheevaluationofdistanceeducation(onlinelearning).Retrievedfromhttp://www.msche.org/publications/Guidelines‐for‐the‐Evaluation‐of‐Distance‐Education‐Programs.pdf

NationalAssociationofCollegeandUniversityBusinessOfficers.(2011).Collegeanduniversitybusinessadministration,informationtechnology(7thed.).Washington,DC:Author.

November,A.,Staudt,C.,Costello,M.A.,&Huske,L.(1996).Guidetodevelopingaschoolordistricttechnologyplan.NorthCentralRegionalTechnologyandEducationalConsortium.Retrievedfromhttp://www.ncrel.org/sdrs/areas/issues/methods/technlgy/te300.htm

O'Brien,B.S.,&Renner,A.L.(2002).Onlinestudentretention:Canitbedone?WorldConferenceonEducationalMultimedia,HypermediaandTelecommunications,1,1479‐1483

Phipps,R.,&Merisotis,J.(2000).Qualityontheline:Benchmarksforsuccessininternet‐baseddistanceeducation.Washington,DC:TheInstituteofHigherEducationPolicy.Retrievedfromhttp://www.ihep.org/research/publications/quality‐line‐benchmarks‐success‐internet‐based‐distance‐education

Tamarkin,M.(2010,Nov‐Dec.).You3.0:Themostimportantevolvingtechnology.EDUCAUSEReview,45(6),31‐44.Retrievedfromhttp://er.educause.edu/articles/2010/11/you‐30‐the‐most‐important‐evolving‐technology

WCET.(2009).Bestpracticestrategiestopromoteacademicintegrityinonlineeducation.Retrievedfromhttp://wcet.wiche.edu/wcet/docs/cigs/studentauthentication/BestPractices.pdf

White,K.W.,&Weight,B.H.(2000).Theonlineteachingguide:Ahandbookofattitudes,strategies,andtechniquesforthevirtualclassroom.Boston,MA:AllynandBacon.

24 Quality Scorecard for Blended Learning Programs

Page 37: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Acoursedevelopmentprocessisfollowedthatensurescoursesaredesignedwithalignmentbetweencoursematerials,assessmentsandlearningobjectivessothatstudentsdevelopthenecessaryknowledgeandskillstomeetmeasurablelearningoutcomesatthecourseandprogramlevel.*

2. Guidelinesregardingminimumrequirementsforcoursedevelopment,design,anddeliveryofblendedinstruction(suchascoursesyllabuselements,coursematerials,assessmentstrategies,facultyfeedback)arefollowed.*

3. Thereisconsistencyincoursedevelopmentforstudentretention(enrollmentandcoursecompletion)andquality(i.e.,coursesinaprogramhaveaconsistentnavigationalstructure).

4. Ablendedcourseshouldbedesignedasonecohesivewhole,incorporatingbothface‐to‐faceandonlineexperiencesincomplementaryways.

5. Instructionalmaterials(bothonlineandin‐class)andcoursesyllabiarereviewedperiodicallytoensuretheymeettheblendedcourse'sandprogram'slearningoutcomes.*

6. Courseassignmentsandactivitiesarereviewedperiodicallytoensuretheymeettheblendedcourses'andprogram'slearningoutcomes.

7. Student‐centeredinstructionisconsideredduringthecoursedevelopmentprocess(i.e.,studentengagement,immersion,andpersonalresponsibility).

8. Coursedesignpromotesbothfacultyandstudentengagement.

9. Courseworkloadsarereviewedtoensureitisappropriatefordesignatedcreditallocation.

10. Aprocessisestablishedandfollowedforevaluatingtheeffectivenessofcurrentandemergingtechnologiestosupporttheachievementoflearningoutcomesanddeliveringcoursecontent.

11. Courseembeddedtechnologyactivelysupportstheachievementoflearningoutcomesanddeliveringcoursecontentandunnecessaryuseoftechnologyisminimized.*

12. UsabilitytestsareconductedandappliedandrecommendationsbaseduponWebContentAccessibilityGuidelines(WCAGs)areincorporatedinthecoursedesignprocess.

13. Curriculumdevelopmentisacoreresponsibilityforfaculty(i.e.,facultyshouldbeinvolvedineitherthedevelopmentorthedecisionmakingfortheblendedcurriculumchoices).

14. Facultysupportandresourcesareprovidedtopromotethebestuseofblendeddeliverymethodincoursedevelopmentandinstructionaldesigntofacilitateteachingandlearning.

Course Development &

Instructional Design

Page 38: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. A course development process is followed that ensures courses are designed with alignment between course materials, assessments and learning objectives so that students develop the necessary knowledge and skills to meet measurable learning outcomes at the course and program level.*

Alignmentisakeyconsiderationforaccreditation;yet,notallcoursesareinstructionallydesignedsothelearningoutcomesareclearlymeasured.Learningoutcomesmustbemeasuredinordertoknowifstudentsaremeetingcoursegoalsandprogramgoals.Thiscanbedonesothat“individualcourseswithinthecurriculumhelplearnersprogressfrombasic,introductorylevelsofknowledgeandskillstohigherlevelobjectivesforcriticalthinking,masteryofskills,anddemonstrationofknowledgecommontoadiscipline”(Porter,2004,p.31).Inordertoaccomplishthis,PuzziferroandShelton(2008)recommendedcoursesbedevelopedwiththefollowinggoalsforlearning:

masteringinformation,suchaskeyconcepts,termsandideasthroughexposuretotheinformationandrecall,

progressivelygraspingtheinformationbypracticinglearningactivitiesthatfocusonrecall,trialanderror,andbuildingthevocabularyandcomprehensionoftheinformation,

applyingtheinformationtoaproblem‐basedsituationthatcanbecollaborative, analyzingtheproblemfurtherbydeconstructingtheinformationandreconstructingit

intoasolutionbyapplyingandanalyzingtheinformation,begintorecognizethepatternsorrelationshipsbetweentheinformationandtheproblem,andcreatingnewknowledgeandtheabilitytoreasonabouttheinformationandapplyitpracticallytosituations.

Thus,coursesshouldbedesignedtoreinforcetheinteractivestagesoflearningtowardsthemasteryofobjectivesintheoverallprogramanddiscipline.

Recommendations 

Clearlydemonstratethatblendedcourselearningoutcomesaremeasuredeachtermandcomparedtolearningoutcomegoals.

Clearlydemonstratethatblendedcourseoutcomescontributetotheprogramlearningoutcomeswithinwhichthecourseoccurs.

Useformativeandnormativeassessmentcriteriaincoursesandprogramstosupportthelearningoutcomebeingmeasuredandtohelpstudentstogaugeprogresstowardsmastery.

Buildpracticeactivitiesintoblendedcoursedesignthatencouragestudentstocheckunderstanding,refineunderstandingbyapplyingit,andtestunderstandingwithfeedbackfrompeersandexperts.

26 Quality Scorecard for Blended Learning Programs

Page 39: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. Guidelines regarding minimum requirements for course development, design, and delivery of blended instruction (such as course syllabus elements, course materials, assessment strategies, faculty feedback) are followed.* 

Althoughfacultymembersareconsideredexpertsintheirdisciplines,theycannotbeexpectedtobeexpertsatdevelopingblendedcoursematerials(althoughcertainlytherearethosewhoareverygood).Forconsistency,theinstitution(orcollege)shouldprovideguidelinesleadingtoconsistentdesignforblendedcoursedevelopmentanddelivery(Mesh,2016).Indevelopinganeffectiveblendedlearningenvironment,justlikeinonlinelearning,itisimportantbeforedevelopingthecourse,todeterminewhatqualitylookslikeincoursedesignandplantheappropriatestandard(Puzziferro&Shelton,2008).Fromtheinstitution’svisionofquality,theframeworkforcoursequalityshouldemergetoguidedevelopmentofcoursematerialsthatarespecificallyeffectiveforblendedteachingandlearning.Providingstandardsoracoursedevelopmentrubricdoesnotmeanthatallcourseshavetobeidentical.Itsimplymeanstherearebasiccomponentsthatarestandardtoallcourses.

Institutionscancreatetheirownstandards,orselectand/oradaptavarietyofstandardsandblendedcourserubricsthatevaluatequality.Forexample,theOnlineLearningConsortiumprovidestheQualityCourseTeachingandInstructionalPractice(QCTIP)scorecardandco‐sponsorstheOSCQRcoursereviewrubric.Bothtoolscanbeusedinpartorinwholetoevaluatetheeffectivenessofcoursedesignaswellasaddressingteachingthecourseandusinggoodpedagogicalstrategiesforengagingblendedstudents.

Recommendations 

Provideclearguidelinesandrecommendationsforminimumstandardsthatshouldbecloselyfollowedinthecoursedevelopmentprocessbeforecoursedevelopmentbegins.

Includeinthestandards,effectivepracticesforblendedteachingandexpectationsforfacultyobligations,includingexpectationsforfacultypresenceinthecourseandcommunicationresponsetime.

Provideeffectivebestpracticesandmodelsofexemplarycoursesbeforethecoursedevelopmentbegins.

Developandimplementaprocessthatcomparesdevelopedblendedcoursestoestablishedstandardsandguidelinesonaregularbasis.

Course Development & Instructional Design 27

Page 40: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. There is consistency in course development for student retention (enrollment and course completion) and quality (i.e., courses in a program have a consistent navigational structure). 

Awell‐designedblendedcourseboostslearningandenhancesstudentretention(Amaral&Shank,2010).ImplementationofdesigntoolssuchastheBlendedLearningToolkit(Moskal,Thompson,&Futch,2015),EDUCAUSELearningInitiativeDiscoveryTool:BlendedWorkshopGuide(EDUCAUSE,Diaz,&Strickland,2009),ortheCommunityofInquiryframework(Garrison&Vaughan,2008)willensureconsistencyinthedesignoftheblendedcourse,tofacilitatelearnerinteractionandengagement.Factorsthatenhancestudents’successinblendedlearning,includeusingstudentsuccessstrategiesidentifiedintheBlendedLearningToolkitsuchaswelcomingstudentstothecourse,consistentsyllabusorganization,guidetonavigatingthecourse,a“gettingstartedmodule”,creatingopportunitiesforcollaborationandengagement,tipsforsuccess,frequentandtimelyfeedback,guidelinesonusingtechnology,linkstoresources,andtimemanagementassistance(UCF&AASCU,n.d.).

Intentionalcoursedevelopmentshouldaddressqualityconcernssuchas:consistencyincourserigorandachievementofoutcomeswithblendedcoursesincoursesdeliveredusingeitherfullyfacetofaceoronline,preparednessoffacultytofacilitatelearninginthecourse,aswellasevaluationactivitiesthatevaluatecourselearningobjectivesinamannercongruentwithsystematicevaluationplanprovidingafeedbackloop(Niemec&Otte,2009).Useofpeerreviewfortheblendedcourse,suchastheBlendedCoursePeerReviewForm(availablefromCreativeCommons),theSUNYOSCQRcoursereviewrubric,QualityCourseTeachingandInstructionalPractice(QCTIP)ortheBlendedQualityScorecardwillfacilitatetheassessmentandmeasurementofquality,facilitatingcontinualqualityimprovementaswellasprovidingdataordirectionforstrategicplanning.

Recommendations 

Useofresources,suchastheBlendKitCourse,CommunityofInquirydesignprinciplesorEDUCAUSELearningInitiativeDiscoveryTool:BlendedLearningWorkshopGuidetoensureconsistencyinthedesignoftheblendedcourse,tofacilitatelearnerinteractionandengagement.

Useofpeerreviewandaqualityrubrictoensurequalityandconsistencyinblendedcourses.RubricsareavailablefromtheOLCBlendedQualityScorecardortheBlendedCourseImplementationChecklist(availablethroughCreativeCommonslicense).

28 Quality Scorecard for Blended Learning Programs

Page 41: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. A blended course should be designed as one cohesive whole, incorporating both face‐to‐face and online experiences in complementary ways. 

Whendesigningablendedcourse,itisimportanttofullyintegrateboththeface‐to‐faceandonlinecomponentsintoeachother.Therecanbeatendencytosimply“bolton”onlinecomponentstoaface‐to‐facecourse,insteadthecourseshouldbeseenasiftwocansofdifferentlycoloredpaintsarebeingmixed.Thismixshouldyieldanewpaintwheretheoriginalcolorsnolongexistasindependentpaints.Itisimportantthatthedesignofablendedlearningcourseisnotjustatemporalconstruct.

Boththeface‐to‐faceandonlinecomponentsofablendedlearningcourseneedtobealignedwiththestatedlearningobjectivesofthecourse.Inaddition,thecourselearningobjectivesshouldbeusedtoguidenotjusttheactivitiesofthecourse,butalsotheportionofthecourseinwhichtheactivityoccurs.Theface‐to‐faceportionshouldbestudent‐centeredwhiletheonlineactivitiesarebothchallengingandengagingtocomplementtheface‐to‐faceportion.

Recommendations 

Thecourseshouldbedesignedinawaythatintegratesboththeonlineandface‐to‐faceportionsinaclearandconciseway.

Onlinematerialsaredesignedtosupporttheface‐to‐facesession. Onlineresourcesareprovidedthatallowsfortheactivitiesthatbeganintheface‐to‐face

sessiontocontinueonline.

Course Development & Instructional Design 29

Page 42: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. Instructional materials (both online and in‐class) and course syllabi are reviewed periodically to ensure they meet the blended course's and program's learning outcomes.* 

AccordingtotheSouthernAssociationofCollegesandSchoolsCoreRequirement2.5(SACS,2012),institutionsshouldengage“inongoing,integrated,andinstitution‐wideresearch‐basedplanningandevaluationprocessesthat(1)incorporateasystematicreviewof…outcomes;(2)resultincontinuingimprovementininstitutionalquality;and(3)demonstratetheinstitutioniseffectivelyaccomplishingitsmission.”Furthermore,ComprehensiveStandard3.3.1prescribesthattheinstitutionwillidentify“expectedoutcomes,assessestheextenttowhichitachievestheseoutcomes,andprovidesevidenceofimprovementbasedonanalysisoftheresults...ineducationalprograms,toincludestudentlearningoutcomes.”Eachoftheregionalaccreditorshassimilarrecommendations.Forcontinuousimprovement,theinstitutionshouldhaveaperiodicreviewoflearningoutcomes,analyzethestudentachievementdata,anddevelopimprovementsbasedonthosedata.Thisshouldincludeblendedcoursesaswellastraditionalandonlinecourses.

Onceblendedcoursematerialsaredeveloped,thecoursedevelopmentprocessshouldnotendatthatpoint.Infact,rarelyshouldblendedcoursematerialsremaincompletelyunaltered.Likewise,instructionaldesignmodelsrecommendanevaluationprocessshouldbeimplementedafterthecourseisfirsttaught(andinsubsequentofferings)andpossiblerevisionsaddressedforcontinuousimprovement,withtheultimategoalbeingtheachievementoflearningoutcomes.Allblendedcoursesshouldbereviewedonaperiodicbasis.

Recommendations 

Engageinasystematicreviewoflearningoutcomestodeterminecurrency,relevance,andmeasurability.

Implementaprocesswhereallcoursesarereviewedandupdatedonarecurringbasis. Systematicallyreviewandrefineblendedinstructionalmaterialstobesuretheysupport

thecoursesyllabusandcurrentlearningoutcomes. Systematicallyreviewblendedcoursematerialsandcoursesyllabitoensurerelevanceof

educationaltechnologyandassessmentactivities.

30 Quality Scorecard for Blended Learning Programs

Page 43: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. Course assignments and activities are reviewed periodically to ensure they meet the blended courses' and program's learning outcomes. 

Asstudentsprogressthroughablendedcourse,assessmentsmustbedevelopedthatmeasurethelevelofstudentsuccessandmasteryofthematerials.Theseassessmentsmaytakemanyforms;however,itiscriticalthatassessmentsmapexactlytothecoursecontent,andthattheyaredefinedearlyinthecoursedesignprocess.Assessmentsfortheblendedenvironmentdonotnecessarilyneedtomatchassessmentsgiveninface‐to‐facecourses,althoughforcomparisonpurposes,thefacultymaychoosetoadministerthesameassessment.Assessmentsshouldbemappedearlyinthedesignprocess.Assessmentsthatworkwellforblendedcourseworkincludepapers/essays,casestudies,problem‐basedactivities,quizzesandtests,andgroupprojects.Papersandotherwrittenassessmentsareofteneffectiveevaluationtoolsforblendedlearning,astheycanindicateastudent’scriticalthinkinganddirectmasteryofthematerial.

Newformsofblendedassessmentareconstantlyevolvingthatallowamorestudent‐centeredapproach.Properlydesignedandimplementedassessmentsshouldprovidetheinstructorwithevidenceofthestudents’learning,theefficacyofthecoursematerials,andprovideinputforfutureenhancementsofthecoursematerials.

Recommendations 

Mapcourseassessmentsearlyinthecoursedesignprocess. Recognizethatasingleassessmentcanmeasureseverallearningobjectives. Encouragetheuseofrubricsforassessmentthatguidestudentsandprovidefeedback. Provideself‐assessmenttools(thatprovideautomaticfeedback)tostudentsasthey

progressthroughtheclass.

Course Development & Instructional Design 31

Page 44: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

7. Student‐centered instruction is considered during the course development process (i.e., student engagement, immersion, and personal responsibility). 

Student‐centeredinstructionacknowledgestheshiftingrolesintheclassroom(Dewey,1938;Piaget,1963;Vygotsky,1978)sothatstudentslearnbestwhentheyarepersonallymotivatedtoobtainnewskills,behaviors,andknowledgetosolveanauthenticchallenge.Thus,blendededucationisparticularlywell‐suitedforthisshiftinrolesbecauseofitscapacitytodrawonindividualperspectivesandcomparethemwithmultipleperspectives.Inlearner‐centereddesign,theteachershiftstothefacilitatoroflearningandamodelofexpertlearning,ratherthanadispenserofknowledge(NorthCentralRegionalEducationalLaboratory,n.d.),andthuspromotesinteractionamonglearnerstosolveauthentic,real‐worldproblems,andcollaborateonsolutionsandlearningpaths.Blendedinstructorsareseenbystudentsasservinganorganizingandcoursedesignfunctionmorethantraditionalrolesoflecturer,discussionfacilitator,orsocialsupporter(Hung&Chou,2015).

Recommendations 

Whendesigningtheblendedcourse,identifychallengequestionsthattriggerindividualresponsesandmotivateinteractionamonglearners.

Connectassignmentswithlearner’slivesandpreviousexperiences. Designactivitiesandassignmentsthatengagestudentsinlearninghowtheylearn. Reducetheroleoflecturesanddirectinstructionactivitiesandfocusonstudentproblem‐

solving,collaboration,andlearningpaths.

32 Quality Scorecard for Blended Learning Programs

Page 45: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

8. Course design promotes both faculty and student engagement. 

Blendedlearningofferstheopportunitytoengagelearnerswitheachotherandwiththecoursecontent.Bothstudentsandfacultycanhavearichandrewardingexperience.

Blendeddeliveryoffersawidearrayofopportunitiesforengagementthroughcoursedesignanddelivery.Coursedesignthatallowsstudentstointeractwithcontent,faculty,andclassmateshelpsmaintainahighlyengagedenvironment(Swan,2004).Infact,severalofChickeringandEhrmann’s(1996)SevenPrinciplesforGoodPracticeinUndergraduateEducationcenteronalllevelsofinteractionandindicatehowtechnologycanbeusedeffectivelyforinteraction.

Today,facultyandstudentsareutilizingmoreengagingandcollaborativeactivitiesthatexpandengagement,suchasimmersiveenvironments,blogs,wikis,audioandvideoproductions,andcollaborativeprojects.TheOnlineLearningConsortium’sEffectivePracticesprovideexcellentexamplesofinteractivecollaborationthatengagesstudents(OnlineLearningConsortium,2016).

Recommendations 

Provideinstructionaldesignsupportforfacultytotransitionfromaclassroom‐based(possiblylecture‐based)coursetoaninteractiveblendedcoursethatemphasizesactivelearning.Thisshouldincludeexamplesandmodelsofbestpractices.

Providefacultywiththeappropriatetrainingrequiredtousetechnologytoincreaseinteraction.

Encouragefacultytoprovidestudentswithclearlydefinedexpectationsregardingmethodsandfrequencyofengagementwithotherstudents,faculty,andcontent.

Providethemeansforstudentstoprovidefeedbackregardingthelevelofengagementofferedbythedesignofthecourse.

Course Development & Instructional Design 33

Page 46: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

9. Course workloads are reviewed to ensure it is appropriate for designated credit allocation. 

Blendedcourseworkloadsshouldbereviewedperiodicallytoensurethattheworkloadisconsistentwithcredit/clockhourasdefinedbytheorganizationandUSDE(2015)toensurecompliancewithregulatoryandaccreditingrequirements.Thiswillincludetraditionalseattimeaswellastimeneededtocompletedesignatedlearningactivitiestoachievethecourselearningoutcomesusingtheblendedformat.Academicorganizationswillneedtobeabletoprovidesupportfortheequitabledistributionofcreditamongblended,onlineandfacetofacecourses(ForumonEducationAbroad,2014).Itisimportanttonotethatwhilethecourseworkloaddoesnotdifferentiatebetweenfacetofaceoronlinelearningactivities,itdoesrefertoanequitableamountofstudentwork(USDE,2013).

Recommendations 

Useastandardprocessfordefiningcredit/clockhourstoensureconsistencyinallocationaswellascongruencywithregulatoryandaccreditationstandards

Periodicallyreviewexistingblendedcoursestoensuretheyhaveappropriatelydesignatedcreditallocation.

Reviewnewblendedcoursestoensuretheyhaveappropriatelydesignatedcreditallocation.

34 Quality Scorecard for Blended Learning Programs

Page 47: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

10.  A process is established and followed for evaluating the effectiveness of current and emerging technologies to support the achievement of learning outcomes and delivering course content. 

Evaluatingandrecommendingtheuseofnewtechnologyfortheblendedclassroombecomesmorechallengingeachdaywiththerapidintroductionofnewapplicationsandfunctionality.Thus,aninstitutionshouldplanfortheappropriatemeasuresneededtosystematicallyanalyzetechnologiesthatwillsupportblendedteachingandlearning.

Newtechnologiesemergeatsucharapidpacethatinstructionaldesignersandacademictechnologistsneedtocontinuouslymonitorandexamineissuessurroundingevaluatingandintegratingemergingtechnologyforteachingandlearning,withafocusonachievinglearningoutcomes.Today'stoolkitofacademictechnologyandinnovativetoolsincludecloudcomputing,videoconferencing,virtuallaboratories,ebooks,virtualworlds,socialnetworking,mobileapplications,openeducationresourcesandadvancedlearningmanagementsystems.Whatisnowinnovative,changesquickly;moreover,what'sinnovativeforoneinstitutionmaybeoldtechnologyforanotheroreventoo“bleeding‐edge”foryetanother.

Notalltools"outofthebox"arenecessarilyusefulforimprovinglearningoutcomes,soitisimportantforeducatorstofirstidentifylearningoutcomesandmethodsforachievementandthenapplytheappropriatetechnology.Integratingnewtechnologiesalsorequiresupdatedperspectivesabouttheiruse.Whileprovidingflexibility,theymaybedifficulttodeploybecauseofissuesrelatedtoscalability,policy,andtechnologyadoptionvariations.Armedwiththisknowledge,onlythencantheeducatorevaluateandrecommendthedeploymentorappropriaterejectionofthetechnology.

Recommendations 

Supporttheuseofemergingtechnologiesbyexplainingtheirvalueforimprovingteachingandlearning.Forexample,Odom(2010)explainedhowWeb2.0technologiessupportconstructivism.

Askwhattheinstitutionneedsbeforeinvestinginnewtechnologies,forexample:o Whatemergingtechnologiesareneededforthisinstitutionatthistime?o Whataretheproblemsassociatedwithintegratingthem?o Whatstandardevaluativetoolswillbeusedtoassessandrecommendtheemerging

technologies? Encourageandenablefacultyandcontentexpertstobefamiliarwithinnovative

technologies. Minimizeskepticismtowardsnewtechnologybyusingwell‐trained,knowledgeable

academics,technologists,andanalystsintheevaluationprocess.

Course Development & Instructional Design 35

Page 48: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

11.  Course embedded technology actively supports the achievement of learning outcomes and delivering course content and unnecessary use of technology is minimized.* 

Therapidlychanginglandscapeoflearningtechnologiesrequiresinstitutionalagilityandadaptation.Managingtheuseoftechnologyintheblendedclassroombecomesmorechallengingeachdaywiththecontinuousintroductionofnewapplicationsandfunctionality.Becauseofthis,wemustthinkbothstrategicallyandtacticallyaboutthebestuseoftechnologytosupportlearningoutcomesforthedeliveryofcoursecontent.

Technologyuseaboundsnotonlytosupportlearning,butalsotosupportstudentinformationsystems,studentfinancialservices,library,facultyrecruitmentanddevelopment,aswellasoutcomesassessment.Strategically,bestpracticesareneededfortheseareastoappropriatelyassimilatetechnologyuseacrosstheentireinstitutionandtobeasreliablyavailableaspossible.However,specialemphasisneedstobeonintegratingbestpracticesintotheuseoftechnologyforteachingandlearningwhiledevelopinganddeliveringcoursecontenttostudents.Theemphasisshouldincludetoolsthatsupportlearningoutcomesandwillproducesustainedresultsoncethestudentcompletesthecourseandprogram.

Wefacemanynewtechnologiesintheareasofweb‐basedmaterials,cloudcomputing,videoconferencing,virtuallaboratories,ebooks,virtualworlds,socialnetworking,mobileapplications,openeducationresources,andofcourse,thenucleusofblendedcoursedelivery,thelearningmanagementsystem.However,notalltoolsarenecessarilyapplicabletolearningoutcomes,soacademicsmustfirstidentifylearning‐relatedissuesandthenapplytechnologiesthataddressthelearningissues.Integratingthesetechnologiesrequiresupdatedperspectivesabouttheiruse.Whiletoolsmayincreaseflexibility,theymayalsobedifficulttodeployduetoissuesofscalability,policy,andtechnologyadoptionvariations.Technologyshouldnotbethedriverinthedeliveryofcourses;instructionaldesignersshouldseekthebestmixoftechnologiestoachievelearning(Palloff&Pratt,1999).

Theinstitutionmustaligntechnologieswithbothprogramoutcomesandteachingandlearninggoals.Thelearningmanagementsystemshouldbeuserfriendly.Forflexibility,technologyadvancesalreadyprovidethepotentialdeliveryofcontentanytime,anywhere.Fordiversity,institutionsmustmakedecisionsaboutapproachestoexternalcontentacquisition,learningmanagementsystemactivities,intelligenttutoringimplications,adaptiveassessments,etc.Forconsistency,technologymustsupportcoursecreationinacentralizedfashioninthedesign,development,andmaintenanceofblendedcoursesthatareenhancedwithacademiccontent,sophisticatedtools,andinnovativetechnologies.

Recommendations 

Developinstitutionalguidelinesandstandardsforinvestigatingtechnologies,notjustforthesakeofintegratingnewtoolsbuttobettersupportlearningoutcomesandtheneedsoflearners.

Developstandardsandpoliciesforappropriatelyacquiringandintegratingopensourcematerialsandlearningobjects,audio,video,andothermultimedia.

Providebestpracticesofwheretolocatequalityopensourcematerials,audio,video,andothermultimedia.

36 Quality Scorecard for Blended Learning Programs

Page 49: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Provideguidanceforusingnewtechnologiesintheformofreports,presentations,demonstrationsorjust‐in‐timetrainingmaterials.

Useselectioncriteriaforevaluatingnewtechnologies,suchasaccessibilityandaffordability,usability,privacyandintellectualpropertyaffordance,workloadandtimemanagement,andinteractivity(Bovard,n.d.).

Course Development & Instructional Design 37

Page 50: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

12. Usability tests are conducted and applied and recommendations based upon Web Content Accessibility Guidelines (WCAGs) are incorporated in the course design process. 

AccordingtotheWebAccessibilityInitiative(2012),theWebContentAccessibilityGuidelines(WCAG)weredevelopedthroughacooperativeprocesswithindividualsandorganizationsaroundtheworld.Thegoalistoprovideasinglesharedstandardforwebcontentaccessibilitythatmeetstheneedsofindividuals,organizations,andgovernmentsinternationally.

TheWebAccessibilityInitiative(2012)providesmanyresourcesinthefollowingareas:

● GettingStarted● DesigningforInclusion● GuidelinesandTechniques● PlanningandImplementing● EvaluatingAccessibility● PresentationsandTutorials● GettingInvolvedwiththeWebAccessibilityInitiative(WAI).

WCAGisatechnicalstandard(WebAccessibilityInitiative,2012)andisnotintendedtoserveasanintroductiontoaccessibilityissues.Asablendededucationadministrator,itmaybenecessaryfortraininganddevelopmentactivitiestobeginwithanintroductionandgraduatetoWCAGdependingontheaudience.However,asmoreandmoreblendedcoursestranscendgeographicboundaries,developingblendedcoursesthatmeettheWCAGtechnicalstandardshouldbeconsideredbestpractice.

Recommendations 

StayabreastofadvanceswithintheWebContentandAccessibilityGuidelines(WCAG)initiative.

Providetrainingopportunitiesforfacultyandcoursedevelopment/instructionaldesignpersonnelwhichhighlighttheimportanceofdesigningforinclusionandshowcaserelevantguidelinesandtechniques.

PlanandimplementblendedcoursesthataredesignedusingtheWCAGfromtheinitialfoundations.

UseachecklistforbestpracticesforblendedcoursedevelopmentaccordingtoWCAG. Identifyanddeployinternalusabilitytestingprocessesthatunderscoretheimportanceof

andincorporatetheWCAG.

 

 

38 Quality Scorecard for Blended Learning Programs

Page 51: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

13. Curriculum development is a core responsibility for faculty (i.e., faculty should be involved in either the development or the decision making for the blended curriculum choices). 

Thecurriculumisthemajorstatementanyinstitutionmakesaboutitself,aboutwhatitcancontributetotheintellectualdevelopmentofstudents,aboutwhatitthinksisimportantinitsteachingservicetosociety(TheCarnegieFoundationfortheAdvancementofTeaching,1977).

Theresponsibilityforcurriculumdevelopmentordecisionmakingandselectionshouldresidewithfacultybecauseoftheirprofessionalcredentialsandcommitmenttoteaching,research,andservice.Thiscoreresponsibilitytypicallyincludesdevelopingcurriculaandkeepingitcurrent;reviewingandevaluatingcoursesandprograms,maintaininggradingstandards,selectinginstructionalresourcesandmedia,anddevelopingnewprograms.Becauseoftheprolificamountofresourcesforblendedcourses,facultyshouldbeinvolvediftheprogramorinstitutionchoosestouseresourcessuchaspurchasingready‐madecoursematerials.

Thefollowingallputpressureontraditionalgovernanceanddecision‐makingprocesses,includingcurriculumdevelopment:

● theproliferationoflearningmediaandchannels;● competency‐basedlearninginitiatives;● regulatorydemandsforgreateraccountability;● federal,corporateandfoundationcallsforbetterworkforcedevelopment;● theavailabilityofopeneducationalresourcesandcommercialcontentandcourse

providers;● standardizationofcourses;● thegrowthofpart‐timefacultyandnon‐traditionallearners;and● fiscalshortfalls.

Recommendations 

Ensurefacultyleadintheblendedcurriculaprocessincludingdecisionstoadoptcontentexperts,andopenresources.

Ensurefacultyleadorareheavilyinvolvedindecisionstochangecurricula,courses,andprograms.

Providefacultywithdataaboutlearningoutcomes,persistence,graduationrates,defaultrates,costefficiencies,andgraduateemployability.

Engageinnationaleffortstodetermineandcomparesuccessratesandqualitystandards.

 

Course Development & Instructional Design 39

Page 52: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

14. Faculty support and resources are provided to promote the best use of blended delivery method in course development and instructional design to facilitate teaching and learning. 

Whendesigningablendedcourse,itisimportanttofullyintegrateboththeface‐to‐faceandonlinecomponentsintoeachother.Therecanbeatendencytosimply“bolton”onlinecomponentstoaface‐to‐facecourse,insteadthecourseshouldbeseenasiftwocansofdifferentlycoloredpaintsarebeingmixed.Thismixshouldyieldanewpaintwheretheoriginalcolorsnolongexistasindependentpaints.Itisimportantthatthedesignofablendedlearningcourseisnotjustatemporalconstruct.

Boththeface‐to‐faceandonlinecomponentsofablendedlearningcourseneedtobealignedwiththestatedlearningobjectivesofthecourse.Inaddition,thecourselearningobjectivesshouldbeusedtoguidenotjusttheactivitiesofthecourse,butalsotheportionofthecourseinwhichtheactivityoccurs.Theface‐to‐faceportionshouldbestudent‐centeredwhiletheonlineactivitiesarebothchallengingandengagingtocomplementtheface‐to‐faceportion.

Recommendations 

Thecourseshouldbedesignedinawaythatintegratesboththeonlineandface‐to‐faceportionsinaclearandconciseway.

Onlinematerialsaredesignedtosupporttheface‐to‐facesession. Onlineresourcesareprovidedthatallowsfortheactivitiesthatbeganintheface‐to‐face

sessiontocontinueonline.

40 Quality Scorecard for Blended Learning Programs

Page 53: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforCourseDevelopmentandInstructionalDesign

Aramal,K.,&Shank,J.(2010).EnhancingStudentLearningandRetentionwithBlendedLearningClassGuides.EducauseReview(December,2010).Retrievedfromhttp://er.educause.edu/articles/2010/12/enhancing‐student‐learning‐and‐retention‐with‐blended‐learning‐class‐guides

Bovard,B.(n.d.).Web2.0selectioncriteria:Savetimechoosinganappropriatetool.Retrievedfromhttp://150.164.100.248/cei/data1/arquivos/Web20selectioncriteria.pdf

Chickering,A.W.,&Ehrmann,S.(1996).ImplementingtheSevenPrinciples:Technologyaslever.AAHEBulletin,October,3‐6.Retrievedfromhttp://www.tltgroup.org/programs/seven.html

Dewey,J.(1938).Experienceandeducation.NewYork,NY:FreePress.

ForumonEducationAbroad.(2014).DefiningtheCreditHourforTitleIVFederalFinancialAidPurposes.Retrievedfromhttp://forumea.org/wp‐content/uploads/2014/10/DefiningtheCreditHourforTitleIVFederalFinancialAidPurposesFinal.pdf

Garrison,D.R.,Vaughan,N.(2008).BlendedLearninginHigherEducation:Framework,Principles,andGuidelines.Jossey‐Bass

Hung,M.L.,&Chou,C.(2015).Students'perceptionsofinstructors'rolesinblendedandonlinelearningenvironments:Acomparativestudy.Computers&Education,81,315‐325.

McGee,P.,&Reis,A.(2012).Blendedcoursedesign:Asynthesisofbestpractices.JournalofAsynchronousLearningNetworks,16(4),7–22.Retrievedfromhttp://sloanconsortium.org/jaln/v16n4/blended‐course‐design‐synthesis‐best‐practices

Mesh,L.I.(2016).Acurriculum‐basedapproachtoblendedlearning.JournalofE‐Learning&KnowledgeSociety,12(3),87‐97.

Moskal,P.,Thompson,K.,&Futch,L.(2015).Enrollment,engagement,andsatisfactionintheBlendKitfacultydevelopmentopen,onlinecourse.OnlineLearning,19(4),Retrieved from http://olj.onlinelearningconsortium.org/index.php/olj/article/view/555/173.

Niemec,M.,&Otte,G.(2009).Anadministrator’sguidetothewhy’sandhow’sofblendedlearning.JournalofAsynchronousLearningNetworks,13(1),19‐30.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ909846.pdf

NorthCentralRegionalEducationalLaboratory.(n.d.).Teachervs.learner‐centeredinstruction.Retrievedfromhttp://www.nclrc.org/essentials/goalsmethods/learncentpop.html

Odom,L.S.(2010).MappingnewlyidentifiedWeb2.0benefitstoknownbestpracticesindistanceeducation.Retrievedfromhttp://instruction.blackhawk.edu/blackboard/pdf/Mapping_Web_2‐0_Benefits_LaddieOdom.pdf

OnlineLearningConsortium.(2016).OurQualityFramework.Retrievedfromhttp://onlinelearningconsortium.org/about/quality‐framework‐five‐pillars/

Palloff,R.,&Pratt,K.(1999).Buildinglearningcommunitiesincyberspace:Effectivestrategiesfortheonlineclassroom.SanFrancisco,CA:Jossey‐Bass.

Piaget,J.(1963).Originsofintelligenceinchildren.NewYorkCity,NY:Norton.

Course Development & Instructional Design 41

Page 54: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Puzziferro,M.,&Shelton,K.(2008,December).Amodelfordevelopinghigh‐qualityonlinecourses:Integratingasystemsapproachwithlearningtheory.JournalofAsynchronousLearningNetworks,12,3‐4.Retrievedfromhttp://olj.onlinelearningconsortium.org/index.php/olj/article/view/58

SouthernAssociationofCollegesandSchoolsCommissiononColleges.(2012).Theprinciplesofaccreditation:Foundationsforqualityenhancement,2012Edition.Decatur,GA:SouthernAssociationofCollegesandSchoolsCommissiononColleges.Retrievedfromhttp://www.sacscoc.org/pdf/2012PrinciplesOfAcreditation.pdf

Swan,K.(2004).Relationshipsbetweeninteractionsandlearninginonlineenvironments.Needham,MA.Retrievedfromhttps://www.researchgate.net/publication/250700769_Relationships_Between_Interactions_and_Learning_In_Online_Environments

TheCarnegieFoundationfortheAdvancementofTeaching.(1977).Missionofthecollegecurriculum:Acontemporaryreviewwithsuggestions.SanFrancisco,CA:Jossey‐Bass.

UniversityofCentralFlorida(UCF),&AmericanAssociationofStateCollegesandUniversities(AASCU).(n.d.)BlendKitCourse:DIYProjectTasks.Retrievedfromhttps://blended.online.ucf.edu/files/2011/06/implementation_checklist.pdf

UnitedStatesDepartmentofEducation(USDE).(2013).LawandGuidance:ProgramIntegrityQuestionandAnswers‐CreditHour.Retrievedfromhttp://www2.ed.gov/policy/highered/reg/hearulemaking/2009/credit.html

UnitedStatesDepartmentofEducation(USDE).(2015).FederalStudentAidHandbook.Retrievedfromhttps://ifap.ed.gov/fsahandbook/attachments/1516FSAHbkActiveIndexMaster.pdf

Vygotsky,L.S.(1978).Mindinsociety.Thedevelopmentofhigherpsychologicalprocesses.Cambridge,MA:HarvardUniversityPress.

WebAccessibilityInitiative.(2012).Webaccessibilityinitiativehomepage.Retrievedfromhttp://www.w3.org/WAI/

42 Quality Scorecard for Blended Learning Programs

Page 55: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Theblendedcourseincludesasyllabusoutliningcourseobjectives,learningoutcomes,evaluationmethods,booksandsupplies,technicalandproctoringrequirements,andotherrelatedcourseinformation,makingcourserequirementsandcoursescheduletransparent.*

2. Thecoursestructureensuresthatallstudents,regardlessoflocation,haveaccesstolibrary/learningresourcesthatadequatelysupporttheblendedcourse.*

3. Expectationsforstudentassignmentcompletion,gradepolicy,andfacultyresponseareclearlyprovidedinthecoursesyllabus.*

4. Linksorexplanationsoftechnicalsupportareavailableinthecourse(i.e.,eachcourseprovidessuggestedsolutionstopotentialtechnicalissuesand/orlinksfortechnicalassistance).

5. Rulesorstandardsforappropriatestudentbehavior,bothonlineandface‐to‐face,areprovidedwithinthecourse.

6. Instructionalmaterialsareeasilyaccessibletothestudent,easytouse,withanabilitytobeaccessedbymultipleoperatingsystemsandapplications.

7. Instructionalmaterialsareeasilyaccessedbystudentswithdisabilitiesviaalternativeinstructionalstrategiesand/orreferraltospecialinstitutionalresources.

8. Theblendedcourseisvisuallyappealingtothestudentandthecourseisnavigationallysound.

Course Structure

Page 56: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. The blended course includes a syllabus outlining course objectives, learning outcomes, evaluation methods, books and supplies, technical and proctoring requirements, and other related course information, making course requirements and course schedule transparent.*

Thesoonerstudentsknowthecourserequirements,theeasiertheirtransitionwillbeintotheblendedclassroom.Themoredetailedthesyllabus,thefewerquestionsaninstructorwillhavetorespondtointheearlystageofparticipantsgettingtoknowandtrusteachotherandbecomecomfortableintheblendedcourseenvironment.

Ataminimum,adetailedcoursesyllabuswillincludecourseobjectives,learningoutcomes,evaluationmethods,informationaboutthetextbook(s)andotherrequiredmaterials,aswellastechnicalrequirements.Inaddition,thetextbookandrequiredcoursematerialinformationshouldbeavailablewellbeforeastudentenrollsinthecourse.Thisallowsadvancepreparationonthepartofthestudent,timeforaccessibilitysupportservicestohelpwithinput/outputissuesforstudentswithdisabilities,andcanalsoserveasanicerecruitingtoolfortheprogram.

Weeklymodules/unitsshouldindicateamoregranularsetoflearningoutcomesthatareclearlylinkedbacktothecourseobjectives.Toenhancetransparency,term‐specificdates(firstdayofclass,lastdaytodrop,finalsweek,gradesdue)andlocaldates(campus,state,andnationalholidays)shouldbeindicated.

Recommendations 

Provide,ataminimum,thetextbookandrequiredmaterialsorresourcestostudentsinadvanceoftheircourseregistration.

Createauser‐friendlysyllabusthatiseasilyaccessible.Separatethesyllabusintosectionstoallowforeasydownloadingandprintingasneeded.

Developawell‐designedsyllabusincludinganswerstocommonquestions,providelinkstopopularcampuswebsites(e.g.,library)andincludeallinstitutionalpolicyrequirements.

Utilizeasetofconsistentstandardsforsyllabusdevelopmentsuchasabestpracticesrubric.

44 Quality Scorecard for Blended Learning Programs

Page 57: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. The course structure ensures that all students, regardless of location, have access to library/ learning resources that adequately support the blended course.* 

Allstudentsareentitledtothesamelearningresourcesprovidedbytheinstitutionregardlessofiftheyareattendingface‐to‐face,blended,orfullyonlinecourses.Thismeansanylearningresourcesupportthatisprovidedoncampusshouldalsobeprovidedoratleastofferedtoblendedstudents.Accessshouldincludetheinstitution’slibraryforbookcheckoutandonlineresearchcapabilitiesandallotherinstitutionallearningresourcessuchasawritingcenter,tutoringsupport,counseling,etc.

Inadditiontoalinktothelibrarywebsite,alinkspageforresourceswithinthecoursecanlistrecommendedjournalsobtainablethroughthecampuslibrary,linkstorelevanthowtouseonlineresources,informationaboutplagiarism,howtogethelp,andlinkstoexternalwebsitesthathaverequiredorrecommendedmaterials(whichmustbeaccessibleoravailableinalternateformats).Instantmessagingorchatprogramsmaybeusedtoprovidejust‐in‐timeservicethroughtheinstitutionalwebsitebecauseblendedstudentsmayneedadditional

Recommendations 

Offerquicklinkstothelibrarywebsite,writingcenter,tutoring,andotherapplicablelearningresourceswithintheblendedcoursematerials.

Ensurealllibraryandacademicsupportservicesareavailabletoalltypesofstudents. Dedicateatleastonelibrariantoprovidededicatedsupportinawaythatisfeasiblegiven

theinstitution’sorganizationalstructureandresources. Communicatewithblendedstudentsusingavarietyofmethodstoremindthemofall

resourcesavailabletothem.

Course Structure 45

Page 58: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. Expectations for student assignment completion, grade policy, and faculty response are clearly provided in the course syllabus.* 

Clearlypostedexpectations,instructions,duedates,pointassignments/rubrics,andinstructorresponsetimesareveryimportantforstudentswhomightnototherwisebeorganizedenoughtokeepupovertime,andareparticularlyimportantforstudentswithexecutivefunctiondisorders.Gradingpoliciesshouldbedesignatedinthecoursesyllabus,includinglinkstoassignmentrubrics.

Instructorresponsetimescoulddifferentiateamongdiscussionboards,email,holidays,anddays“off”andshouldclearlyshowthattheinstructorwillnotbeavailable24x7.However,thereshouldbeguidelinesthathelpwithstudentexpectationsofwhentoanticipatearesponsefromtheinstructor.Onlineofficehourscanbehelpfulformoststudents,thoughsynchronousrequirementswillbeanadvantagetosome.Synchronousmethodsincludeinstantmessaging,webconferencing,face‐to‐facemeetings,andchat.Considerprovidingaone‐on‐onesessionwithastudentwhoneedsextratimetoinputtextorprocessquestions/answers.

“Head’sup”remindersofupcomingeventsareusefulforallstudents,andcangenerallybelinkedtotheemailsystemoftheLMSsothattheinformationwillsimultaneouslybecomeanannouncementwithinthecourseandanemailtoeachstudent.

Recommendations 

Provideclearinstructorofficehoursaswellasexpectedresponsetimetoemails,discussionboardpostings,andtextmessages(ifencouragedbyinstructor)inthecoursesyllabus.

Designateinthecourseschedulewhencoursesessionsbegin,andincludetopics,readingassignments,andstudentdeliverableswithduedateandtime.

Ensureinstitutional,program,and/orspecificcoursegradingpoliciesareincludedinthesyllabus.

Encouragefacultytoprovidegradingrubricsforeachassignment. Developandprovidealatepolicythatclearlycommunicateswhatthepenaltywillbe(if

acceptedatall)whenassignmentsaresubmittedlate.

46 Quality Scorecard for Blended Learning Programs

Page 59: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. Links or explanations of technical support are available in the course (i.e., each course provides suggested solutions to potential technical issues and/or links for technical assistance). 

Allofthetechnicalaspectsofablendedcoursearenotalwaysobvioustostudents,whojustwanttoquicklygetintothecoursematerialsandstartthecourse.Instructorsshouldnothavetospendvaluableclasstimetryingtotroubleshoottechnicalissuesforstudentswhoneedhelp.Theinstitutionshould,ataminimum,provideahelppageortutorialforthelearningmanagementsystemandthecampusemailwebsite.Acampuspolicyon“safecomputing”isbeneficial.Students(andinstructors)areoftennotsureifaproblemiswiththeirInternetbrowser,theirblendedInternetprovider,thelearningmanagementsystem,theinstitution’sserver,ortheInternetitself.Instructorsshouldnotbetheprimaryresourceforresolvingtheseissues,especiallyfortheirstudents.

Accesstospecifictechnologysupportcanbelinkedwithinthecoursematerials.Forexample,ifanaudiofileistobeplayed,alinktothecorrespondingplayersoftwareshouldbeprovidedwithinstructionsonhowtodownloadit,installit,andplaythefile.

Recommendations 

Provideacoursediscussionthreadfocusedontechnicalissueswherestudentsareencouragedtopostandanswerquestionstohelpeachother.

Provideafrequentlyaskedquestionsareainthecoursethataddressestypicaltechnicalsupportissues.

Includebasicsupportlinksinthecoursesyllabusandalsoincludealinktothestudentaccessibilitycenter/resources(alongwithafriendlystatementonrequestinghelpfordisabilityaccommodations).

Remindstudentstostorecopiesoftheirelectronicfilesinmorethanoneplaceandhaveanupdatedviruscheckerinstalledontheircomputer.

Ensurecampustechnicalresourcesandsupportsareavailableforstudentsinface‐to‐faceaspectsoftheblendedcourseorprogram.

Course Structure 47

Page 60: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. Rules or standards for appropriate student behavior, both online and face‐to‐face, are provided within the course. 

Becauseoftheubiquityoftechnology‐mediatedinteractionsthatblendedstudentsface,itisreallyimportanttobeawareof,andprovidesupportfor,eachaspectoftheseinteractions.Fromthetimeastudentfirstlogsintothecourseandfindsthesyllabus,throughoutalltheassignmentsubmissions,discussions,anduntilthelastexamorpaperishandedin,technologyiseverpresentandcanpossiblytocreatebarriersorbeasourceofstudentfrustration.Thisinturn,canleadtostudentmisbehaviorinthecourseandwhilefacultyaremoreaccustomedtodealingwiththatface‐to‐face,theonlineaspectoftheblendedclassroomcanbeintimidating.Facultyneedaframeworkofclassroomrulestoreferenceandreceiveguidance.

Someinstitutionsaredevelopingresourcesguidestoassistfacultyinmanagingstudentbehaviorblended.Examplesofsomeoftheseguidesorcodesofconductspecificallyalignedwithblendedcoursesinclude:

FoothillCollege(n.d.).http://www.foothill.edu/services/honorint.php VirginiaCommonwealthUniversity(2009).

https://www.vcu.edu/cte/resources/OTLRG/04_12_Behavior.html

Recommendations 

Developrules,standards,orcodesofconductfocusingonappropriatestudentbehaviorforblendedstudents.

Trainfacultytobepreparedforpotentialstudentmisbehaviorwithrecommendationsforhowtoaddressit.Thiscouldincludepracticessuchasblendedroleplayingforhowaninstructorshouldrespondwhenastudentverbally(viatext)attacksanotherstudentinthediscussionforums.

48 Quality Scorecard for Blended Learning Programs

Page 61: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. Instructional materials are easily accessible to the student, easy to use, with an ability to be accessed by multiple operating systems and applications. 

Inclusivedesignshouldbethefoundationforblendedcoursedesignandcoursestructure.Whilestudentswithdocumenteddisabilitiespresentthemostobviousdesignchallenges,itisusefultoexpandthedefinitionof“at‐risk”beyondthat.Problemswithvision,mobility,hearing,andcognitioncanaffectanyoneatanytime(andnearlyeveryoneastheyage),ascanillnessandthesideeffectsofmedicationandfatigue.Otherbarriersincludehavingtoreadandwriteusinganunfamiliarlanguage,havingtodeciphernewlyencounteredacademicwritingandsubjectareaterminology,andbalancingmanydemandsontimeandenergy.

Usabledesignincludespresentingcoursematerialswith“writingfortheweb”rulessuchaschunking,plentyofwhitespace,simpleandobviousnavigation(withasfewclicksaspossible),clearlylabeledbuttonsandmenuitems,judicioususeofcolorwithanemphasisongoodcontrast,andnotusingcoloralonetoprovideinformationforthosewhoarecolor‐blind(forexample,adisplaywitharedbarshowingfemales,thegreenbarshowingmalesisindecipherabletosomestudents).

AccordingtotheUsability.gov(2014)website“UXisagrowingfieldthatisverymuchstillbeingdefined”(para.4).TheUserExperienceProfessionalAssociation(2014)definesuserexperiencedesignas“adisciplineconcernedwithalltheelementsthattogethermakeupthatinterface,includinglayout,visualdesign,text,brand,sound,andinteraction”(para.2).Gube(2010)assertedthatthe“userexperienceishowapersonfeelswheninterfacingwithasystem.Thesystemcouldbeawebsite,awebapplicationordesktopsoftwareand,inmoderncontexts,isgenerallydenotedbysomeformofhuman‐computerinteraction(HCI)”(para.4).Forblendedadministrators,focusingresourcesontheUXasitrelatestocoursestructureisincreasinglybecomingabestpractice.

Recommendations 

Writethecoursesyllabus,instructions,andweeklyobjectivesineasytounderstandlanguage.

Usefileformatsthatstudentsaremostlikelytouse,i.e.pdforhtmlfiles. ProvidelinkswithintheblendedcoursetoInternetbrowserplug‐inswhenusingless

commonfileformats.Forexample,pdffilesneedtheAdobeReaderplug‐intoallowthefiletobereadwithinthebrowser.

ConsiderhowUXdesignprinciplesbecomeinherentlyembeddedwithinthecoursestructure.

Course Structure 49

Page 62: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

7. Instructional materials are easily accessed by students with disabilities via alternative instructional strategies and/or referral to special institutional resources. 

Adisabilitycanoccursuddenlyandunexpectedlytoanyone.Planningaheadbydesigningaccessiblecoursematerialsnotonlyservesallstudentsandinstructors,italsocreatespossibilitiesforclassescontinuingevenunderconditionsofnaturalorotherdisasters.Infact,therearemanypeoplewhodonotquitemeetthethresholdfor“disability”butwho,nonetheless,facechallenges.

Learningstyles,preferences,andabilitiesalsovarywidelyamongstudentsandinstructors,soawell‐designedcourseshouldhavemultiplemodesofcoursematerialformatsandalternativewaystosubmitwork.

Researchisclear—brainsandbodiesdiffergreatly,andaone‐size‐fits‐allapproachtocoursedesignbenefitsrelativelyfewstudents.If,ontheotherhand,thecourseisdesignedtobeasinclusiveaspossible,allstudents(andinstructors)benefit.

Recommendations 

Provideaccessiblecoursedesignbyensuringthatrequiredcoursematerialscanbeusedwithassistivetechnologysuchasclosed‐captionedvideosandsoundfilesthathavetextscriptsprovided.

Ensurethatfaculty/professionaldevelopmentoffices,inconjunctionwithlibraryandotherrelevantunitsprovideongoing“bestpractices”incoursedesigntrainingandsupportservicestoaddressstudentswhorequireaccommodations.

Welcomestudentswithdisabilities;invitethemtodiscusstheirneedswithadvisors,andprovidelinkstoadditionalrelevantresources.

50 Quality Scorecard for Blended Learning Programs

Page 63: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

8. The blended course is visually appealing to the student and the course is navigationally sound. 

Oftenacourseshellismadeavailabletostudentpriortothefirstface‐to‐facesessionofablendedlearningcourse.Thiscourseshellcanbethefirstimpressionastudentmayhaveofacourseanditisimportantthatthecoursebebothwelcomingandinformative.

Acourseshellthatcontainsprimarilytextcanbeintimidatingtostudents.Therefore,thecourseshellshouldappropriatelyuseaudio,images,andvideoinameaningfulwaythatenhancesthestudentexperience(whileconsideringtheneedsofdiversestudents).Foranexample,whenthecourseinstructorprovidesanintroductionofthemselvestheyshouldincludeeitheravideointroduction(withtranscript)oratext‐basedonewithaphoto.

Astudentshouldnotbeoverwhelmedbytoomanymenuchoices.Menuitemsshouldbebothclearandsimpleandthiscanbeachievedbyreducingthenumberofchoicesonthemainmenu.Themainmenushouldrepresentthekeyareasofthecoursethatastudentmayneedtoaccess.

Thecourseshelldesignshouldalsoinformthestudentsthatthisisacoursethatutilizesboththeface‐to‐faceandonlinelearningenvironments.Indoingthis,theintegrationofthetwoenvironmentsshouldbehighlightedbyexplicitlyidentifyingwhatastudentistodopriortoaface‐to‐facesessionandhowtheface‐to‐facesessionconnectsbacktotheonlineportionofthecourse.Inaddition,thecourseshelldesignshouldgroupcontentbyweekormodulekeepingthecoursecontentandcomponentsgroupedbycontextwithinthecourseinaconsistentmanner.

Recommendations 

Courseshouldutilizeasimilarcourseshelldesignwithintheprogram. Navigationmenurepresentsthehierarchyofthecourse. Coursecontentispresentedincontexttohowitwillbeused. Thecourseshellclearlydistinguisheswhatistobecompletedbythestudentpriorto

comingtoaface‐to‐facesession.

Course Structure 51

Page 64: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforCourseStructure

Gube,J.(October5,2010).Whatisuserexperiencedesign?Overview,tools,andresources.SmashingMagazine.Retrievedfromhttps://www.smashingmagazine.com/2010/10/what‐is‐user‐experience‐design‐overview‐tools‐and‐resources/.

Usability.gov.(2014).Userexperiencebasics.Retrievedfromhttp://www.usability.gov/what‐and‐why/user‐experience.html

UserExperienceProfessionalsAssociation.(2014).Definitionofuserexperiencesandusability.Retrievedfromhttp://uxpa.org/resources/definitions‐user‐experience‐and‐usability

52 Quality Scorecard for Blended Learning Programs

Page 65: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Student‐to‐StudentandFaculty‐to‐Studentinteractionareessentialcharacteristicsandareencouragedandfacilitated.*

2. Instructorfeedbackonstudentassignmentsandquestionsisconstructiveandprovidedinatimelymanner.*

3. Instructorsusespecificstrategiestocreateanengaged,learning‐focusedpresenceinbothmodalitiesofthecourse.

4. Facultyteachthecourseasonecohesivewhole,with"presence"inboththeface‐to‐faceandtheonlineportionsofthecourse.

5. Resourcesareprovidedtoassiststudentsinconductingresearchonlineandassessingthevalidityofonlineresources.*

Teaching and Learning

Page 66: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Student‐to‐Student and Faculty‐to‐Student interaction are essential characteristics and are encouraged and facilitated.*

Bothstudent‐to‐studentandfaculty‐to‐studentinteractionsarerecommendedforaricher,moreengagingblendedenvironment.Infact,Boettcher(2006)recommendedbothasbestpracticesforteaching,inordertoalleviatestudentsfromfeelingaloneorabandonedandtocreateapositivelearningcommunity.Bothtypesofinteractionscanbeaccomplishedwithavarietyofasynchronousandsynchronousactivities.

Faculty‐to‐studentinteractionservestwoprimarypurposes:supportiveandinstructional.Facultyteachingblendedcoursescancreateasupportiveclassroomenvironmentbyensuringthattheyareactiveinthecourseearlyandoften.Inaddition,asupportiveenvironmentmaybecreatedthroughintroductory,welcoming,andcontinuedmessagesandannouncementstoindividualstudentsandthewholeclassthroughoutthecourseterm.Thesemessagescanbedeliveredthroughavarietyofforms,includingtext,socialmedia,audio,in‐person,andvideo.Providingmultipleoptionsforeachinteractionallowsstudentstoselecttheaccessibleand/orthemostpersonallevelofengagement.

Facultycanprovideanengaginginstructionalenvironmentbyprovidinginteractioninclassdiscussions,emails,face‐to‐facemeetings,andthroughdirectionswithintheclasscontentsuchaslectures,PowerPointpresentations,articles,podcasts,etc.Theseinteractionsmayalsobeofamoreindividualnaturesuchasprovidingfeedbackonindividualassignments,providingguidanceonthedevelopmentofapaper,providingfeedbacktostudents’postingsinthreadeddiscussions,etc.Coursecontentmaylenditselftogroupworkinwhichstudentscollaboratetocompleteassignments.Inthesesituations,facultyoversightandinputareoftenrequiredforameaningfuloutcomeforthestudentssuchashelpingtoformthegroups,facilitatingcollaboration,andrequiringindividualparticipation.

Thelevelofstudent‐to‐studentinteractionmaydependonthenatureandoutcomesofthecourse.Careshouldbetakentoensurethattheinteractionsareameanstoeitherprovideasupportive/socialenvironmentorallowstudentstoachievethelearningoutcomes.Facultycanrequirestudent‐to‐studentinteractionthroughthetypesofassignmentsandassessmentsdevelopedforthecourse.Additionally,facultycancreateassignmentsthatrequirepeerreviewandgroupwork.

Modernlearningmanagementsystemsprovideastructureforbothfaculty‐to‐studentandstudent‐to‐studentinteractions.Theseinclude,butarenotlimitedto,threadeddiscussions,chatrooms,grouptoolsandpages,andvideo.Astechnologiesaredeveloped,itisimportantforthefacultymemberstodecideifandhowthetechnologycanbeusedintheircourses.However,choosingtheappropriatetechnologyforaparticularactivityisnotasimportantasthedesignoftheactivityitself.

Recommendations 

Createandimplementpolicyregardingfacultyinteractioninblendedclassessuchasparticipatinginthediscussionboards,offeringofficehours,andprovidingrobustfeedbackonassignments.

Monitorcourseevaluationsforstudentcommentsonlackoffacultyinteraction. Providefacultytipsforhowtoencouragestudent‐to‐studentinteractions. Providefacultybestpracticesforgroupworkandrecommendedactivitiesforblended

teachingandlearning. Providestudentswithrequirementsforinteractionandmonitorstudentinteraction. Providesupportforfacultytoevaluateandadoptnewtechnologiestoenhanceinteraction.

54 Quality Scorecard for Blended Learning Programs

Page 67: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. Instructor feedback on student assignments and questions is constructive and provided in a timely manner.* 

Forthisqualityindicator,thedefinitionoffeedbackistheconstructiveinformationtheinstructorprovidestostudentsregardingcourseassignmentsandactivitiesandshouldincludedirectinstructionsothatstudentscancorrectandimproveunderstanding.Oneoftheprimarywaysfacultyprovideinstructionintheonlineaspectsoftheblendedclassroomisthroughfeedback(Shaw,2014).Facultyplayacriticalroleinthisprocessas“effectivefeedbackmovesstudentsbeyondreflectiononwhattheyhaveaccomplished;itmovesthemforwardbyhelpingthemtoidentifygapsinknowledgeandgoalsandstrategiesforfuturelearning,bothinthecourseandinnon‐courseactivitiesintheirlives”(Getzlaf,Perry,Toffner,Lamarche,&Edward,2009,p.16).Infact,Getzlafetal.(2009)foundthemesforeffectiveinstructorfeedback:

● thereshouldbestudentinvolvementtocreateafeedbackprocesswithindividualizedfeedback;itshouldbegentleguidance;

● itshouldbepositiveandconstructive;● itshouldbetimely;and,● itshouldhelpstudentstoidentifygapsinknowledgeandgoalsandstrategiesfor

futurelearning.Theinstitutioncanplayanimportantpartinthisprocessbyprovidingtoolssuchasfeedbackformsandrubrics,aswellasprofessionaldevelopmentonstrategiesandprocessesforgoodfeedback.Thefeedbackprovidedbytheinstructormustbewithinarelevanttimeframeastoprovideconstructivecorrectionbecauseitiseasyforstudentstobecomefrustratedanddisengagediftheydonotreceivefeedbackintimetoapplyittothenextassignment.

Recommendations 

Encourageorrequireifpossiblethatfacultyprovideconstructivefeedbackoncourseassignmentswithinastatedtimeframe.Forexample:includingthestatementthatallgradedassignmentswillbereturnedwithinoneweekaftersubmissionwillhelpwithstudentexpectationsandmaypreventmultiplestudentemailsaskingwhentoexpectthereturn.

Providefacultyresourcessuchasmodelsofgoodfeedback,feedbackforms,andrubricstohelpformulateconstructivefeedback.

Whentrainingfacultytoteachblendedcourses,emphasizetheimportanceanduseofconstructivefeedback.

Considerprovidingdiversefeedbacktypesoneachgradedassignmenttoencouragecontentmastery.

Teaching and Learning 55

Page 68: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. Instructors use specific strategies to create an engaged, learning‐focused presence in both modalities of the course. 

Thiscriterionrefersspecificallytoinstructorpresenceratherthanacollectivepresenceofinstructorandstudents.Studentsinblendedcoursesmustknowthattheirinstructorispayingattention,scaffolding,challenging,assessing,communicating,monitoring,andenhancingtheirlearningexperience.Theyneedtofeelaconnectiontotheirinstructorandknowthatheorsheisaccessibleandinterestedintheirsuccess.Byusingspecificstrategiestoestablishapresence,instructorscanprovidemotivation,improvelearningoutcomes,avoidmisunderstandingsandfeelingsofisolation,andmaximizestudentsatisfaction.Instructorscanhelpstudentsbecomeeffectiveblendedlearnersaswellascanalsoexperienceamorerewardingteachingexperience.

Withoutrobustinstructorpresenceinablendedcourse,studentsoftenfeelisolated,unsupported,unmotivated,unrecognized,andunsuccessful.Inaddition,theironlywayofknowingiftheyadequatelycomprehendthematerialandarecompletingassignmentsasexpectedisthegradestheyreceive,whenitistoolate.Formativefeedbackfromanactiveinstructorisacriticalsupportforthelearningprocess(Shaw,2014).Thelearningexperienceisenrichedfurtherwheninstructorsparticipateinandhelpguideblendeddiscussionsandcollaborativeprojects.Studentscanbechallengedtothinkmoredeeply,critically,andcreativelyiftheinstructorisobservant,responsive,andengaged.

InastudybyGaytonandMcEwen(2007),bothfacultyandstudentsperceivedthefollowingasimportantstrategiesforensuringinstructionalquality:maintainingopencommunicationwithstudents,requiringstudentstointeractwiththeinstructor(andeachother)tofostergroupcohesiveness,andbuildingastronglearningcommunity.Timelyanddetailedfeedbackregardingthequalityofstudentworkandbuildinggoodrapportandcollaborationwithstudentswerealsoconsideredimportant.

Instructorpresencecanbeaccomplishedinmanyways,andisbestaccomplishedbyusingacombinationofstrategies.Somearebuiltintothecourse,butmanyaredynamicactivitiesthatrequirethattheinstructorbefullyengagedwiththecoursedeliveryandstudentinvolvement.

Recommendations 

Establishawelcomingclassenvironmentwithpersonaltouches,suchasthese:o Sendawelcomemessagebeforethecoursebeginso Postanintroductorytext,audio,orvideomessage,andinvitestudentstodothe

sameo Continuetouseaudio/videoinmessagestoindividualstoconveypresenceo Useaconversational,plainEnglishstyleo Addressstudentsbynameo Besensitivetocultures,genders,religions,nationalities,learnerconfidencelevels,

anddifferingabilitiesinallcommunicationso Calleachstudentduringtheclasstoconveypersonalinterest

Uselearningmanagementsystemfeaturesregularlytoconveyinstructorpresence,interest,andconsiderationtoannounceprogressandevents,topostofficehoursandcontactinformationincludingmethodsandtimes,andtoencourage,motivate,andchallengestudents.

Encourageinstructorstouse(butdon’trequire)socialnetworkingmediatocommunicatewithstudents.

Encouragefacultytoalwaysprovidestudentstimely,individualfeedbackonassignments.

56 Quality Scorecard for Blended Learning Programs

Page 69: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. Faculty teach the course as one cohesive whole, with "presence" in both the face‐to‐face and the online portions of the course. 

Facultypresencemeansmorethanjustprovidingcontenttostudentsinpersonoronline.Facultyneedtobeengagedwithstudentsinbothportionsofthecourseandrespondtostudents’questionsandconcerns.Whenteachingablendedcourse,therecanbeatendencytoutilizetheonlineportiontodelivercontentina“boltedon”componenttoaface‐to‐faceportion;therefore,itisimportantthatthefacultyispresentinthebothportionsofthecourse.

Asenseofcommunityshouldbeestablishedthatutilizesboththeonlineandface‐to‐faceportionsofthecourse.Facultyshouldestablishsocialpresenceinbothportionsoftheblendedcourseinordertocreateasenseofconnectedness.Thedialogshouldflowfromtheface‐to‐faceportiontotheonlineportionandbacktotheface‐to‐faceportion.

Recommendations 

Setasideaportionoftheface‐to‐facesessiontoanswerstudentquestionsandparticipateinadiscussionboardwherestudentscanaskquestions.

Activelyencourageasenseofcommunitywithinthecourse.

Teaching and Learning 57

Page 70: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. Resources are provided to assist students in conducting research online and assessing the validity of online resources.* 

Studentsinblendedlearningcoursesareoftenaskedtoconductresearch,rangingfromastraightforwardundergraduateclassresearchprojecttoathesisordissertationforagraduateprogram.Becausestudentsinblendedcourseshavelessaccesstoaface‐to‐faceinstructor,resourcesmustbeprovidedwiththeonlinecoursematerialsforinformationonthisprocess,includingtheinstructor’sexpectationsregardingresearch.Studentsmustknowhowtofind,organize,evaluate,andciteinformationusedintheirresearch.

Studentsinblendedcoursesmustunderstandthattheyareaccountablefortheinformationtheyuseincompletingcourseassignments.Instructorsmustemphasizethatthestandardsfortheonlinecomponentofthecourseareasrigorousasthosefortheface‐to‐faceportion.Maintainingconsistencyandintegrityinresearchexpectationswillhelpmaintaincourseandprogramintegrity.Itwillalsopreparestudentstoeffectivelyuseonlineinformationbeyondthescopeofthecourse.Intheonlinecoursematerials,instructorscanembedorlinktoadequate,clearinstructiononhowtoconducteffectiveresearchoftheprofessionalliteraturethroughwebsearches,andlibraryholdingsordatabases.Onceinformationhasbeenlocated,studentsmustalsounderstandhowtoevaluatethatinformation.Withtheeaseofonlinesearchesandtheplethoraofinformationavailable,studentsmustlearntobecriticalconsumersofinformationandrecognizethecriteriaforreliablesources.Theymustknowhowtodiscernthedifferencebetweencredibleinformationfrompeer‐reviewedarticlesandbiasedinformationfromsubjectivesourcessuchasblogs,wikis,andcommercialsites(Naufel,Briley,Harackiewicz,Johnson,Marzec,&Nielsen,2010).

Recommendations 

Stateexpectationsandresearchrequirements,clearlyandindetail,inthecoursesyllabusandonlinelearningmanagementsystem.

Providestudentsinformation:o OnhowtouseBooleansearchtermstofindabstracts,o Informationoncitation,andfull‐textarticlesinlibrarydatabaseso AboutthevariouslibrarydatabasesrelatedtothecoursetopicsaboutGoogle

Scholar(orothers)asausefulsourcesofinformationo Aboutthemostreliableandcrediblejournals(bydiscipline)o OnhowtocorrectlyciteInternet‐basedsources,includingtheDOIo Onhowtoevaluatetheinformationfoundonlinebystudentsforsource,accuracy,

credibility,focus,biasandrelevanceo Aboutmethodsandethicsofresearch,includingpropercitationand

acknowledgementofsources,plagiarism,andacademicintegrity. Considerusingaformtohelpstudentsevaluateonlinesources. Createanonlinetutorialonresearchtechniquesusingarecordedpresentationincluding

voiceandvideoifpossible,sothatstudentscanaccessandrevisitasneeded.

58 Quality Scorecard for Blended Learning Programs

Page 71: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforTeachingandLearning

Boettcher,J.(2006).Tenbestpracticesforteachingonlinequickguidefornewonlinefaculty.Retrievedfromhttp://www.designingforlearning.info/services/writing/ecoach/tenbest.html

Getzlaf,B.,Perry,B.,Toffner,G.,Lamarche,K.,&Edward,M.(2009).Effectiveinstructorfeedback:Perceptionsofonlinegraduatestudents.TheJournalofEducatorsOnline,6(2).Retrievedfromhttp://www.thejeo.com/Archives/Volume6Number2/GetzlafetalPaper.pdf

Naufel,K.Z.,Briley,K.E.,Harackiewicz,A.S.,Johnson,K.P.,Marzec,K.P.,&Nielsen,M.E.(2010).Howdopsychologystudentsuseweb‐basedinformation:Trendsandimplicationsfromadescriptivestudy.NorthAmericanJournalofPsychology,12(1),1‐14.

Shaw,M.(2014).Teachingthroughfeedback.Retrievedfromhttp://olc.onlinelearningconsortium.org/effective_practices/teaching‐through‐feedback

Teaching and Learning 59

Page 72: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Technicalassistanceisprovidedforfacultybeforeandduringblendedcoursedevelopmentandteaching.*

2. Theinstitutionensuresfacultyreceivetraining,assistance,andsupporttopreparefacultyforcoursedevelopmentandeffectiveteachingwithtechnologyinavarietyofmodalities.*

3. FacultyreceivetrainingandmaterialsrelatedtoFairUse,plagiarism,andotherrelevantlegalandethicalconcepts.*

4. Facultyareprovidedon‐goingprofessionaldevelopmentrelatedtoblendedteachingandlearning.

5. Clearstandardsareestablishedforfacultyengagementandexpectationsaroundblendedteaching(e.g.responsetime,contactinformation,etc.).

6. Facultyareprovidedtraininginblendedteaching.

Faculty Support

Page 73: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Technical assistance is provided for faculty before and during blended course development and teaching.*

Asblendedlearningrequiresfacultytoadoptvariouslearningtechnologies,itismoreimportantthaneverthatinstitutionsprovidefacultywithtechnicalassistanceandongoingsupportforintegratingtechnologyintoboththeface‐to‐faceandonlineclassroom.Studentsexpecttoutilizethesetechnologiestointeractandcollaborateinbothoftheselearningenvironments.

Institutionsarechallengednotonlytoprovideanadequateleveloftechnicalsupportandassistanceforfaculty,butalsoappropriatesupportandassistance.Institutionsmustthinkstrategicallyabouthowtostaff,promote,structure,anddeliverthetechnicalassistancefacultyneedforteachinginablendedlearningcourse.Often,institutionsassociate“technical”with“informationtechnology,”andthetechnicalassistanceforcoursedevelopmentandblendedcoursesishousedwithinITdepartments.Technicalsupportshouldbeprovidedforthelearningmanagementsystem,webpagedevelopment,socialmediatools,audioandvideorecordings,graphicdevelopment,classroomwifi,classroomtechnologies,andnewemergingtechnologies.Supportmustalsobecontextual,pedagogy‐based,andeasilyaccessible,andfitwithintheoverallorganizationalstructuretomaximizeefficienciesandresources.Thesupportshouldcontinuethroughthetimeofteachingblendedcourses,andnotsolelyfocusoncoursedevelopment.

Recommendations 

Embedtechnicalsupportpracticesforbothface‐to‐faceandonlineeducationwithintheinstitutionalmissionandculturetoencouragefacultyacceptance.Supportshouldbeallocatedtoassistfacultyinblendedcoursedevelopmentsotheywillbecomemorefamiliarwithonlineeducationanditsroleinthecontextoftheirowndepartment.

Carefullyconsiderwheretolocatetechnicalsupportservices.Thinkaboutexistingorganizationalstructuresanddecidewhethertocentralizeordecentralizeservices.Keepinmindaccessibility,approachability,consistency,maintainingeducationalcontext,andensuringexpertiseisavailable.

Technicalsupportwillbeneeded,attimes,forfacultyandtheirhomecomputers.Providearesourceinorderforfacultytobeabletotroubleshoothomecomputerswiththeonlineteachingenvironment.

Technicalsupportwillbeneeded,attimes,intheface‐to‐faceclassroom.Providearesourceforfacultytobeablerequestclassroomsupportduringaclasssothatthesessionisnotdisrupted.Thisisimportantduetothereducednumberofclasssessions.

Faculty Support 61

Page 74: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. The institution ensures faculty receive training, assistance, and support to prepare faculty for course development and effective teaching with technology in a variety of modalities.* 

Whenaninstitutionplacesvalueonblendedlearningbyprovidingpoliciesandsupportmechanismsthatsupportsfaculty,facultysatisfactionwithblendedlearningwillincrease.Thisincludesprovidingaccesstoeducationaltechnologies,instructionalteams,andguidanceastohowfacultycancommunicatethecourseformattostudents.

Theeducationaltechnologiesselectedbytheinstitutionshouldbewidelyadoptedwithconsiderationtohowitwillenhanceblendedlearning.Theselectionprocessofthesetechnologiesneedstoconsidertheprocessofteachingandlearning,aswellasfacultyandstudentusage.Asufficientstaffneedstobeavailabletosupportandmaintainthesetechnologies.

Aninstructionalteamofinstructionaldesigners,instructionaltechnologists,mediadevelopers,andotherspecialistsshouldjoinwiththefacultytodevelopblendedlearningcourses.By“unbundling”thevariousfunctionsofthefacultyanddistributingtheserolestotheinstructionalteam,thecoursecanbedevelopedandtaughtmoreefficientlyandeffectively.

Thissupportmayincludeinformationforfacultytocommunicatetostudentsabouttheblendedlearningformatandtipsforsuccess.

Theseresourcesneedtobeprovidedviadifferentmodalitiesinordertosupportbothpart‐andfull‐timefaculty.Thismayincludehelpguides,in‐personconsultation,andthroughonlinetechnologies.

Recommendations 

Providecomprehensivecoursedevelopmentsupport,training,andassistanceforblendedlearninginstructors.Thefacultydevelopmentopportunitiesshouldbefrequentandongoing.

Examinedifferentmodelsofdelivery(virtualmodules,handouts,livetraining,self‐paced,etc.)andprovideavarietyofresourcesforprofessionaldevelopment.

Adoptavarietyoffaultytrainingandsupportstrategies,e.g.facultymentoringprograms,web‐basedknowledgebases,collaborativewikis,trainingworkshops,onlinesupportmaterials,andothers.

62 Quality Scorecard for Blended Learning Programs

Page 75: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. Faculty receive training and materials related to Fair Use, plagiarism, and other relevant legal and ethical concepts.* 

Theblendedenvironmentpresentseducatorswithconsiderablelegalandethicalchallenges.Understandingofissuesrelatedtoownership,privacy,identity,andcopyrightlawarecritical.CopyrightlawandFairUseareessentialconceptsforallfacultymemberstounderstand.Facultygenerallyapproachtheseissuesfromtwodistinctperspectives:asconsumerandcreator.Theirconcernsandperceptionsaboutcopyrightlawmaydifferdependingonthescenariobeingconsidered.Onecommoncopyrightissuerelatedtoblendedlearningisfacultyownershipofmaterialcreatedforblendedcourses.Eachinstitutionmustbeproactiveinsettingacoreintellectualpropertypolicy,andprovidingguidelinesforcompliance.

Fairuseisperhapstheleastunderstoodaspectofblendedcoursedevelopmentandteachingamongfaculty.Inarecentstudy,Sweeney(2006)foundthatfacultymembersweregenerallyunclearaboutcopyrightlawandwereuncertainofwhereandhowtolocateuniversitypolicies.Inaddition,facultynotedthattheywereconcernedabouttheircoursecontent,butcontinuedtousematerialforwhichtheydidnotseekcopyrightrelease.Theconsequencesofcopyrightinfringementcanbedireforanindividualandaninstitution,andcanincludedisciplinaryaction,termination,legalfees,andanassortmentofcriminalpenalties.Institutionsmustsupportfacultywiththeneededinformationandtrainingaboutcopyrightlawinordertoavoidabreach.Inadditiontotheriskofinfringement,alackofknowledgeaboutFairUsecanresultinfacultyavoidanceofincorporationofdiversecontentintoablendedcourse.

Anothermajorethicalissueforfacultyisplagiarism.Facultymustunderstandthenatureofplagiarismintheblendedenvironment,andlearntorecognizeitinordertodeterfrustratingandchallengingincidents.Facultymustunderstandtheirroleisnotto“catch”students;rather,facultyshouldhavetheknowledgeandtoolstoeducatestudentsaboutacademicintegrity,ethicalissues,andappropriateconduct.

Recommendations 

Createchecklists,tipsheets,examples,workshops,andchunkedinformationtomakecopyrightcomplianceeasierforfacultytoapply.Seeforexample,CopyrightComplianceMadeSimple–SixRulesforCourseDesign(Enghagen,2001).

Ensurefacultyreceivecopyrightinformationwithinrelevantcontexts.Forexample,copyrightinformationinrelationtocoursedevelopmentshouldbeincludedinblendedcoursedevelopmentsupportmaterials.

Ensurethatcopyrightlawdoesnotbecomeonedepartment’sresponsibility,butrather,universalandintegratedthroughouttheinstitution.

Providefacultywithworkshops,discussions,roundtables,andotheropportunitiestocollaborativelydiscussethicalissuesinblendededucation.

Faculty Support 63

Page 76: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. Faculty are provided on‐going professional development related to blended teaching and learning. 

Blendededucationhasevolvedquicklyoverthepastdecade.Thefrequencywithwhichnewtechnologiesbecomeavailableisstaggering.Inaddition,asmoreandmoreresearchisconductedonelectronicallymediatededucationalpracticeandoutcomes,furtherevidenceemergesthatcompelsustochangeoureducationalmethodsandpedagogicalapproaches.

Anongoingprogramofprofessionaldevelopmentisanecessarypartofsupportingqualityinblendededucation.Stayingcurrentwiththebodyofresearchandtheavailabilityofnewtechnologiesiscriticaltosuccessintheblendedenvironment.Inaddition,focusonadditionalareasforfacultydevelopmentcanenhanceaninstructor’seffectiveness,suchasgenerationalstudies,learningtheory,timemanagement,andretentionstrategies.Professionaldevelopmentcanalsohelptoavoidinstructorburnout,andgiveinstructorsimportanttimeforreflection,learning,andpersonaldevelopment.

Thetraditionaldefinitionofprofessionaldevelopmentencompassesactivitiessuchassabbaticals,grantfunding,conferenceattendance,andactivitiesdirectlyrelatedtodevelopmentasaprofessionalorresearcher.However,amorestudent‐centereddefinitionisemerging,andthisisespeciallytrueofprofessionaldevelopmentforblendedfaculty.Professionaldevelopmentisnotsimplyaseriesofworkshopsorclassestohoneskills,thoughthosearecertainlyapartofprofessionaldevelopment.Rather,professionaldevelopmentisaholisticprocessthatshouldincludefaculty’slong‐termgoalsforimprovingteachingandlearning.

Itisamyththatprofessionaldevelopmentforfacultyiscost‐prohibitive.Althoughprofessionaldevelopmentrequiressomeinstitutionalinvestmentforstaff,materials,andprograms,theessenceofprofessionaldevelopmentthriveswithinacultureofcontinuousimprovement,studentlearning,andqualityfocus.

Recommendations 

Defineandcommunicatetheimportanceofprofessionaldevelopmentforfaculty. Approachprofessionaldevelopmentasaprocessofindividualgoalsettingorself‐

improvementwithafocusonimprovingteachingforimprovinglearning. Ensurethatstaffselectedtoleadprofessionaldevelopmenteffortshavebuy‐inandsupport

fromfacultyandthatallinitiativesanddepartmentsworkcollaborativelyontheprograms. Proactivelyengagefacultybyensuringtheirawarenessofdevelopmentopportunities

throughvariouscommunicationchannels. Provideflexibleanddiverseprofessionaldevelopmentstrategiestomeetthedemandsof

thediversefacultypopulation.Engageadjunctfacultyintheteachingcultureoftheinstitution.

64 Quality Scorecard for Blended Learning Programs

Page 77: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. Clear standards are established for faculty engagement and expectations around blended teaching (e.g. response time, contact information, etc.). 

Inanageofaccountability,theestablishmentandimplementationofinstructionalstandardsprovidesinstitutionsawaytomeasureperformanceandeffectstudentlearningoutcomes.However,apotentialpitfallofstandardsisthattheysometimesinhibitinnovationandcreativityandbecomecodifiedasrules.

Institutionsshouldestablishstandardsthataredirectlyandpositivelytiedtostudentlearningoutcomes.Theonlywaytotrulyknowwhethercertaineducationalmethodscontributepositivelytostudentoutcomesisthroughthecollectionandanalysisofdata.Thus,individualswhoaredevelopingstandardsshouldaskthemselveswhattheywanttoaccomplishthrougheachstandardanduseevidencetojustifythestandard.

Acultureofqualityisbuiltonsharedvaluesaboutgoodpractice.Often,institutionsspendtoolittletimebuildingtheculture,andtoomuchtimecodifyingandenforcingstandards.Intheblendedenvironment,thisisespeciallytrue,asoftenthereisafocusonmonitoringfacultyperformanceforadherencetoresponsetimeandengagementindicators.

Oncestandardsareestablishedandunderstood,theremustbeaprocessandstaffinplacetocommunicatestandardsandenforcethem.Blendedfacultymustbemadeawareoftheperformanceexpectationsandstandardsofteachingperformance.Thecommunicationmustbeclear,consistent,andprecise.

Recommendations 

Documentstandardsforfacultyexpectationsandalwaysincludetheminteachingcontractsandfacultypolicyhandbooks.Supportstandardsbyincludinginformalmessagesaboutspecificstandardsinadiverseselectionofmaterials(websites,collateralmaterial,meetingagendas,andsoforth).

Ensurethatappropriatepersonnelandsystemsareinplacetocommunicatestandardsandmonitorfacultyperformance.Inculcatethebenefitsofassessmentandperformanceevaluationintheinstitutionalculturesothatmethodsareembracedratherthancondemned.

Createandimplementblendedfacultycertificationcoursesthatimparttheessentialcomponentsofblendedteachingstandards,andprovidethenecessarytoolsandskillstoachievequalityblendedteaching.

Createaconsistentperformancereviewprocesswithdetaileddocumentation.Providefacultywithperformancefeedback,inwriting,aswellasspecificinformationonhowtorectifyanynotedissues.

Establishmeasuresforstandardsthatcomefromvarioussourcessothatperformanceassessmentscanbeimproved.Forexample,measurefacultyengagementinaclassbyadministrativeobservation,peerreview,studentreports,andloginactivity.

Assesscompetenceintheblendedclassroomthroughcombinedmeasures,includingwrittenskills,technicalskills,pedagogicalawareness,compliancewithadministrativedutiesandmore.

Faculty Support 65

Page 78: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. Faculty are provided training in blended teaching. 

Facultyneedtobepreparedtopurposefullyintegratetheonlineandface‐to‐faceportionsofablendedcourse.Thistrainingprogramshouldallowforfacultytoimmediatelytransfertheirnewknowledgetotheirpracticeandshouldincludetopicssuchascoursedesign,teachingstrategies,formativeandsummativeassessments,educationaltechnology,andtheintegrationofthesetopicsintooneanother.

Modelsforprovidingtrainingtofacultyincludeprogramsthataredeliveredface‐to‐face,online,orblended.Theseprogramsmaybeofferedasan“intensive”oneortwo‐weekprogramormaybespreadoutovermultipleweeks.Anotheroptionmayincludeprovidingmentoringtofacultynewtoblendedlearning.Theseprogramsshouldbeavailabletobothfull‐andpart‐timefaculty.

Recommendations 

Provideafacultydevelopmentprogramtopreparefacultytoteachinablendedlearningenvironment.

Facultydevelopmentprogramshouldincludetopicssuchascoursedesign,teachingstrategies,formativeandsummativeassessments,andeducationaltechnologyasrelatedtoblendedlearning.

 

66 Quality Scorecard for Blended Learning Programs

Page 79: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforFacultySupport

Enghagen,L.J.(2001).Copyrightcompliancemadesimple–sixrulesforcoursedesign.Needham,MA:TheOnlineLearningConsortium.

Sweeney,P.C.(2006,Spring).Faculty,copyrightlaw,andonlinecoursematerials.OnlineJournalofDistanceLearningAdministration,9(1).Retrievedfromhttp://www.westga.edu/%7Edistance/ojdla/spring91/sweeney91.htm

Faculty Support 67

Page 80: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Beforestartingablendedlearningprogram,studentscompleteanorientationorself‐assessmenttodetermineiftheypossesstheself‐motivationandcommitmenttolearn.*

2. Beforestartingablendedlearningprogram,studentsareadvisedabouttheprogramtodetermineiftheyhaveaccesstotheminimumtechnologyskillsandequipmentrequiredbythecoursedesign.*

3. Beforestartingablendedlearningprogram,studentsreceive(orhaveaccessto)informationaboutprograms,includingadmissionrequirements,tuitionandfees,booksandsupplies,technicalandproctoringrequirements,andstudentsupportservices.*

4. Throughoutthedurationofthecourse/program,studentshaveaccesstotrainingandinformationtheywillneedtosecurerequiredmaterialsthroughelectronicdatabases,interlibraryloans,governmentarchives,newsservices,andothersources.*

5. Throughoutthedurationofthecourse/program,studentshaveaccesstoappropriatetechnicalassistanceandtechnicalsupportstaff.*

6. Supportpersonnelareavailable(24/7)toaddressstudentquestionsandproblemsofatechnicalnature.*

7. Policy,processes,andresourcesareinplacetosupportstudentswithdisabilities.

8. Studentshaveaccesstoinformationregardingrequiredcoursematerialsinprintand/ordigitalformat,suchasISBNnumbersfortextbooks,booksuppliers,anddeliverymodespriortocourseenrollment.

9. Programdemonstratesastudent‐centeredfocusandintentionalityintheintegrationofonlineandface‐to‐faceresources.

10. Theinstitutionprovidesguidance/tutorialsforstudentsintheuseofallformsoftechnologiesusedforcoursedelivery.

11. Studentsareprovidedclearinformationforenlistinghelpfromtheinstitution.

Student Support

Page 81: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Before starting a blended learning program, students complete an orientation or self‐assessment to determine if they possess the self‐motivation and commitment to learn.*

Asonlineandblendedlearningmatures,studentsuccesshasbeenlinkedtocertainmotivationsandskillsets.Alem,Plaisent,Bernard,&Chitu(2014)foundinasystematicreviewthate‐learningreadinesswasrepresentedthroughattributessuchas:computerinternetself‐efficacy,self‐direction,motivation,interactionandattitude.Tosupportstudentretention,institutionsshouldtakeactiontoensurestudentsarebothcapableandequippedtosucceedinablendedlearningenvironment.AlthoughstudentsmaybefamiliarwithtechnologythatGarrisonandVaughan(2008)suggestprovidestheessentiallinkageinblendedlearning,andareabletofind,createandshareinformationviatheweb,TangandChaw(2015)foundthatdigitalliteracythatfacilitatesaccessing,understandingandapplyinginformationobtainedfromthewebisalsoimportant.Studentsshouldbeawarethat:

● Theyareenrolledinablendedlearningcoursethatfacilitateslearninginboththeonlineandfacetofacelearningenvironments

● Whileblendedcoursesgivemoreflexibilityinscheduling,theycanrequiremoreself‐disciplinethanatraditionalfacetofaceclass.

● Theywillhavetheopportunitytointeractwiththecourseinstructorinboththeonlineandfacetofacelearningenvironment.

● Onlinelearningmayrequirethestudenttoworkfromwrittendirectionsforcourseassignments.

● Itmaytakeaslittleasafewhoursorasmuchasseveraldaystogetfeedbackfrominstructors.

● Printmaterialsmaytheprimarysourceofdirectionsandinformationfortheonlineweeks.

● Studentsshouldfeelcomfortabletocontacttheinstructorforhelpassoonastheyneedhelpforthecourse.

Recommendations 

Encouragestudentstoassesstheirmotivationandcommitmenttoblendedandonlinelearning(seebelow;MooreandShelton,2013).

o StudentOnlineReadinessTool(availablefromUniversitySystemofGeorgia)throughCreativeCommonslicenseandassesses:Technologyexperience,accesstotools,studyhabits,mylifestyle,goalsandpurposes,andlearningexperiences.

o SmarterMeasure™LearningReadinessIndicator(2016)isacommercialtoolthatassesses:Individualattributes,lifefactors,learningstyles,technicalcompetency,technicalknowledge,on‐screenreadingrateandrecall,andtypingspeedandaccuracy.

Developprocessestomeasurestudentsuccessandmotivation(i.e.,usinga“droppedstudent”surveytodeterminecommonalities).

Student Support 69

Page 82: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. Before starting a blended learning program, students are advised about the program to determine if they have access to the minimum technology skills and equipment required by the course design.* 

Onlinestudentsrelyontechnologytointeractwithinstructors,classmatesandinstitutionalsupportstaffinanonlineandblendedlearningenvironment.Therefore,itisimportanttoadvisestudentsoftheneedforbasiccomputerandtechnologyskillsandequipmentbeforetakingblendedclasses.Thisadvicecancomefromtheadmissionsoffice,viatheinstitutionalwebsiteandmarketingmedia.Manycollegeshaveimplementedonlineself‐assessmentactivitiesthatallowstudentstoascertainiftheypossessthetechnicalskillsneededtosucceedinanonlinecourse,asdiscussedinthepreviousindicator.Studentsuccessinablendedlearningenvironmentnecessitatesstudentshavethenecessaryknowledgeandabilitytouseinformationandcommunicationtechnology(Poon,2013),aswellasaccesstoanadequatecomputersystem.Toassessifastudenthasaccesstoadequatecomputersystem,institutionsneedtodisseminatetechnologyrequirements.s(Simonson,Smaldino,Albright,&Zvacek,2009,p.202).

Simonsonetal.(2009)remindsusthatthatthiscommunicationshouldstartbeforestudentsbeginthecourse.Technologyrequirementsshouldbeclearlydisplayedandarticulated,beforethestudentregistersforthecourse,includingrequirementsforbrowsers,browsersettingsandbrowseradd‐ons.Technologyrequirementsshouldalsoincludeanyspecialequipmentthatisspecifictothecourserequirements,suchascameras,microphones,CD‐ROM,DVDdrives,specialsoftware,ortheneedtodownloadadditionalsoftwareprograms.Allspecificonlinecoursetechnologyrequirementswillneedtobeincludedinthesourcesyllabuscourseschedule,andotherappropriateplaceswithinthecoursesiteaswellasbeeasilyaccessibleonthewebsitesoexpectationsareclearbeforeastudentsignsupforacourse.

Recommendations 

Provideinformationontheprogramwebsite,throughemail,socialmedia,etc.concerningallrequiredonlinetechnologiesfortheblendedcoursebeforethecoursebegins.

Provideweblinkstohelpstudentslocatetheresourcestheyneed(i.e.,bookstore,vendors,websites,software,downloads).

Providealistoftroubleshootingtipsandfrequentlyaskedquestionsforstudentstoaccessonline.

Provideinformationontechnicalskillsandstandardsspecifictotheinstitution.Forexample,“Studentsshouldhavebasicskillsusingwordprocessingsoftware.”Disseminatetheinformationtostudentsthroughseveralchannelsofcommunication.

Addressspecifictechnologiesrequiredbytheinstitutionsuchasemailononlinelearningmanagementsystems.

Provideademoinvideooratutorialthatexplainsthetechnologiesneededforsuccess.

70 Quality Scorecard for Blended Learning Programs

Page 83: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. Before starting a blended learning program, students receive (or have access to) information about programs, including admission requirements, tuition and fees, books and supplies, technical and proctoring requirements, and student support services.* 

Studentsneedaccesstorelevantinformationregardingblendedprogramofferingsbeforemakingadecisiontoapplytoadegreeprogramorenrollinablendedcourse.Communicationofavailablesupportservicesandnecessaryinformationisanimportantelementineffectiveprogramadministration.

Themoreoptionsthestudenthastoaccessrelevantinformationandsupportservices,thebetter.Sincemanystudentswishtoconductbusinesswiththeinstitutionduringnon‐standardhours,providinganswerstofrequentlyaskedquestionsthatcanbeaccessedanytimehasprovenhelpfulforstudents.Furthermore,eachquestionclearlyansweredmayreducethenumberofresponsesneededfromsupportstafforinstructors.Studentservicessupportstaffandinstructorsshouldbeawareofalllocations(physicalandonline)todirectstudentsforanswerstoanyadditionalquestions.

Recommendations 

Makeinformationrelatedtoin‐classandonlineattendanceexpectationsforblendedcoursesandprogramseasilyaccessibletoprospectiveandenrolledstudents.

Provideonlineinformationrelatedtoprogramadmission,enrollment,costandfinancialaid,books,policies,andstudentsupportservices.Keepthewebsiteupdated,organized,andeasilyaccessible.

Considerthefollowingquestionswhendevelopingtheinformationalwebsite:o Whatrequirementsmustbesatisfiedforstudentstogainadmittancetoacourse?o Howshouldstudentscommunicate(phone,email,fax,etc.)withtheschoolto

acquireinformation/support?o Whatproceduresshouldstudentsfollowtoaccessservices,suchasthelibrary

(identification,login,URL)?o Howdostudentscommunicatewithadvisorsandotherfacultymembers?o Whatkindsofbarriersmightstudentsencounterinsuccessfullycompletingtheir

programandhowwilltheinstitutionremovethesebarriers?

Student Support 71

Page 84: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. Throughout the duration of the course/program, students have access to training and information they will need to secure required materials through electronic databases, interlibrary loans, government archives, news services, and other sources.* 

Learningtouseonlinelibrarydatabasesforthefirsttimewithoutexperthelpcanbeintimidating.Andevenifstudentssuccessfullylocateappropriateresources,findingfull‐textversionsofjournalarticlesorarrangingfore‐documentdeliverycanoftenbeachallenge.Libraryprofessionalsshouldprovidetrainingandeducationalopportunitiesforstudentsandfaculty,andmaywanttoviewteachinginformationliteracyaspartoftheirjob.Instructorsshouldmakeuseoftheexperthelpthatlibraryprofessionalscanprovidesothattheycansupportandencourageuseoflibraryresources.

Recommendations 

Provideaccesstoinformationonaccessinglibrarydatabases(i.e.,alinktolibraryresourcesandlibrarystaffcontactinformation).

Provideservicesthathelpstudentslocaterelevantinformationsuchasaself‐pacedtutorialorlibraryorientationmodule.

Createanonlinelibraryresearcharticlescavengerhuntforyourstudentsasaclassorsubjectorientationassignment.

Designatealibrarianforeachdisciplineareaandprovidestudentswiththedesignatedlibrarian’scontactinformation,sostudentshaveaprimarypointofcontactforquestionsandtrouble‐shooting.

72 Quality Scorecard for Blended Learning Programs

Page 85: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. Throughout the duration of the course/program, students have access to appropriate technical assistance and technical support staff.* 

Studentsmayencounterdifficultyworkingwithrequiredtechnologyinablendedcourseorprogram,andmayneedsupporttosuccessfullyfunctionusingsuchtechnology.However,theystillmustbeabletocompletecourselearningactivitiesandassignmentswithoutunduerelatedstress.

Bothstudentsandinstructorsworkinginablendedteachingandlearningenvironmentwillneedtechnicalsupportfromtimetotime.Ifastudentisunabletoresolvetechnicalproblemsinthecourseofstudy,frustrationandisolationwilloccur,which,inturn,mayimpactstudentretentionandlearning(Shelton&Saltsman,2005).

Facultyintroducinganewtechnologyorapplicationinablendedcoursemusttakeresponsibilityforworkingwithcampusinformationtechnologyservicestoensureappropriatedocumentationandtrainingisavailabletosupportthestudentsworkingwiththattechnologyorapplication.

However,“instructorscannotberesponsibleforotherthancursorytrainingoftheirstudentsintheuseoftechnology”(Miller&King,2003,p.292).Therefore,theinstitutionshouldprovideonlinesupportforthestudents.Ideally,thesupportshouldbe24x7whencombiningself‐helpandaccesstoapersonforsupport.Asmuchinformationaspossibleshouldbeavailableinaself‐helpformatsostudentscanaccesshelpatanytime.However,theremustbeaprocessforcontactingsupportapersonifadditionalhelpisneeded.Someinstitutionsprovideadirecttelephonenumber(oftentoll‐free);othershaveanonlineticketingsystemorchatroom.Thesupportshouldincludeeverythingfromhowtoresetapasswordtowhattodoifastudentgetslockedoutofanonlineexamination.

Recommendations 

Provideaccessibleinformationonhowtoaccesstechnicalsupportonthewebsiteand/ortheLMS,atollfreephonenumber,and/orlivechat.

Providetechnicalsupportservicesascloseto24hours,7daysaweekaspossible. Clearlyspecifywhenvarioussupportservicesareavailableiftheyarenotoffered24x7. Createachannelthroughwhichfacultycanrequestthedevelopmentanddeliveryofnew

documentationandtraininginsupportofblendedcourseintegratedtechnologies. Createatechnicalsupportrequestformthatisweb‐basedwithdetailsaboutexpected

turnaroundtime,evening/weekendavailabilityandtelephoneand/orchatavailabilityfor“live”help.

Student Support 73

Page 86: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. Support personnel are available (24/7) to address student questions and problems of a technical nature.* 

Studentfrustrationwithtechnologyinlearningenvironmentsneedstobeavoided,andresearchsupportsthatfrustrationwithtechnologyhasbeencitedasakeyfactorinstudentfailure(Moore&Kearsley,1996).Inablendedteachingandlearningenvironment,asupportsystemneedstobeinplaceforaddressingtechnologyrelatedquestionsoutsideoftheclassroom.Inthelearningmanagementsystemorstudentportal,itshouldbeclearwhomtocontactifastudentforgetshis/herpassword,iftheLMSisnotworkingproperly,ifaninstructorisnotrespondingtothestudents,orifthecomplaintisnotgettingaddressedinatimelymanner.Students“needtounderstandwheretheresponsibilityoftheinstitutionliesregardingreliability.Policiesneedtobearticulatedsothatallpartiesareclearaboutresponsibilities”(Simonsonetal.,2009,p.201).

Recommendations 

Providesupportstafftoaddressspecificquestionsaboutaccessingcourseandprogramcomponents.

Presentaclearprocessforstudentstoreportproblemsandmakecomplaints,preferablyonline.

Createasearchableonlineknowledgebasethataddressescurrentissuesthatmightaffectstudentsusingblendedlearningtechnologiesandapplications.

74 Quality Scorecard for Blended Learning Programs

Page 87: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

7. Policy, processes, and resources are in place to support students with disabilities. 

Althoughstudentswithdisabilitieshavealegalandfundamentalrighttobeabletoenrollinallclassesandprograms,theywillneedawaytoprocureassistancefromtheinstitutionwhennecessary.Theinstitutionshoulddesignatesupportpersonnel(oftenintheareaofstudentaffairs)whodocumentstudents’needs,provideguidanceinallocatingresources,andhelpdeterminehowbesttomeetthoseneeds.

Theinstitutionmusttakeusabilityandaccessibilityoftechnologiesandapplicationsintoconsiderationwhenofferingoptionsandsolutionstofacultycreatinganddeliveringblendedteachingandlearningcomponents.

Recommendations 

Putinplaceprocessthatdeterminesthetypesofservicesthatshouldbeavailablefordisabledstudentsinteractingwithblendedlearningcomponents.

Putinplacepolicywhichsupportsstudentswithdisabilitiesacrossallmodesofdelivery,includingblendedteachingandlearning.

Assurethattheprocessfordocumentingadisabilityisaccessibletoalllearners. Providededicatedsupportstaffforservingtheneedsofdisabledstudents. Assurethatfacultyandstaffareawareofstudentsupportservicesandunderstandthebest

methodsforreferral.

Student Support 75

Page 88: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

8. Students have access to information regarding required course materials in print and/or digital format, such as ISBN numbers for textbooks, book suppliers, and delivery modes prior to course enrollment. 

Informationonthematerialsneededforeachcourseshouldbemadeavailabletostudentsatthetimeofregistrationorbeforesothatstudentscan“shop”forthecoursematerials.

TobeincompliancewithfederalHigherEducationOpportunityActof2008(HEOA)regulations:

Tothemaximumextentpracticable,aninstitutionmustincludeonitsInternetcoursescheduleforrequiredandrecommendedtextbooksandsupplementalmaterialtheInternationalStandardBookNumber(ISBN)andretailprice;iftheISBNisnotavailable,theauthor,title,publisher,andcopyrightdate;orifsuchdisclosureisnotpracticable,thedesignation“ToBeDetermined.”

Ifapplicable,theinstitutionmustincludeonitswrittencoursescheduleareferencetothetextbookinformationavailableonitsInternetscheduleandtheInternetaddressforthatschedule.

Institutionsdisclosingtheinformationtobeincludedontheircourseschedulesforrequiredandrecommendedtextbooks,andsupplementalmaterialareencouragedtoprovideinformationon:

• rentingtextbooks;• purchasingusedtextbooks;• textbookbuy‐backprograms;and• alternativecontentdeliveryformatssuchasebooks.

TheHEOAalsorequirestheGovernmentAccountabilityOffice(GAO)tostudytheimplementationofthissectionandreporttoCongress.

Recommendations 

Developandcontinuouslyupdateonlineandprintmaterialsthatprovideinformationoncourseschedulesandallrelevanttextbooksandsupplementarymaterials.

Providestudentsandfacultywiththelocationofthecampusbookstore,includingdirections,hoursofoperation,andcontactinformation.

Provideeasyaccesstoonlinebookstoresthatstudentscansearchforthebestprices.(Manyinstitutionswillnotwanttodothissincetheyareoftenbusinesspartnerswiththecampusbookstore.Inthatcase,providelinkstothebooksonthecampusbookstorewebsite.)

76 Quality Scorecard for Blended Learning Programs

Page 89: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

9. Program demonstrates a student‐centered focus and intentionality in the integration of online and face‐to‐face resources. 

Blendedteachingandlearningapproachesaresteadilyevolvingalongsideexistingandemergingtechnologies.Toensureastudent‐centeredfocus,systems,services,resources,andinstructionmustbeintegratedtocreateaseamlessexperience,removingallobstaclestofocusthelearner'sattentiononknowledgetobeacquiredandapplied.

Theinstitutionshouldprovideadvertising,recruiting,andadmissionsinformationtostudentsthatadequatelyandaccuratelyrepresentstheprograms,requirements,andservicesavailable.Thereshouldbeasetofwrittenpolicies,procedures,andguidelinespertainingtodegree‐relatedcoursesandprogramswhichincludesinformationonadmissions,curriculum,requirementsforcompletionofthecoursesorprograms,costsandpaymentpolicies,financialaid,andanyotherpertinentinformation.

Institutionsshouldprovideinformationaboutthenatureofblendedofferings,detailingexpectationsrelatedtobothface‐to‐faceandblendedcomponents,includingthenatureoffaculty/studentinteraction,assumptionsaboutstudents’technologicalcompetenceandskills,technicalequipmentandsoftwarerequirements,andavailabilityofacademicandstudentsupportservices.

Recommendations 

Provideequalaccesstostudentservicesappropriatetosupporttheonlineandface‐to‐facecomponentsofblendedcoursesprograms,includingfinancialaid,academicadvising,registration,accesstocoursematerials,proctoringrequirements,placementandcounseling,andcareersupport.

Committothecontinuationofaprogramorcertificateforitspublicizedtimeframetoallowalladmittedstudentstocomplete.

Ensurethatstudentservicepersonnelcanaccuratelyandquicklyanswerallstudentquestionsandhaveastructuredsystemforaddressingstudentcomplaints.

Provideprospectivestudentswitharealisticpreviewoftheblendedlearningexperiencethroughanorientationorcoursedemonstration.

Providetime‐managementresourcesforstudentstoeffectivelymanagethebalanceofonlineandface‐to‐faceworkloadwithintheblendedlearningframework.

Student Support 77

Page 90: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

10. The institution provides guidance/tutorials for students in the use of all forms of technologies used for course delivery. 

Blendedteachinglearningtakesadvantageoftechnologytoenhancetheclassroomexperience.Asstudentslearninablendedenvironment,theymayneedtrainingandguidancetomasterthetechnologiesusedforblendedcomponentdelivery.

Theinstitution'slearningmanagementsystemmaybeonecomponentoftheblendedcourseorprogram.“Thecomplexityofamodern[learning]managementsystem(LMS)andtherelatedinfrastructurecanbedaunting,yetprovidingastableandfunctionallearningenvironmentisabsolutelynecessary”(Shelton&Saltsman,2005,p.117).

AttheheartoftheLMSareclassroomadministrativefunctionslikestudentauthentication,discussionthreads,chat,gradebooks,andin‐classmessagingandannouncements.Allofthesystemsavailabletodaycontainthesecorecapabilities.However,additionalcontentdeliverymethods,interactioncomponents,andassessmentcapabilitiesmayalsobeutilizedaspartofablendedcourseorprogram.ThesetechnologiesmayresidewithintheLMS,orbeaccessiblefromtheopenWebormobiledevices.

Studentsmustbecomfortablewithusingthesetechnologiessothattheyarenotatalldistractedfromthelearningprocess.

Recommendations 

Provideaccesstodocumentationandtrainingforalltechnologieswithinthescopeofblendedcourseandprogramofferings.

Developanddistributeminimumtechnologyrequirementsforstudentsinblendedcoursesorprograms.

Provideopportunitiestocrowdsourceknowledgebyhavingstudentstocontributetotrainingmaterialsrelatedtoblendedcourseandprogramtechnologies.

Enablefacultytocreatetheirowninstructionalvideosinsupportoftechnologiesusedinblendedcourses.

78 Quality Scorecard for Blended Learning Programs

Page 91: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

11.  Students are provided clear information for enlisting help from the institution. 

Forinstitutionstobesuccessfulandcompetitive,theymustprovideafullrangeofacademicandadministrativesupportservices,alongwithclearcommunicationtostudentsonhowtoaccessthoseservices.

LinkstovariousacademicandadministrativesupportservicesoftheinstitutioncanbeplacedthroughouttheLMS,oninstitutionalwebpages,andadministrativeportals.Allrelevantcontactinformationshouldbelisted,includinghoursofoperation.Carefulattentionmustbemadetoensurethattheinformationlistedthroughoutthesevenuesisaccurateandup‐to‐date.

Recommendations 

Providelinksincourseinstructions,schoolwebpages,andportalstocleardescriptionsofavailablesupportandtutorials/resourcesthatanswerbasicquestionsconcerningresearch,writing,technology,etc.

Remindstudentsviaemailorlearningmanagementsystemannouncementsthatthereisadistinctprocessforstudentstousetocontactadministrativeandacademicsupportservices.

Developaninstitution‐widestrategyforkeepingallpublishedadministrativeandacademicsupportserviceinformationup‐to‐dateandrelevant.

 

Student Support 79

Page 92: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforStudentSupport

Alem,F.,Plaisent,M.,Bernard,P.,&Chitu,O.(2014).StudentOnlineReadinessAssessmentTools:ASystematicreviewapproach.TheElectronicJournalofE‐Learning,12(4),375‐382.Retrievedfromhttps://www.ejel.org

Garrison,D.R.,Vaughan,N.(2008).BlendedLearninginHigherEducation:Framework,Principles,andGuidelines.Jossey‐Bass

H.R.4137‐‐110thCongress.(2008).HigherEducationOpportunityAct.Retrievedfromhttp://www.govtrack.us/congress/bills/110/hr4137

Miller,T.,&King,F.(2003).Distanceeducation:Pedagogyandbestpracticesinthenewmillennium.InternationalJournalofLeadershipinEducation,6(3),283‐‐‐297.

Moore,J.,&Shelton,K.(2013).Socialandstudentengagementandsupport:TheSloan‐cQualityScorecardfortheadministrationofonlineprograms.TheJournalofAsynchronousNetworks,17(1),53‐72.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ1011372.pdf

Moore,M.,&Kearsley,G.(1996).Distanceeducation:Asystemsview.Belmont,CA:WadsworthPublishing.

Poon,J.(2013).Blendedlearning:Aninstitutionalapproachforenhancingstudents’learningexperience.MERLOTJournalofOnlineLearningandTeaching,9(2).Retrievedfromhttp://jolt.merlot.org/vol9no2/poon_0613.htm

Shelton,K.,&Saltsman,G.(2005).Anadministrator’sguidetoonlineeducation.Greenwich,CT:InformationAgePublishing.

Simonson,M.,Smaldino,S.,Albright,M.,&Zvacek,S.(2009).Teachingandlearningatadistance:Foundationsofdistanceeducation(4thed.).Boston,MA:Pearson.

SmarterMeasure™(2016).Retrievedfromhttp://readi.info/about/

Tang,C.&Chaw,L.(2015).Digitalliteracyandeffectivelearninginblendedlearning.EuropeanConferenceone‐Learning:601‐XVIII.KidmoreEnd:AcademicConferencesInternationalLimited.Retrievedfromhttp://search.proquest.com/openview/

5e52b0f4a3bbbfcc4179c9f323e0a18e/1?pq‐origsite=gscholar

UniversitySystemofGeorgia.(2010).StudentOnlineReadinessTools.Retrievedfromhttp://ci.ed1stop.net/sort/html/tool.html

80 Quality Scorecard for Blended Learning Programs

Page 93: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. Theprogramisassessedthroughanevaluationprocessthatappliesspecificestablishedstandards(i.e.,accreditorguidelinesand/orotherrecognizedagencysuchastheOLCScorecard).*

2. Avarietyofdata(academicandadministrativeinformation)areusedtoregularlyandfrequentlyevaluateprogrameffectivenessinordertoguidechangestowardcontinualimprovement.*

3. Intendedlearningoutcomesatthecourseandprogramlevelarereviewedregularlytoensurealignment,clarity,utility,appropriateness,andeffectiveness.*

4. Aprocessisinplaceandfollowedforthecomprehensiveassessmentofsupportservicesforfacultyandstudents.

5. Aprocessisinplaceandfollowedfortheassessmentofstudentretentioninblendedcoursesandprograms.

6. Programdemonstratescomplianceandreviewofaccessibilitystandards(Section508,etc.).

7. Courseevaluationscollectstudentfeedbackontheeffectivenessofinstructioninrelationtofacultyperformanceevaluations.

8. Courseevaluationscollectstudentfeedbackonqualityofblendedcoursematerials.

9. Aprocessisinplaceandfollowedfortheinstitutionalassessmentoffacultyblendedteachingperformance.

10. Aprocessisinplaceandfollowedfortheassessmentofstakeholder(e.g.,learners,faculty,staff)satisfactionwiththeblendedlearningprograms.

Evaluation and Assessment

Page 94: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

1. The program is assessed through an evaluation process that applies specific established standards (i.e., accreditor guidelines and/or other recognized agency such as the OLC Scorecard).*

Theblendedprogramevaluationmaybedefinedas“themeansbywhichtheinstitutionsorproviderssettheirprogramgoalsandmeasureresultsagainstthosegoals”(CouncilforHigherEducationAccreditation,1998,p.1).Acommonperceptionhasbeenthatablendededucationprogramis“oftenpoorlydesignedand/orunderfunded;itismoreofanafterthoughtratherthananintegralpartofplanningandimplementation”(Thompson&Irele,2007,p.419).Butasblendededucationcontinuestogrow,asstudentsareattractedtothepromiseofconvenienceandflexibility,andaswelearnmoreaboutthepedagogicalandlearningadvantagesofblendededucation,blendedprogramsmustbeabletodemonstratethattheyareguidedbyandaccountabletothehigheststandards.Demonstratingevidenceofaccountabilitymeansimplementing“evaluationactivitiesthatassessalignmentofpedagogy,educationalactivities,anddesiredlearningoutcomes,plusaddressspecificissuesofusabilityandbenchmarkachievement,[which]providevaluableinformationforcontinualimprovement”(Balanko,2002,p.7).Thesesuggestedactivitiesforevaluationareagoodstartingpointforanyblendededucationprogram.Inaddition,anyregionalordisciplinespecificaccreditinggroupwillhavestandardsthatwillneedtobeadheredtoandcansupportanestablishedevaluationprocess.

Recommendations 

Useanevaluationprocessthatisalignedtoestablishedstandardstoassessblendedprograms,anduseevaluationresultstoimproveprograms.Theevaluationshouldbecompletedwithmultipleassessmentsandavarietyofdatacollectedandanalyzed.

Documentevidenceofprogramevaluationandimprovementtouseforsupport.Thisshouldbedoneatleastannually.

Useapplicableaccreditingstandardsanddemonstrateadherencetothesestandardsforblendedprogramwithartifactsanddataanalysis.Forexample,usingtheinstitution’sregionalaccreditorstandards,showcomplianceforthosethatapplytotheblendededucationprogramandhowprogramimprovementsaredevelopedandimplementedafterdatacollectionandanalysis.

82 Quality Scorecard for Blended Learning Programs

Page 95: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

2. A variety of data (academic and administrative information) are used to regularly and frequently evaluate program effectiveness in order to guide changes toward continual improvement.* 

Asingleevaluationprocess(studentcoursesurveys,forexample)cannoteffectivelyorfullyevaluateablendededucationprogram.Qualityprogramevaluationwillrequiremultipleprocesses,instruments,evidenceandanalysis.Inaddition,inputsandoutputsshouldbeperiodicallyreviewed.Infact,“effectiveprogramevaluationisthebestwaytoachievecontinuousprogramimprovement”(Chapman,2006,para.33).

Aholistic,360‐degreeapproachtoprogramevaluationthatsupportscontinuousimprovementincludesdatacollectionandevidenceofanalysisinthefollowingareas:

•Perceptionofdepartmentcommitmenttotheprogram;

•Effectivenessoftheadministrator/coordinator;

•Extentandqualityofsupportservices;

•Qualityoftechnicalsupport;

•Perspectiveonstudentlearning;

•Qualityofstudentsandcomparisonwithtraditionalcampusstudents;

•Rigorofcoursesandprograms;

•Effectiveuseoffacultytime;

•Personalandprofessionalsatisfactionwithinvolvementintheprogram(Lesht,Montague,Page,Shaik,&Smith,2006,pp.98‐99).

Otherdatamaybeused;however,thesearetheprimaryareasthatshouldbefocusedupon.

Recommendations 

Usemorethanasinglereviewprocesstoevaluatequality.Thefollowingareasshouldbecombinedforuseintheevaluationprocess:

o Studentsatisfactionandstudentcomplaints(programandsupportservices);o Studentsuccess/persistence/withdrawal/dropratesincludingfeedbackreceived;o Courseevaluations;o Facultysatisfaction;o Blendedcoursematerialsandmethodsreviews;o Learningoutcomesachievement(atthecourseandprogramlevel);o Evidenceofinstitutionalsupportforblendededucation.

Establishgoalsforeachoftheevaluationprocessesanddevelopimprovementplanswhenresultsfallshortofintendedgoals.Recognizeandsharepositiveresultswithallthoseinvolved.

Engageineffortstodetermineandcomparesuccessratesandqualitystandardswithotherprogramsoflikesizewithsimilarinstitutions.

Evaluation and Assessment 83

Page 96: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

3. Intended learning outcomes at the course and program level are reviewed regularly to ensure alignment, clarity, utility, appropriateness, and effectiveness.* 

Learningoutcomesshouldbethefoundationforbothcoursesandprograms.Alearningoutcomeis“ageneralstatementofwhatlearnerswillbeabletodoaftertheinstructioniscompleted”(Davidson‐Shivers&Rasmussen,2006,p.84).Likewise,learningoutcomesshoulddescribeexplicitstudentbehavior.Theyshouldbecurrent,relevant,appropriateforsupportingtheprogramgoals,andclearlystated—thesecharacteristicshelpwithdevelopingstudentassessmentactivities.Learningoutcomesshouldbereviewedforrelevanceandcourseactivitiesandassessmentsshouldbeupdatedaccordingly.

Effectiveassessmentdependsonstandardsandclearobjectives(Holtz&Radner,2006).Becauseclarityissoimportanttostudentunderstanding,thereshouldbeaperiodicrecurrentreviewtodetermineiflearningoutcomesareclearlydefinedandstillapplicableandrelevanttothelearninggoals.

Recommendations 

Mapcourselearningoutcomestodemonstratehowtheysupportacademicprogramgoalsandobjectives.

Updatelearningoutcomesduringregularcurriculumreviewtoensuretheyareclear,current,andrelevant.

Regularlyreviewlearningoutcomesinblendedcoursesandprograms(includepeerreviewintheprocess)andensurelearningoutcomesineachcoursesupportoverallprogramgoals,objectives,andexpectedstudentproficiencies.

Demonstratealearningoutcomereviewprocesswithdocumentedchangesornotesofreviewprocess.

84 Quality Scorecard for Blended Learning Programs

Page 97: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

4. A process is in place and followed for the comprehensive assessment of support services for faculty and students. 

Evaluatingthequalityofablendededucationprogramshouldincludeanexaminationofallsupportservicesforbothstudentandfaculty.Thereviewprocessshouldincludeannualgoalsforimprovedservicesandsatisfactionlevels.Inaddition,theinstitutionshoulddemonstrateacommitmenttoaprocessforcontinuousimprovement.Forexample,agoalforthetechnologysupportservicescouldbetooffer24x7supporttostudentswith90%ofthosestudentsbelievingtheirneedsweremetafterthefirstphonecalloremail.Ifonly70%ofstudentsreportedthattheirneedsweremet,thenastrategyshouldbedevelopedtoimprovetheservicebeingprovidedtostudents.Allresultsofsurveysandimprovementstrategiesshouldbedocumentedforevidence.Supportserviceswouldincludelibraryservices,tutoring,bookstore,counseling,advising,blendedstudentorientations,financialaid,andcashierservices.Facultysupportserviceswouldincludetechnicalsupport,coursedevelopmentsupport,professionaldevelopmentactivities,andongoingsupportduringtheteachingprocess.

Recommendations 

Surveyfacultyteachingblendedcoursesandannuallyassessallfacultysupportservicesforteachingandcreatingblendedcoursematerials,suchascoursedevelopmentsupport,training,andongoingteachingsupportforpossibleimprovementandfacultysatisfaction.Establishagoalforfacultysatisfaction.Ifthegoalisnotmet,developastrategyforimprovement.

Attheleast,annuallysurveyblendedstudentstoreviewstudentsatisfactionwithsupportservicestodeterminehowwellstudents’needsaremet,andusedatafromthereviewprocessforcontinuousimprovements.Forexample,findoutfromstudentsiftheyusethecampusbookstorefortextbookpurchasesandhowtheyfeelabouttheserviceprovided.

Evaluation and Assessment 85

Page 98: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

5. A process is in place and followed for the assessment of student retention in blended courses and programs. 

Programretentioncanbedefinedas“thenumberoflearnersorstudentswhoprogressfromonepartofaneducationalprogramtothenext.Inhighereducation,studentretentionisnormallymeasuredasenrollmentfromacademicyeartoacademicyear”(Martinez,2003,p.3).Retentionmaybemeasuredbyblendedcoursecompletionrateswhichshouldalsobecloselymonitored;however,retentioninvolvesreturningbackeachsemesterandbeingsuccessfullyenrolledincourses.

Whileretentionratesdovaryfromprogramtoprogram,therearemanyprogramsthatexperiencenormalorevenbetterretentionrates.Programleadersandinstructorscantakeproactivestepstoincreasestudentretention.

Itisclearthatdatamustbecollectedfromavarietyofareastoformulateastrategyforimprovingretention.Studentswhodropoutoftheprogramshouldbesurveyedandfeedbackcollectedontheirreasonsfornotfinishingacourseorprogram.

Recommendations 

Examineanddocumentstudentretentionincoursesandprogramsfromsemestertosemesterandyeartoyear.Comparetheresultingdatatothefeedbackcollectedonpriorcourseevaluations.

Develop,implement,andassessstrategiestocontinuouslyimprovestudentretention. Providefacultyandadvisorswithstrategiesandtechniquesforimprovingstudent

retention. Surveystudentsforfeedbackregardingwhytheydroppedcoursesandreviewfeedbackfor

possibleimprovementinprocesses.

86 Quality Scorecard for Blended Learning Programs

Page 99: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

6. Program demonstrates compliance and review of accessibility standards (Section 508, etc.). 

Studentswithdisabilitieshavealegalandfundamentalrighttoaccesshighereducation,whichincludesblendedcoursesanddegreeprograms.IntheUnitedStates,legislationsuchastheAmericanswithDisabilitiesAct(ADA)of1990,Section504oftheRehabilitationAct(1973),andthe1998AmendmenttoSection508oftheRehabilitationActprovidefoundationrequirementsformakingblendededucationaccessibletoallstudents.However,inspiteoftechnologicalimprovementsinblendededucation,therearestillreportsthatstudentswithdisabilitiesarenotverywellserved(Kinash&Crichton,2007).Yet,thenumberofstudentswithdisabilitiesthatpursueapostsecondaryeducationcontinuestoincrease(Simoncelli&Hinson,2010).Becauseoflegislation,students’rights,andtheincreasingnumberofdisabledstudentsenrollinginblendedprograms,institutionsmustbeabletodemonstratecompliancewithaccessibilitystandards.Complianceincludesprovidingstudentsupportservicesanddevelopingwebsiteandblendedcoursematerialsthatareaccessiblebyadaptivetechnologiessuchasscreenreadersandothertechnologiesforassistingstudents.

Recommendations 

Provideclearinformationonthewebsitetostudentswithdisabilitieswhoareinterestedinorenrolledinblendedclassesandlinkstospecialsupportpagesintheblendedcoursematerials.

DesignatesupportpersonnelthatareknowledgeableofADAcomplianceissuesinblendededucationprograms.

Providecontinuoustrainingforallthoseinvolvedwithservingstudentswithdisabilities. Publishcompliancewithaccessibilitystandardsontheblendededucationprogram’s

website. Applyuniversaldesignstandardsinblendedcoursedevelopmentandcoursematerials.

Evaluation and Assessment 87

Page 100: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

7. Course evaluations collect student feedback on the effectiveness of instruction in relation to faculty performance evaluations. 

Whilefacultyshouldneverbeevaluatedsolelyondatacollectedfromcourseevaluations,thefeedbackfromstudentsshouldbecarefullyexaminedbecauseitmayprovidespecificinformationthatcouldimprovepolicydevelopmentandpointtopotentialfacultybehaviorsthatwillneedcorrectionoradditionaltraining.Courseevaluationresultsmayalsosuggestlevelsofinstructoractivity,courseengagement,andtimelinessofresponsetostudents’questions.Infact,“byusingevaluationstoguaranteerapidturnaroundtimes,providecustomquestionsthatfocusoncurrentorevenplannedactivities,andaddresstheinnovativeteachingstrategiesandindividualcharacteristicsthatdistinguishdifferentcourses,facultyandstudentscancometoreflectmoreeasilyuponthoseuniqueaspectsofinstructionthatmattertothem”(Anderson,Brown,&Spaeth,2006,para.20).

Recommendations 

Determineifcurrentcourseevaluationsforblendedcoursescollectstudentfeedbackoninstructorperformanceintheblendedclassroom.Ifnot,modifytheinstrumentsinordertocollectfeedbackspecifictoinstructorperformance.

Carefullyreviewblendedcourseevaluationresultsasapartoftheoverallperformanceevaluationprocessforfacultyteachingblended.Lookfortrendsorsimilarcommentsaboutindividualinstructors’blendedteachingperformance.

Baseduponthecourseevaluationresults,gooverthefeedbackwiththefacultymemberanddevelopaplantominimizesimilarnegativefeedbackthenexttimethecourseistaught.

88 Quality Scorecard for Blended Learning Programs

Page 101: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

8. Course evaluations collect student feedback on quality of blended course materials. 

Evaluationis“theformaldeterminationofthequality,effectivenessorvalueofaprogram,product,project,process,objective,orcurriculum”(Worthen&Sanders,1987,p.23).Blendedcourseevaluationshelpdeterminetheeffectivenessandvalueofacourseaswellasprovidefeedbackregardinginstructorperformance.Thus,Cavanaugh(2002)recommended,“studentsshouldbeaskedtoexplaintheirsatisfactionwiththeirexperience,includinglikesanddislikes”(p.184).Thedatacollectedfromcourseevaluations,ifreviewed,analyzed,andreflectedupon,supportsbetterawarenessoftheinstructionalpractice(Huba&Freed,2000).

Recommendations 

Gatherinformationbeyondhowmuchthestudentlikedthecourse.Useevaluationstocollectinformationaboutthespecificblendedcoursematerialsandtextbooks,aswellashowtheinstructorperformedasthementorandfacilitatorofthecourse.

Periodicallyreviewthequestionsinthecourseevaluationsforrelevanceandeffectiveness. Inblendedcourseevaluations,askstudentshowtheyfeelabouttheeffectivenessofthe

blendedinstruction. Collectstudentfeedbackabouttechnologysupportingthecoursedeliveryandcourse

activities. Considerthefollowingquestionsforevaluations(Achtemeier,Morris,&Finnegan,2003,p.

8):o Werethecoursegoals,learningobjectivesandoutcomesmadecleartoyouatthe

beginningofthecourse?o Didyouhavethenecessarytechnologicalequipmentandskillsrequiredforthis

course?o Wasthereadequatetechnicalsupportifyouencountereddifficulties?o Wastheformatandpagedesignofthecourseeasytouse?o Weretheresufficientinstructionsgivenforyoutocompleteallassignments?o Didyoufeelhinderedinyourcourseexperienceinanyway?Pleasedescribe.o Werestandardsforevaluationofassignmentsmadeclear?o Didyoureceivepromptfeedbackonyourcompletedassignments?o Whatlearningactivitiesmostinfluencedyourlearninginthiscourse?

Evaluation and Assessment 89

Page 102: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

9. A process is in place and followed for the institutional assessment of faculty blended teaching performance. 

Thesuccessofablendededucationprogramcanoftendependlargelyuponfacultyinvolvedastheycandirectlyaffectstudentretention,studentsatisfaction,andstudentsuccess.Infact,Tinto(1999)foundinthetraditionalclassroom,thatthefrequencyandqualityoffacultyandstudentcontactisdirectlyrelatedtostudentpersistence.Thisalsoappliestotheblendedclassroom.Becauseoftheimpactfacultycanhaveontheblendedclassroomandstudents,facultyperformanceshouldbeassessedandtheresultsshouldbeprovidedtothefacultymemberaftereachcoursetheyteach.Thisshouldincludeavarietyofmeasurements,includingstudentevaluationsanddirectperformancereviewintheblendedclassroom.

Recommendations 

Createpoliciestoguideexpectationsforfacultyperformanceinblendedclasses. Assessfacultyperformanceeachtimetheyteachablendedcourse.Thismaymeanadean

orprogramdirectorwillneedtoaccessthelearningmanagementsystemtoverifyinstructoractivity.

Provideperformancestandardsforfaculty.Developanimprovementplanandfollow‐upevaluationforthosewhodonotmeettheestablishedstandards.

90 Quality Scorecard for Blended Learning Programs

Page 103: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

10. A process is in place and followed for the assessment of stakeholder (e.g., learners, faculty, staff) satisfaction with the blended learning programs. 

Everyeducationprogramexiststomeettheneedsofexternalconstituentssuchasstudentsandfutureemployersandtoadvancesociety'sneedsoverall.Blendededucationprogramsuseinternalconstituenciessuchasadministrators,technologists,faculty,andstafftoachievethegoalsdefinedfortheprogram.Eachofthesegroupsofconstituentsisastakeholderoftheblendedprogram.

Establishingaprocessthatcollectsfeedbackfromstakeholdersregardingtheeffectivenessoftheblendedprogramgathersthedataneededtoestablishaprogramofcontinuousimprovement.Thesolicitationoffeedbackfrommultipleperspectivesallowstheprogramtoimplementmulti‐raterfeedback,or360‐degreeevaluation.Ideally,thisevaluationshouldincludeavarietyofmeasurementsandcovermostorallservicesofferedbytheprogram,inadditiontotheoverallsatisfaction.Oncefeedbackiscollected,correctiveactioncanbetakentoensuretheprogramcontinuestobeeffectiveinservingstakeholdersneeds.

Recommendations 

Establisharegulartimeframeinwhichtoapplyquestionnaires/surveystodiscoverthestudents’andallexternalclients’levelofsatisfactionwiththeblendedprogram.Forexample,everyfallandspring,surveyblendedstudentstodeterminetheirsatisfactionlevelwiththeprogram.

Solicitinformalfeedbackfromallstakeholders.

Evaluation and Assessment 91

Page 104: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

ReferencesforEvaluationandAssessment

Achtemeier,S.D.,Morris,L.V.,&Finnegan,C.L.(2003).Considerationsfordevelopingevaluationsofonlinecourses.JournalofAsynchronousLearningNetworks,7(1),1‐13.Retrievedfromonlinelearningconsortium.org/sites/default/files/v7n1_achtemeier_1.pdf

Anderson,J.,Brown,G.,&Spaeth,S.(2006).Onlinestudentevaluationsandresponseratesreconsidered.Innovate,2(6).Retrievedfromnsuworks.nova.edu/cgi/viewcontent.cgi?article=1124&context=innovate

Balanko,S.L.(2002).Reviewandresources:Onlineeducationimplementationandevaluation(Report02‐11).Seattle,WA:UniversityofWashingtonOfficeofEducationalAssessment.Retrievedfromhttp://www.washington.edu/oea/pdfs/reports/OEAReport0211.pdf

Cavanaugh,C.(2002).Distanceeducationquality:Successfactorsforresources,practicesandresults.InR.Discenza,C.D.Howard,&K.Schenk(Eds.),Thedesign&managementofeffectivedistancelearningprograms(pp.171‐189).Hershey,PA:InformationScience.

Chapman,D.D.(2006).Buildinganevaluationplanforfullyonlinedegreeprograms.OnlineJournalofDistanceLearningAdministration,9(1).Retrievedfromhttp://www.westga.edu/~distance/ojdla/spring91/chapman91.htm

CouncilforHigherEducationAccreditation.(1998).Assuringqualityindistancelearning:Apreliminaryview.WashingtonDC:Author.Retrievedfromhttp://www.chea.org/pdf/HED_Apr1998.pdf

Davidson‐Shivers,G.V.,&Rasmussen,K.L.(2006).Web‐basedlearning:Design,implementation,andevaluation.UpperSaddleRiver,NJ:PearsonMerrillPrenticeHall.

Diaz,D.,&Cartnal,R.(2006).Termlengthasanindicatorofattritioninonlinelearning.Innovate:JournalofOnlineEducation,2(6).

Holtz,J.K.,Radner,B.(2006).Assessmentandcollegeprogress:Capacitybuilding.InD.D.Williams,S.L.Howell,&M.Hricko(Eds.),Onlineassessment,measurement,andevaluation:Emergingpractices,(pp.275‐291).Hershey,PA:InformationScience.

Huba,M.E.,&Freed,J.E.(2000).Learner‐Centeredassessmentoncollegecampuses:Shiftingthefocusfromteachingtolearning.Boston,MA:Allyn&Bacon.

Kinash,S.,&Crichton,S.(2007).Supportingthedisabledstudent.InM.G.Moore(Ed.),Handbookofdistanceeducation,(2nded.,pp.193‐204).Mahwah,NJ:LawrenceErlbaumAssociates

Lesht,F.L.,Montague,R.A.,Page,V.J.,Shaik,N.,&Smith,L.C.(2006).Onlineprogramassessment:AcasestudyoftheUniversityofIllinoisatUrbana‐Champaignexperience.InD.D.Williams,S.L.Howell&M.Hricko(Eds.),Onlineassessment,measurementandevaluation:Emergingpractices(pp.92‐108).Hershey,PA:InformationScience.

Lynch,M.M.(2001).Effectivestudentpreparationforonlinelearning.TheTechnologySourceArchivesattheUniversityofNorthCarolina.Nov‐Dec.Retrievedfromhttp://technologysource.org/article/effective_student_preparation_for_online_learning/

Martinez,M.(2003).Highattritionratesine‐Learning:Challenges,predictors,andsolutions.TheeLearningDeveloper'sJournal,17(11),1‐9.Retrievedfromhttp://www.elearningguild.com

Rovai,A.P.(2003).Insearchofhigherpersistenceratesindistanceeducationblendedprograms.TheInternetandHigherEducation,6(1),1‐16.doi:10.1016/S1096‐7516(02)00158‐6

92 Quality Scorecard for Blended Learning Programs

Page 105: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Simoncelli,A.,&Hinson,J.(2010).Designingonlineinstructionforpostsecondarystudentswithlearningdisabilities.JournalofEducationalMultimediaandHypermedia,19(2),211‐220.Chesapeake,VA:AACE.Retrievedfromhttp://www.editlib.org/p/33205

Thompson,M.M.,&Irele,M.E.(2007).Evaluatingdistanceeducationprograms.InM.G.Moore(Ed.),Handbookofdistanceeducation(2nded.,pp.419‐432).Mahwah,NJ:LawrenceErlbaum&Associates.

Tinto,V.(1999,Fall).Takingstudentretentionseriously:Rethinkingthefirstyearofcollege.NationalAcademicAdvisingAssociationJournal,19(2),5‐9.

Willging,P.,&Johnson,S.D.(2009).Factorsthatinfluencestudents’decisiontodropoutofcourses.JournalofAsynchronousLearning,13(3).Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ862360.pdf

Worthen,B.,&Sanders,J.R.(1987).Educationalevaluation:Alternativeapproachesandpracticalguidelines.WhitePlains,NY:Longman.

Evaluation and Assessment 93

Page 106: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Appendix 

Page 107: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

Blen

ded

Qua

lity

Scor

ecar

d Ru

bric

Page 108: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

he in

stitu

tion

has

had

no

disc

ussi

ons

abou

t the

ble

nded

go

vern

ance

str

uctu

re.

Gov

erna

nce,

res

pons

ibili

ties,

de

cisi

on m

akin

g au

thor

ity a

nd

orga

niza

tion

of b

lend

ed o

pera

tions

is

haph

azar

d an

d in

hibi

ting

syst

emat

ic

and

cont

inuo

us im

prov

emen

t

Gov

erna

nce,

res

pons

ibili

ties,

de

cisi

on m

akin

g au

thor

ity a

nd

orga

niza

tion

of b

lend

ed o

pera

tions

is

som

ewha

t am

orph

ous

with

mul

tiple

un

its h

avin

g sp

ecifi

c re

spon

sibi

litie

s co

mpl

icat

ing

syst

emat

ic a

nd

cont

inuo

us im

prov

emen

t

Gov

erna

nce,

res

pons

ibili

ties,

de

cisi

on m

akin

g au

thor

ity a

nd

orga

niza

tion

of b

lend

ed o

pera

tions

is

delib

erat

e an

d cl

ear;

line

s of

aut

horit

y fo

r su

ppor

ting

units

are

del

inea

ted

(e.g

., in

a s

hare

d se

rvic

e ag

reem

ent)

.

2T

he in

stitu

tion

has

had

no

disc

ussi

ons

abou

t the

ble

nded

go

vern

ance

str

uctu

re a

nd d

ecis

ion

mak

ing

Gov

erna

nce,

res

pons

ibili

ties,

de

cisi

on m

akin

g au

thor

ity a

nd

orga

niza

tion

of b

lend

ed o

pera

tions

is

haph

azar

d an

d it

is n

ot a

lway

s cl

ear

whi

ch u

nit i

s ta

king

the

lead

Gov

erna

nce,

res

pons

ibili

ties,

de

cisi

on m

akin

g au

thor

ity a

n or

gani

zatio

n of

ble

nded

ope

ratio

ns is

so

mew

hat a

mor

phou

s w

ith m

ultip

le

units

hav

ing

spec

ific

resp

onsi

bilit

ies.

Gov

erna

nce,

res

pons

ibili

ties,

de

cisi

on m

akin

g au

thor

ity a

nd

orga

niza

tion

of b

lend

ed o

pera

tions

is

delib

erat

e an

d cl

ear;

line

s of

aut

horit

y fo

r su

ppor

ting

units

are

del

inea

ted

(e.g

., in

a s

hare

d se

rvic

e ag

reem

ent)

.

3N

o st

rate

gic

plan

exi

sts.

The

ble

nded

pro

gram

is in

the

proc

ess

of d

evel

opin

g its

ow

n st

rate

gic

plan

tied

to th

e in

stitu

tiona

l pl

an.

The

ble

nded

pro

gram

has

its

own

stra

tegi

c pl

an ti

ed to

the

inst

itutio

nal

plan

and

acc

redi

tatio

n st

anda

rds;

the

plan

is p

erio

dica

lly r

evie

wed

to

asce

rtai

n its

con

tinui

ng r

elev

ance

.

The

ble

nded

pro

gram

has

its

own

stra

tegi

c pl

an ti

ed to

the

inst

itutio

nal

plan

and

acc

redi

tatio

n st

anda

rds;

the

plan

is p

erio

dica

lly r

evie

wed

to

asce

rtai

n its

con

tinui

ng r

elev

ance

; ba

sed

on th

e re

view

, it i

s im

prov

ed

and

upda

ted.

4T

he in

stitu

tion

is e

ngag

ing

in

expl

orat

ory

conv

ersa

tions

abo

ut th

e po

tent

ial o

f ble

nded

lear

ning

.

The

inst

itutio

n is

wor

king

to b

uild

its

capa

bilit

ies

(e.g

., le

ader

ship

, bu

dget

ary

supp

ort,

supp

ort s

ervi

ces)

in

ord

er to

rea

lize

the

stra

tegi

c va

lue

of b

lend

ed le

arni

ng.

The

inst

itutio

n es

tabl

ishe

s se

nior

le

ader

ship

pos

ition

s to

ste

war

d th

e bl

ende

d in

itiat

ive

with

cle

ar b

udge

tary

su

ppor

t gui

delin

es; t

here

is a

co

mm

itmen

t to

deve

lop

the

nece

ssar

y se

rvic

es to

sup

port

st

uden

ts a

nd fa

culty

mem

bers

; but

bl

ende

d le

arni

ng h

as y

et to

bec

ome

a cl

ear

part

of t

he in

stitu

tion'

s m

issi

on/v

isio

n.

Ble

nded

lear

ning

is p

art o

f the

in

stitu

tion'

s m

issi

on/v

isio

n; th

e in

stitu

tion

has

esta

blis

hed

seni

or

lead

ersh

ip p

ositi

ons

to s

tew

ard

the

wor

k w

ith c

lear

bud

geta

ry s

uppo

rt

guid

elin

es; t

here

is a

com

mitm

ent t

o de

velo

p th

e ne

cess

ary

serv

ices

to

supp

ort s

tude

nts

and

facu

lty

mem

bers

.

5T

he in

stitu

tion

is e

ngag

ing

in

expl

orat

ory

conv

ersa

tions

abo

ut

blen

ded

lear

ning

.

The

inst

itutio

n is

in th

e be

ginn

ing

stag

es o

f rea

lizin

g ho

w a

ble

nded

le

arni

ng p

rogr

am s

uppo

rts

the

inst

itutio

n's

mis

sion

, val

ues,

and

st

rate

gic

plan

.

The

ble

nded

lear

ning

pro

gram

is

clea

rly ti

ed to

the

inst

itutio

n's

mis

sion

, val

ues,

and

str

ateg

ic p

lan.

The

ble

nded

lear

ning

pro

gram

is

posi

tione

d w

ithin

the

orga

niza

tiona

l st

ruct

ure

to e

nsur

e su

cces

s; it

is

clea

rly ti

ed to

the

inst

itutio

n's

mis

sion

, val

ues,

and

str

ateg

ic p

lan.

6T

he b

lend

ed p

rogr

am is

con

side

red

a 's

tart

-up'

and

is n

ot ti

ed to

the

inst

itutio

n's

stra

tegi

c pl

anni

ng

proc

ess

spec

ifica

lly w

ith r

egar

d to

re

sour

ce a

lloca

tions

.

The

ble

nded

pro

gram

str

uggl

es to

re

aliz

e st

rate

gic

impo

rtan

ce w

hen

it co

mes

to p

roce

sses

ass

ocia

ted

with

in

stitu

tiona

l pla

nnin

g an

d re

sour

ce

allo

catio

n

Pla

nnin

g an

d re

sour

ce a

lloca

tion

(e.g

., fin

anci

al r

esou

rces

) pr

oces

ses

asso

ciat

ed w

ith b

lend

ed le

arni

ng a

re

tied

to th

e in

stitu

tion'

s ov

eral

l st

rate

gic

stra

tegy

.

Ble

nded

lear

ning

is c

onsi

dere

d 'm

issi

on c

ritic

al';

plan

ning

and

re

sour

ce a

lloca

tion

(e.g

., fin

anci

al

reso

urce

s) p

roce

sses

ass

ocia

ted

with

onl

ine

lear

ning

are

tied

to th

e in

stitu

tion'

s ov

eral

l str

ateg

ic p

lan.

INS

TIT

UT

ION

AL

SU

PP

OR

T (

27 p

oint

s)

Th

e in

stit

uti

on

has

a g

ove

rnan

ce s

tru

ctu

re t

o

enab

le c

lear

, tim

ely,

eff

ecti

ve, a

nd

co

mp

reh

ensi

ve d

ecis

ion

mak

ing

rel

ated

to

b

len

ded

lear

nin

g c

ou

rses

/pro

gra

ms.

Th

e b

len

ded

lear

nin

g p

rog

ram

’s s

trat

egic

pla

n

is r

evie

wed

fo

r it

s co

nti

nu

ing

rel

evan

ce,

com

plia

nce

wit

h a

ccre

dit

atio

n o

bje

ctiv

es, a

nd

is

per

iod

ical

ly im

pro

ved

an

d u

pd

ated

.

Th

e in

stit

uti

on

has

def

ined

th

e st

rate

gic

val

ue

of

ble

nd

ed le

arn

ing

to

its

ente

rpri

se a

nd

st

akeh

old

ers

(stu

den

ts, f

acu

lty,

par

ents

, etc

.).

Qu

alit

y S

core

card

C

riter

ia fo

r E

xcel

lenc

e in

Ble

nded

Lea

rnin

g P

rogr

ams

Th

e o

rgan

izat

ion

al s

tru

ctu

re o

f th

e b

len

ded

le

arn

ing

pro

gra

m s

up

po

rts

the

inst

itu

tio

n's

m

issi

on

, val

ues

, an

d s

trat

egic

pla

n.

Th

e in

stit

uti

on

has

a p

roce

ss f

or

pla

nn

ing

an

d

reso

urc

e al

loca

tio

n f

or

the

ble

nd

ed le

arn

ing

p

rog

ram

, in

clu

din

g f

inan

cial

res

ou

rces

, in

ac

cord

ance

wit

h s

trat

egic

pla

nn

ing

.

Th

e in

stit

uti

on

has

a g

ove

rnan

ce s

tru

ctu

re t

o

enab

le s

yste

mat

ic a

nd

co

nti

nu

ou

s im

pro

vem

ent

rela

ted

to

th

e ad

min

istr

atio

n o

f b

len

ded

ed

uca

tio

n.

96 Quality Scorecard for Blended Learning Programs

Page 109: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

7T

he b

lend

ed p

rogr

am is

con

side

red

a 's

tart

-up'

and

is n

ot ti

ed to

the

inst

itutio

n's

stra

tegi

c pl

anni

ng

proc

esse

s sp

ecifi

cally

with

reg

ard

to

reso

urce

allo

catio

ns

The

ble

nded

pro

gram

con

sist

ently

st

rugg

les

to g

arne

r su

ffici

ent

reso

urce

allo

catio

n (e

.g.,

finan

cial

re

sour

ces)

to e

ffect

ivel

y su

stai

n th

e pr

ogra

m

The

inst

itutio

n de

mon

stra

tes

suffi

cien

t res

ourc

e al

loca

tion

to

effe

ctiv

ely

sust

ain

the

blen

ded

prog

ram

Ble

nded

lear

ning

is c

onsi

dere

d 'm

issi

on c

ritic

al';

the

inst

itutio

n de

mon

stra

tes

suffi

cien

t res

ourc

e al

loca

tion

to e

ffect

ivel

y su

stai

n an

d al

so g

row

the

blen

ded

prog

ram

.

8T

he in

stitu

tion

is e

ngag

ing

in

expl

orat

ory

conv

ersa

tions

abo

ut

blen

ded

lear

ning

.

The

inst

itutio

n is

in th

e be

ginn

ing

stag

es o

f rea

lizin

g ho

w a

ble

nded

le

arni

ng p

rogr

am s

uppo

rts

the

inst

itutio

n's

mis

sion

, val

ues,

and

st

rate

gic

plan

.

The

ble

nded

lear

ning

pro

gram

is

clea

rly ti

ed to

the

inst

itutio

n's

mis

sion

, val

ues,

and

str

ateg

ic p

lan.

The

ble

nded

lear

ning

pro

gram

is

posi

tione

d w

ithin

the

orga

niza

tiona

l st

ruct

ure

to e

nsur

e su

cces

s; it

is

clea

rly ti

ed to

the

inst

itutio

n's

mis

sion

, val

ues,

and

str

ateg

ic p

lan.

9N

o pr

oces

s is

est

ablis

hed

to e

nsur

e th

at p

erm

issi

ons

(Cre

ativ

e C

omm

ons,

Cop

yrig

ht, F

air

Use

, P

ublic

Dom

ain,

etc

.) a

re in

pla

ce fo

r ap

prop

riate

use

of a

ll co

urse

m

ater

ials

.

The

inst

itutio

n/ p

rogr

am is

in th

e be

ginn

ing

stag

es o

f dev

elop

ing

a co

nsis

tent

pro

cess

to e

nsur

e th

at

perm

issi

ons

(Cre

ativ

e C

omm

ons,

C

opyr

ight

, Fai

r U

se, P

ublic

Dom

ain,

et

c.)

are

in p

lace

for

appr

opria

te u

se

in a

ll co

urse

mat

eria

ls.

The

inst

itutio

n/ p

rogr

am h

as

deve

lope

d a

cons

iste

nt p

roce

ss to

en

sure

that

per

mis

sion

s (C

reat

ive

Com

mon

s, C

opyr

ight

, Fai

r U

se,

Pub

lic D

omai

n, e

tc.)

are

in p

lace

for

appr

opria

te u

se in

all

cour

se

mat

eria

ls; r

ando

m c

heck

s ar

e co

nduc

ted

to v

erify

adh

eren

ce.

The

inst

itutio

n/ p

rogr

am h

as

deve

lope

d a

cons

iste

nt p

roce

ss to

en

sure

that

per

mis

sion

s (C

reat

ive

Com

mon

s, C

opyr

ight

, Fai

r U

se,

Pub

lic D

omai

n, e

tc.)

are

in p

lace

for

appr

opria

te u

se in

all

cour

se

mat

eria

ls; c

ours

es a

re c

heck

ed fo

r ad

here

nce

to p

roce

ss a

nd g

uida

nce

prov

ided

whe

n is

sues

iden

tifie

d.

A p

roce

ss is

fo

llow

ed t

hat

en

sure

s th

at

per

mis

sio

ns

(Cre

ativ

e C

om

mo

ns,

Co

pyr

igh

t,

Fai

r U

se, P

ub

lic D

om

ain

, etc

.) a

re in

pla

ce f

or

app

rop

riat

e u

se o

f al

l co

urs

e m

ater

ials

.

Th

e in

stit

uti

on

dem

on

stra

tes

suff

icie

nt

reso

urc

e al

loca

tio

n, i

ncl

ud

ing

tec

hn

olo

gy

and

fi

nan

cial

res

ou

rces

, in

ord

er t

o e

ffec

tive

ly

sup

po

rt t

he

mis

sio

n o

f b

len

ded

ed

uca

tio

n.

Th

e o

rgan

izat

ion

al s

tru

ctu

re o

f th

e b

len

ded

le

arn

ing

pro

gra

m s

up

po

rts

the

inst

itu

tio

n's

m

issi

on

, val

ues

, an

d s

trat

egic

pla

n.

Blended Learning Scorecard Rubric 97

Page 110: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

he b

lend

ed c

ours

e de

liver

y te

chno

logy

is n

ot c

onsi

dere

d im

port

ant;

it is

mor

e lik

e an

afte

r th

ough

t tha

n a

plan

ned

serv

ice.

The

ble

nded

cou

rse

deliv

ery

tech

nolo

gy is

con

side

red

impo

rtan

t, bu

t not

yet

vie

wed

as

‘mis

sion

crit

ical

’ an

d is

sup

port

ed a

s su

ch.

The

ble

nded

cou

rse

deliv

ery

tech

nolo

gy is

con

side

red

‘mis

sion

cr

itica

l’ an

d is

sup

port

ed a

s su

ch.

The

ble

nded

cou

rse

deliv

ery

tech

nolo

gy is

con

side

red

‘mis

sion

crit

ical

’ and

is s

uppo

rted

as

such

; the

pro

gram

/in

stitu

tion

uses

met

rics

and

benc

hmar

king

for

upgr

adin

g an

d im

prov

ing

tech

nolo

gies

.

2T

he in

stitu

tion

has

yet t

o es

tabl

ish

para

met

ers

or a

pla

n w

ith r

egar

d to

da

ta m

anag

emen

t sta

ndar

ds.

No

docu

men

tatio

n is

pro

vide

d to

kn

ow w

heth

er d

ata

cent

ers

(loca

l or

outs

ourc

ed, h

oste

d or

clo

ud

serv

ices

) ar

e ad

min

iste

red

in

com

plia

nce

with

est

ablis

hed

data

m

anag

emen

t pra

ctic

es s

uch

as IT

SM

st

anda

rds.

Dat

a ce

nter

s (lo

cal o

r ou

tsou

rced

, ho

sted

or

clou

d se

rvic

es)

are

adm

inis

tere

d in

com

plia

nce

with

es

tabl

ishe

d da

ta m

anag

emen

t pr

actic

es s

uch

as IT

SM

sta

ndar

ds.

Dat

a ce

nter

s (lo

cal o

r ou

tsou

rced

, ho

sted

or

clou

d se

rvic

es)

are

adm

inis

tere

d in

com

plia

nce

with

es

tabl

ishe

d da

ta m

anag

emen

t pr

actic

es s

uch

as IT

SM

sta

ndar

ds;

the

blen

ded

prog

ram

adm

inis

trat

ion

docu

men

ts c

ontin

ual c

ompl

ianc

e.

3T

o da

te, t

he te

chno

logy

sys

tem

pe

rfor

man

ce is

not

con

tinua

lly

mon

itore

d, tr

acke

d, a

nd r

epor

ted.

Tec

hnol

ogy

deliv

ery

syst

em

perf

orm

ance

is c

ontin

ually

mon

itore

d,

trac

ked,

and

rep

orte

d.

The

pro

gram

/ ins

titut

ion

view

s its

te

chno

logy

del

iver

y s

yste

ms

as

‘mis

sion

crit

ical

’; sy

stem

per

form

ance

is

con

tinua

lly m

onito

red,

trac

ked,

and

re

port

ed.

The

pro

gram

/inst

itutio

n v

iew

s its

te

chno

logy

del

iver

y sy

stem

s as

‘mis

sion

crit

ical

’; sy

stem

pe

rfor

man

ce

is c

ontin

ually

mon

itore

d, tr

acke

d, a

nd

repo

rted

; sys

tem

bac

kups

are

in

plac

e fo

r al

l nec

essa

ry te

chni

cal

syst

ems

(pre

fera

bly

with

off-

site

bac

kups

in

case

of a

clo

sure

due

to d

isas

ter)

.

4N

o ev

iden

ce is

pro

vide

d w

hich

ill

umin

ates

that

the

prog

ram

/in

stitu

tion

has

disc

usse

d or

is

enga

ging

in a

tech

nolo

gy p

lann

ing

proc

ess.

The

pro

gram

/ ins

titut

ion

is in

the

proc

ess

of d

evel

opin

g a

tech

nolo

gy p

lan.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d an

d di

ssem

inat

ed

a te

chno

logy

pla

n to

stu

dent

s an

d fa

culty

whi

ch in

clud

es e

lect

roni

c se

curit

y m

easu

res;

the

tech

nolo

gy p

lan

is

deve

lope

d in

acc

orda

nce

with

es

tabl

ishe

d st

anda

rds

and

reg

ulat

ory

requ

irem

ents

.

The

pro

gram

/inst

itutio

n ha

s de

velo

ped

and

diss

emin

ated

a te

chno

logy

pla

n to

stu

dent

s an

d fa

culty

whi

ch in

clud

es

elec

tron

ic s

ecur

ity m

easu

res;

the

tech

nolo

gy p

lan

is d

evel

oped

in

acco

rdan

ce w

ith e

stab

lishe

d st

anda

rds

and

reg

ulat

ory

requ

irem

ents

; fac

ulty

m

embe

rs a

re tr

aine

d on

str

ateg

ies

used

to e

nsur

e qu

ality

and

ad

here

nce

to s

tand

ards

.

5T

he in

stitu

tion

has

yet t

o ex

plor

e de

velo

ping

a c

ontin

genc

y pl

an in

the

even

t of a

pro

long

ed s

ervi

ce

disr

uptio

n.

The

inst

itutio

n is

in th

e pr

oces

s of

es

tabl

ishi

ng a

con

tinge

ncy

plan

in th

e ev

ent o

f a p

rolo

nged

ser

vice

di

srup

tion.

The

inst

itutio

n ha

s es

tabl

ishe

d a

cont

inge

ncy

plan

in th

e ev

ent o

f a p

rolo

nged

ser

vice

di

srup

tion.

The

inst

itutio

n ha

s es

tabl

ishe

d a

cont

inge

ncy

plan

in th

e ev

ent o

f a

prol

onge

d se

rvic

e di

srup

tion;

moc

k di

sast

er d

rills

are

con

duct

ed

perio

dica

lly to

ens

ure

back

-up

and

rest

orat

ion

proc

esse

s ar

e vi

able

and

m

aint

ain

data

inte

grity

; g

oals

hav

e be

en e

stab

lishe

d fo

r tim

ely

syst

em r

ecov

ery.

Th

e te

chn

olo

gy

syst

ems

rela

ted

to

th

e d

eliv

ery

of

ble

nd

ed le

arn

ing

pro

gra

ms

are

hig

hly

re

liab

le a

nd

op

erab

le w

ith

mea

sura

ble

st

and

ard

s b

ein

g u

tiliz

ed s

uch

as

syst

em

do

wn

tim

e tr

acki

ng

an

d t

ask

ben

chm

arki

ng

.*

Wh

eth

er t

he

inst

itu

tio

n m

ain

tain

s lo

cal d

ata

cen

ters

(se

rver

s), a

nd

/or

con

trac

ts f

or

ou

tso

urc

ed, h

ost

ed s

ervi

ces

or

clo

ud

ser

vice

s,

tho

se s

yste

ms

are

adm

inis

tere

d in

co

mp

lian

ce

wit

h e

stab

lish

ed d

ata

man

agem

ent

pra

ctic

es

such

as

the

Info

rmat

ion

Tec

hn

olo

gy

Ser

vice

M

anag

emen

t (I

TS

M)

stan

dar

ds,

wh

ich

incl

ud

e ap

pro

pri

ate

po

wer

pro

tect

ion

, bac

kup

so

luti

on

s, d

isas

ter

reco

very

pla

ns,

etc

.

Th

e co

urs

e d

eliv

ery

tech

no

log

y is

co

nsi

der

ed a

m

issi

on

cri

tica

l en

terp

rise

sys

tem

an

d

sup

po

rted

as

such

.

A d

ocu

men

ted

tec

hn

olo

gy

pla

n t

hat

incl

ud

es

elec

tro

nic

sec

uri

ty m

easu

res

(e.g

., p

assw

ord

p

rote

ctio

n, e

ncr

ypti

on

, sec

ure

on

line

or

pro

cto

red

exa

ms

if a

pp

licab

le, e

tc.)

is in

pla

ce

and

op

erat

ion

al t

o e

nsu

re q

ual

ity,

in

acco

rdan

ce w

ith

est

ablis

hed

sta

nd

ard

s an

d

reg

ula

tory

req

uir

emen

ts.*

Th

e in

stit

uti

on

has

an

est

ablis

hed

(u

pd

ated

an

d

con

tin

uo

usl

y re

view

ed)

con

tin

gen

cy p

lan

fo

r th

e co

nti

nu

ance

of

dat

a ce

nte

rs a

nd

su

pp

ort

se

rvic

es in

th

e ev

ent

of

pro

lon

ged

ser

vice

d

isru

pti

on

.

TE

CH

NO

LO

GY

SU

PP

OR

T (

21 p

oint

s)

98 Quality Scorecard for Blended Learning Programs

Page 111: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

6T

he b

lend

ed p

rogr

am is

in ‘s

tart

-up’

an

d ha

s ye

t to

deve

lop

a sy

stem

for

build

ing

and

supp

ortin

g th

e bl

ende

d in

fras

truc

ture

.

The

pro

gram

/ ins

titut

ion

has

a di

strib

uted

sys

tem

(e.

g., s

ever

al

func

tiona

l uni

ts a

re in

volv

ed)

for

build

ing

and

mai

ntai

ning

the

blen

ded

infr

astr

uctu

re.

The

pro

gram

/ ins

titut

ion

has

inve

sted

in

a c

entr

aliz

ed s

yste

m fo

r bu

ildin

g an

d m

aint

aini

ng th

e bl

ende

d in

fras

truc

ture

.

The

pro

gram

/inst

itutio

n ha

s in

vest

ed

in a

cen

tral

ized

sys

tem

for

build

ing

and

mai

ntai

ning

the

blen

ded

infr

astr

uctu

re;

it is

team

bas

ed,

colla

bora

tive,

com

preh

ensi

ve, a

ctio

n or

ient

ed a

nd n

onhi

erar

chic

al;

pers

pect

ives

of k

ey s

take

hold

ers

are

cont

inua

lly a

sses

sed

to im

prov

e th

e sy

stem

.

7N

o ev

iden

ce is

pro

vide

d th

at fa

culty

, st

aff o

r st

uden

t use

new

tech

nolo

gies

an

d ho

w th

ey a

re s

uppo

rted

.

Ent

repr

eneu

rial f

acul

ty m

embe

rs a

re

expl

orin

g ne

w te

chno

logi

es a

nd

deve

lopi

ng th

eir

indi

vidu

al s

kills

with

lim

ited

inst

itutio

nal s

uppo

rt.

The

pro

gram

/inst

itutio

n pr

ovid

es

suffi

cien

t ser

vice

s to

sup

port

facu

lty,

staf

f and

stu

dent

s in

the

use

of n

ew

tech

nolo

gies

.

The

pro

gram

/inst

itutio

n pr

ovid

es

suffi

cien

t ser

vice

s to

sup

port

facu

lty,

staf

f and

stu

dent

s in

the

use

of n

ew

tech

nolo

gies

; gui

des,

info

rmat

ion

shee

ts, o

r tu

toria

ls a

re d

evel

oped

; he

lp d

esk

supp

ort s

taff

are

know

ledg

eabl

e; fa

culty

wor

ksho

ps o

r ot

her

in-s

ervi

ce tr

aini

ng is

con

duct

ed

to e

nsur

e fa

culty

hav

e th

e ne

cess

ary

skill

s to

use

new

tech

nolo

gies

.

Fac

ult

y, s

taff

, an

d s

tud

ents

are

su

pp

ort

ed in

th

e d

evel

op

men

t an

d u

se o

f n

ew t

ech

no

log

ies

and

ski

lls a

pp

licab

le t

o b

len

ded

lear

nin

g.

A c

entr

aliz

ed s

yste

m p

rovi

des

su

pp

ort

fo

r b

uild

ing

an

d m

ain

tain

ing

th

e b

len

ded

ed

uca

tio

n in

fras

tru

ctu

re.*

Blended Learning Scorecard Rubric 99

Page 112: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

here

is n

o in

dica

tion

that

cou

rses

ar

e de

sign

ed to

ens

ure

that

lear

ning

ou

tcom

es a

re m

et.

Ble

nded

cou

rses

are

des

igne

d to

m

eet l

earn

ing

outc

omes

, but

no

cons

iste

nt m

echa

nism

exi

sts

to

ensu

re c

ours

e an

d pr

ogra

m le

arni

ng

outc

omes

are

met

.

Ble

nded

cou

rses

are

des

igne

d to

m

eet l

earn

ing

outc

omes

and

a

mec

hani

sm e

xist

s to

ens

ure

cour

se

and

prog

ram

lear

ning

out

com

es a

re

met

; how

ever

, adh

eren

ce v

arie

s ac

ross

cou

rses

and

pro

gram

s.

All

blen

ded

cour

ses

are

desi

gned

to

ensu

re s

tude

nts

deve

lop

the

nece

ssar

y kn

owle

dge

and

skill

s at

bo

th th

e co

urse

and

pro

gram

leve

l; em

phas

is is

pla

ced

on b

oth

form

ativ

e an

d su

mm

ativ

e as

sess

men

t crit

eria

.

2N

o ev

iden

ce is

pro

vide

d w

hich

indi

cate

s gu

idel

ines

reg

ardi

ng m

inim

um s

tand

ards

ex

ist.

The

pro

gram

/inst

itutio

n ha

s be

gun

the

proc

ess

of d

evel

opin

g gu

idel

ines

co

ncer

ning

min

imum

sta

ndar

ds.

Gui

delin

es e

xist

, but

the

prog

ram

/inst

itutio

n ha

s ye

t to

fully

de

ploy

them

acr

oss

all c

ours

es.

Gui

delin

es e

xist

and

evi

denc

e (e

.g.,

exem

plar

y co

urse

s) is

pre

sent

ed

whi

ch s

how

case

s th

at s

tand

ards

are

fu

lly d

eplo

yed

acro

ss a

ll co

urse

s.

3T

he in

stitu

tion

has

yet t

o es

tabl

ish

cons

iste

ncy

in c

ours

e de

velo

pmen

t fo

r st

uden

t ret

entio

n, q

ualit

y or

hav

e a

cons

iste

nt n

avig

atio

nal s

truc

ture

.

The

inst

itutio

n is

exp

lorin

g m

etho

ds

to e

stab

lish

cons

iste

ncy

in c

ours

e de

velo

pmen

t for

stu

dent

ret

entio

n,

qual

ity o

r ha

ve a

con

sist

ent

navi

gatio

nal s

truc

ture

.

Inte

ntio

nal c

ours

e de

velo

pmen

t and

de

sign

add

ress

es c

once

rns

in

cons

iste

ncy

in c

ours

e rig

or,

achi

evem

ent o

f out

com

es, a

nd

cons

iste

nt n

avig

atio

nal s

truc

ture

. E

valu

atio

n of

cou

rse

outc

omes

in

cons

iste

nt w

ith S

yste

mat

ic E

valu

atio

n P

lan

that

incl

udes

a fe

edba

ck lo

op.

Inte

ntio

nal c

ours

e de

velo

pmen

t and

de

sign

add

ress

es c

once

rns

in

cons

iste

ncy

in c

ours

e rig

or,

achi

evem

ent o

f out

com

es, a

nd

cons

iste

nt n

avig

atio

nal s

truc

ture

w

hile

faci

litat

ing

lear

ner

inte

ract

ion

and

enga

gem

ent.

Eva

luat

ion

of

cour

se o

utco

mes

in c

onsi

sten

t with

S

yste

mat

ic E

valu

atio

n P

lan

that

in

clud

es a

feed

back

loop

. P

eer

revi

ew u

sing

a s

core

card

pro

vide

s di

rect

ion

for

stra

tegi

c pl

anni

ng a

s w

ell a

s fa

cilit

atin

g co

ntin

ual q

ualit

y im

prov

emen

t.

4T

he o

nlin

e co

mpo

nent

s of

the

blen

ded

cour

se a

re u

sed

for

to

deliv

er c

onte

nt th

at is

als

o re

peat

ed

in th

e fa

ce-t

o-fa

ce c

ours

e.

Onl

ine

cont

ent i

s ut

ilize

d to

pre

pare

st

uden

ts fo

r th

e fa

ce-t

o-fa

ce s

essi

on.

The

cou

rse

utili

zes

the

onlin

e co

mpo

nent

s th

at in

clud

e co

nten

t de

liver

y, fo

rmat

ive

asse

ssm

ents

, an

d/or

act

ive

lear

ning

to p

repa

re

stud

ents

for

the

face

-to-

face

se

ssio

ns.

The

cou

rse

utili

zes

the

onlin

e co

mpo

nent

s th

at in

clud

e co

nten

t de

liver

y, fo

rmat

ive

asse

ssm

ents

, an

d/or

act

ive

lear

ning

to p

repa

re

stud

ents

for

the

face

-to-

face

se

ssio

ns. I

n ad

ditio

n, th

e fa

ce-t

o-fa

ce

sess

ion

utili

zes

onlin

e en

viro

nmen

t in

a va

riety

of w

ays

both

dur

ing

and

afte

r th

e se

ssio

n en

ds.

5N

o pe

riodi

c re

view

pro

cess

of

inst

ruct

iona

l mat

eria

ls a

nd c

ours

e sy

llabi

exi

sts.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d a

plan

to p

erio

dica

lly r

evie

w c

ours

e sy

llabi

bu

t few

rev

iew

s ha

ve o

ccur

red.

Per

iodi

c re

view

of i

nstr

uctio

nal m

ater

ials

an

d co

urse

syl

labi

occ

urs,

but

on

an a

d ho

c ba

sis.

The

pro

gram

/inst

itutio

n ha

s a

cons

iste

nt

and

syst

em-w

ide

perio

dic

revi

ew p

roce

ss

of in

stru

ctio

nal m

ater

ials

and

cou

rse

sylla

bi to

det

erm

ine

curr

ency

, rel

evan

ce,

and

mea

sura

bilit

y.

6N

o pe

riodi

c re

view

pro

cess

of c

ours

e as

sign

men

ts a

nd a

ctiv

ities

exi

sts.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d a

plan

to p

erio

dica

lly r

evie

w c

ours

e as

sign

men

ts a

nd a

ctiv

ities

but

few

re

view

s ha

ve o

ccur

red.

Per

iodi

c re

view

of c

ours

e as

sign

men

ts

and

activ

ities

occ

urs,

but

on

an a

d ho

c ba

sis.

The

pro

gram

/inst

itutio

n ha

s a

cons

iste

nt

and

syst

em-w

ide

perio

dic

revi

ew p

roce

ss

of c

ours

e as

sign

men

ts a

nd a

ctiv

ities

; ru

bric

s or

sel

f-as

sess

men

t too

ls a

re

prov

ided

; iss

ues

of a

cade

mic

dis

hone

st

are

rais

ed.

Co

urs

e as

sig

nm

ents

an

d a

ctiv

itie

s ar

e re

view

ed p

erio

dic

ally

to

en

sure

th

ey m

eet

the

ble

nd

ed c

ou

rses

' an

d p

rog

ram

's le

arn

ing

o

utc

om

es.

Th

ere

is c

on

sist

ency

in c

ou

rse

dev

elo

pm

ent

for

stu

den

t re

ten

tio

n (

enro

llmen

t an

d c

ou

rse

com

ple

tio

n)

and

qu

alit

y (i

.e.,

cou

rses

in a

p

rog

ram

hav

e a

con

sist

ent

nav

igat

ion

al

stru

ctu

re).

A b

len

ded

co

urs

e sh

ou

ld b

e d

esig

ned

as

on

e co

hes

ive

wh

ole

, in

corp

ora

tin

g b

oth

fac

e-to

-fa

ce a

nd

on

line

exp

erie

nce

s in

co

mp

lem

enta

ry

way

s.

Inst

ruct

ion

al m

ater

ials

(b

oth

on

line

and

in-

clas

s) a

nd

co

urs

e sy

llab

i are

rev

iew

ed

per

iod

ical

ly t

o e

nsu

re t

hey

mee

t th

e b

len

ded

co

urs

e's

and

pro

gra

m's

lear

nin

g o

utc

om

es.*

Gu

idel

ines

reg

ard

ing

min

imu

m r

equ

irem

ents

fo

r co

urs

e d

evel

op

men

t, d

esig

n, a

nd

del

iver

y o

f b

len

ded

inst

ruct

ion

(su

ch a

s co

urs

e sy

llab

us

elem

ents

, co

urs

e m

ater

ials

, ass

essm

ent

stra

teg

ies,

fac

ult

y fe

edb

ack)

are

fo

llow

ed.*

A c

ou

rse

dev

elo

pm

ent

pro

cess

is f

ollo

wed

th

at

ensu

res

cou

rses

are

des

ign

ed w

ith

alig

nm

ent

bet

wee

n c

ou

rse

mat

eria

ls, a

sses

smen

ts a

nd

le

arn

ing

ob

ject

ives

so

th

at s

tud

ents

dev

elo

p

the

nec

essa

ry k

no

wle

dg

e an

d s

kills

to

mee

t m

easu

rab

le le

arn

ing

ou

tco

mes

at

the

cou

rse

and

pro

gra

m le

vel.*

CO

UR

SE

DE

VE

LO

PM

EN

T A

ND

INS

TR

UC

TIO

NA

L D

ES

IGN

(42

poi

nts)

100 Quality Scorecard for Blended Learning Programs

Page 113: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

7T

he in

stitu

tion

has

yet t

o de

mon

stra

te th

at c

ours

es a

re

deve

lope

d w

ith a

stu

dent

-cen

tere

d fo

cus.

No

form

al tr

aini

ng e

xist

s, b

ut s

ome

cour

ses

are

desi

gned

with

a s

tude

nt-

cent

ered

focu

s (u

se o

f stu

dent

-ce

nter

ed a

ssig

nmen

ts a

nd

cons

truc

tivis

t act

iviti

es).

Tra

inin

g is

pro

vide

d an

d th

e pr

ogra

m/in

stitu

tion

is m

akin

g an

ef

fort

to e

nsur

e co

urse

s ar

e de

velo

ped

with

a s

tude

nt-c

ente

red

focu

s (u

se o

f stu

dent

-cen

tere

d as

sign

men

ts a

nd c

onst

ruct

ivis

t ac

tiviti

es).

App

ropr

iate

trai

ning

is p

rovi

ded

and

the

prog

ram

/inst

itutio

n en

sure

s st

uden

t-ce

nter

ed in

stru

ctio

n (e

.g.,

assi

gnm

ents

con

nect

with

the

lear

ners

' life

, eng

age

stud

ents

in

lear

ning

how

they

lear

n) is

at t

he

hear

t of t

he c

ours

e de

velo

pmen

t pr

oces

s fo

r al

l cou

rses

.

8N

o ev

iden

ce e

xist

s th

at c

ours

es a

re

desi

gned

to p

rom

ote

facu

lty a

nd s

tude

nt

enga

gem

ent.

The

pro

gram

/ ins

titut

ion

incl

udes

facu

lty-

stud

ent e

ngag

emen

t in

the

cour

se d

esig

n pr

oces

s. F

acul

ty r

ecei

ve li

ttle

to n

o tr

aini

ng fo

r co

urse

des

ign.

The

pro

gram

/ ins

titut

ion

incl

udes

facu

lty-

stud

ent e

ngag

emen

t in

the

cour

se d

esig

n pr

oces

s T

rain

ing

is p

rovi

ded,

but

no

eval

uatio

n oc

curs

for

leve

ls o

f fac

ulty

and

st

uden

t eng

agem

ent.

App

ropr

iate

trai

ning

is p

rovi

ded

to

show

case

str

ateg

ies

that

pro

mot

e bo

th

facu

lty a

nd s

tude

nt e

ngag

emen

t and

a

proc

ess

is in

pla

ce to

eva

luat

e/as

sess

le

vels

, met

hods

and

freq

uenc

y of

en

gage

men

t.

9T

he in

stitu

tion

has

not i

dent

ified

a

stan

dard

pro

cess

for

defin

ing

cloc

k/cr

edit

hour

s in

con

grue

ncy

with

re

gula

tory

and

acc

redi

tatio

n st

anda

rds.

The

inst

itutio

n ha

s no

pr

oces

s to

ens

ure

that

cou

rse

wor

kloa

ds a

re r

evie

wed

to e

nsur

e ap

prop

riate

allo

catio

n of

cre

dit.

The

inst

itutio

n ha

s id

entif

ied

a st

anda

rd p

roce

ss fo

r de

finin

g cl

ock/

cred

it ho

urs

in c

ongr

uenc

y w

ith

regu

lato

ry a

nd a

ccre

dita

tion

stan

dard

s.

A s

tand

ard

proc

ess

for

defin

ing

cred

it/cl

ock

hour

s to

ens

ure

cons

iste

ncy

in a

lloca

tion

as w

ell a

s co

ngru

ency

with

reg

ulat

ory

and

accr

edita

tion

stan

dard

s as

def

ined

by

the

orga

niza

tion

and

US

DE

is

used

to

ens

ure

com

plia

nce

with

reg

ulat

ory

and

accr

editi

ng r

equi

rem

ents

. B

lend

ed c

ours

es a

re p

erio

dica

lly

revi

ewed

to e

nsur

e ap

prop

riate

al

loca

tion

of c

redi

t and

new

ble

nded

co

urse

s ar

e re

view

ed to

ens

ure

appr

opria

te a

lloca

tion

of c

redi

t.

A s

tand

ard

proc

ess

for

defin

ing

cred

it/cl

ock

hour

s to

ens

ure

cons

iste

ncy

in a

lloca

tion

as w

ell a

s co

ngru

ency

with

reg

ulat

ory

and

accr

edita

tion

stan

dard

s as

def

ined

by

the

orga

niza

tion

and

US

DE

is

used

to

ens

ure

com

plia

nce

with

reg

ulat

ory

and

accr

editi

ng r

equi

rem

ents

. B

lend

ed c

ours

es a

re p

erio

dica

lly

revi

ewed

to e

nsur

e ap

prop

riate

al

loca

tion

of c

redi

t and

new

ble

nded

co

urse

s ar

e re

view

ed to

ens

ure

appr

opria

te a

lloca

tion

of c

redi

t. A

cade

mic

org

aniz

atio

ns a

re a

ble

to

prov

ide

facu

lty s

uppo

rt to

ens

ure

the

equi

tabl

e di

strib

utio

n of

cre

dit a

mon

g bl

ende

d, o

nlin

e an

d fa

ce to

face

co

urse

s, w

ith a

n eq

uita

ble

dist

ribut

ion

of s

tude

nt w

ork.

10T

here

is n

o ev

iden

ce th

at th

e pr

ogra

m/

inst

itutio

n ha

s a

proc

ess

in p

lace

for

eval

uatin

g th

e ef

fect

iven

ess

of c

urre

nt

and

emer

ging

tech

nolo

gies

.

A p

lan

is in

pla

ce to

eva

luat

e th

e ef

fect

iven

ess

of c

urre

nt a

nd e

mer

ging

te

chno

logi

es, b

ut it

is y

et to

be

depl

oyed

.

A p

roce

ss is

in p

lace

to e

valu

ate

curr

ent

and

emer

ging

tech

nolo

gies

, but

it is

not

fo

llow

ed c

onsi

sten

tly a

cros

s al

l cou

rses

an

d pr

ogra

ms.

A c

onsi

sten

t pro

cess

is in

pla

ce a

nd

follo

wed

acr

oss

all c

ours

es a

nd p

rogr

ams

to e

valu

ate

the

effe

ctiv

enes

s of

cur

rent

an

d em

ergi

ng te

chno

logi

es to

sup

port

le

arni

ng o

utco

mes

and

enc

oura

ge c

onte

nt

expe

rts

to b

e fa

mili

ar w

ith in

nova

tive

te

chno

logi

es.

11T

he p

rogr

am/in

stitu

tion

does

not

pr

ovid

e gu

idan

ce r

egar

ding

ap

prop

riate

and

con

stru

ctiv

e us

e of

te

chno

logy

for

blen

ded

cour

ses.

The

pro

gram

/inst

itutio

n pr

ovid

es

limite

d gu

idan

ce o

n us

ing

tech

nolo

gy

cons

truc

tivel

y, a

nd/o

r th

ere

is

wid

espr

ead

use

of s

uper

fluou

s te

chno

logy

.

The

pro

gram

/inst

itutio

n is

mak

ing

prog

ress

tow

ard

follo

win

g be

st

prac

tices

for

the

use

of te

chno

logy

in

blen

ded

cour

ses

and

is li

miti

ng th

e su

perf

luou

s us

e of

tech

nolo

gy.

The

pro

gram

/inst

itutio

n ha

s gu

idel

ines

and

sta

ndar

ds fo

r in

vest

igat

ing

tech

nolo

gies

; cou

rse

embe

dded

tech

nolo

gies

act

ivel

y su

ppor

t the

ach

ieve

men

t of l

earn

ing

outc

omes

in a

ll co

urse

s an

d su

perf

luou

s us

e of

tech

nolo

gy is

m

inim

ized

.

Co

urs

e d

esig

n p

rom

ote

s b

oth

fac

ult

y an

d

stu

den

t en

gag

emen

t.

Co

urs

e em

bed

ded

tec

hn

olo

gy

acti

vely

su

pp

ort

s th

e ac

hie

vem

ent

of

lear

nin

g

ou

tco

mes

an

d d

eliv

erin

g c

ou

rse

con

ten

t an

d

un

nec

essa

ry u

se o

f te

chn

olo

gy

is m

inim

ized

.*

Co

urs

e w

ork

load

s ar

e re

view

ed t

o e

nsu

re it

is

app

rop

riat

e fo

r d

esig

nat

ed c

red

it a

lloca

tio

n.

A p

roce

ss is

est

ablis

hed

an

d f

ollo

wed

fo

r ev

alu

atin

g t

he

effe

ctiv

enes

s o

f cu

rren

t an

d

emer

gin

g t

ech

no

log

ies

to s

up

po

rt t

he

ach

ieve

men

t o

f le

arn

ing

ou

tco

mes

an

d

del

iver

ing

co

urs

e co

nte

nt.

Stu

den

t-ce

nte

red

inst

ruct

ion

is c

on

sid

ered

d

uri

ng

th

e co

urs

e d

evel

op

men

t p

roce

ss (

i.e.,

stu

den

t en

gag

emen

t, im

mer

sio

n, a

nd

per

son

al

resp

on

sib

ility

).

Blended Learning Scorecard Rubric 101

Page 114: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

12N

o ev

iden

ce e

xist

s th

at u

sabi

lity

test

s ar

e be

ing

cons

ider

ed o

r co

nduc

ted.

Dis

cuss

ions

are

und

erw

ay a

nd p

lans

ar

e be

ing

deve

lope

d co

ncer

ning

how

to

con

duct

usa

bilit

y te

stin

g, b

ut n

o ac

tion

has

been

take

n to

dat

e.

The

pro

gram

/ ins

titut

ion

is c

ondu

ctin

g us

abili

ty te

sts,

but

the

reco

mm

enda

tions

hav

e ye

t to

be

incl

uded

in a

ll co

urse

s an

d pr

ogra

ms.

The

pro

gram

/inst

itutio

n ha

s a

form

al

and

cons

iste

ntly

app

lied

proc

ess

of

cond

uctin

g us

abili

ty te

sts

acro

ss a

ll co

urse

s an

d pr

ogra

ms;

rev

iew

re

com

men

datio

ns a

re in

corp

orat

ed

into

cou

rses

.

13F

acul

ty m

embe

rs h

ave

no c

onsi

sten

t in

volv

emen

t in

curr

icul

um

deve

lopm

ent.

Adm

inis

trat

ive

pers

onne

l hav

e an

ac

tive

role

in th

e de

velo

pmen

t and

de

cisi

on m

akin

g fo

r bl

ende

d cu

rric

ula,

but

facu

lty m

embe

rs

prov

ide

inpu

t.

Fac

ulty

mem

bers

hav

e an

act

ive

role

in

the

deve

lopm

ent a

nd d

ecis

ion

mak

ing

for

blen

ded

curr

icul

a, w

ith

adm

inis

trat

ive

over

sigh

t.

Fac

ulty

mem

bers

are

inte

gral

in b

oth

the

deve

lopm

ent a

nd d

ecis

ion

mak

ing

for

all b

lend

ed c

urric

ula.

14N

o ev

iden

ce e

xist

s th

at fa

culty

su

ppor

t and

res

ourc

es a

re p

rovi

ded

to p

rom

ote

the

best

use

of b

lend

ed

lear

ning

in c

ours

e de

velo

pmen

t and

in

stru

ctio

nal d

esig

n.

Som

e fa

culty

sup

port

and

res

ourc

es

are

in p

lace

and

bei

ng im

plem

ente

d an

d us

ed o

n an

ad

hoc

basi

s.

The

pro

gram

/inst

itutio

n pr

ovid

es

facu

lty s

uppo

rt a

nd r

esou

rces

on

a re

gula

r ba

sis

to s

uppo

rt s

peci

fic

cour

se o

r pr

ogra

m n

eeds

.

The

pro

gram

/inst

itutio

n pr

ovid

es

regu

lar,

com

preh

ensi

ve a

nd

cons

iste

nt fa

culty

sup

port

and

re

sour

ces

to s

uppo

rt th

e be

st u

se o

f bl

ende

d de

liver

y m

etho

d in

cou

rse

deve

lopm

ent a

nd in

stru

ctio

nal d

esig

n ac

ross

the

inst

itutio

n.

Cu

rric

ulu

m d

evel

op

men

t is

a c

ore

re

spo

nsi

bili

ty f

or

facu

lty

(i.e

., fa

cult

y sh

ou

ld b

e in

volv

ed in

eit

her

th

e d

evel

op

men

t o

r th

e d

ecis

ion

mak

ing

fo

r th

e b

len

ded

cu

rric

ulu

m

cho

ices

).F

acu

lty

sup

po

rt a

nd

res

ou

rces

are

pro

vid

ed t

o

pro

mo

te t

he

bes

t u

se o

f b

len

ded

del

iver

y m

eth

od

in c

ou

rse

dev

elo

pm

ent

and

in

stru

ctio

nal

des

ign

to

fac

ilita

te t

each

ing

an

d

lear

nin

g.

Usa

bili

ty t

ests

are

co

nd

uct

ed a

nd

ap

plie

d a

nd

re

com

men

dat

ion

s b

ased

up

on

Web

Co

nte

nt

Acc

essi

bili

ty G

uid

elin

es (

WC

AG

s) a

re

inco

rpo

rate

d in

th

e co

urs

e d

esig

n p

roce

ss.

102 Quality Scorecard for Blended Learning Programs

Page 115: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

he p

rogr

am/in

stitu

tion

does

not

ha

ve r

equi

rem

ents

or

stan

dard

s fo

r sy

llabi

.

Tex

tboo

k an

d re

quire

d le

arni

ng

mat

eria

ls a

re m

ade

avai

labl

e to

st

uden

ts in

adv

ance

of t

heir

cour

se

regi

stra

tion;

the

use

of s

ylla

bi

requ

irem

ents

or

stan

dard

s va

ries

acro

ss th

e co

urse

s or

pro

gram

.

Syl

labi

par

amet

ers/

stan

dard

s ar

e pr

ovid

ed to

all

facu

lty m

embe

rs a

nd

all b

lend

ed c

ours

es in

clud

e a

sylla

bus;

text

book

and

any

req

uire

d le

arni

ng m

ater

ials

are

mad

e av

aila

ble

to s

tude

nts

in a

dvan

ce o

f the

ir co

urse

re

gist

ratio

n.

Tra

inin

g an

d sy

llabi

st

anda

rds/

para

met

ers,

bas

ed o

n pr

ogra

m/in

stitu

tiona

l req

uire

men

ts)

are

prov

ided

to a

ll fa

culty

mem

bers

; al

l ble

nded

cou

rses

incl

ude

a sy

llabu

s; te

xtbo

ok a

nd r

equi

red

lear

ning

mat

eria

ls a

re m

ade

avai

labl

e to

stu

dent

s in

adv

ance

of t

heir

cour

se

regi

stra

tion;

the

prog

ram

/inst

itutio

n en

sure

con

sist

ency

in s

ylla

bi

plac

emen

t in

the

LMS

for

all b

lend

ed

cour

ses.

2T

he p

rogr

am/in

stitu

tion

has

no p

lan

to e

nsur

e th

at b

lend

ed s

tude

nts

have

ac

cess

to n

eces

sary

libr

ary/

lear

ning

re

sour

ces.

The

pro

gram

/ ins

titut

ion

has

no p

lan

to e

nsur

e bl

ende

d st

uden

ts b

lend

ed

stud

ents

hav

e ac

cess

to n

eces

sary

lib

rary

/ lea

rnin

g re

sour

ces

(e.g

., tu

torin

g, w

ritin

g ce

nter

).

The

pro

gram

/ ins

titut

ion

is b

uild

ing

out i

ts a

cces

s to

ens

ure

ble

nded

st

uden

ts h

ave

acce

ss to

nec

essa

ry

libra

ry/ l

earn

ing

reso

urce

s (e

.g.,

tuto

ring,

writ

ing

cent

er).

The

pro

gram

/ ins

titut

ion

ensu

re a

ll bl

ende

d st

uden

ts h

ave

acce

ss to

ne

cess

ary

libra

ry/ l

earn

ing

reso

urce

s (e

.g.,

tuto

ring,

writ

ing

cent

er)

rega

rdle

ss o

f geo

grap

hic

loca

tion.

3T

he p

rogr

am/in

stitu

tion

has

no p

lan

to e

nsur

e th

at r

equi

red

elem

ents

for

sylla

bi a

re in

clud

ed in

cou

rse

sylla

bi.

Info

rmat

ion

is p

rovi

ded

for

all f

acul

ty

mem

bers

, so

cour

se s

ylla

bi in

clud

e ke

y ex

pect

atio

ns a

nd g

radi

ng

polic

ies,

but

facu

lty r

espo

nse

time

is

not a

ddre

ssed

.

Fac

ulty

mem

bers

are

pro

vide

d in

form

atio

n co

ncer

ning

key

ex

pect

atio

ns fo

r sy

llabi

and

gra

ding

po

licie

s; th

e pr

ogra

m/in

stitu

tion

has

a re

com

men

ded

facu

lty r

espo

nse

time.

Info

rmat

ion/

trai

ning

is p

rovi

ded

for

all

facu

lty m

embe

rs s

o co

urse

syl

labi

in

clud

e ke

y ex

pect

atio

ns a

nd g

radi

ng

polic

ies;

the

prog

ram

/inst

itutio

n ha

s a

requ

ired

facu

lty r

espo

nse

time;

the

resp

onse

tim

e is

reg

ular

ly e

valu

ated

.

4N

o ev

iden

ce e

xist

s w

hich

illu

stra

tes

that

pro

vidi

ng li

nks

or e

xpla

natio

ns o

f te

chni

cal s

uppo

rt a

re u

sed

in b

lend

ed

clas

ses.

Som

e bl

ende

d co

urse

s pr

ovid

e lin

ks

or e

xpla

natio

ns o

f tec

hnic

al s

uppo

rt.

The

maj

ority

of b

lend

ed c

ours

es

prov

ide

links

or

expl

anat

ions

of

tech

nica

l sup

port

.

All

blen

ded

cour

ses

cons

iste

ntly

pr

ovid

e lin

ks, e

xpla

natio

ns o

r F

AQ

's

of te

chni

cal s

uppo

rt.

5N

o ev

iden

ce is

pro

vide

d w

hich

su

ppor

ts a

focu

s on

rul

es o

r st

anda

rds

of b

ehav

ior

with

in th

e pr

ogra

m/ i

nstit

utio

n.

Whi

le r

ules

or

stan

dard

s ex

ist w

ithin

th

e pr

ogra

m/ i

nstit

utio

n, th

ey a

re

broa

d w

ith li

mite

d ap

plic

atio

n to

bl

ende

d co

urse

s.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d ru

les,

sta

ndar

ds, o

r co

des

of c

ondu

ct fo

cusi

ng o

n ap

prop

riate

st

uden

t beh

avio

r fo

r bl

ende

d st

uden

ts.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d ru

les,

sta

ndar

ds, o

r co

des

of c

ondu

ct fo

cusi

ng o

n ap

prop

riate

st

uden

t beh

avio

r fo

r bl

ende

d st

uden

ts a

nd m

akes

thos

e co

nsis

tent

ly a

vaila

ble

(thr

ough

the

LMS

or

othe

r m

eans

) fo

r al

l cou

rses

.

6A

t the

pre

sent

tim

e, n

o at

tent

ion

has

been

giv

en to

the

acce

ssib

ility

of

inst

ruct

iona

l mat

eria

ls.

The

pro

gram

/ ins

titut

ion

has

a pl

an to

en

sure

inst

ruct

iona

l mat

eria

ls fo

r al

l co

urse

s ar

e ea

sily

acc

essi

ble;

eas

y to

use

; can

be

acce

ssed

by

mul

tiple

op

erat

ing

syst

ems.

The

pro

gram

/ ins

titut

ion

has

a pr

oces

s to

ens

ure

inst

ruct

iona

l m

ater

ials

for

all c

ours

es a

re e

asily

ac

cess

ible

; eas

y to

use

; can

be

acce

ssed

by

mul

tiple

ope

ratin

g sy

stem

s; h

owev

er, i

t is

not

cons

iste

ntly

dep

loye

d ac

ross

all

cour

ses.

The

pro

gram

/ ins

titut

ion

has

cons

iste

nt b

usin

ess

proc

esse

s to

en

sure

inst

ruct

iona

l mat

eria

ls fo

r al

l co

urse

s ar

e ea

sily

acc

essi

ble;

eas

y to

use

; can

be

acce

ssed

by

mul

tiple

op

erat

ing

syst

ems;

and

UX

des

ign

prin

cipl

es a

re in

here

nt to

cou

rse

desi

gn.

Th

e co

urs

e st

ruct

ure

en

sure

s th

at a

ll st

ud

ents

, re

gar

dle

ss o

f lo

cati

on

, hav

e ac

cess

to

lib

rary

/ le

arn

ing

res

ou

rces

th

at a

deq

uat

ely

sup

po

rt t

he

ble

nd

ed c

ou

rse.

*

Th

e b

len

ded

co

urs

e in

clu

des

a s

ylla

bu

s o

utl

inin

g c

ou

rse

ob

ject

ives

, lea

rnin

g

ou

tco

mes

, eva

luat

ion

met

ho

ds,

bo

oks

an

d

sup

plie

s, t

ech

nic

al a

nd

pro

cto

rin

g

req

uir

emen

ts, a

nd

oth

er r

elat

ed c

ou

rse

info

rmat

ion

, mak

ing

co

urs

e re

qu

irem

ents

an

d

cou

rse

sch

edu

le t

ran

spar

ent.

*

Exp

ecta

tio

ns

for

stu

den

t as

sig

nm

ent

com

ple

tio

n, g

rad

e p

olic

y, a

nd

fac

ult

y re

spo

nse

ar

e cl

earl

y p

rovi

ded

in t

he

cou

rse

sylla

bu

s.*

Lin

ks o

r ex

pla

nat

ion

s o

f te

chn

ical

su

pp

ort

are

av

aila

ble

in t

he

cou

rse

(i.e

., ea

ch c

ou

rse

pro

vid

es s

ug

ges

ted

so

luti

on

s to

po

ten

tial

te

chn

ical

issu

es a

nd

/or

links

fo

r te

chn

ical

R

ule

s o

r st

and

ard

s fo

r ap

pro

pri

ate

stu

den

t b

ehav

ior,

bo

th o

nlin

e an

d f

ace-

to-f

ace,

are

p

rovi

ded

wit

hin

th

e co

urs

e.

Inst

ruct

ion

al m

ater

ials

are

eas

ily a

cces

sib

le t

o

the

stu

den

t, e

asy

to u

se, w

ith

an

ab

ility

to

be

acce

ssed

by

mu

ltip

le o

per

atin

g s

yste

ms

and

ap

plic

atio

ns.

CO

UR

SE

ST

RU

CT

UR

E (

24 p

oint

s)

Blended Learning Scorecard Rubric 103

Page 116: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

7N

o ev

iden

ce e

xist

s w

hich

illu

stra

tes

that

the

prog

ram

/ ins

titut

ion

is

prep

ared

to o

ffer

inst

ruct

iona

l m

ater

ials

that

are

eas

ily a

cces

sibl

e by

stu

dent

s w

ith d

isab

ilitie

s.

The

pro

gram

/ ins

titut

ion

has

ad h

oc

busi

ness

pro

cess

es to

ens

ure

inst

ruct

iona

l mat

eria

ls fo

r al

l cou

rses

ar

e ea

sily

acc

essi

ble

by s

tude

nts

with

di

sabi

litie

s.

The

pro

gram

/ ins

titut

ion

has

cons

iste

nt b

usin

ess

proc

ess

to

ensu

re in

stru

ctio

nal m

ater

ials

for

all

cour

ses

are

easi

ly a

cces

sibl

e by

st

uden

t with

dis

abili

ties.

Tra

inin

g is

pro

vide

d to

facu

lty a

nd th

e pr

ogra

m/ i

nstit

utio

n ha

s co

nsis

tent

bu

sine

ss p

roce

ss to

ens

ure

inst

ruct

iona

l mat

eria

ls fo

r al

l cou

rses

ar

e ea

sily

acc

essi

bly

by s

tude

nts

with

di

sabi

litie

s; p

rovi

de li

nks

to r

elev

ant

reso

urce

s.

8T

he p

rogr

am/in

stitu

tion

does

not

ove

rsee

th

e ae

sthe

tics

and

navi

gatio

n in

the

onlin

e co

mpo

nent

of b

lend

ed c

ours

es.

The

pro

gram

/inst

itutio

n pr

ovid

es

som

e tr

aini

ng a

bout

aes

thet

ics

and

navi

gatio

n fo

r fa

culty

des

igni

ng

blen

ded

cour

ses,

but

ther

e is

no

mon

itorin

g pr

oces

s to

ens

ure

qual

ity.

The

pro

gram

/inst

itutio

n pr

ovid

es

som

e tr

aini

ng a

bout

aes

thet

ics

and

navi

gatio

n fo

r fa

culty

des

igni

ng

blen

ded

cour

ses,

alo

ng w

ith

reco

mm

enda

tions

to m

eet

requ

irem

ents

.

The

pro

gram

/inst

itutio

n pr

ovid

es

trai

ning

with

spe

cific

gui

danc

e ab

out

aest

hetic

s an

d na

viga

tion

in th

e on

line

envi

ronm

ent,

incl

udin

g re

com

men

datio

ns to

mee

t re

quire

men

ts. T

he p

rogr

am/

inst

itutio

n re

gula

rly r

evie

ws

cour

ses.

Th

e b

len

ded

co

urs

e is

vis

ual

ly a

pp

ealin

g t

o t

he

stu

den

t an

d t

he

cou

rse

is n

avig

atio

nal

ly s

ou

nd

.

Inst

ruct

ion

al m

ater

ials

are

eas

ily a

cces

sed

by

stu

den

ts w

ith

dis

abili

ties

via

alt

ern

ativ

e in

stru

ctio

nal

str

ateg

ies

and

/or

refe

rral

to

sp

ecia

l in

stit

uti

on

al r

eso

urc

es.

104 Quality Scorecard for Blended Learning Programs

Page 117: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

he p

rogr

am/in

stitu

tion

does

not

ha

ve a

an

orga

nize

d m

eans

of

com

mun

icat

ing

expe

ctat

ions

re

gard

ing

inte

ract

ion

to fa

culty

.

Ent

repr

eneu

rial f

acul

ty m

embe

rs a

re

expl

orin

g va

rious

opp

ortu

nitie

s/to

ols

to e

ncou

rage

stu

dent

s an

d st

uden

t-fa

culty

inte

ract

ion

on a

cla

ss-b

y-cl

ass

basi

s.

Tra

inin

g is

pro

vide

d to

facu

lty to

sh

owca

se o

ppor

tuni

ties/

tool

s av

aila

ble

to e

ncou

rage

stu

dent

-st

uden

t and

stu

dent

-fac

ulty

in

tera

ctio

n an

d th

e pr

ogra

m/in

stitu

tion

enco

urag

es a

nd

faci

litat

es in

tera

ctio

n.

Tra

inin

g is

pro

vide

d to

facu

lty to

sh

owca

se o

ppor

tuni

ties/

tool

s av

aila

ble

to e

ncou

rage

stu

dent

-st

uden

t and

stu

dent

-fac

ulty

in

tera

ctio

n; th

e pr

ogra

m/in

stitu

tion

enco

urag

es a

nd fa

cilit

ates

in

tera

ctio

n; s

tude

nts

are

prov

ided

re

quire

men

ts o

r st

anda

rds

for

inte

ract

ion;

inte

ract

ion

is a

sses

sed;

su

ppor

t is

prov

ided

as

need

ed to

as

sist

facu

lty m

embe

rs in

eva

luat

ing

and

adop

ting

new

tech

nolo

gies

.

2T

he p

rogr

am/in

stitu

tion

prov

ides

no

polic

y or

rec

omm

enda

tion

to fa

culty

m

embe

rs c

once

rnin

g pr

ovid

ing

stud

ents

in b

lend

ed c

ours

es,

cons

truc

tive

and

timel

y fe

edba

ck.

Fac

ulty

mem

bers

wor

k in

depe

nden

tly

to e

nsur

e fe

edba

ck is

con

stru

ctiv

e an

d pr

ovid

ed in

a ti

mel

y m

anne

r.

Fac

ulty

mem

bers

rec

eive

trai

ning

ab

out p

rovi

ding

tim

ely

(wha

t are

the

reco

mm

ende

d tim

efra

mes

) an

d co

nstr

uctiv

e fe

edba

ck (

use

of r

ubric

s,

etc.

); m

ost c

ours

e sy

llabi

pro

vide

an

over

view

of f

eedb

ack

timel

ines

.

Fac

ulty

mem

bers

rec

eive

trai

ning

co

ncer

ning

pro

vidi

ng ti

mel

y (w

hat a

re

the

reco

mm

ende

d tim

efra

mes

) an

d co

nstr

uctiv

e fe

edba

ck (

use

of r

ubric

s,

etc.

); a

ll co

urse

syl

labi

pro

vide

an

over

view

of f

eedb

ack

timel

ines

.

3T

he p

rogr

am/in

stitu

tion

prov

ides

no

supp

ort,

trai

ning

, res

ourc

es, o

r po

licy

for

facu

lty m

embe

rs c

once

rnin

g st

rate

gies

for

crea

ting

cour

se

pres

ence

.

The

pro

gram

/inst

itutio

n ha

s co

mm

unic

ated

to fa

culty

the

impo

rtan

ce o

f ins

truc

tor

pres

ence

in

both

the

face

-to-

face

and

onl

ine

mod

aliti

es o

f the

cou

rse,

but

has

not

pr

ovid

ed tr

aini

ng, g

uide

lines

, or

reso

urce

s.

The

pro

gram

/inst

itutio

n ha

s co

mm

unic

ated

to fa

culty

the

impo

rtan

ce o

f ins

truc

tor

pres

ence

in

both

the

face

-to-

face

and

onl

ine

mod

aliti

es o

f the

cou

rse;

som

e re

sour

ces,

but

not

trai

ning

or

guid

elin

es, h

as b

een

prov

ided

.

The

pro

gram

/inst

itutio

n ha

s co

mm

unic

ated

to fa

culty

the

impo

rtan

ce o

f ins

truc

tor

pres

ence

in

both

cou

rse

mod

aliti

es, a

nd h

as

prov

ided

mod

els,

res

ourc

es, t

rain

ing,

an

d gu

idel

ines

. Ins

truc

tor

cour

se

pres

ence

is c

onsi

dere

d ve

ry

impo

rtan

t.

4T

he p

rogr

am/in

stitu

tion

prov

ides

no

supp

ort,

trai

ning

, res

ourc

es, o

r po

licy

for

facu

lty m

embe

rs c

once

rnin

g "c

ohes

ion"

in a

ble

nded

cou

rse.

The

pro

gram

/inst

itutio

n ha

s co

mm

unic

ated

to fa

culty

the

impo

rtan

ce o

f a c

ohes

ive

pres

ence

in

a bl

ende

d co

urse

, but

has

not

pr

ovid

ed tr

aini

ng, g

uide

lines

, or

reso

urce

s.

The

pro

gram

/inst

itutio

n ha

s co

mm

unic

ated

to fa

culty

the

impo

rtan

ce o

f coh

esiv

enes

s be

twee

n m

odal

ities

of t

he c

ours

e; s

ome

reso

urce

s, b

ut n

ot tr

aini

ng o

r gu

idel

ines

, has

bee

n pr

ovid

ed.

The

pro

gram

/inst

itutio

n ha

s co

mm

unic

ated

to fa

culty

the

impo

rtan

ce o

f coh

esiv

enes

s be

twee

n co

urse

mod

aliti

es, a

nd h

as p

rovi

ded

mod

els,

res

ourc

es, t

rain

ing,

and

gu

idel

ines

. Ins

truc

tor

cour

se

pres

ence

is c

onsi

dere

d ve

ry

impo

rtan

t.

5T

he p

rogr

am/in

stitu

tion

prov

ides

no

supp

ort t

o fa

culty

with

reg

ard

to

help

ing

stud

ents

lear

n ap

prop

riate

m

etho

ds fo

r ef

fect

ive

rese

arch

.

The

pro

gram

/inst

itutio

n ha

s a

plan

to

prov

ide

tuto

rials

, web

res

ourc

es, o

r ot

her

form

s to

hel

p st

uden

ts le

arn

appr

opria

te m

etho

ds fo

r ef

fect

ive

rese

arch

; how

ever

, the

y ha

ve y

et to

de

ploy

the

plan

.

The

pro

gram

/inst

itutio

n cr

eate

s va

rious

tuto

rials

, web

res

ourc

es, o

r fo

rms

to h

elp

stud

ents

lear

n ap

prop

riate

met

hods

for

effe

ctiv

e re

sear

ch a

nd fa

culty

are

offe

red

trai

ning

.

The

pro

gram

/inst

itutio

n cr

eate

s va

rious

tuto

rials

, web

res

ourc

es, o

r fo

rms

to te

ach

appr

opria

te m

etho

ds

for

effe

ctiv

e re

sear

ch; f

acul

ty

mem

bers

are

aw

are

of th

ese

reso

urce

s an

d co

nsis

tent

ly p

rovi

de

links

to th

ese

tool

s; fa

culty

als

o pr

ovid

e in

form

atio

n on

how

to a

sses

s th

e va

lidity

of r

esou

rces

. Fac

ulty

are

of

fere

d tr

aini

ng.

Stu

den

t-to

-Stu

den

t an

d F

acu

lty-

to-S

tud

ent

inte

ract

ion

are

ess

enti

al c

har

acte

rist

ics

and

ar

e en

cou

rag

ed a

nd

fac

ilita

ted

.*

Inst

ruct

or

feed

bac

k o

n s

tud

ent

assi

gn

men

ts

and

qu

esti

on

s is

co

nst

ruct

ive

and

pro

vid

ed in

a

tim

ely

man

ner

.*

Inst

ruct

ors

use

sp

ecif

ic s

trat

egie

s to

cre

ate

an

eng

aged

, lea

rnin

g-f

ocu

sed

pre

sen

ce in

bo

th

mo

dal

itie

s o

f th

e co

urs

e.

Res

ou

rces

are

pro

vid

ed t

o a

ssis

t st

ud

ents

in

con

du

ctin

g r

esea

rch

on

line

and

ass

essi

ng

th

e va

lidit

y o

f o

nlin

e re

sou

rces

.*

Fac

ult

y te

ach

th

e co

urs

e as

on

e co

hes

ive

wh

ole

, wit

h "

pre

sen

ce"

in b

oth

th

e fa

ce-t

o-f

ace

and

th

e o

nlin

e p

ort

ion

s o

f th

e co

urs

e.

TE

AC

HIN

G A

ND

LE

AR

NIN

G (

15 p

oint

s)

Blended Learning Scorecard Rubric 105

Page 118: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

he p

rogr

am/in

stitu

tion

has

limite

d te

chni

cal r

esou

rces

and

can

pro

vide

lit

tle te

chni

cal s

uppo

rt.

The

pro

gram

/inst

itutio

n ha

s m

inim

al

tech

nica

l sup

port

res

ourc

es to

ass

ist

facu

lty m

embe

rs d

urin

g bo

th c

ours

e de

velo

pmen

t and

teac

hing

.

The

pro

gram

/ ins

titut

ion

has

adeq

uate

tech

nica

l sup

port

re

sour

ces

to a

ssis

t fac

ulty

mem

bers

du

ring

both

cou

rse

deve

lopm

ent a

nd

teac

hing

.

The

pro

gram

/inst

itutio

n ha

s ad

equa

te

tech

nica

l sup

port

res

ourc

es to

ass

ist

facu

lty m

embe

rs d

urin

g bo

th c

ours

e de

velo

pmen

t and

teac

hing

; tut

oria

ls

and

web

res

ourc

es h

ave

been

cr

eate

d to

aug

men

t fac

ulty

nee

ds;

the

tech

nica

l sup

port

res

ourc

es a

re

cons

ider

ed ‘m

issi

on c

ritic

al.’

2N

o ev

iden

ce is

pro

vide

d co

ncer

ning

fa

culty

trai

ning

.A

facu

lty d

evel

opm

ent p

lan

is in

pl

ace

and

som

e ad

hoc

trai

ning

oc

curs

.

The

pro

gram

/inst

itutio

n pr

ovid

es

regu

lar

trai

ning

bas

ed o

n sp

ecifi

c co

urse

or

prog

ram

nee

ds.

The

pro

gram

/inst

itutio

n pr

ovid

es

regu

lar,

com

preh

ensi

ve a

nd

cons

iste

nt c

ours

e de

velo

pmen

t su

ppor

t, tr

aini

ng a

nd o

ngoi

ng

assi

stan

ce (

e.g.

, men

torin

g pr

ogra

ms)

and

the

trai

ning

is

prov

ided

usi

ng d

iffer

ent m

odel

s of

de

liver

y (e

.g.,

virt

ual m

odul

es,

hand

outs

, liv

e tr

aini

ng).

3N

o ev

iden

ce e

xist

s th

at fa

culty

m

embe

rs a

re p

rovi

ded

trai

ning

or

othe

r m

ater

ials

rel

ated

to F

air

Use

, pl

agia

rism

, etc

.

Som

e lim

ited

trai

ning

exi

sts

for

facu

lty m

embe

rs.

Fac

ulty

mem

bers

are

pro

vide

d tr

aini

ng, c

heck

lists

, tip

she

ets

and

enga

ge in

dis

cuss

ions

con

cern

ing

Fai

r U

se, p

lagi

aris

m a

nd o

ther

re

leva

nt le

gal a

nd e

thic

al c

once

pts.

Fac

ulty

mem

bers

are

pro

vide

d tr

aini

ng, c

heck

lists

, tip

she

ets

and

enga

ge in

dis

cuss

ions

con

cern

ing

Fai

r U

se, p

lagi

aris

m a

nd o

ther

re

leva

nt le

gal a

nd e

thic

al c

once

pts;

th

e pr

ogra

m/ i

nstit

utio

n ha

s a

busi

ness

pro

cess

in p

lace

to a

sses

s co

mpl

ianc

e.

4N

o ev

iden

ce is

pro

vide

d w

hich

su

ppor

ts a

com

mitm

ent t

o on

goin

g pr

ofes

sion

al d

evel

opm

ent f

or fa

culty

m

embe

rs te

achi

ng b

lend

ed c

ours

es.

The

pro

gram

/inst

itutio

n pr

ovid

es

prof

essi

onal

dev

elop

men

t for

facu

lty

mem

bers

on

an a

d ho

c ba

sis.

The

pro

gram

/inst

itutio

n pr

ovid

es

ongo

ing

prof

essi

onal

dev

elop

men

t for

fa

culty

mem

bers

focu

sed

on b

lend

ed

teac

hing

and

lear

ning

.

The

pro

gram

/inst

itutio

n pr

ovid

es

ongo

ing

prof

essi

onal

dev

elop

men

t for

fa

culty

mem

bers

focu

sed

on b

lend

ed

teac

hing

and

lear

ning

; dev

elop

men

t op

port

uniti

es a

re p

rovi

ded

thro

ugh

vario

us d

eliv

ery

form

ats;

the

need

s of

fa

culty

mem

bers

are

con

side

red

whe

n de

velo

ping

a tr

aini

ng s

ched

ule

of to

pics

.

5T

o da

te, n

o di

scus

sion

s or

pla

nnin

g w

as e

vide

nt w

ith r

egar

d to

es

tabl

ishi

ng c

lear

ing

stan

dard

s.

The

pro

gram

/inst

itutio

n is

in th

e pr

oces

s of

est

ablis

hing

cle

ar

stan

dard

s fo

r fa

culty

eng

agem

ent

and

expe

ctat

ions

.

The

pro

gram

/inst

itutio

n ha

s es

tabl

ishe

d cl

ear

stan

dard

s fo

r fa

culty

eng

agem

ent a

nd

expe

ctat

ions

; ens

ure

appr

opria

te

pers

onne

l and

sys

tem

s ar

e in

pla

ce

to c

omm

unic

ate

stan

dard

s an

d m

onito

r fa

culty

per

form

ance

.

The

pro

gram

/inst

itutio

n ha

s es

tabl

ishe

d cl

ear

stan

dard

s fo

r fa

culty

eng

agem

ent a

nd

expe

ctat

ions

; ens

ure

appr

opria

te

pers

onne

l and

sys

tem

s ar

e in

pla

ce

to c

omm

unic

ate

stan

dard

s an

d m

onito

r fa

culty

per

form

ance

; cre

ate

and

impl

emen

t ble

nded

facu

lty

cert

ifica

tion

cour

ses;

eng

age

a co

nsis

tent

per

form

ance

rev

iew

pr

oces

s.

FA

CU

LT

Y S

UP

PO

RT

(18

poi

nts)

Cle

ar s

tan

dar

ds

are

esta

blis

hed

fo

r fa

cult

y en

gag

emen

t an

d e

xpec

tati

on

s ar

ou

nd

ble

nd

ed

teac

hin

g (

e.g

. res

po

nse

tim

e, c

on

tact

in

form

atio

n, e

tc.)

.

Fac

ult

y ar

e p

rovi

ded

on

-go

ing

pro

fess

ion

al

dev

elo

pm

ent

rela

ted

to

ble

nd

ed t

each

ing

an

d

lear

nin

g.

Th

e in

stit

uti

on

en

sure

s fa

cult

y re

ceiv

e tr

ain

ing

, as

sist

ance

, an

d s

up

po

rt t

o p

rep

are

facu

lty

for

cou

rse

dev

elo

pm

ent

and

eff

ecti

ve t

each

ing

w

ith

tec

hn

olo

gy

in a

var

iety

of

mo

dal

itie

s.*

Fac

ult

y re

ceiv

e tr

ain

ing

an

d m

ater

ials

rel

ated

to

Fai

r U

se, p

lag

iari

sm, a

nd

oth

er r

elev

ant

leg

al

and

eth

ical

co

nce

pts

.*

Tec

hn

ical

ass

ista

nce

is p

rovi

ded

fo

r fa

cult

y b

efo

re a

nd

du

rin

g b

len

ded

co

urs

e d

evel

op

men

t an

d t

each

ing

.*

106 Quality Scorecard for Blended Learning Programs

Page 119: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

6N

o ev

iden

ce is

pro

vide

d w

hich

su

ppor

ts a

com

mitm

ent t

o on

goin

g pr

ofes

sion

al d

evel

opm

ent f

or fa

culty

m

embe

rs te

achi

ng b

lend

ed c

ours

es.

The

pro

gram

/inst

itutio

n pr

ovid

es

prof

essi

onal

dev

elop

men

t for

facu

lty

mem

bers

on

an a

d ho

c ba

sis.

The

pro

gram

/inst

itutio

n pr

ovid

es

ongo

ing

prof

essi

onal

dev

elop

men

t for

fa

culty

mem

bers

focu

sed

on b

lend

ed

teac

hing

and

lear

ning

.

The

pro

gram

/inst

itutio

n pr

ovid

es

ongo

ing

prof

essi

onal

dev

elop

men

t for

fa

culty

mem

bers

focu

sed

on b

lend

ed

teac

hing

and

lear

ning

; dev

elop

men

t op

port

uniti

es a

re p

rovi

ded

thro

ugh

vario

us d

eliv

ery

form

ats;

the

need

s of

fa

culty

mem

bers

are

con

side

red

whe

n de

velo

ping

a tr

aini

ng s

ched

ule

of to

pics

.

Fac

ult

y ar

e p

rovi

ded

tra

inin

g in

ble

nd

ed

teac

hin

g.

Blended Learning Scorecard Rubric 107

Page 120: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

here

is n

o ev

iden

ce to

sup

port

st

uden

t's a

re a

dvis

ed a

bout

ble

nded

pr

ogra

m r

eadi

ness

A d

efin

ed a

dvis

ing

proc

ess

is in

pl

ace

and

embe

dded

into

bus

ines

s pr

oces

ses

befo

re a

lear

ner

star

ts

blen

ded

clas

ses

A d

efin

ed a

dvis

ing

proc

ess

is in

pl

ace

and

embe

dded

into

bus

ines

s pr

oces

ses

befo

re a

lear

ner

star

ts

blen

ded

clas

ses;

dev

elop

s pr

oces

ses

for

lear

ners

to s

elf-

asse

ss th

eir

mot

ivat

ion

and

com

mitm

ent

A d

efin

ed a

dvis

ing

proc

ess

is in

pl

ace

and

embe

dded

into

bus

ines

s pr

oces

ses

befo

re a

lear

ner

star

ts

blen

ded

clas

ses;

pro

cess

es a

re

deve

lope

d fo

r le

arne

rs to

sel

f-as

sess

m

otiv

atio

n an

d co

mm

itmen

t; tu

toria

ls

and

chec

klis

ts a

re p

rovi

ded;

'tes

t dr

ives

' or

othe

r de

cisi

on s

uppo

rt to

ols

are

offe

red.

2T

here

is n

o ev

iden

ce th

at s

tude

nt's

ar

e ad

vise

d ab

out m

inim

um

tech

nolo

gy s

kills

or

equi

pmen

t

Info

rmat

ion

avai

labl

e to

pro

spec

tive

stud

ents

is in

com

plet

e an

d/or

not

av

aila

ble

befo

re e

nrol

ling

Info

rmat

ion

avai

labl

e to

pro

spec

tive

stud

ents

is c

ompl

ete

but n

ot

avai

labl

e pr

ior

to e

nrol

ling

and

star

ting

a bl

ende

d le

arni

ng p

rogr

am

Pro

spec

tive

stud

ents

are

pro

vide

d pr

ogra

m in

form

atio

n so

they

can

m

ake

info

rmed

dec

isio

ns b

efor

e en

rolli

ng; n

eces

sary

info

rmat

ion

is

easy

to fi

nd o

n th

e w

eb s

ite.

3T

here

is n

o ev

iden

ce th

at s

tude

nts

rece

ive

impo

rtan

t inf

orm

atio

n be

fore

st

artin

g a

blen

ded

prog

ram

.

Info

rmat

ion

avai

labl

e to

pro

spec

tive

stud

ents

is in

com

plet

e an

d/or

not

av

aila

ble

befo

re e

nrol

ling.

Info

rmat

ion

avai

labl

e to

pro

spec

tive

stud

ents

is c

ompl

ete,

but

not

alw

ays

avai

labl

e pr

ior

to e

nrol

ling

and

star

ting

a bl

ende

d pr

ogra

m.

Pro

spec

tive

stud

ents

are

pro

vide

d pr

ogra

m in

form

atio

n so

they

can

m

ake

info

rmed

dec

isio

ns b

efor

e en

rolli

ng; n

eces

sary

info

rmat

ion

is

easy

to fi

nd o

n th

e w

ebsi

te.

4T

he li

brar

y ha

s no

tuto

rials

, web

re

sour

ces

or fo

rms

to s

uppo

rt

stud

ents

enr

olle

d in

ble

nded

cou

rses

or

pro

gram

s.

Som

e bl

ende

d co

urse

s in

clud

e in

form

atio

n on

how

to a

cces

s lib

rary

pr

ofes

sion

als

whe

ther

they

are

ge

nera

lists

or

spec

ialis

ts; l

imite

d tu

toria

ls, w

eb r

esou

rces

or

form

s ar

e m

ade

avai

labl

e on

line

for

stud

ents

in

blen

ded

cour

ses

or p

rogr

ams.

The

libr

ary

has

vario

us tu

toria

ls, w

eb

reso

urce

s or

form

s to

hel

p st

uden

ts

lear

n ho

w to

use

all

of th

e lib

rary

re

sour

ces

(e.g

., in

terli

brar

y lo

an,

elec

tron

ic d

ocum

ent d

eliv

ery,

e-

rese

rve

syst

em);

mos

t ble

nded

co

urse

s in

clud

e in

form

atio

n on

how

to

acc

ess

libra

ry p

rofe

ssio

nals

w

heth

er th

ey a

re g

ener

alis

ts o

r sp

ecia

lists

.

The

libr

ary

has

vario

us tu

toria

ls, w

eb

reso

urce

s or

form

s to

hel

p st

uden

ts

lear

n ho

w to

use

all

of th

e lib

rary

re

sour

ces

(e.g

., in

terli

brar

y lo

an,

elec

tron

ic d

ocum

ent d

eliv

ery,

e-

rese

rve

syst

em)

embe

dded

in e

ach

cour

se; a

ll bl

ende

d co

urse

s in

clud

e in

form

atio

n on

how

to a

cces

s lib

rary

pr

ofes

sion

als

whe

ther

they

are

ge

nera

lists

or

spec

ialis

ts.

5T

here

is n

o ev

iden

ce th

at s

tude

nts

have

acc

ess

to te

chni

cal a

ssis

tanc

e an

d su

ppor

t.

Stu

dent

s ha

ve a

cces

s to

tech

nica

l su

ppor

t, bu

t sup

port

is in

cons

iste

nt

and/

or

not p

rovi

ded

durin

g pe

ak

hour

s ba

sed

on th

e st

uden

t’s ti

me

zone

.

Stu

dent

s ha

ve a

cces

s to

con

sist

ent

tech

nica

l ass

ista

nce

and

supp

ort b

ut

inst

ruct

ions

for

acce

ssin

g su

ppor

t are

un

clea

r an

d ho

urs

are

limite

d bu

t are

pr

ovid

ed d

urin

g pe

ak h

ours

.

Stu

dent

s ha

ve a

cces

s to

con

sist

ent

tech

nica

l sup

port

and

hav

e cl

ear

inst

ruct

ions

on

how

to a

cces

s;

tech

nica

l sup

port

ser

vice

s ar

e pr

ovid

ed 2

4X7X

365.

6N

o ev

iden

ce is

pro

vide

d of

how

st

uden

ts m

ay a

ddre

ss th

eir

ques

tions

an

d is

sues

.

It is

unc

lear

how

all

type

s of

stu

dent

qu

estio

ns, p

robl

ems,

and

bug

re

port

ing

is h

andl

ed a

cros

s co

urse

s an

d pr

ogra

ms.

Cle

ar a

nd c

onsi

sten

t sup

port

is

avai

labl

e an

d bu

sine

ss p

roce

sses

su

rrou

ndin

g th

e fu

ll ra

nge

of s

tude

nt

issu

es (

e.g.

, que

stio

ns, p

robl

ems,

bu

g re

port

s, c

ompl

aint

s) is

do

cum

ente

d.

Cle

ar a

nd c

onsi

sten

t sup

port

is

avai

labl

e an

d bu

sine

ss p

roce

sses

su

rrou

ndin

g th

e fu

ll ra

nge

of s

tude

nt

issu

es (

e.g.

, que

stio

ns, p

robl

ems,

bu

g re

port

s, c

ompl

aint

s) is

do

cum

ente

d; s

uppo

rt s

taff

mon

itor

reem

ergi

ng is

sues

to e

nsur

e m

itiga

tion

of a

ny s

yste

mic

pro

blem

s.

Bef

ore

sta

rtin

g a

ble

nd

ed le

arn

ing

pro

gra

m,

stu

den

ts r

ecei

ve (

or

hav

e ac

cess

to

) in

form

atio

n a

bo

ut

pro

gra

ms,

incl

ud

ing

ad

mis

sio

n r

equ

irem

ents

, tu

itio

n a

nd

fee

s,

bo

oks

an

d s

up

plie

s, t

ech

nic

al a

nd

pro

cto

rin

g

req

uir

emen

ts, a

nd

stu

den

t su

pp

ort

ser

vice

s.*

Bef

ore

sta

rtin

g a

ble

nd

ed le

arn

ing

pro

gra

m,

stu

den

ts a

re a

dvi

sed

ab

ou

t th

e p

rog

ram

to

d

eter

min

e if

th

ey h

ave

acce

ss t

o t

he

min

imu

m

tech

no

log

y sk

ills

and

eq

uip

men

t re

qu

ired

by

the

cou

rse

des

ign

.*

Th

rou

gh

ou

t th

e d

ura

tio

n o

f th

e co

urs

e/p

rog

ram

, stu

den

ts h

ave

acce

ss t

o

trai

nin

g a

nd

info

rmat

ion

th

ey w

ill n

eed

to

se

cure

req

uir

ed m

ater

ials

th

rou

gh

ele

ctro

nic

d

atab

ases

, in

terl

ibra

ry lo

ans,

go

vern

men

t ar

chiv

es, n

ews

serv

ices

, an

d o

ther

so

urc

es.*

Th

rou

gh

ou

t th

e d

ura

tio

n o

f th

e co

urs

e/p

rog

ram

, stu

den

ts h

ave

acce

ss t

o

app

rop

riat

e te

chn

ical

ass

ista

nce

an

d t

ech

nic

al

sup

po

rt s

taff

.*

Su

pp

ort

per

son

nel

are

ava

ilab

le (

24/7

) to

ad

dre

ss s

tud

ent

qu

esti

on

s an

d p

rob

lem

s o

f a

tech

nic

al n

atu

re.*

Bef

ore

sta

rtin

g a

ble

nd

ed le

arn

ing

pro

gra

m,

stu

den

ts c

om

ple

te a

n o

rien

tati

on

or

self

- as

sess

men

t to

det

erm

ine

if t

hey

po

sses

s th

e se

lf-m

oti

vati

on

an

d c

om

mit

men

t to

lear

n.*

ST

UD

EN

T S

UP

PO

RT

(33

poi

nts)

108 Quality Scorecard for Blended Learning Programs

Page 121: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

7T

here

is n

o ev

iden

ce th

at s

tude

nts

with

dis

abili

ties

are

supp

orte

d.T

here

is e

vide

nce

of s

uppo

rt fo

r st

uden

ts w

ith d

isab

ilitie

s, b

ut p

olic

ies

and

proc

esse

s ar

e no

t evi

dent

.

The

pro

gram

/ ins

titut

ion

has

polic

ies,

pr

oces

ses

and

reso

urce

s to

sup

port

st

uden

ts w

ith d

isab

ilitie

s; h

owev

er

stud

ents

with

dis

abili

ties

rece

ive

inco

nsis

tent

sup

port

or

inst

ruct

ions

fo

r ac

cess

are

unc

lear

.

The

pro

gram

/inst

itutio

n ha

s co

nsis

tent

pol

icie

s, p

roce

sses

, and

re

sour

ces

to s

uppo

rt s

tude

nts

with

di

sabi

litie

s; it

is c

lear

that

the

inst

itutio

n is

wor

king

to in

corp

orat

e th

e re

leva

nt s

tand

ards

and

/or

best

pr

actic

es; d

ocum

enta

tion

incl

udes

sp

ecifi

c co

urse

info

rmat

ion

as w

ell a

s ho

w s

tude

nts

acce

ss r

elev

ant

serv

ices

.

8T

here

is n

o ev

iden

ce th

at s

tude

nts

have

acc

ess

to r

equi

red

mat

eria

ls

prio

r to

enr

ollm

ent.

Acc

ess

to r

equi

red

cour

se m

ater

ials

va

ries

by c

ours

e an

d is

inco

nsis

tent

.A

cces

s to

req

uire

d co

urse

mat

eria

ls

is c

onsi

sten

t acr

oss

the

prog

ram

, but

st

uden

ts m

ust b

e en

rolle

d fo

r ac

cess

.

Acc

ess

to r

equi

red

cour

se m

ater

ials

is

con

sist

ent a

nd s

tude

nts

have

ac

cess

prio

r to

enr

ollm

ent.

9T

here

is n

o ev

iden

ce o

f a s

tude

nt-

cent

ered

focu

s or

inte

ntio

nal

inte

grat

ion

of r

esou

rces

.

Stu

dent

-cen

tere

d su

ppor

t ser

vice

s ar

e ta

rget

ed p

rimar

ily fo

r on

-cam

pus

stud

ents

.

Stu

dent

-cen

tere

d su

ppor

t ser

vice

s ar

e av

aila

ble

for

stud

ents

in b

lend

ed

cour

ses

or p

rogr

ams,

but

evi

denc

e is

in

com

plet

e an

d/or

har

d to

loca

te.

Stu

dent

-cen

tere

d su

ppor

t ser

vice

s ar

e av

aila

ble

for

stud

ents

in b

lend

ed

cour

ses

or p

rogr

ams.

The

inst

itutio

n ac

tivel

y in

tegr

ates

sup

port

ser

vice

s ac

ross

the

cont

inuu

m o

f lea

rnin

g m

odes

, inc

ludi

ng b

lend

ed c

ours

es

and

prog

ram

s. S

ervi

ces

are

cont

inuo

usly

mon

itore

d an

d up

date

d to

ser

ve th

e sh

ift in

ble

nded

lear

ning

ap

proa

ches

.

10T

here

is n

o ev

iden

ce th

at s

tude

nts

are

prov

ided

with

gui

danc

e co

ncer

ning

use

of t

echn

olog

y.

The

re is

evi

denc

e th

at g

uida

nce

is

avai

labl

e fo

r st

uden

ts b

ut g

uida

nce

prov

ided

is c

onfu

sing

or

uncl

ear.

In

stru

ctio

ns fo

r ac

cess

may

be

need

ed.

Gui

danc

e is

pro

vide

d fo

r on

ly th

e pr

imar

y te

chno

logy

use

d fo

r co

urse

de

liver

y. In

stru

ctio

ns fo

r ac

cess

are

cl

ear.

Gui

danc

e, fa

ct s

heet

s, in

fogr

aphi

cs

are

prov

ided

for

all t

ypes

of

tech

nolo

gies

use

d in

cou

rsew

ork

and

may

incl

ude

vide

o tu

toria

ls.

Inst

ruct

ions

for

acce

ss a

re c

lear

and

co

nsis

tent

acr

oss

all c

ours

es

rega

rdle

ss o

f del

iver

y m

ode.

11T

here

is n

o ev

iden

ce th

at s

tude

nts

are

prov

ided

inst

ruct

ions

for

enlis

ting

help

.

Stu

dent

s ar

e pr

ovid

ed in

form

atio

n to

en

list h

elp

but s

ervi

ces

are

limite

d,

acce

ss is

unc

lear

and

/ or

min

imal

ch

anne

ls a

re a

vaila

ble.

Stu

dent

s ha

ve a

cces

s to

bot

h on

e-tim

e se

rvic

es a

nd r

epea

ted

serv

ices

, bu

t inf

orm

atio

n is

not

cle

ar a

nd/o

r ch

anne

ls a

re li

mite

d.

Stu

dent

s ha

ve c

lear

info

rmat

ion

in

orde

r to

acc

ess

both

one

-tim

e an

d re

peat

ed s

ervi

ces;

mul

tiple

cha

nnel

s (e

.g.,

e-m

ail,

phon

e, c

hat,

web

co

nfer

enci

ng)

are

prov

ided

for

enlis

ting

assi

stan

ce; s

ervi

ce

stan

dard

s ar

e m

onito

red

over

tim

e an

d im

prov

emen

ts m

ade.

Stu

den

ts h

ave

acce

ss t

o in

form

atio

n r

egar

din

g

req

uir

ed c

ou

rse

mat

eria

ls in

pri

nt

and

/or

dig

ital

fo

rmat

, su

ch a

s IS

BN

nu

mb

ers

for

text

bo

oks

, b

oo

k su

pp

liers

, an

d d

eliv

ery

mo

des

pri

or

to

cou

rse

enro

llmen

t.

Th

e in

stit

uti

on

pro

vid

es g

uid

ance

/tu

tori

als

for

stu

den

ts in

th

e u

se o

f al

l fo

rms

of

tech

no

log

ies

use

d f

or

cou

rse

del

iver

y.

Pro

gra

m d

emo

nst

rate

s a

stu

den

t-ce

nte

red

fo

cus

and

inte

nti

on

alit

y in

th

e in

teg

rati

on

of

on

line

and

fac

e-to

-fac

e re

sou

rces

.

Stu

den

ts a

re p

rovi

ded

cle

ar in

form

atio

n f

or

enlis

tin

g h

elp

fro

m t

he

inst

itu

tio

n.

Po

licy,

pro

cess

es, a

nd

res

ou

rces

are

in p

lace

to

su

pp

ort

stu

den

ts w

ith

dis

abili

ties

.

Blended Learning Scorecard Rubric 109

Page 122: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

1T

he p

rogr

am/ i

nstit

utio

n do

es n

ot

have

a p

lan

or p

roce

ss to

eva

luat

e th

e bl

ende

d pr

ogra

m

The

pro

gram

/ ins

titut

ion

has

deve

lope

d a

prog

ram

eva

luat

ion

plan

to

per

iodi

cally

ass

ess

the

prog

ram

. M

ostly

per

form

ed o

n ad

hoc

bas

is.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d or

follo

ws

spec

ific

stan

dard

s w

hich

are

con

sist

ently

and

pe

riodi

cally

use

d to

ass

ess/

eval

uate

th

e pr

ogra

m.

The

pro

gram

/inst

itutio

n ha

s de

velo

ped

or fo

llow

s sp

ecifi

c st

anda

rds

whi

ch a

re c

onsi

sten

tly a

nd

perio

dica

lly u

sed

to a

sses

s/ev

alua

te

the

prog

ram

; eva

luat

ion

resu

lts a

re

used

to im

prov

e bl

ende

d pr

ogra

m(s

).

2N

o ev

iden

ce e

xist

s th

at th

e pr

ogra

m/

inst

itutio

n ev

alua

tes

its b

lend

ed

prog

ram

usi

ng a

var

iety

of d

ata.

Ble

nded

pro

gram

eva

luat

ion

is ju

st

begi

nnin

g an

d lim

ited

acad

emic

or

adm

inis

trat

ive

mea

sure

s ar

e us

ed to

as

sess

pro

gram

effe

ctiv

enes

s.

Ble

nded

pro

gram

eva

luat

ion

incl

udes

so

me

acad

emic

(e.

g., c

ours

e ev

alua

tions

, lea

rnin

g ou

tcom

es

achi

evem

ent)

and

adm

inis

trat

ive

(e.g

., sa

tisfa

ctio

n su

rvey

s, s

tude

nt

succ

ess/

per

sist

ence

rat

es)

mea

sure

s to

ass

ess

effe

ctiv

enes

s.

Ble

nded

pro

gram

eva

luat

ion

incl

udes

a

wid

e va

riety

of a

cade

mic

(e.

g.,

cour

se e

valu

atio

ns, l

earn

ing

outc

omes

ach

ieve

men

t) a

nd

adm

inis

trat

ive

(e.g

., sa

tisfa

ctio

n su

rvey

s, s

tude

nt s

ucce

ss/

pers

iste

nce

rate

s) m

easu

res

to

asse

ss e

ffect

iven

ess;

the

eval

uatio

n oc

curs

reg

ular

ly a

nd fr

eque

ntly

; and

, re

sults

are

use

d to

gui

de c

hang

es.

3T

he p

rogr

am/in

stitu

tion

does

not

ha

ve a

rev

iew

pro

cess

to a

sses

s le

arni

ng o

utco

mes

.

Lear

ning

out

com

es a

re a

sses

sed

on

an a

d ho

c ba

sis

to e

nsur

e al

ignm

ent,

clar

ity, u

tility

, app

ropr

iate

ness

and

ef

fect

iven

ess.

Lear

ning

out

com

es a

re a

sses

sed

on

a re

gula

r ba

sis

to e

nsur

e al

ignm

ent,

clar

ity, u

tility

, app

ropr

iate

ness

and

ef

fect

iven

ess.

Lear

ning

out

com

es a

re a

sses

sed

on

a re

gula

r ba

sis

to e

nsur

e al

ignm

ent,

clar

ity, u

tility

, app

ropr

iate

ness

and

ef

fect

iven

ess;

a p

eer

revi

ew p

roce

ss

is u

sed;

out

com

es fr

om th

e pr

oces

s dr

ive

upda

tes.

4N

o pr

oces

s is

in p

lace

to a

sses

s su

ppor

t ser

vice

s fo

r fa

culty

and

st

uden

ts.

A p

roce

ss is

in p

lace

and

eng

aged

on

an

ad h

oc b

asis

to a

sses

s su

ppor

t se

rvic

es fo

r fa

culty

and

stu

dent

s.

A p

roce

ss is

in p

lace

and

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to

asse

ss s

uppo

rt s

ervi

ces

for

facu

lty

and

stud

ents

.

A p

roce

ss is

in p

lace

and

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to

asse

ss s

uppo

rt s

ervi

ces

for

facu

lty

and

stud

ents

; out

com

es s

erve

as

a fo

unda

tion

for

impr

ovem

ents

.

5N

o pr

oces

s is

in p

lace

to a

sses

s st

uden

t ret

entio

n.A

pro

cess

is in

pla

ce a

nd e

ngag

ed

on a

n ad

hoc

bas

is to

stu

dent

re

tent

ion.

A p

roce

ss is

in p

lace

and

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to

asse

ss s

tude

nt r

eten

tion.

A p

roce

ss is

in p

lace

and

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to

asse

ss s

tude

nt r

eten

tion;

out

com

es

serv

e as

a fo

unda

tion

for

impr

ovem

ents

.

6N

o pr

oces

s is

in p

lace

to r

egul

arly

re

view

acc

essi

bilit

y st

anda

rds/

Pro

gram

/ ins

titut

ion

has

desi

gnat

ed

pers

onne

l to

supp

ort a

cces

sibi

lity

need

s; a

pro

cess

is in

pla

ce a

nd

follo

wed

on

an a

d ho

c ba

sis

to

asse

ss a

cces

sibi

lity

stan

dard

s.

Pro

gram

/ ins

titut

ion

has

desi

gnat

ed

pers

onne

l to

supp

ort a

cces

sibi

lity

need

s; a

pro

cess

is in

pla

ce a

nd

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to a

sses

s ac

cess

ibili

ty

stan

dard

s.

Pro

gram

/ ins

titut

ion

has

desi

gnat

ed

pers

onne

l to

supp

ort a

cces

sibi

lity

need

s; a

pro

cess

is in

pla

ce a

nd

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to a

sses

s ac

cess

ibili

ty

stan

dard

s w

ith o

utco

mes

pub

lishe

d on

the

prog

ram

's w

ebsi

te; o

utco

mes

al

so s

erve

as

a fo

unda

tion

for

impr

ovem

ent.

Pro

gra

m d

emo

nst

rate

s co

mp

lian

ce a

nd

rev

iew

o

f ac

cess

ibili

ty s

tan

dar

ds

(Sec

tio

n 5

08, e

tc.)

.

Th

e p

rog

ram

is a

sses

sed

th

rou

gh

an

ev

alu

atio

n p

roce

ss t

hat

ap

plie

s sp

ecif

ic

esta

blis

hed

sta

nd

ard

s (i

.e.,

accr

edit

or

gu

idel

ines

an

d/o

r o

ther

rec

og

niz

ed a

gen

cy

such

as

the

OL

C S

core

card

).*

A p

roce

ss is

in p

lace

an

d f

ollo

wed

fo

r th

e co

mp

reh

ensi

ve a

sses

smen

t o

f su

pp

ort

se

rvic

es f

or

facu

lty

and

stu

den

ts.

A p

roce

ss is

in p

lace

an

d f

ollo

wed

fo

r th

e as

sess

men

t o

f st

ud

ent

rete

nti

on

in b

len

ded

co

urs

es a

nd

pro

gra

ms.

A v

arie

ty o

f d

ata

(aca

dem

ic a

nd

ad

min

istr

ativ

e in

form

atio

n)

are

use

d t

o r

egu

larl

y an

d

freq

uen

tly

eval

uat

e p

rog

ram

eff

ecti

ven

ess

in

ord

er t

o g

uid

e ch

ang

es t

ow

ard

co

nti

nu

al

imp

rove

men

t.*

Inte

nd

ed le

arn

ing

ou

tco

mes

at

the

cou

rse

and

p

rog

ram

leve

l are

rev

iew

ed r

egu

larl

y to

en

sure

al

ign

men

t, c

lari

ty, u

tilit

y, a

pp

rop

riat

enes

s, a

nd

ef

fect

iven

ess.

*

EV

AL

UA

TIO

N A

ND

AS

SE

SS

ME

NT

(30

poi

nts)

110 Quality Scorecard for Blended Learning Programs

Page 123: CRITERIA FOR EXCELLENCE IN BLENDED LEARNING …info2.onlinelearningconsortium.org › rs › 897-CSM-305...programs, the OLC introduced the Quality Scorecard for Blended Learning Programs

0=D

efic

ient

1=D

evel

opin

g2=

Acc

ompl

ishe

d3=

Exe

mpl

ary

7C

ours

e ev

alua

tions

are

not

reg

ular

ly

revi

ewed

.C

ours

e ev

alua

tions

and

oth

er

feed

back

(e.

g., p

eer

revi

ew r

esul

ts)

are

revi

ewed

on

an a

d ho

c ba

sis

to

asse

ss e

ffect

iven

ess

of in

stru

ctio

n;

feed

back

gle

aned

from

the

revi

ew is

us

ed to

sha

pe fa

culty

per

form

ance

ev

alua

tions

.

Cou

rse

eval

uatio

ns a

nd o

ther

fe

edba

ck (

e.g.

, pee

r re

view

res

ults

) ar

e re

view

ed r

egul

arly

to a

sses

s ef

fect

iven

ess

of in

stru

ctio

n; fe

edba

ck

glea

ned

from

the

revi

ew is

use

d to

sh

ape

facu

lty p

erfo

rman

ce

eval

uatio

ns.

Cou

rse

eval

uatio

ns a

nd o

ther

fe

edba

ck (

e.g.

, pee

r re

view

res

ults

) ar

e re

view

ed r

egul

arly

to a

sses

s ef

fect

iven

ess

of in

stru

ctio

n; fe

edba

ck

glea

ned

from

the

revi

ew is

use

d to

sh

ape

facu

lty p

erfo

rman

ce

eval

uatio

ns; a

lso

used

as

outli

ne fo

r fu

ture

facu

lty d

evel

opm

ent t

opic

s or

ot

her

impr

ovem

ents

.

8C

ours

e ev

alua

tions

are

not

col

lect

ed

and

used

to a

sses

s th

e qu

ality

of

blen

ded

cour

se m

ater

ials

.

A c

ompr

ehen

sive

cou

rse

eval

uatio

n is

dep

loye

d pe

riodi

cally

to c

olle

ct

stud

ent f

eedb

ack;

out

com

es s

erve

as

a fo

unda

tion

for

impr

ovem

ents

.

A c

ompr

ehen

sive

cou

rse

eval

uatio

n is

dep

loye

d fo

r ea

ch b

lend

ed c

ours

e to

col

lect

stu

dent

feed

back

; ou

tcom

es s

erve

as

a fo

unda

tion

for

impr

ovem

ents

.

A c

ompr

ehen

sive

cou

rse

eval

uatio

n is

dep

loye

d fo

r ea

ch b

lend

ed c

ours

e to

col

lect

stu

dent

feed

back

; ou

tcom

es s

erve

as

a fo

unda

tion

for

impr

ovem

ents

to c

ours

e qu

ality

; the

ev

alua

tion

tool

is p

erio

dica

lly

revi

ewed

for

effe

ctiv

enes

s an

d re

leva

nce.

9N

o pr

ogra

m/ i

nstit

utio

nal p

olic

ies

have

bee

n de

velo

ped

to g

uide

ex

pect

atio

ns fo

r fa

culty

per

form

ance

.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d po

licie

s to

gui

de

expe

ctat

ions

for

facu

lty p

erfo

rman

ce

and

the

polic

y/ s

tand

ards

are

sha

red

with

facu

lty m

embe

rs.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d po

licie

s to

gui

de

expe

ctat

ions

for

facu

lty p

erfo

rman

ce;

polic

y/ s

tand

ards

are

sha

red

with

fa

culty

; sta

ndar

ds a

re u

sed

perio

dica

lly to

eva

luat

e a

facu

lty

mem

ber's

per

form

ance

.

The

pro

gram

/ ins

titut

ion

has

deve

lope

d po

licie

s to

gui

de

expe

ctat

ions

for

facu

lty p

erfo

rman

ce;

polic

y/ s

tand

ards

are

sha

red

with

fa

culty

; the

se s

tand

ards

are

use

d to

ev

alua

te a

facu

lty m

embe

r's

perf

orm

ance

eac

h tim

e th

ey te

ach

in

a bl

ende

d m

odal

ity.

10N

o pr

oces

s is

in p

lace

to a

sses

s st

akeh

olde

r sa

tisfa

ctio

n.A

pro

cess

is in

pla

ce a

nd fo

llow

ed o

n an

ad

hoc

basi

s to

ass

ess

stak

ehol

der

satis

fact

ion.

A p

roce

ss is

in p

lace

and

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to

asse

ss s

take

hold

er s

atis

fact

ion.

A p

roce

ss is

in p

lace

and

follo

wed

on

a re

gula

r ba

sis

(e.g

., an

nual

ly)

to

asse

ss s

take

hold

er s

atis

fact

ion;

ou

tcom

es s

erve

as

a fo

unda

tion

for

impr

ovem

ents

.

Co

urs

e ev

alu

atio

ns

colle

ct s

tud

ent

feed

bac

k o

n

qu

alit

y o

f b

len

ded

co

urs

e m

ater

ials

.

A p

roce

ss is

in p

lace

an

d f

ollo

wed

fo

r th

e in

stit

uti

on

al a

sses

smen

t o

f fa

cult

y b

len

ded

te

ach

ing

per

form

ance

.

A p

roce

ss is

in p

lace

an

d f

ollo

wed

fo

r th

e as

sess

men

t o

f st

akeh

old

er (

e.g

., le

arn

ers,

fa

cult

y, s

taff

) sa

tisf

acti

on

wit

h t

he

ble

nd

ed

lear

nin

g p

rog

ram

s.

Co

urs

e ev

alu

atio

ns

colle

ct s

tud

ent

feed

bac

k o

n

the

effe

ctiv

enes

s o

f in

stru

ctio

n in

rel

atio

n t

o

facu

lty

per

form

ance

eva

luat

ion

s.

* A

dapt

ed fr

om In

stitu

te fo

r H

ighe

r E

duca

tion

Pol

icy'

s Q

ualit

y on

the

Line

: Ben

chm

ark

for S

ucce

ss in

Inte

rnet

-bas

ed D

ista

nce

Edu

catio

n (2

000)

.

Blended Learning Scorecard Rubric 111


Recommended