B L E N D E D L E A R N I N G P R O G R A M SC R I T E R I A F O R E X C E L L E N C E I N
Quality ScorecardO N L I N E L E A R N I N G C O N S O R T I U M
Quality Scorecard Handbook:
Criteria for Excellence in
Blended Learning Programs
Editors:
Jennifer Mathes
Karen Pedersen
The Online Learning Consortium
ThepurposeoftheOnlineLearningConsortiumistohelplearningorganizationscontinuallyimprovethequality,scale,andbreadthoftheironlineprogramsaccordingtotheirowndistinctivemissions,sothateducationwillbecomeapartofeverydaylife,accessibleandaffordableforanyone,anywhere,atanytime,inawidevarietyofdisciplines.
Thispublicationcontainsinformationobtainedfromauthenticandhighlyregardedsources.Reprintedmaterialisquotedwithpermission,andsourcesareindicated.Awidevarietyofreferencesarelisted.Reasonableeffortshavebeenmadetopublishreliabledataandinformation,buttheauthorsandthepublishercannotassumeresponsibilityforthevalidityofallmaterialsorfortheconsequencesoftheiruse.
Neitherthispublicationnoranypartmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,includingphotocopying,microfilming,andrecording,orbyanyinformationstorageorretrievalsystem,withoutpriorpermissioninwritingfromthepublisher.
TheconsentoftheOnlineLearningConsortiumdoesnotextendtocopyingforgeneraldistribution,forpromotion,forcreatingnewworks,orforresale.SpecificpermissionmustbeobtainedinwritingfromtheOnlineLearningConsortiumforsuchcopyingoruseinotherworks.Directallinquiriestopublisher@onlinelearning‐c.org.
Coverdesigncopyright©2016byOnlineLearningConsortium™Allrightsreserved.Published2016
PrintedintheUnitedStatesofAmerica
InternationalStandardBookNumber978‐1‐934505‐06‐9(digitalebook)
Permissions
TheQualityScorecardforBlendedLearningProgramshasmanyindicatorsthatareverysimilartotheQualityScorecardfortheAdministrationofOnlineLearning.Assuch,thereaderwillnoticethatdescriptionsformanyindicatorsinthisHandbookandaccompanyingrubricareidenticalorhighlysimilartothosefoundintheQualityScorecard2014Handbook:CriteriaforExcellenceintheAdministrationofOnlinePrograms.AsbotharepublishedbytheOnlineLearningConsortium,theeditorshavebeengrantedpermissiontoreuseappropriatecontentintheQualityScorecardHandbook:CriteriaforExcellenceinBlendedLearningProgramsrevisingasneededtomaintainrelevancetoblendedlearning.
Table of Contents
ExpertPanelists........................................................................................................................................................................vi
AbouttheEditors.....................................................................................................................................................................ix
Foreword.....................................................................................................................................................................................x
IntroductiontotheQualityScorecardforBlendedLearningPrograms...........................................................1
InstitutionalSupport...............................................................................................................................................................3
TechnologySupport.............................................................................................................................................................16
CourseDevelopment&InstructionalDesign.............................................................................................................25
CourseStructure....................................................................................................................................................................43
TeachingandLearning........................................................................................................................................................53
FacultySupport......................................................................................................................................................................60
StudentSupport.....................................................................................................................................................................68
EvaluationandAssessment...............................................................................................................................................81
BlendedQualityScorecardRubric..................................................................................................................................95
Expert Panelists OursincerestthanksgoestothefollowingblendedlearningprofessionalswhocontributedtothefirsteditionofQualityScorecardHandbook:CriteriaforExcellenceinBlendedLearningPrograms.Throughyourcontributionthisfirsteditionwillprovideagoodfoundationtosupportqualityinblendedlearningthroughouthighereducation.
PhyliseBannerClarksonUniversity
JenBradleyNorthamptonCommunityCollege
StevenCrawfordArizonaStateUniversity
DebMerriamDaemenCollege
MelanieShawNorthcentralUniversity
ThomasSlomkaUniversityofBuffalo
CourtneyStewartUtahStateUniversity
vi Quality Scorecard for Blended Learning Programs
TheOnlineLearningConsortiumQualityScorecardfortheAdministrationofBlendedLearningProgramsistheresultofaDelphiresearchstudyconductedbyDr.KayeSheltonin2015.Thankyoutothefollowingblendedlearningeducatorswhowillinglyparticipatedinthestudyandcontributedtheirexpertise:
VictoriaAbramenka
FortHaysStateUniversity
ReneeAitken
WrightStateUniversity
BarbaraAltman
TexasA&MUniversityCentralTexas
ElizabethAnderson
UniversityofDenver
PhyliseBanner
PhyliseBannerConsulting
RachelBehling
GrandCanyonUniversity
JenBradley
NorthamptonCommunityCollege
BrookeBurkle
RelayGraduateSchoolofEducation
LisaBunkowski
TexasA&MUniversityCentralTexas
TerryBuxton
RegisUniversity
MauricioCadavid
CaliforniaStateUniversity,SanBernardino
MargaretChristmas
ConcordiaUniversityIrvine
MarieCini
UniversityofMarylandUniversityCollege
RosemaryCleveland
GrandValleyStateUniversity
KristyConger
UniversityofTennessee
StevenCrawford
ArizonaStateUniversity
DarrylDraper
OldDominionUniversity
TimmoDugdale
UniversityofWisconsin‐Madison
LindaFutch
UniversityofCentralFlorida
JulieGolden
FloridaAtlanticUniversity
CharlesGraham
BrighamYoungUniversity
DougGrove
ConcordiaUniversityIrvine
NancyHale
PaceUniversity
LisaHalverson
BrighamYoungUniversity
CindyHayden
EasternKentuckyUniversity
JulieHinkle
UniversityofCentralFlorida
LynetteHudiburgh
MiamiUniversity
SandraJohnson
NewMexicoStateUniversity
Expert Panelists vii
KimberlyKenward
GrandValleyStateUniversity
ShannonMcCarty
RioSaladoCollege
PaigeMcDonald
TheGeorgeWashingtonUniversity
TawnyaMeans
UniversityofFlorida
DeborahMerriam
DaemenCollege
PatMielke
WesternTechnicalCollege
NoreenBarajas‐Murphy
UniversityofLaVerne
JasonNeben
ConcordiaUniversityIrvine
MaryNiemiec
UniversityofNebraska
SunayPalsole
TheUniversityofTexasatSanAntonio
RobertPaz
NewMexicoStateUniversity
BrendaPerea
ColoradoCommunityCollegeSystem
KarenPollack
ThePennsylvaniaStateUniversity
LauriePosey
TheGeorgeWashingtonUniversity
DonnaRedman
UniversityofLaVerne
LindaRuest
SouthernNewHampshireUniversity
MelanieShaw
NorthcentralUniversity
ChadShorter
UniversityofWisconsin‐Madison
ThomasSlomka
UniversityofBuffalo
CourtneyStewart
UtahStateUniversity
KarenSwan
UniversityofIllinoisSpringfield
KristenTaubman
NorthernArizonaUniversity
AlexisTerrell
OregonStateUniversity
JanetThompson
TheCollegeofWestchester
KelvinThompson
UniversityofCentralFlorida
BrianUdermann
UniversityofWisconsin–LaCrosse
viii Quality Scorecard for Blended Learning Programs
About the Editors Jennifer Mathes ‐ Dr.JenniferMathesistheDirectorofStrategicPartnershipsattheOnlineLearningConsortium(OLC).Inthisrole,sheprovidesleadershipinresearching,scoping,managing,andevaluatingkeyprojects,programs,andcustomizedsolutionsthatarealignedwithOLC’sAdvisoryServicesdomesticallyandinternationally.Dr.Matheshas20yearsofexperienceinbothpublicandprivatehighereducationwhereshehassupportedonlinelearninginitiativessinceshetaughtherfirstonlinecoursein1997.Shehasbeeninstrumentalinworkingwithstart‐uponlineinitiativesaswellasleadinggrowthininstitutionswithanexistingonlineprogram.SheholdsadoctorofphilosophydegreeineducationfromtheUniversityofIllinoisatUrbana‐Champaignwhereshewroteherdissertationon“PredictorsforStudentSuccessinOnlineEducation.”ShealsohasearnedamasterofsciencedegreeinbusinesseducationandabachelorofsciencedegreeinmasscommunicationsfromIllinoisStateUniversity.
Karen Pedersen ‐Dr.KarenPederseniscurrentlytheChiefKnowledgeOfficerfortheOnlineLearningConsortium(OLC)wheresheservesasathoughtleaderandpresenteronthetopicsofquality,learnersuccess,militaryeducationprograms,institutionaltransformation,andexcellence.PriortojoiningOLC,Pedersenservedasaseniorleaderinvariousadministrativepositionsforbothpublicandprivatehighereducationinstitutions.Sheledaward‐winningacademicoperations,marketingunits,technologyteams,andstudentsupportservicesdivisions.Duringhercareer,shehasenvisionedandbuiltasuccessfulonlineoperationfromthegroundupandlaunchedover25innovativeonlinedegreeprograms.Otherexperiencesincludeleadingasystem‐wideenrollmentmanagementtransformation,servingonthelaunchteamforacompetency‐basededucationinitiative,andbuildingacademicpartnershipsdomesticallyandinternationallyincludinginBrazil,HongKong,Indonesia,Malaysia,andSingapore.EvenbeforecomingtoOLC,herinterestinacademicqualityledhertodeploytheOLCQualityScorecardfortheAdministrationofOnlineProgramsattwoinstitutionswhereshealsoservedasa“Leanchampion”torealizebusinessprocessimprovementsacrosskeyoperationalareassuchasrecruitment,advising,andretention.Priortostartingheradministrativecareer,Pedersenservedasafull‐timefacultymemberandcurrentlyservesasafacultymemberfortheOLC/PennStateInstituteforEmergingLeadershipinOnlineLearning(IELOL).SheholdsbachelorofscienceandmasterofsciencedegreesfromtheUniversityofNebraska‐LincolnandadoctorofphilosophydegreefromOklahomaStateUniversity.
About the Editors ix
Foreword
In2011,theOnlineLearningConsortium(OLC)begansharingourfirstcomprehensiveframeworkusedtoevaluateonlineprograms.DevelopedbyDr.KayeSheltonaspartofherdoctoraldissertation,theQualityScorecardfortheAdministrationofOnlineProgramshasbeenusedbymanyhighereducationinstitutionstovalidatethequalityoftheirprogramforavarietyofstakeholders.Recognizingthatthetimehadcometomeettheneedforasimilartooltoevaluateblendedlearningprograms,theOLCintroducedtheQualityScorecardforBlendedLearningProgramsin2015.
ThishandbookwasdevelopedtoaccompanytheQualityScorecardforBlendedLearningProgramsandprovideblendedlearningprogramadministratorswitharesourcebasedonthebestpracticesidentifiedbyourpanelofexpertsinblendedlearning.Inthishandbook,descriptionshavebeenprovidedforall70indicatorsalongwithrecommendationsformeetingeachstandard.AspecialthankstoDr.SheltonwholedtheresearchtodeveloptheQualityScorecardforBlendedLearningProgramsandcontinuestoidentifynewwaystosupportfaculty,staffandadministratorsintheirquesttocreatequalitylearningexperiencesinonline,blendedanddigitallearningasawhole.
Wewouldalsoliketothankthe54experiencedblendedprogramadministratorsthatservedasourexpertpanelfortheDelphistudy.TheQualityScorecardforBlendedLearningPrograms,andthishandbook,wouldnotbepossiblewithouttheirwillingnesstosharetheirknowledgeofblendedlearningprogramadministrationandparticipateinthestudy.
Aswefindnewwaystomeetstudentneedsandtechnologyisincreasinglyreliedontodelivercoursecontent,weanticipatecontinuedgrowthinblendedlearning.Withthisgrowth,itishighlyimperativethatinstitutionscreateaneffectivelearningenvironmentthatsupportsstudentsuccess.WiththeQualityScorecardforBlendedLearningPrograms,institutionsnowhaveawaytovalidatetheirblendedlearningprogramwhichincludestheoptionforaformalreviewprovidedbytheOLCwhichcouldleadtoanofficialOLCendorsement.
JenniferMathes,Ph.D.DirectorofStrategicPartnershipsOnlineLearningConsortiumKarenPedersen,Ph.D.ChiefKnowledgeOfficerOnlineLearningConsortium
The Online Learning Consortium’s Quality Scorecard for Blended Learning Programs (QSC-‐‑BL) is an easy to use process for measuring and quantifying elements of quality within blended learning programs in higher education. By evaluating each of the respective quality indicators within the established eight categories, an administrator of blended learning programs can determine strengths and areas of improvement within their program. The results of this tool can be used to support program improvement and strategic planning initiatives. The Quality Scorecard for Blended Learning Programs could also be used to demonstrate to accrediting bodies, elements of quality within the program as well as an overall level of quality.
The Quality Scorecard for Blended Learning Programs contains 70 unique quality indicators each indicator is worth up to three points. The 70 quality indicators are within eight different categories: Institutional Support; Technology Support; Course Development and Instructional Design; Course Structure; Teaching and Learning; Faculty Support; Student Support; and Evaluation and Assessment.
The administrator will first determine at what level their program meets the intent of each of the quality indicators after examining all procedures and processes. The next step is to determine how that score can be substantiated or demonstrated to others. Each point value is described below.
0 points = Deficient. The administrator does not observe any indications of the quality standard in place.
1 point = Developing. The administrator has found a slight existence of the quality standard but difficult to substantiate. Much improvement is still needed in this area.
2 points = Accomplished. The administrator has found there to be moderate use and can substantiate the quality standard. Some improvement is still needed in this area.
3 points = Exemplary. The administrator has found that the quality standard is being fully implemented, can be fully substantiated, and there is little to no need for improvement in this area.
The Quality Scorecard for Blended Learning Programs is versatile enough to be used to demonstrate the overall quality of blended learning programs, no matter what size or type of institution. Using the Delphi research methodology, a panel of 54 experts came to consensus concerning which of the possible indicators should be included in the scorecard. The indicators were derived from the research literature on blended in 2015. The following range of scores should be applied after scoring, developing justifications, and providing artifacts for substantiation:
90-‐‑100% (189—210 points) = Exemplary
80-‐‑89% (168—188 points) = Acceptable
70-‐‑79% (147—167 points) = Marginal
60-‐‑69% (126—146 points) = Inadequate
< 59% (< 125 points) = Unacceptable
Introduction to the Quality Scorecard
for Blended Learning Programs
ContacttheOnlineLearningConsortiumat(617)716‐1414ifinterestedinaformalprogramreviewofblendedlearningprograms.
Note:TheQualityScorecardforBlendedLearningProgramscontainsadaptationsofthe24qualitystandardsidentifiedbytheInstituteforHigherEducationPolicyreport,QualityontheLine:BenchmarksforSuccessinInternetbasedDistanceEducation(2000).
1. Theinstitutionhasagovernancestructuretoenablesystematicandcontinuousimprovementrelatedtotheadministrationofblendededucation.
2. Theinstitutionhasagovernancestructuretoenableclear,timely,effective,andcomprehensivedecisionmakingrelatedtoblendedlearningcourses/programs.
3. Theblendedlearningprogram’sstrategicplanisreviewedforitscontinuingrelevance,compliancewithaccreditationobjectives,andisperiodicallyimprovedandupdated.
4. Theinstitutionhasdefinedthestrategicvalueofblendedlearningtoitsenterpriseandstakeholders(students,faculty,parents,etc.).
5. Theorganizationalstructureoftheblendedlearningprogramsupportstheinstitution'smission,values,andstrategicplan.
6. Theinstitutionhasaprocessforplanningandresourceallocationfortheblendedlearningprogram,includingfinancialresources,inaccordancewithstrategicplanning.
7. Theinstitutiondemonstratessufficientresourceallocation,includingtechnologyandfinancialresources,inordertoeffectivelysupportthemissionofblendededucation.
8. Theinstitutionhaspolicyandguidelinesthatconfirmastudentwhoregistersinablendedcourseorblendedlearningprogramisthesamestudentwhoparticipatesinandcompletesthecourseorprogramandreceivesacademiccredit.Thisisdonebyverifyingtheidentityofastudentbyusingmethodssuchas(a)asecureloginandpasscode,(b)proctoredexaminations,or(c)othertechnologiesandpracticesthatareeffectiveinverifyingstudentidentification.
9. Aprocessisfollowedthatensuresthatpermissions(CreativeCommons,Copyright,FairUse,PublicDomain,etc.)areinplaceforappropriateuseofallcoursematerials.
Institutional Support
1. The institution has a governance structure to enable clear systematic and continuousimprovement related to the administration of blended education.
Thisqualityindicatorexaminestheassessmentofblendededucationwithinthegovernancestructuretoensurecontinuousimprovement.Qualityoftheblendededucationprogramshouldbeevaluatedannually,andnotwithjustaone‐timeexamination.Workingtowardscontinuousimprovementshouldbeafocusthroughoutallareasoftheblendededucationprogram.Withinthisconceptualframework,itisunderstoodthattheleadershipofanyinstitutionisbasedonapplyingmanagementtechniquestoimplementitsactivitiesandachieveitsobjectives.Programadministrationrequiresastrongcommitmentbyallthoseinvolved(faculty,staff,technicians,andstudents)guaranteeingtheefficacyandefficiencyofactionsimplementedforprogramdevelopmentareeffective.Becausegoalsandobjectivesarepartoftheoverallinstitutionalstrategicplan,theseactivitiesandstrategiesforachievingthosegoalsshouldbemeasuredwithintheinstitutionalgovernancestructureonaperiodicbasis.TheOnlineLearningConsortium(formerlySloan‐C)Pillarsoflearningeffectiveness,access,costeffectiveness,studentsatisfactionandfacultysatisfactioncanbeusedtodevelopanevaluationframeworkastheycloselyalignwiththreeprimarymotivationsforadoptingblendedlearningof:enhancedlearningeffectiveness,enhancesaccessandflexibilitytolearningandenhancedcosteffectiveness(Laumakis,Graham,&Dziuban,2009).
Useofbothfacetofaceandonlinelearningnecessitatesmorecarefulconsiderationofappropriatemethodsofassessment.Whenusingblendedlearning,theuseoffacetofaceclassroomobservationscanbecombinedwithlearnertrackingtoolswithincoursemanagementsystemstoprovideacorecomplexassessmentview(Laumakisetal.,2009).Datacapturepossibilitieswithinonlinelearningfacilitatestheabilitytonotonlyassessstudentlearningthroughassignments,test,etc.,butalsostudentengagementthroughanalysisofstudenttransactions(Picciano,2015).Thosesamedatacapturepossibilitieswouldalsoapplytothoseonlinecomponentsofablendedcourse.Forgoalsnotbeingmet,animprovementplanshouldbedevelopedinordertomeetthegoalthenexttimemeasurementwilloccur.Thesystematicevaluationplanshouldprovidevaluablefeedbackthatwillinformandsupportblendedlearning,aswellasidentifyareastoimprove(Laumakisetal.,2009).
Recommendations
Establishseniorpositionstoguidestrategicandoperationalobjectivesandcreateaccountabilitymeasuresforalldesignatedstafftoensuretheorganization’sabilitytoachieveobjectivesforblendedprograms.
Committodevelopingandofferingnecessaryservicesandresourcesinsupportofthemyriadoftechnical,social,andlearningissues(bothinternalandexternal)encounteredbystudents,faculty,andstaffengagedinblendededucation.
Distinguishhowprogramgoalsandobjectivesarepartoftheoverallstrategicplananddevelopstrategiesformeasuringthosegoalswithintheinstitutionalgovernancestructure.Keepaccuraterecordsofwhengoalsaremet;iftheyarenotachieved,developanimprovementstrategyinordertoreachthegoalsforthenextmeasurementperiod.
Datafromsurveysshouldbemadeavailabletofacultyaswellasinstructionaldesigners. Developareportingcyclethatidentifiesmeasuredgoalsandimprovementstrategiesfor
meetingthosegoalsannually. Chooseoneareaoftheprogramandfocusonitsimprovementforaone‐yearperiod.Once
thegoalsareachieved,selectanotherareatoimprovethenextyear.
4 Quality Scorecard for Blended Learning Programs
2. The institution has a governance structure to enable clear, timely, effective, and comprehensive decision making related to blended learning courses/programs.
Academicgovernanceisaregulatorytermthatclarifieshowinstitutionsareorganizedandhowresponsibilitiesaredividedandassessed.Commonacrossbothpublicandprivateinstitutions,governancestructuresensureorderlyandcontinuousoperation.Thisqualityindicatorexaminesthegovernanceofblendedlearningeducationtoensureitsorderlyandcontinuousoperationandcleardecisionmakingprocess.
Duringtheemergenceofblendedlearningeducationinhighereducation,institutionsoftenstructuredblendedlearningeducationasanauxiliaryserviceortemporaryentity.Theseearlyprogramswereoftenseenastangential,ratherthanasintegralpartsoftheinstitution’smissionandstrategicplan.Asaresult,governancestructuresforblendedlearningeducationwerehaphazardandinstitutionaldecisionmakingregardingblendedlearningeducationwasneithereffectivenorcomprehensive.However,withrapidgrowthandacceptance,blendedlearningeducationquicklybecameacoreeducationalservice.Continuedgrowthoverthepasttwodecadeshasdemonstratedthatblendedlearningeducationhasenteredthemainstream(Graham,Woodfield,&Harrison,2013).Consequently,blendedlearningprogramsrequireattentiveplanningandstructuringthatreflectsthecoreeducationalroleonlineeducationnowplaysinhighereducation.
Thisqualityindicatormakesnoprescriptionaboutthenatureofthegovernancestructure.Institutionsenjoythelibertytoorganizeblendededucationgovernancestructuresdifferentlybasedonsize,mission,androle;however,aninstitutionmustdemonstratethatstrategicdecisionmakingregardingblendededucationisappropriatelymadeattheinstitutionlevel.
Recommendations
Adoptaninstitutionalapproachtowardthegovernanceandorganizationofonlineandblendededucationprograms.
Includeallinstitutionaldivisionsthatarelikelytobeinvolvedinand/oraffectedbythedecisionmakingprocessforblendededucationinthegovernanceframework.
Clarifyresponsibilitiesforallauthoritiesoverblendededucationprogramsandcommunicatethatclearlytostakeholders.
Developpoliciesandpracticesforgovernanceviaasteeringcommitteewithrepresentativesfromalldivisionsimpactedbythedeliveryofblendededucation.
Institutional Support 5
3. The blended learning program’s strategic plan is reviewed for its continuing relevance, compliance with accreditation objectives, and is periodically improved and updated.
Toprovidearigorousblendedlearningprogram,aninstitutionmustbeginwithastrategicplanthatdemonstratesthatvaluesthatadministrationandfacultyplaceonblendedlearning.Astrategicplandirectshowoperationsfunctionandresourcesareallocatedinaccordancewiththegoalsandobjectivesestablished(Moskal,Dziuban,&Hartman,2013).Infact,thestrategicplanprovidesinformationontheresultsthattheorganizationistryingtoachieve.Becauseofthis,itisimportantforthestrategicplantoremaincurrentandrelevant.Someinstitutionswilldevelopafiveorten‐yearstrategicplan;however,thatdoesnotmeanthatitisleftunchangedforthattimeperiod.Eachyear,itshouldbereviewedforrelevancyandcurrentapplication.
Whendevelopingastrategicplanforablendedlearningprogram,itisnecessarytoconsiderboththepresentandthepastactivitiesoftheorganizationinordertoreachitsgoals;inotherwords,itisvitaltoknowthelocationofthedestinationinordertogetthere.Becausethestrategicplanshouldoutlineprincipalactivities,implementation,andtimeframes,theseshouldbeidentifiedtoreviewforcompletionandcontinuedrelevance.Performancefunctionanddutiesidentifiedfortheinstitutionalmembersshouldbeperiodicallyreviewedinordertoidentifypotentialweaknessesandbarrierforcompletionofthefunctionsandactivities
Thestrategicplanshouldincludespecificactivitiesthatsupporttheidentifiedgoalsandobjectives.Itishelpfultotakethestartingandendingdatesofactivitiesandtoevaluatethemperiodicallytoseeiftheyarebeingfullyimplemented,orwhetheritisnecessarytomakechangestoachievewhatwasplannedand/ortoupdatethestrategicplaninordertoachievecontinuousimprovement.
Recommendations
Includespecificactivitieswithinthestrategicplanthatshouldbeevaluatedperiodically.Theseactivitiesshouldbescheduledonatimelineforevaluationandupdates.
Evaluatethestrategicplanatleastonceayearandproposenecessaryimprovementstomaintainrelevancy.
Withintheconceptualframework,itisunderstoodthattheleadershipofanyinstitutionisbasedonapplyingmanagementtechniquestoimplementitsactivitiesandachieveitsobjectives.Developareportingcyclethatreportsmeasuredgoalsandimprovementstrategiesformeetingthosegoalsannually.
Chooseoneareaoftheprogramandfocusonitsimprovementforaone‐yearperiod.Oncethegoalsareachieved,selectanotherareatoimprove.
6 Quality Scorecard for Blended Learning Programs
4. The institution has defined the strategic value of blended learning to its enterprise and stakeholders (students, faculty, parents, etc.).
Theextenttowhichqualitymaybeachievedinblendedlearningprogramsdependsonmanyfactors;chiefamongthemisthedegreetowhichtheorganization’sstrategicplan,mission,visiondefineandsuccessfullycommunicatetothestakeholders.Systematicplanningfocusesonexpressingtheblendededucationstrategicobjectiveswithintheinstitutionalvision,developingguidelinesandoperationalproceduresforprogrammaticoversight,committingtothequalitydeliveryofservices,andestablishingbudgetaryandfinancialprotocolsthataddresstheuniqueaspectsofblendededucation.Theseactionsprovideaframeworkfortherelationshipsamongprogramdevelopment,strategyandgrowth,andoperationalobjectivesaswellashelptoestablishtheinstitutionorprogram’scommitmenttoquality.
Thereisatendencyamongcollegesanduniversitiestofocussolelyonthemodeofdeliveryforblendedlearning.Suchalimitedapproachdoesnotconveythevalueofblendededucationtotheinstitutionasawhole.Theextenttowhichtheinstitution’svisionforandoperationalapproachtoblendededucationisarticulatedtoallstakeholdersiscriticalforthequalityofblendededucationprograms.Regardlessoftheoperationalmodel,thesystemcanmirrorthecampusvirtually.Blendedlearningshouldfitwithintheinstitutionalgoalsandbealignedwithspecificstrategies(Niemiec&Otte,2009).
Strategicdevelopmentofblendedprogramsthatleveragetheexpertiseofmultiplestakeholdersacrossthecollegeoruniversityprovidesthenecessarycollectiveeffortsthatcontributetoqualityinblendededucation.Clarityofvision,buy‐in,oversight,services,andcostarekeyareasforconsiderationwithrespecttodefiningtheinstitution‐widestrategicvalueofblendedprograms.
Recommendations
Affirmthevalueofblendededucationwithintheinstitutionalvisionbylinkingoperationalactivitiestostrategicobjectives.Alsoemphasizetheinstitution’scommitmenttothevalueofblendededucationbyannouncingitinprominentplacessuchastheinstitution’sandprogram’shomepagesonthewebsite,thepresident’swelcomepage,andstudentandfacultydevelopmentcentersoncampus.
Establishseniorpositionstoguidestrategicandoperationalobjectivesandcreateaccountabilitymeasuresforalldesignatedstafftoensuretheorganization’sabilitytoachieveobjectivesforblendedprograms.
Institutional Support 7
5. The organizational structure of the blended learning program supports the institution's mission, values, and strategic plan.
Inanyorganization,itisimportanttomaintainanopenorganizationalculture,andforitsmemberstoshowcommitmentandactresponsibly,therebyenabletheorganizationtoachieveexcellenceandsuccess.Typically,whenwespeakoforganizationalculture,wearereferringtospecificpatternsofbehaviorthatmayberecognized,transmitted,andlearned(Schein,1984).Likewise,Mintzberg(1979)definedtheorganizationalstructureasthesumofallthewaysinwhichworkisdividedintodifferentactivitiesandcoordinated.Itisthesumofthevaluesthatdefinetherelationshipsbetweenitsmembers.
Itisessentialforeveryorganizationtodevelopastrategicplanthatoutlinestheprincipleactivities,implementation,andtimeframes,includingblendededucationprograms.Inaddition,itisimportantforthefunctionscarriedoutbythemembersofaninstitutiontobeclearlydefined,sothattogether,theycanmeettheinstitutionalgoals;i.e.eachpersonshouldknowhisorherrights,dutiesandactivities,sothatoptimalresultsmaybeachieved.Withoutastrategicplan,everymemberoftheorganizationwouldbeworkinginisolation,andunawareoftheresultsthattheorganizationisaimingtoachieve.
Thepoliciesoftheinstitutionshouldclearlyreflecttheinstitution’svaluesandthepartplayedbyitspersonnel.Iftheinstitutionalpoliciesareclearandcoherent,theywillbekeytoachievingeducationalqualityand,moreover,enableallprocessestobecarriedoutinaneffectiveandefficientmanner.Forallthesereasons,itisessentialfortheblendedprogramstructuretosupportthepolicies,thestrategicplan,thevaluesandthecultureoftheorganization.
Recommendations
Provideinformativeopportunitiestoensureallmembersoftheorganizationbecomefamiliarwiththestrategicplan,supportingpolicies,andthevaluesandcultureoftheorganizationinwhichtheywork.
Surveymembersoftheorganizationtofindouthowclearlythecommunicationregardingthestrategicplanhasbeenandhowitsupportsblendededucation.
Createanorganizationalstructurechartthatisvisuallysupportedbythemissionandvisionoftheinstitution.Developjustificationsastowhytheorganizationalstructureisappropriatelyaligned.
8 Quality Scorecard for Blended Learning Programs
6. The institution has a process for planning and resource allocation for the blended learning program, including financial resources, in accordance with strategic planning.
Planninginvolvesaprocessfordeterminingthegoalsofanorganization,resourcesthataretobeutilized,andgeneralpoliciesthatwillguidetheacquisitionandadministrationofsuchresources,supportingtheorganizationasawhole.Withinthisprocess,variousactionsaretaken,oneofwhichisthattheorganizationasawholeanalyzesandevaluatesthefinancialresourcesneededtodevelopitsactivitiesandsatisfytheneedsofitsstakeholders,usingdifferentstrategiesaccordingtotheparticularstructureoftheprogram.Thestrategicplanfortheblendedprogramshouldsupporttheneedforresourceallocation.
Eachblendededucationprogramshouldhaveitsownbudget,sothattheoriginandapplicationofeconomic,material,andendowmentresourcesmaybeidentified.Theblendedprogramcannotbesuccessfulwithoutanadequatebudgetandothernecessaryresourcessuchaspersonnel.Intheearlydaysofblendededucation,manyprogramstriedtooperatewithwhateverfundscouldbegeneratedfromgrantsorextrafunding.Whileseekingoutsidefundingisstillanoption,wenowunderstandthatablendededucationprogramwillnothavelong‐termviabilitywithoutitsownlongtermfundingsources.
Recommendations
Engageallmembersinvolvedintheblendededucationprogramintheprocess,sothatafinancialplanisdevelopedbyindividualsthroughouttheorganizationatdifferentlevelsaffectedbytheblendedprogramandindifferentsupportunitssotheirspecificneedsmaybebudgeted.
Developclearbudgetaryguidelinesforunits,takingintoaccountcostandtheextenttowhichbudgetingandaccountingpracticescanclearlydelineateatruereturnoninvestment.Distinguishbetweenbenefitsofblendededucationandhoweachmayormaynotimpacttheoverallunitcostintermsofdeliveringservices,infrastructure,andstaffing.
Estimatetheresourcesassignedtoeachofthemainactivitiesdescribedintheblendedprogram’sstrategicplan.Checkhalfwaythroughthebudgetyeartobesureadequateresourcesstillexistforallactivitiesidentifiedinthestrategicplanforthatyear.
Institutional Support 9
7. The institution demonstrates sufficient resource allocation, including technology and financial resources, in order to effectively support the mission of blended education.
Theinstitutiondemonstratessufficientresourceallocation,includingtechnologyandfinancialresources,inordertoeffectivelysupportthemissionofblendededucation.
Planninginvolvesaprocessfordeterminingthegoalsofanorganization,resourcesthataretobeutilized,andgeneralpoliciesthatwillguidetheacquisitionandadministrationofsuchresources,supportingtheorganizationasawhole.Withinthisprocess,variousactionsaretaken,oneofwhichisthattheorganizationasawholeanalyzesandevaluatesthefinancialresourcesneededtodevelopitsactivitiesandsatisfytheneedsofitsstakeholder,usingdifferentstrategiesaccordingtotheparticularstructureofeachprogram.Thestrategicplanningfortheblendedprogramshouldsupporttheneedforresourceallocationasacomprehensiveframeworkofsupportisoneoftheessentialsuccessfactorsinblendedlearning(Niemiec&Otte,2009).
Eachblendedprogramshouldhaveitsownbudget,sothattheoriginandapplicationofeconomic,material,andendowmentresourcesmaybeidentified.Theblendedprogramcannotbesuccessfulwithoutanadequatebudgetandothernecessaryresourcessuchaspersonnel.Blendedlearning,supportedbypedagogy,necessitatesthatoneoftheessentialresourceswillincludepeopleaswellastechnology.Instructorsmustacquiretheskillsandabilitiestoleveragetechnologytofacilitateeffectivestudentlearning(Poon,2013).Asblendedlearningtargetscorestudentsandhastheabilitytopositivelyimpactallstudents,itmaynotnecessitateaspecialstreamoffundingifitisseenasacoreexpense(Niemec&Otte,2009).Seekingadditionaloutsidefundingisalwaysencouraged.
Recommendations
Engageallmembersinvolvedintheblendedprogramintheprocess,sothatafinancialplanisdevelopedbyindividualsthroughouttheorganizationatdifferentlevelsaffectedbytheblendedprogramandindifferentsupportunitssotheirspecificneedsmaybebudgeted.
Developclearbudgetaryguidelinesforunits,takingintoaccountcostandtheextenttowhichbudgetingandaccountingpracticescanclearlydelineateatruereturnoninvestment.Distinguishbetweenbenefitsofblendededucationandhoweachmayormaynotimpacttheoverallunitcostintermsofdeliveringservices,infrastructure,andstaffing.
Estimatetheresourcesassignedtoeachtothemainactivitiesdescribedintheblendedprogram’sstrategicplan.Checkhalfwaythroughthebudgetyeartobesureadequateresourcesstillexistforallactivitiesidentifiedinthestrategicplanforthatyear.
10 Quality Scorecard for Blended Learning Programs
8. The institution has policy and guidelines that confirm a student who registers in a blended course or blended learning program is the same student who participates in and completes the course or program and receives academic credit. This is done by verifying the identity of a student by using methods such as (a) a secure login and pass code, (b) proctored examinations, or (c) other technologies and practices that are effective in verifying student identification.
Regardingstudentauthenticationandtheonlinecomponentofblendedcoursesandprograms,theUSHigherEducationandOpportunityActof2208(HEOA)mandatedthataccreditingagenciesmustrequire:
Aninstitutionthatoffersdistanceeducationorcorrespondenceeducationtohaveprocessesthroughwhichtheinstitutionestablishesthatthestudentwhoregistersinadistanceeducationorcorrespondenceeducationcourseorprogramisthesamestudentwhoparticipatesinandcompletestheprogramandreceivestheacademiccredit.(H.R.4137‐‐110thCongress,2008)
InMay2009,theNegotiatedRuleMakingCommitteeonaccreditationclarifiedtherequirementbyinstructingregionalaccreditorstorequirethatinstitutionswhichrequiredistanceeducationprogramshaveprocessesinplacefordistanceeducation.Theaccreditingagencyisincomplianceifit:
1.Requiresinstitutionstoverifytheidentityofastudentwhoparticipatesinclassorcourseworkbyusing,attheoptionoftheinstitution,methodssuchas‐
a.Asecureloginandpasscode;
b.Proctoredexaminations;and
c.Identificationtechnologiesthatareeffectiveinverifyingstudentidentification;
2.Makesclearinwritingthatinstitutionsmustuseprocessesthatprotectstudentprivacyandnotifystudentsofanyprojectedadditionalchargesassociatedwiththeverificationofstudentidentityatthetimeofregistrationorenrollment(20U.S.C.1099b).
Authenticationisanalogoustotakingattendanceinaface‐to‐facecourse.Thesophisticationwithwhichstudentsareauthenticatediflargelyafunctionoftheeffortoftheinstructorandinstitution(Bailie&Jortberg,2009).Forexample,forallclasses,institutionsnormallyproduceclassrosterslistingstudentswhohaveregisteredandpaidforcourses.Thisrosterisprovidedtotheinstructor,whointurn,hastheresponsibilityofmakingsurestudentsontherosterarewhotheysaytheyare,andthattheyaretheonesperformingtheacademicworkthroughoutthecourse.Thesesameprinciplesapplytotheonlinecomponentofblendededucation;however,themethodsofauthenticationaredifferentaccordingtowhetherstudentsarephysicallyorvirtuallypresent.
Technologyplaysamoreimportantroleintheauthenticationofstudentsintheonlineenvironment.Asidentificationtechnologyevolvesandbecomesmoresophisticated,theabilitytocheckstudentidentitywillbecomeeasierinbothvirtualandphysicalenvironments.Ontheotherhand,thetechnologicalsophisticationofstudentsdeterminedtocommitacademicfraudwillalsoincrease(Bedford,Gregg,&Clinton,2009).Consequently,allinstitutionsshouldregularlyexaminetheirstudentauthenticationprocessandbecognizantoftechnologicaladvancesthatmakestudentauthenticationmoreaccurate,affordable,andeasytouse.
Institutional Support 11
Recommendations
Employasecurestudentloginandpasswordprocessforstudentstoaccesstheonlinecoursematerials,studentinformation,financialaidprocessing,andsoon.
Adoptaproctoringsystemforstudentexaminationscompletedonline.Thismayincludebothofflineand/oronlineauthentication.Onlinemethodsmayincludewebcams,typingheuristics,orotherbiometricidentification.
Authenticatestudentsuponloginwithsecondaryauthentication,uptoandincludingbiometricauthentication.
12 Quality Scorecard for Blended Learning Programs
9. A process is followed that ensures that permissions (Creative Commons, Copyright, Fair Use,Public Domain, etc.) are in place for appropriate use of all course materials.
Thegrowthofopenlysharedcontentcreatedbyauthorswhoencouragethere‐mixingoftheirworkhasforgednewfrontiersintotheavailabilityofqualityeducationalresourcesavailableatlowornocost.Thisnewsharingeconomybringswithitawholenewmixofquestionsaboutwhatconstitutesafairuse.Facultyareincreasinglyunsureofwhatroutetotakewhenconsideringauthoringandsourcingcontentforblendedcourses.Facultyfinditdifficulttonavigatethepermissionsforcontentcollectedanddeterminingifthosepermissionspermitthefaculty’sintendeduseofthecontent.
Theinstitutionhasastandardfordistributionofcoursecontent,throughtheteachingandlearningprocess,withspecificstandardsforblendedcoursecontent:
● Courseintellectualpropertyownershipisdefinedfortheinstitution,department,andfaculty;
● Apolicyforcitingpermissionsforcontentintendedfordistributioninablendedcourse.
Institutionalsupportisavailabletostaffandfacultywhowishtoauthorneworderivativeworkfordistributionorcreateacompilationofresourcedcontentinablendedcourse:
● Aspecialistofcopyrightandfairuse.● Aspecialistofinstructionaldesign.● Aspecialistofcourseandprogramassessment.
Recommendations
Provideinstructionaldesignsupporttofacultyoneffectiveuseofeducationalresourcesfordistributedlearningactivitiesandpairingappropriatecontentwithappropriatelearningactivities.
Provideeducation(guides,tools,training,policy)onauthorshipofnewandderivativeworks,thedifferencesbetweenopen/sharedworksandworksprotectedbycopyrightorpatent,andthebenefitsopeneducationresourcesmayprovideincontrasttouseofresourcesprotectedbycopyright.
Provideeducation[guides,tools,training]onopeneducationresourceclearinghouses,howtoassessthequalityofopeneducationresources,anddiscerningthetypesresourcesandtheirbestuse.
Provideeducation[guides,tools,training]onthenatureoffairuseandunderstandingthelegalscopeofdifferentusemethodologies:freelyshared,limitedbyanopencopyrightsuchasCreativeCommons,orprotectedbyanationalcopyrightorpatent.
Provideeducation(guides,tools,training,policy)onproperlycitingopenandprotectedcontentwithinblendedcourses.
Institutional Support 13
References for Institutional Support
Bailie,J.L.,&Jortberg,M.A.(2009).Onlinelearnerauthentication:Verifyingtheidentityofonlineusers.MERLOTJournalofOnlineLearningandTeaching,5(2).Retrievedfromhttp://jolt.merlot.org/vol5no2/gregg_0609.htm
Bedford,W.,Gregg,J.,&Clinton,S.(2009).Implementingtechnologytopreventonlinecheating:AcasestudyataSmallSouthernRegionalUniversity(SSRU).MERLOTJournalofOnlineLearningandTeaching,5(2),230–238.Retrievedfromhttp://jolt.merlot.org/vol5no2/gregg_0609.htm
CornellLegalInformationInstitute.(2016).Copyright:Anoverview.Retrievedfromhttp://www.law.cornell.edu/topics/copyright.html
CreativeCommons.(2016).ExploretheCreativeCommonslicenses.Retrievedfromhttps://creativecommons.org/choose/
CreativeCommons.(2016).WhatisOER?Retrievedfromhttps://wiki.creativecommons.org/wiki/What_is_OER%3F
Graham,C.R.,Woodfield,W.,&Harrison,J.B.(2013).Aframeworkforinstitutionaladoptionandimplementationofblendedlearninginhighereducation.InternetandHigherEducation,18,4‐14.
H.R.4137—110thCongress.(2008).HigherEducationOpportunityAct.Retrievedfromwww.GovTrack.usRetrievedfromhttp://www.govtrack.us/congress/bills/110/hr4137
Laumakis,M.,Graham,C.,Dziuban,C.(2009).TheSloan‐Cpillarsandboundaryobjectsasaframeworkforevaluatingblendedlearning.JournalofAsynchronousLearningNetworks,13(1),75‐87.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ837582.pdf
LibraryofCongress.(2011).CopyrightLawoftheUnitedStates.Retrievedfromhttp://www.copyright.gov/title17/
Mintzberg,H.(1979).Thestructuringoforganizations:Asynthesisoftheresearch.EnglewoodCliffs,NJ:PrenticeHall.
Moskal,P.,Dziuban,C.,&Hartman,J.(2013).Blendedlearning:Adangerousidea?InternetandHigherEducation,18,15‐23.
Niemec,M.,&Otte,G.(2009).Anadministrator’sguidetothewhy’sandhow’sofblendedlearning.JournalofAsynchronousLearningNetworks,13(1),19‐30.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ909846.pdf
Picciano,A.(2015).Planningforonlineeducation:Asystemsmodel.OnlineLearning,19(5),Retrievedfromhttp://onlinelearningconsortium.org/read/journal‐issues/
Poon,J.(2013).Blendedlearning:Aninstitutionalapproachforenhancingstudents’learningexperience.MERLOTJournalofOnlineLearningandTeaching,9(2).Retrievedfromhttp://jolt.merlot.org/vol9no2/poon_0613.htm
Schein,E.H.(1984,Winter).Comingtoanewawarenessoforganizationalculture.SloanManagementReview,25(2),3‐15.
Shelton,K.,&Saltsman,G.(2005).Anadministrator'sguidetoonlineeducation.Greenwich,CT:InformationAge.
14 Quality Scorecard for Blended Learning Programs
StateUniversityofNewYorkAcademicAffairs.(2016).CopyrightandFacultyOwnershipofIntellectualProperty.Retrievedfromhttp://system.suny.edu/academic‐affairs/faculty/faculty‐ownership/UnitedStatesCopyRightOfficeWebSite.LibraryofCongress.(April,2016).Retrievedfromhttp://www.copyright.gov/
UniversityatBuffalo.(2016).FairUseinHigherEducation.Retrievedfromhttp://www.buffalo.edu/ubit/service‐guides/safe‐computing/dmca/fair‐use‐higher‐education.html
Institutional Support 15
1. Thecoursedeliverytechnologyisconsideredamissioncriticalenterprisesystemandsupportedassuch.
2. Whethertheinstitutionmaintainslocaldatacenters(servers),and/orcontractsforoutsourced,hostedservicesorcloudservices,thosesystemsareadministeredincompliancewithestablisheddatamanagementpracticessuchastheInformationTechnologyServiceManagement(ITSM)standards,whichincludeappropriatepowerprotection,backupsolutions,disasterrecoveryplans,etc.
3. Thetechnologysystemsrelatedtothedeliveryofblendedlearningprogramsarehighlyreliableandoperablewithmeasurablestandardsbeingutilizedsuchassystemdowntimetrackingandtaskbenchmarking.*
4. Adocumentedtechnologyplanthatincludeselectronicsecuritymeasures(e.g.,passwordprotection,encryption,secureonlineorproctoredexamsifapplicable,etc.)isinplaceandoperationaltoensurequality,inaccordancewithestablishedstandardsandregulatoryrequirements.*
5. Theinstitutionhasanestablished(updatedandcontinuouslyreviewed)contingencyplanforthecontinuanceofdatacentersandsupportservicesintheeventofprolongedservicedisruption.
6. Acentralizedsystemprovidessupportforbuildingandmaintainingtheblendededucationinfrastructure.*
7. Faculty,staff,andstudentsaresupportedinthedevelopmentanduseofnewtechnologiesandskillsapplicabletoblendedlearning.
Technology Support
1. The course delivery technology is considered a mission critical enterprise system and supportedas such.
Theinstitutionmustconsidertheblendedprograms’technologicalinfrastructureasmissioncriticalevidencedbyacommitmentofresources,continuousimprovementprocesses,andacommitmenttonear100%uptime.Studentsandfacultymusthaveaccesstoblendedcoursematerialswhenneeded.Learningmanagementsystemsforcoursedelivery,inwhateverformat,mustbeaninstitutionalpriority.Coordinatingriskmanagementandtechnologyupgradestrategieswithonlineprogramsisawaytoleverageexistinginstitutionalizedcultureandinfrastructure.Theremustbededicatedtechnologysupportpersonnelavailable24x7x365forsolvingissues,asonlineandblendedstudentsaccesscoursematerialsatvarioustimesoftheday,typicallyoutsideofthedaytimeMondaythroughFridayhours.Bothonlineandblendedclassstudentsusecoursetechnologyduringtheeveningsandonweekends(Moskal,Dzuiban,&Hartman,2013).
Recommendations
Seekcollaborationbetweenblendedprogramadministratorsandinstitutionaltechnologyservicesinordertobeinvolvedwiththedevelopmentandupdatingofthetechnologysupportplan.
Establishandclearlycommunicatetoallpersonnelinvolvedemergencyresponseproceduresforunexpecteddowntime.
Budgetandplaninordertomaintainthetechnologicalinfrastructureforcoursedelivery. Utilizemetricsandbenchmarkingforupgradingandimprovingtechnologiesusedto
supporttheblendededucationprogram.
Technology Support 17
2. Whether the institution maintains local data centers (servers), and/or contracts for outsourced,hosted services or cloud services, those systems are administered in compliance withestablished data management practices such as the Information Technology ServiceManagement (ITSM) standards, which include appropriate power protection, backup solutions,disaster recovery plans, etc.
Maintainingthemyriadoftechnologicalinfrastructuresystemsrequiredforthedeliveryandsupportofblendededucationprogramsrequiresenterprise‐levelcommitmenttoensuresystemavailability,responsetime,dataintegrity,studentprivacy,andsystemsecurity.Suchpracticeschangequicklyastechnologyadvancesandservicesevolve.InstitutionsshouldcreateandadheretoInformationTechnology(IT)anddatamanagement.TheInformationTechnologyServiceManagement(ITSM)standards(orsimilar),maintainestablishedandevolvingbestpracticesforthemanagementofqualityinformationtechnologyservices.TheITSMstandardsshouldbefollowedaccordingtotheinstitution’sacademicpolicies.Thisqualityindicatorrecognizesthatblendedprogramtechnologicalsupportisbestprovidedbytechnologistswhosubscribetoandadheretoestablishedtechnologyindustryqualitystandards.
Recommendations
Basecurrenttechnologicalsupportoperationsonestablishedinformationtechnologyindustrystandardsthatensurequalityinformationtechnologyservices.
Providesufficientbudgetsupportinordertoensuretechnologysupportstaffpersonnelareabletoparticipateinongoingprofessionaldevelopment.
Monitorregularlyandadheretobestpracticesininformationtechnologyinordertoupdateandexpandservicesasneeded.
18 Quality Scorecard for Blended Learning Programs
3. The technology systems related to the delivery of blended learning programs are highly reliable and operable with measurable standards being utilized such as system downtime tracking and task benchmarking.*
Ifthelightsgooutinatraditionalclassroom,itmaybeaninconvenience.Ifthesystemcrashes,it’sacalamity!(Phipps&Merisotis,2000,p.15).
Whenstudentsareenrolledinblendedcourses,theimportanceofreliabletechnicaldeliverysystemsisascriticaltothequalityofeducationasphysicalclassroomshavebeentotraditionaleducation.Infact,foranInternet‐deliveredclassroomtobeoffline,itisanalogoustolockingtheclassroomdoortostudentsbeforetheyarriveforclass.Technologydeliverysystemsshouldberesponsiveandoperationalduringpeakloadtimessothatstudentsareabletofocusonacademiccontentandbeminimallydisruptedbyunresponsivetechnology.Providingastableandreliabletechnicalinfrastructureisessentialasitinfluenceslearning(Gibbons&Wentworth,2002),studentsatisfaction(O'Brien&Renner,2002),andstudentretention(Fetzner,2003).Blendededucationprogramsrequireastrongtechnicalinfrastructuretoprovidethefoundationalstructureswithintheonlinecomponentforhowstudentsandfacultyinteractwitheachotherandwithcoursecontent.
Intheearlydaysofonlineeducation,itwascommonforinstitutionstohostthelearningmanagementsoftwareandothercriticalinfrastructureonsiteusinginstitutionallyownedhardwareandstaff.Itisnowmorecommontoencounteronline,andtherebyblended,programsthathavecontractedwithexternalserviceprovidersthatofferhostedorcloud‐basedservicesexternaloftheinstitutionalnetwork.Regardlessoftheservicemethodselected,blendedprogramsshouldbeawareofthequalityofserviceandworktoensuretheleastdisruptivelearningenvironmentisprovided.
Notechnologycanbeexpectedtoprovide100%uptime.Allsystemswillneedtobetakendownforupdates,upgrades,andrepair.Largerprogramsmaybeabletoestablishredundantserversystemsthatallowforcontinuousuptime,orprogramsofanysizemaycontractforsuchwithanexternaldatacenter.Ataminimum,expecteddowntimeshouldbescheduledwellinadvance,completedduringtimesofleastdisruption,andcommunicatedtostudents,faculty,andblendedprogramstaffwithsufficientpriornotice.
Recommendations
Providestudentsandfaculty24x7x365access(asidefrompre‐scheduledmaintenanceperiods)forallnecessarytechnicalsystemssuchasthelearningmanagementsystem,mediaservers,etc.thatsupporttheblendededucationprogram.
Carefullymonitorsystemdowntimeanddevelopdatameasurestoensurefuturereliabilitywithagoalof100%uptime.
Establishstandardmetricsforsystemperformancesuchaswebpageloadtimesandincludeconsistentmeasurementsformonitoringgoalachievement.
Monitorsystemutilizationduringpeakusagetimesinordertoensurethatsystemstabilityismaintained.
Trackanddocumentprogramgrowthandtechnologyusageinordertoprojectandscalefuturetechnologyneeds.
Determinelowestuseperiodsandusethemfornecessarymaintenanceperiodsifredundantsystemsarenotfinanciallypossible.PublishthedowntimescheduleontheLMS,institutionalwebsite,viaemail,tweets,andthroughotherapplicablecommunicationmethodssothatstudentsandfacultyarefullyawareandcanplanaccordingly.
Technology Support 19
4. A documented technology plan that includes electronic security measures (e.g., passwordprotection, encryption, secure online or proctored exams if applicable, etc.) is in place andoperational to ensure quality, in accordance with established standards and regulatoryrequirements.*
Atechnologyplanshouldensurethattheblendedprogram’stechnologicalinfrastructureadequatelysupportstheacademicenvironment.Thetechnologyplanshouldencompassalltechnologicalinfrastructurerequiredbytheprogram,includingsystemshostedlocallyandoffsite,orthroughcontractedservices.Ataminimum,suchsystemsincludethelearningmanagementsystem,userauthenticationsystems,collaborationsoftware,andtestingsoftware.Asconsumerexpectationscoincidewithcontinuouslyemergingtechnologyapplications,technologyplansshouldreflectvirtuallyeverybusinessandacademicprocess(NationalAssociationofCollegeandUniversityBusinessOfficers,2011)andthusmustbeflexibletoadapttofiscalandenvironmentalcontexts.Therefore,thehallmarksofeffectivetechnologyplansincludeexplicitlylinkingtheplantotheinstitution’smissionandvision;engagingstakeholdersinidentifyingcurrentconditionsandassumptions;prioritizingstakeholderneedsanddesires;andpublicizingtimelinesandbenchmarksviaclearcommunicationschannels(Geer,2006;November,Staudt,Costello,&Huske,1996;White&Weight,2000).
WICHECooperativeforEducationalTechnologies(WCET)(2009)recommendedthatinstitutionsprovide“securestudentloginsandpasswordstoaccessonlinecoursesandrelatedresources,discussions,assignmentsandassessments”(p.1).Currently,withtheubiquityofcommerciallearningmanagementsystems,muchoftheresponsibilityforestablishinglogincredentialsandencryptioncanbeappropriatelymanagedbythesevendors.However,accordingtotheHEOActof2008,asthetechnologyforauthenticationbecomesmoreprevalent,institutionswillberequiredtodomorethanjustprovideasecureloginprocess(H.R.B4137‐110thCongress,2008).
TheHEOActof2008requirementspurrednumeroushardwareandsoftwareoptionstoauthenticatestudentsandtheelectronicproctoringofexams.Theseincludeoptionsforusingbiometricdata,secondaryauthenticationwithchallengequestions,keystrokeheuristics,useofwebcams,andbrowserlockdownsoftware(Foster,2008).
Recommendations
DemonstratefullprogramcompliancewithrequirementsoutlinedbytheHEOActof2008fortheonlineportionofthecourse.
Providestudentssecurelogincredentialstoonlinecoursematerialsusingcurrentauthenticationstandards.
DevelopanddisseminatetoallstudentsandfacultypoliciesthatclearlyindicateexpectationsforAcceptableUse,includingaccesstocourses,assignments,andrelatedmaterials.
EstablishandenforcepenaltiesforstudentsandfacultyviolatingAcceptableUsepolicy. Providefacultytraininginhowtousetechnologyandinstructionalmethodsthatsupport
academicintegrity. Monitorinnovationsinauthenticationtechnologiesandproctoringandevaluatetheirfitfor
theprogram.
20 Quality Scorecard for Blended Learning Programs
5. The institution has an established (updated and continuously reviewed) contingency plan for the continuance of data centers and support services in the event of prolonged service disruption.
Becausethetechnologyinfrastructureismissioncriticaltotheblendededucationprogram,theinstitutionshouldhaveinplaceatechnologysystemthatensurestheintegrityofprogramdataandprotectsagainstcatastrophicloss.Similarly,theinstitutionshouldhaveacontingencyplaninplaceintheeventofmajorsystemsdisruption.Institutionalservicesthatprovideforconstantavailability,backup,archival,datasecurity,andbusinesscontinuanceintheeventofcatastrophicfailuresdemonstrateinstitutionalcommitmenttotheprogram.
Recommendations
Providebudgetsupportforasystembackupsolutionwithoff‐sitevaulting(physical,electronic,orboth).
Develop,maintain,andperiodicallyreviewandupdateadisasterrecoveryandbusinesscontinuityplan.
Conductdisasterrecoveryormockdisasterdrillsperiodicallytoverifysystemdatabackupandthatrestorationprocessisviableandmaintainsdataintegrity.
Establishgoalsforhowquicklysystemdataisrecoveredincaseofsystemfailure.
Technology Support 21
6. A centralized system provides support for building and maintaining the blended education infrastructure.*
Commitmentfromtheinstitutionalchiefinformationofficer(CIO)ensurestheblendedprogramisalignedwithoverallcollegeoruniversityprioritiesanddemonstratestechnologycommitmentatthehighestlevelregardlesswhethertheservicesareultimatelyprovidedin‐house,viaaconsortium,orbyout‐sourcing.Forthisqualityindicator,acentralizedsystemmeansthatthetechnologythatsupportstheblendedprogramisoverseenbyaninstitutionaltechnologydecision‐makingandsupportstructure.Havingacentralizedsystemfortheinfrastructurecanbeeffectiveinseveralways:itreducescomplexityandmisunderstandingbetweentechnicalstaff,itstandardizessupport(singlehelpdesksupportanduserexperience).Centralizedsupportenablesa"focusonfaculty,student,andlocalprocessissues"(Fetzner,2003,p.240).Thecentralizedsystemneedstobecoordinated,orclearcommunicationneedstobeinplace,betweentheinstitution’sCIO/ITinfrastructureandthechiefacademicofficers(CAO)settingclearexpectationsofdesiredsupportfornewandongoingacademicbusiness,suchasblendedlearning.
Thisdoesnotmeanthatalltechnologyservicesneedtoresideinonespaceorthattheybedirectlymanagedbyinstitutionalstaff.Infact,theremaybeamixtureofoutsourcedandinhousetechnologysolutionsthatsupportablendedprogram.Thequalityindicatorexpectsthattheinstitutionaltechnologysupporthasknowledgeofthetechnologyandnecessarysupporttobuildandsustaintheprogramandactivelyadvisestoensurecontinuedreliability.
Recommendations
Designthecentralizedsystemtoparallelthebestaspectsofonlinelearning;makeit“team‐based,collaborative,comprehensive,action‐oriented,andnon‐hierarchical”(Fetzner,2003,p.232).
Includethefollowinginthecentralizedsystem:“academicsupport,trainingcoordination,instructionaldesignandsupport,librarysupport,technicalsupport,andstudentservices”(Fetzner,2003,p.234).
Collectandreviewtheperspectivesofprogramstakeholdersinordertocontinuouslyimprovethetechnologysystems.
Utilizethecentralizedsystemasaninformationexchangeandforbenchmarkingservicecalls,responsetimes,andproblemresolution.
22 Quality Scorecard for Blended Learning Programs
7. Faculty, staff, and students are supported in the development and use of new technologies andskills applicable to blended learning.
Theincreasingchangesinworkforcerequirementsandstudents’needforflexiblelearningoptionswillcontinuetodrivehighereducationintheuseofnewtechnologiestobetterserveitsstudents.Infact,students“expecttheireducationtobepersonalizedtotheirindividualwantsandneeds;theyexpecttheireducationtobeeasilyachievedandfocusedontheskillstheywillrequireintheirfutureprofession.Inotherwords,moststudentsarenotlearningforlearning'ssakeorforgettingadegree;instead,theywanttheskillsneededtoentertheworkforce”(Tamarkin,2010,para.11).Becauseofthisneedforstudentpreparation,aninstitutionshouldsupportnewtechnologyinitiativesthatwillbettersupportlearningandpreparestudentsforthecurrentworkforce.
Whennewtechnologiesareadoptedandimplemented,allthoseinvolved(students,faculty,andinstitutionalstaffmemberswhowillbesupportingallthoseinvolved)willmostlikelyneedsomeformoftraining.Dalziel(2003)pointedoutthat“facultyneedtoknowhowtousetechnologytoincorporatedifferenttypesoflearningmediatocreateacompleteeducationalpackagethatmakessensetotheirstudents”(p.669).Studentswillbenefitfromhelpdesksupport,tutorials,anddemonstrations,andaccesstoaknowledgebaseforself‐‐‐help.Staffwillneedtraininginhowbesttosupportboththefacultyandstudentswiththenewtechnology.
Recommendations
Provideongoingtrainingforfacultyandotherrelevantpersonnel.Thetimesanddaysshouldbevariedsothatallstakeholderscanbereached.
Encouragefacultytosharehowtheyembracenewtechnologiesforteachingandlearningandhowitbetterservesorbetterengagestheirstudents.
Providefacultyandstudentswithopportunitiestoexplorenewandexistingtechnologiesforcreatingandcuratingblendedcoursecomponents.
Providestudentswithtechnicalsupport,opportunitiesforself‐‐‐trainingmodules,andtutorialsonnewtechnologiesthatareimplementedforteachingandlearning.
Technology Support 23
ReferencesforTechnologySupport
Dalziel,C.(2003).Communitycollegesanddistanceeducation.InM.G.Moore,&W.G.Anderson(Eds.),Handbookofdistanceeducation(pp.663‐671).Mahwah,NJ:LawrenceErlbaum.
Fetzner,M.(2003).Institutionalsupportforonlinefaculty.Expandingthemodel.InJ.Bourne&J.Moore(Eds.),Elementsofqualityonlineeducation:Practiceanddirection:Volume4intheSloan‐CQualitySeries(pp.229‐241).Needham,MA:TheSloanConsortium.
Foster,A.(2008,July21).Keepaneyeononlinestudents.TheChronicleofHigherEducation.Retrievedfromhttp://chronicle.com/blogs/wiredcampus/keeping‐an‐eye‐on‐online‐students/4109
Geer,W.T.(2006,November).Craftingtheperfecttechnologyplan.EdTechFocusonHigherEducation.Retrievedfromhttp://www.edtechmag.com/higher/october‐november‐2006/crafting‐the‐perfect‐game‐plan.html
Gibbons,H.,&Wentworth,G.(2002,August).Processesformotivatingonlinelearnersfromrecruitmentthroughdegreecompletion.VirtualUniversityGazette:MotivatingandRetainingAdultLearnersOnline,127‐135.
H.R.4137‐‐110thCongress.(2008).HigherEducationOpportunityAct.Retrievedfromhttp://www.govtrack.us/congress/bills/110/hr4137
MiddleStatesCommissiononHigherEducation.(2011).Distanceeducationprograms:Interregionalguidelinesfortheevaluationofdistanceeducation(onlinelearning).Retrievedfromhttp://www.msche.org/publications/Guidelines‐for‐the‐Evaluation‐of‐Distance‐Education‐Programs.pdf
NationalAssociationofCollegeandUniversityBusinessOfficers.(2011).Collegeanduniversitybusinessadministration,informationtechnology(7thed.).Washington,DC:Author.
November,A.,Staudt,C.,Costello,M.A.,&Huske,L.(1996).Guidetodevelopingaschoolordistricttechnologyplan.NorthCentralRegionalTechnologyandEducationalConsortium.Retrievedfromhttp://www.ncrel.org/sdrs/areas/issues/methods/technlgy/te300.htm
O'Brien,B.S.,&Renner,A.L.(2002).Onlinestudentretention:Canitbedone?WorldConferenceonEducationalMultimedia,HypermediaandTelecommunications,1,1479‐1483
Phipps,R.,&Merisotis,J.(2000).Qualityontheline:Benchmarksforsuccessininternet‐baseddistanceeducation.Washington,DC:TheInstituteofHigherEducationPolicy.Retrievedfromhttp://www.ihep.org/research/publications/quality‐line‐benchmarks‐success‐internet‐based‐distance‐education
Tamarkin,M.(2010,Nov‐Dec.).You3.0:Themostimportantevolvingtechnology.EDUCAUSEReview,45(6),31‐44.Retrievedfromhttp://er.educause.edu/articles/2010/11/you‐30‐the‐most‐important‐evolving‐technology
WCET.(2009).Bestpracticestrategiestopromoteacademicintegrityinonlineeducation.Retrievedfromhttp://wcet.wiche.edu/wcet/docs/cigs/studentauthentication/BestPractices.pdf
White,K.W.,&Weight,B.H.(2000).Theonlineteachingguide:Ahandbookofattitudes,strategies,andtechniquesforthevirtualclassroom.Boston,MA:AllynandBacon.
24 Quality Scorecard for Blended Learning Programs
1. Acoursedevelopmentprocessisfollowedthatensurescoursesaredesignedwithalignmentbetweencoursematerials,assessmentsandlearningobjectivessothatstudentsdevelopthenecessaryknowledgeandskillstomeetmeasurablelearningoutcomesatthecourseandprogramlevel.*
2. Guidelinesregardingminimumrequirementsforcoursedevelopment,design,anddeliveryofblendedinstruction(suchascoursesyllabuselements,coursematerials,assessmentstrategies,facultyfeedback)arefollowed.*
3. Thereisconsistencyincoursedevelopmentforstudentretention(enrollmentandcoursecompletion)andquality(i.e.,coursesinaprogramhaveaconsistentnavigationalstructure).
4. Ablendedcourseshouldbedesignedasonecohesivewhole,incorporatingbothface‐to‐faceandonlineexperiencesincomplementaryways.
5. Instructionalmaterials(bothonlineandin‐class)andcoursesyllabiarereviewedperiodicallytoensuretheymeettheblendedcourse'sandprogram'slearningoutcomes.*
6. Courseassignmentsandactivitiesarereviewedperiodicallytoensuretheymeettheblendedcourses'andprogram'slearningoutcomes.
7. Student‐centeredinstructionisconsideredduringthecoursedevelopmentprocess(i.e.,studentengagement,immersion,andpersonalresponsibility).
8. Coursedesignpromotesbothfacultyandstudentengagement.
9. Courseworkloadsarereviewedtoensureitisappropriatefordesignatedcreditallocation.
10. Aprocessisestablishedandfollowedforevaluatingtheeffectivenessofcurrentandemergingtechnologiestosupporttheachievementoflearningoutcomesanddeliveringcoursecontent.
11. Courseembeddedtechnologyactivelysupportstheachievementoflearningoutcomesanddeliveringcoursecontentandunnecessaryuseoftechnologyisminimized.*
12. UsabilitytestsareconductedandappliedandrecommendationsbaseduponWebContentAccessibilityGuidelines(WCAGs)areincorporatedinthecoursedesignprocess.
13. Curriculumdevelopmentisacoreresponsibilityforfaculty(i.e.,facultyshouldbeinvolvedineitherthedevelopmentorthedecisionmakingfortheblendedcurriculumchoices).
14. Facultysupportandresourcesareprovidedtopromotethebestuseofblendeddeliverymethodincoursedevelopmentandinstructionaldesigntofacilitateteachingandlearning.
Course Development &
Instructional Design
1. A course development process is followed that ensures courses are designed with alignment between course materials, assessments and learning objectives so that students develop the necessary knowledge and skills to meet measurable learning outcomes at the course and program level.*
Alignmentisakeyconsiderationforaccreditation;yet,notallcoursesareinstructionallydesignedsothelearningoutcomesareclearlymeasured.Learningoutcomesmustbemeasuredinordertoknowifstudentsaremeetingcoursegoalsandprogramgoals.Thiscanbedonesothat“individualcourseswithinthecurriculumhelplearnersprogressfrombasic,introductorylevelsofknowledgeandskillstohigherlevelobjectivesforcriticalthinking,masteryofskills,anddemonstrationofknowledgecommontoadiscipline”(Porter,2004,p.31).Inordertoaccomplishthis,PuzziferroandShelton(2008)recommendedcoursesbedevelopedwiththefollowinggoalsforlearning:
masteringinformation,suchaskeyconcepts,termsandideasthroughexposuretotheinformationandrecall,
progressivelygraspingtheinformationbypracticinglearningactivitiesthatfocusonrecall,trialanderror,andbuildingthevocabularyandcomprehensionoftheinformation,
applyingtheinformationtoaproblem‐basedsituationthatcanbecollaborative, analyzingtheproblemfurtherbydeconstructingtheinformationandreconstructingit
intoasolutionbyapplyingandanalyzingtheinformation,begintorecognizethepatternsorrelationshipsbetweentheinformationandtheproblem,andcreatingnewknowledgeandtheabilitytoreasonabouttheinformationandapplyitpracticallytosituations.
Thus,coursesshouldbedesignedtoreinforcetheinteractivestagesoflearningtowardsthemasteryofobjectivesintheoverallprogramanddiscipline.
Recommendations
Clearlydemonstratethatblendedcourselearningoutcomesaremeasuredeachtermandcomparedtolearningoutcomegoals.
Clearlydemonstratethatblendedcourseoutcomescontributetotheprogramlearningoutcomeswithinwhichthecourseoccurs.
Useformativeandnormativeassessmentcriteriaincoursesandprogramstosupportthelearningoutcomebeingmeasuredandtohelpstudentstogaugeprogresstowardsmastery.
Buildpracticeactivitiesintoblendedcoursedesignthatencouragestudentstocheckunderstanding,refineunderstandingbyapplyingit,andtestunderstandingwithfeedbackfrompeersandexperts.
26 Quality Scorecard for Blended Learning Programs
2. Guidelines regarding minimum requirements for course development, design, and delivery of blended instruction (such as course syllabus elements, course materials, assessment strategies, faculty feedback) are followed.*
Althoughfacultymembersareconsideredexpertsintheirdisciplines,theycannotbeexpectedtobeexpertsatdevelopingblendedcoursematerials(althoughcertainlytherearethosewhoareverygood).Forconsistency,theinstitution(orcollege)shouldprovideguidelinesleadingtoconsistentdesignforblendedcoursedevelopmentanddelivery(Mesh,2016).Indevelopinganeffectiveblendedlearningenvironment,justlikeinonlinelearning,itisimportantbeforedevelopingthecourse,todeterminewhatqualitylookslikeincoursedesignandplantheappropriatestandard(Puzziferro&Shelton,2008).Fromtheinstitution’svisionofquality,theframeworkforcoursequalityshouldemergetoguidedevelopmentofcoursematerialsthatarespecificallyeffectiveforblendedteachingandlearning.Providingstandardsoracoursedevelopmentrubricdoesnotmeanthatallcourseshavetobeidentical.Itsimplymeanstherearebasiccomponentsthatarestandardtoallcourses.
Institutionscancreatetheirownstandards,orselectand/oradaptavarietyofstandardsandblendedcourserubricsthatevaluatequality.Forexample,theOnlineLearningConsortiumprovidestheQualityCourseTeachingandInstructionalPractice(QCTIP)scorecardandco‐sponsorstheOSCQRcoursereviewrubric.Bothtoolscanbeusedinpartorinwholetoevaluatetheeffectivenessofcoursedesignaswellasaddressingteachingthecourseandusinggoodpedagogicalstrategiesforengagingblendedstudents.
Recommendations
Provideclearguidelinesandrecommendationsforminimumstandardsthatshouldbecloselyfollowedinthecoursedevelopmentprocessbeforecoursedevelopmentbegins.
Includeinthestandards,effectivepracticesforblendedteachingandexpectationsforfacultyobligations,includingexpectationsforfacultypresenceinthecourseandcommunicationresponsetime.
Provideeffectivebestpracticesandmodelsofexemplarycoursesbeforethecoursedevelopmentbegins.
Developandimplementaprocessthatcomparesdevelopedblendedcoursestoestablishedstandardsandguidelinesonaregularbasis.
Course Development & Instructional Design 27
3. There is consistency in course development for student retention (enrollment and course completion) and quality (i.e., courses in a program have a consistent navigational structure).
Awell‐designedblendedcourseboostslearningandenhancesstudentretention(Amaral&Shank,2010).ImplementationofdesigntoolssuchastheBlendedLearningToolkit(Moskal,Thompson,&Futch,2015),EDUCAUSELearningInitiativeDiscoveryTool:BlendedWorkshopGuide(EDUCAUSE,Diaz,&Strickland,2009),ortheCommunityofInquiryframework(Garrison&Vaughan,2008)willensureconsistencyinthedesignoftheblendedcourse,tofacilitatelearnerinteractionandengagement.Factorsthatenhancestudents’successinblendedlearning,includeusingstudentsuccessstrategiesidentifiedintheBlendedLearningToolkitsuchaswelcomingstudentstothecourse,consistentsyllabusorganization,guidetonavigatingthecourse,a“gettingstartedmodule”,creatingopportunitiesforcollaborationandengagement,tipsforsuccess,frequentandtimelyfeedback,guidelinesonusingtechnology,linkstoresources,andtimemanagementassistance(UCF&AASCU,n.d.).
Intentionalcoursedevelopmentshouldaddressqualityconcernssuchas:consistencyincourserigorandachievementofoutcomeswithblendedcoursesincoursesdeliveredusingeitherfullyfacetofaceoronline,preparednessoffacultytofacilitatelearninginthecourse,aswellasevaluationactivitiesthatevaluatecourselearningobjectivesinamannercongruentwithsystematicevaluationplanprovidingafeedbackloop(Niemec&Otte,2009).Useofpeerreviewfortheblendedcourse,suchastheBlendedCoursePeerReviewForm(availablefromCreativeCommons),theSUNYOSCQRcoursereviewrubric,QualityCourseTeachingandInstructionalPractice(QCTIP)ortheBlendedQualityScorecardwillfacilitatetheassessmentandmeasurementofquality,facilitatingcontinualqualityimprovementaswellasprovidingdataordirectionforstrategicplanning.
Recommendations
Useofresources,suchastheBlendKitCourse,CommunityofInquirydesignprinciplesorEDUCAUSELearningInitiativeDiscoveryTool:BlendedLearningWorkshopGuidetoensureconsistencyinthedesignoftheblendedcourse,tofacilitatelearnerinteractionandengagement.
Useofpeerreviewandaqualityrubrictoensurequalityandconsistencyinblendedcourses.RubricsareavailablefromtheOLCBlendedQualityScorecardortheBlendedCourseImplementationChecklist(availablethroughCreativeCommonslicense).
28 Quality Scorecard for Blended Learning Programs
4. A blended course should be designed as one cohesive whole, incorporating both face‐to‐face and online experiences in complementary ways.
Whendesigningablendedcourse,itisimportanttofullyintegrateboththeface‐to‐faceandonlinecomponentsintoeachother.Therecanbeatendencytosimply“bolton”onlinecomponentstoaface‐to‐facecourse,insteadthecourseshouldbeseenasiftwocansofdifferentlycoloredpaintsarebeingmixed.Thismixshouldyieldanewpaintwheretheoriginalcolorsnolongexistasindependentpaints.Itisimportantthatthedesignofablendedlearningcourseisnotjustatemporalconstruct.
Boththeface‐to‐faceandonlinecomponentsofablendedlearningcourseneedtobealignedwiththestatedlearningobjectivesofthecourse.Inaddition,thecourselearningobjectivesshouldbeusedtoguidenotjusttheactivitiesofthecourse,butalsotheportionofthecourseinwhichtheactivityoccurs.Theface‐to‐faceportionshouldbestudent‐centeredwhiletheonlineactivitiesarebothchallengingandengagingtocomplementtheface‐to‐faceportion.
Recommendations
Thecourseshouldbedesignedinawaythatintegratesboththeonlineandface‐to‐faceportionsinaclearandconciseway.
Onlinematerialsaredesignedtosupporttheface‐to‐facesession. Onlineresourcesareprovidedthatallowsfortheactivitiesthatbeganintheface‐to‐face
sessiontocontinueonline.
Course Development & Instructional Design 29
5. Instructional materials (both online and in‐class) and course syllabi are reviewed periodically to ensure they meet the blended course's and program's learning outcomes.*
AccordingtotheSouthernAssociationofCollegesandSchoolsCoreRequirement2.5(SACS,2012),institutionsshouldengage“inongoing,integrated,andinstitution‐wideresearch‐basedplanningandevaluationprocessesthat(1)incorporateasystematicreviewof…outcomes;(2)resultincontinuingimprovementininstitutionalquality;and(3)demonstratetheinstitutioniseffectivelyaccomplishingitsmission.”Furthermore,ComprehensiveStandard3.3.1prescribesthattheinstitutionwillidentify“expectedoutcomes,assessestheextenttowhichitachievestheseoutcomes,andprovidesevidenceofimprovementbasedonanalysisoftheresults...ineducationalprograms,toincludestudentlearningoutcomes.”Eachoftheregionalaccreditorshassimilarrecommendations.Forcontinuousimprovement,theinstitutionshouldhaveaperiodicreviewoflearningoutcomes,analyzethestudentachievementdata,anddevelopimprovementsbasedonthosedata.Thisshouldincludeblendedcoursesaswellastraditionalandonlinecourses.
Onceblendedcoursematerialsaredeveloped,thecoursedevelopmentprocessshouldnotendatthatpoint.Infact,rarelyshouldblendedcoursematerialsremaincompletelyunaltered.Likewise,instructionaldesignmodelsrecommendanevaluationprocessshouldbeimplementedafterthecourseisfirsttaught(andinsubsequentofferings)andpossiblerevisionsaddressedforcontinuousimprovement,withtheultimategoalbeingtheachievementoflearningoutcomes.Allblendedcoursesshouldbereviewedonaperiodicbasis.
Recommendations
Engageinasystematicreviewoflearningoutcomestodeterminecurrency,relevance,andmeasurability.
Implementaprocesswhereallcoursesarereviewedandupdatedonarecurringbasis. Systematicallyreviewandrefineblendedinstructionalmaterialstobesuretheysupport
thecoursesyllabusandcurrentlearningoutcomes. Systematicallyreviewblendedcoursematerialsandcoursesyllabitoensurerelevanceof
educationaltechnologyandassessmentactivities.
30 Quality Scorecard for Blended Learning Programs
6. Course assignments and activities are reviewed periodically to ensure they meet the blended courses' and program's learning outcomes.
Asstudentsprogressthroughablendedcourse,assessmentsmustbedevelopedthatmeasurethelevelofstudentsuccessandmasteryofthematerials.Theseassessmentsmaytakemanyforms;however,itiscriticalthatassessmentsmapexactlytothecoursecontent,andthattheyaredefinedearlyinthecoursedesignprocess.Assessmentsfortheblendedenvironmentdonotnecessarilyneedtomatchassessmentsgiveninface‐to‐facecourses,althoughforcomparisonpurposes,thefacultymaychoosetoadministerthesameassessment.Assessmentsshouldbemappedearlyinthedesignprocess.Assessmentsthatworkwellforblendedcourseworkincludepapers/essays,casestudies,problem‐basedactivities,quizzesandtests,andgroupprojects.Papersandotherwrittenassessmentsareofteneffectiveevaluationtoolsforblendedlearning,astheycanindicateastudent’scriticalthinkinganddirectmasteryofthematerial.
Newformsofblendedassessmentareconstantlyevolvingthatallowamorestudent‐centeredapproach.Properlydesignedandimplementedassessmentsshouldprovidetheinstructorwithevidenceofthestudents’learning,theefficacyofthecoursematerials,andprovideinputforfutureenhancementsofthecoursematerials.
Recommendations
Mapcourseassessmentsearlyinthecoursedesignprocess. Recognizethatasingleassessmentcanmeasureseverallearningobjectives. Encouragetheuseofrubricsforassessmentthatguidestudentsandprovidefeedback. Provideself‐assessmenttools(thatprovideautomaticfeedback)tostudentsasthey
progressthroughtheclass.
Course Development & Instructional Design 31
7. Student‐centered instruction is considered during the course development process (i.e., student engagement, immersion, and personal responsibility).
Student‐centeredinstructionacknowledgestheshiftingrolesintheclassroom(Dewey,1938;Piaget,1963;Vygotsky,1978)sothatstudentslearnbestwhentheyarepersonallymotivatedtoobtainnewskills,behaviors,andknowledgetosolveanauthenticchallenge.Thus,blendededucationisparticularlywell‐suitedforthisshiftinrolesbecauseofitscapacitytodrawonindividualperspectivesandcomparethemwithmultipleperspectives.Inlearner‐centereddesign,theteachershiftstothefacilitatoroflearningandamodelofexpertlearning,ratherthanadispenserofknowledge(NorthCentralRegionalEducationalLaboratory,n.d.),andthuspromotesinteractionamonglearnerstosolveauthentic,real‐worldproblems,andcollaborateonsolutionsandlearningpaths.Blendedinstructorsareseenbystudentsasservinganorganizingandcoursedesignfunctionmorethantraditionalrolesoflecturer,discussionfacilitator,orsocialsupporter(Hung&Chou,2015).
Recommendations
Whendesigningtheblendedcourse,identifychallengequestionsthattriggerindividualresponsesandmotivateinteractionamonglearners.
Connectassignmentswithlearner’slivesandpreviousexperiences. Designactivitiesandassignmentsthatengagestudentsinlearninghowtheylearn. Reducetheroleoflecturesanddirectinstructionactivitiesandfocusonstudentproblem‐
solving,collaboration,andlearningpaths.
32 Quality Scorecard for Blended Learning Programs
8. Course design promotes both faculty and student engagement.
Blendedlearningofferstheopportunitytoengagelearnerswitheachotherandwiththecoursecontent.Bothstudentsandfacultycanhavearichandrewardingexperience.
Blendeddeliveryoffersawidearrayofopportunitiesforengagementthroughcoursedesignanddelivery.Coursedesignthatallowsstudentstointeractwithcontent,faculty,andclassmateshelpsmaintainahighlyengagedenvironment(Swan,2004).Infact,severalofChickeringandEhrmann’s(1996)SevenPrinciplesforGoodPracticeinUndergraduateEducationcenteronalllevelsofinteractionandindicatehowtechnologycanbeusedeffectivelyforinteraction.
Today,facultyandstudentsareutilizingmoreengagingandcollaborativeactivitiesthatexpandengagement,suchasimmersiveenvironments,blogs,wikis,audioandvideoproductions,andcollaborativeprojects.TheOnlineLearningConsortium’sEffectivePracticesprovideexcellentexamplesofinteractivecollaborationthatengagesstudents(OnlineLearningConsortium,2016).
Recommendations
Provideinstructionaldesignsupportforfacultytotransitionfromaclassroom‐based(possiblylecture‐based)coursetoaninteractiveblendedcoursethatemphasizesactivelearning.Thisshouldincludeexamplesandmodelsofbestpractices.
Providefacultywiththeappropriatetrainingrequiredtousetechnologytoincreaseinteraction.
Encouragefacultytoprovidestudentswithclearlydefinedexpectationsregardingmethodsandfrequencyofengagementwithotherstudents,faculty,andcontent.
Providethemeansforstudentstoprovidefeedbackregardingthelevelofengagementofferedbythedesignofthecourse.
Course Development & Instructional Design 33
9. Course workloads are reviewed to ensure it is appropriate for designated credit allocation.
Blendedcourseworkloadsshouldbereviewedperiodicallytoensurethattheworkloadisconsistentwithcredit/clockhourasdefinedbytheorganizationandUSDE(2015)toensurecompliancewithregulatoryandaccreditingrequirements.Thiswillincludetraditionalseattimeaswellastimeneededtocompletedesignatedlearningactivitiestoachievethecourselearningoutcomesusingtheblendedformat.Academicorganizationswillneedtobeabletoprovidesupportfortheequitabledistributionofcreditamongblended,onlineandfacetofacecourses(ForumonEducationAbroad,2014).Itisimportanttonotethatwhilethecourseworkloaddoesnotdifferentiatebetweenfacetofaceoronlinelearningactivities,itdoesrefertoanequitableamountofstudentwork(USDE,2013).
Recommendations
Useastandardprocessfordefiningcredit/clockhourstoensureconsistencyinallocationaswellascongruencywithregulatoryandaccreditationstandards
Periodicallyreviewexistingblendedcoursestoensuretheyhaveappropriatelydesignatedcreditallocation.
Reviewnewblendedcoursestoensuretheyhaveappropriatelydesignatedcreditallocation.
34 Quality Scorecard for Blended Learning Programs
10. A process is established and followed for evaluating the effectiveness of current and emerging technologies to support the achievement of learning outcomes and delivering course content.
Evaluatingandrecommendingtheuseofnewtechnologyfortheblendedclassroombecomesmorechallengingeachdaywiththerapidintroductionofnewapplicationsandfunctionality.Thus,aninstitutionshouldplanfortheappropriatemeasuresneededtosystematicallyanalyzetechnologiesthatwillsupportblendedteachingandlearning.
Newtechnologiesemergeatsucharapidpacethatinstructionaldesignersandacademictechnologistsneedtocontinuouslymonitorandexamineissuessurroundingevaluatingandintegratingemergingtechnologyforteachingandlearning,withafocusonachievinglearningoutcomes.Today'stoolkitofacademictechnologyandinnovativetoolsincludecloudcomputing,videoconferencing,virtuallaboratories,ebooks,virtualworlds,socialnetworking,mobileapplications,openeducationresourcesandadvancedlearningmanagementsystems.Whatisnowinnovative,changesquickly;moreover,what'sinnovativeforoneinstitutionmaybeoldtechnologyforanotheroreventoo“bleeding‐edge”foryetanother.
Notalltools"outofthebox"arenecessarilyusefulforimprovinglearningoutcomes,soitisimportantforeducatorstofirstidentifylearningoutcomesandmethodsforachievementandthenapplytheappropriatetechnology.Integratingnewtechnologiesalsorequiresupdatedperspectivesabouttheiruse.Whileprovidingflexibility,theymaybedifficulttodeploybecauseofissuesrelatedtoscalability,policy,andtechnologyadoptionvariations.Armedwiththisknowledge,onlythencantheeducatorevaluateandrecommendthedeploymentorappropriaterejectionofthetechnology.
Recommendations
Supporttheuseofemergingtechnologiesbyexplainingtheirvalueforimprovingteachingandlearning.Forexample,Odom(2010)explainedhowWeb2.0technologiessupportconstructivism.
Askwhattheinstitutionneedsbeforeinvestinginnewtechnologies,forexample:o Whatemergingtechnologiesareneededforthisinstitutionatthistime?o Whataretheproblemsassociatedwithintegratingthem?o Whatstandardevaluativetoolswillbeusedtoassessandrecommendtheemerging
technologies? Encourageandenablefacultyandcontentexpertstobefamiliarwithinnovative
technologies. Minimizeskepticismtowardsnewtechnologybyusingwell‐trained,knowledgeable
academics,technologists,andanalystsintheevaluationprocess.
Course Development & Instructional Design 35
11. Course embedded technology actively supports the achievement of learning outcomes and delivering course content and unnecessary use of technology is minimized.*
Therapidlychanginglandscapeoflearningtechnologiesrequiresinstitutionalagilityandadaptation.Managingtheuseoftechnologyintheblendedclassroombecomesmorechallengingeachdaywiththecontinuousintroductionofnewapplicationsandfunctionality.Becauseofthis,wemustthinkbothstrategicallyandtacticallyaboutthebestuseoftechnologytosupportlearningoutcomesforthedeliveryofcoursecontent.
Technologyuseaboundsnotonlytosupportlearning,butalsotosupportstudentinformationsystems,studentfinancialservices,library,facultyrecruitmentanddevelopment,aswellasoutcomesassessment.Strategically,bestpracticesareneededfortheseareastoappropriatelyassimilatetechnologyuseacrosstheentireinstitutionandtobeasreliablyavailableaspossible.However,specialemphasisneedstobeonintegratingbestpracticesintotheuseoftechnologyforteachingandlearningwhiledevelopinganddeliveringcoursecontenttostudents.Theemphasisshouldincludetoolsthatsupportlearningoutcomesandwillproducesustainedresultsoncethestudentcompletesthecourseandprogram.
Wefacemanynewtechnologiesintheareasofweb‐basedmaterials,cloudcomputing,videoconferencing,virtuallaboratories,ebooks,virtualworlds,socialnetworking,mobileapplications,openeducationresources,andofcourse,thenucleusofblendedcoursedelivery,thelearningmanagementsystem.However,notalltoolsarenecessarilyapplicabletolearningoutcomes,soacademicsmustfirstidentifylearning‐relatedissuesandthenapplytechnologiesthataddressthelearningissues.Integratingthesetechnologiesrequiresupdatedperspectivesabouttheiruse.Whiletoolsmayincreaseflexibility,theymayalsobedifficulttodeployduetoissuesofscalability,policy,andtechnologyadoptionvariations.Technologyshouldnotbethedriverinthedeliveryofcourses;instructionaldesignersshouldseekthebestmixoftechnologiestoachievelearning(Palloff&Pratt,1999).
Theinstitutionmustaligntechnologieswithbothprogramoutcomesandteachingandlearninggoals.Thelearningmanagementsystemshouldbeuserfriendly.Forflexibility,technologyadvancesalreadyprovidethepotentialdeliveryofcontentanytime,anywhere.Fordiversity,institutionsmustmakedecisionsaboutapproachestoexternalcontentacquisition,learningmanagementsystemactivities,intelligenttutoringimplications,adaptiveassessments,etc.Forconsistency,technologymustsupportcoursecreationinacentralizedfashioninthedesign,development,andmaintenanceofblendedcoursesthatareenhancedwithacademiccontent,sophisticatedtools,andinnovativetechnologies.
Recommendations
Developinstitutionalguidelinesandstandardsforinvestigatingtechnologies,notjustforthesakeofintegratingnewtoolsbuttobettersupportlearningoutcomesandtheneedsoflearners.
Developstandardsandpoliciesforappropriatelyacquiringandintegratingopensourcematerialsandlearningobjects,audio,video,andothermultimedia.
Providebestpracticesofwheretolocatequalityopensourcematerials,audio,video,andothermultimedia.
36 Quality Scorecard for Blended Learning Programs
Provideguidanceforusingnewtechnologiesintheformofreports,presentations,demonstrationsorjust‐in‐timetrainingmaterials.
Useselectioncriteriaforevaluatingnewtechnologies,suchasaccessibilityandaffordability,usability,privacyandintellectualpropertyaffordance,workloadandtimemanagement,andinteractivity(Bovard,n.d.).
Course Development & Instructional Design 37
12. Usability tests are conducted and applied and recommendations based upon Web Content Accessibility Guidelines (WCAGs) are incorporated in the course design process.
AccordingtotheWebAccessibilityInitiative(2012),theWebContentAccessibilityGuidelines(WCAG)weredevelopedthroughacooperativeprocesswithindividualsandorganizationsaroundtheworld.Thegoalistoprovideasinglesharedstandardforwebcontentaccessibilitythatmeetstheneedsofindividuals,organizations,andgovernmentsinternationally.
TheWebAccessibilityInitiative(2012)providesmanyresourcesinthefollowingareas:
● GettingStarted● DesigningforInclusion● GuidelinesandTechniques● PlanningandImplementing● EvaluatingAccessibility● PresentationsandTutorials● GettingInvolvedwiththeWebAccessibilityInitiative(WAI).
WCAGisatechnicalstandard(WebAccessibilityInitiative,2012)andisnotintendedtoserveasanintroductiontoaccessibilityissues.Asablendededucationadministrator,itmaybenecessaryfortraininganddevelopmentactivitiestobeginwithanintroductionandgraduatetoWCAGdependingontheaudience.However,asmoreandmoreblendedcoursestranscendgeographicboundaries,developingblendedcoursesthatmeettheWCAGtechnicalstandardshouldbeconsideredbestpractice.
Recommendations
StayabreastofadvanceswithintheWebContentandAccessibilityGuidelines(WCAG)initiative.
Providetrainingopportunitiesforfacultyandcoursedevelopment/instructionaldesignpersonnelwhichhighlighttheimportanceofdesigningforinclusionandshowcaserelevantguidelinesandtechniques.
PlanandimplementblendedcoursesthataredesignedusingtheWCAGfromtheinitialfoundations.
UseachecklistforbestpracticesforblendedcoursedevelopmentaccordingtoWCAG. Identifyanddeployinternalusabilitytestingprocessesthatunderscoretheimportanceof
andincorporatetheWCAG.
38 Quality Scorecard for Blended Learning Programs
13. Curriculum development is a core responsibility for faculty (i.e., faculty should be involved in either the development or the decision making for the blended curriculum choices).
Thecurriculumisthemajorstatementanyinstitutionmakesaboutitself,aboutwhatitcancontributetotheintellectualdevelopmentofstudents,aboutwhatitthinksisimportantinitsteachingservicetosociety(TheCarnegieFoundationfortheAdvancementofTeaching,1977).
Theresponsibilityforcurriculumdevelopmentordecisionmakingandselectionshouldresidewithfacultybecauseoftheirprofessionalcredentialsandcommitmenttoteaching,research,andservice.Thiscoreresponsibilitytypicallyincludesdevelopingcurriculaandkeepingitcurrent;reviewingandevaluatingcoursesandprograms,maintaininggradingstandards,selectinginstructionalresourcesandmedia,anddevelopingnewprograms.Becauseoftheprolificamountofresourcesforblendedcourses,facultyshouldbeinvolvediftheprogramorinstitutionchoosestouseresourcessuchaspurchasingready‐madecoursematerials.
Thefollowingallputpressureontraditionalgovernanceanddecision‐makingprocesses,includingcurriculumdevelopment:
● theproliferationoflearningmediaandchannels;● competency‐basedlearninginitiatives;● regulatorydemandsforgreateraccountability;● federal,corporateandfoundationcallsforbetterworkforcedevelopment;● theavailabilityofopeneducationalresourcesandcommercialcontentandcourse
providers;● standardizationofcourses;● thegrowthofpart‐timefacultyandnon‐traditionallearners;and● fiscalshortfalls.
Recommendations
Ensurefacultyleadintheblendedcurriculaprocessincludingdecisionstoadoptcontentexperts,andopenresources.
Ensurefacultyleadorareheavilyinvolvedindecisionstochangecurricula,courses,andprograms.
Providefacultywithdataaboutlearningoutcomes,persistence,graduationrates,defaultrates,costefficiencies,andgraduateemployability.
Engageinnationaleffortstodetermineandcomparesuccessratesandqualitystandards.
Course Development & Instructional Design 39
14. Faculty support and resources are provided to promote the best use of blended delivery method in course development and instructional design to facilitate teaching and learning.
Whendesigningablendedcourse,itisimportanttofullyintegrateboththeface‐to‐faceandonlinecomponentsintoeachother.Therecanbeatendencytosimply“bolton”onlinecomponentstoaface‐to‐facecourse,insteadthecourseshouldbeseenasiftwocansofdifferentlycoloredpaintsarebeingmixed.Thismixshouldyieldanewpaintwheretheoriginalcolorsnolongexistasindependentpaints.Itisimportantthatthedesignofablendedlearningcourseisnotjustatemporalconstruct.
Boththeface‐to‐faceandonlinecomponentsofablendedlearningcourseneedtobealignedwiththestatedlearningobjectivesofthecourse.Inaddition,thecourselearningobjectivesshouldbeusedtoguidenotjusttheactivitiesofthecourse,butalsotheportionofthecourseinwhichtheactivityoccurs.Theface‐to‐faceportionshouldbestudent‐centeredwhiletheonlineactivitiesarebothchallengingandengagingtocomplementtheface‐to‐faceportion.
Recommendations
Thecourseshouldbedesignedinawaythatintegratesboththeonlineandface‐to‐faceportionsinaclearandconciseway.
Onlinematerialsaredesignedtosupporttheface‐to‐facesession. Onlineresourcesareprovidedthatallowsfortheactivitiesthatbeganintheface‐to‐face
sessiontocontinueonline.
40 Quality Scorecard for Blended Learning Programs
ReferencesforCourseDevelopmentandInstructionalDesign
Aramal,K.,&Shank,J.(2010).EnhancingStudentLearningandRetentionwithBlendedLearningClassGuides.EducauseReview(December,2010).Retrievedfromhttp://er.educause.edu/articles/2010/12/enhancing‐student‐learning‐and‐retention‐with‐blended‐learning‐class‐guides
Bovard,B.(n.d.).Web2.0selectioncriteria:Savetimechoosinganappropriatetool.Retrievedfromhttp://150.164.100.248/cei/data1/arquivos/Web20selectioncriteria.pdf
Chickering,A.W.,&Ehrmann,S.(1996).ImplementingtheSevenPrinciples:Technologyaslever.AAHEBulletin,October,3‐6.Retrievedfromhttp://www.tltgroup.org/programs/seven.html
Dewey,J.(1938).Experienceandeducation.NewYork,NY:FreePress.
ForumonEducationAbroad.(2014).DefiningtheCreditHourforTitleIVFederalFinancialAidPurposes.Retrievedfromhttp://forumea.org/wp‐content/uploads/2014/10/DefiningtheCreditHourforTitleIVFederalFinancialAidPurposesFinal.pdf
Garrison,D.R.,Vaughan,N.(2008).BlendedLearninginHigherEducation:Framework,Principles,andGuidelines.Jossey‐Bass
Hung,M.L.,&Chou,C.(2015).Students'perceptionsofinstructors'rolesinblendedandonlinelearningenvironments:Acomparativestudy.Computers&Education,81,315‐325.
McGee,P.,&Reis,A.(2012).Blendedcoursedesign:Asynthesisofbestpractices.JournalofAsynchronousLearningNetworks,16(4),7–22.Retrievedfromhttp://sloanconsortium.org/jaln/v16n4/blended‐course‐design‐synthesis‐best‐practices
Mesh,L.I.(2016).Acurriculum‐basedapproachtoblendedlearning.JournalofE‐Learning&KnowledgeSociety,12(3),87‐97.
Moskal,P.,Thompson,K.,&Futch,L.(2015).Enrollment,engagement,andsatisfactionintheBlendKitfacultydevelopmentopen,onlinecourse.OnlineLearning,19(4),Retrieved from http://olj.onlinelearningconsortium.org/index.php/olj/article/view/555/173.
Niemec,M.,&Otte,G.(2009).Anadministrator’sguidetothewhy’sandhow’sofblendedlearning.JournalofAsynchronousLearningNetworks,13(1),19‐30.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ909846.pdf
NorthCentralRegionalEducationalLaboratory.(n.d.).Teachervs.learner‐centeredinstruction.Retrievedfromhttp://www.nclrc.org/essentials/goalsmethods/learncentpop.html
Odom,L.S.(2010).MappingnewlyidentifiedWeb2.0benefitstoknownbestpracticesindistanceeducation.Retrievedfromhttp://instruction.blackhawk.edu/blackboard/pdf/Mapping_Web_2‐0_Benefits_LaddieOdom.pdf
OnlineLearningConsortium.(2016).OurQualityFramework.Retrievedfromhttp://onlinelearningconsortium.org/about/quality‐framework‐five‐pillars/
Palloff,R.,&Pratt,K.(1999).Buildinglearningcommunitiesincyberspace:Effectivestrategiesfortheonlineclassroom.SanFrancisco,CA:Jossey‐Bass.
Piaget,J.(1963).Originsofintelligenceinchildren.NewYorkCity,NY:Norton.
Course Development & Instructional Design 41
Puzziferro,M.,&Shelton,K.(2008,December).Amodelfordevelopinghigh‐qualityonlinecourses:Integratingasystemsapproachwithlearningtheory.JournalofAsynchronousLearningNetworks,12,3‐4.Retrievedfromhttp://olj.onlinelearningconsortium.org/index.php/olj/article/view/58
SouthernAssociationofCollegesandSchoolsCommissiononColleges.(2012).Theprinciplesofaccreditation:Foundationsforqualityenhancement,2012Edition.Decatur,GA:SouthernAssociationofCollegesandSchoolsCommissiononColleges.Retrievedfromhttp://www.sacscoc.org/pdf/2012PrinciplesOfAcreditation.pdf
Swan,K.(2004).Relationshipsbetweeninteractionsandlearninginonlineenvironments.Needham,MA.Retrievedfromhttps://www.researchgate.net/publication/250700769_Relationships_Between_Interactions_and_Learning_In_Online_Environments
TheCarnegieFoundationfortheAdvancementofTeaching.(1977).Missionofthecollegecurriculum:Acontemporaryreviewwithsuggestions.SanFrancisco,CA:Jossey‐Bass.
UniversityofCentralFlorida(UCF),&AmericanAssociationofStateCollegesandUniversities(AASCU).(n.d.)BlendKitCourse:DIYProjectTasks.Retrievedfromhttps://blended.online.ucf.edu/files/2011/06/implementation_checklist.pdf
UnitedStatesDepartmentofEducation(USDE).(2013).LawandGuidance:ProgramIntegrityQuestionandAnswers‐CreditHour.Retrievedfromhttp://www2.ed.gov/policy/highered/reg/hearulemaking/2009/credit.html
UnitedStatesDepartmentofEducation(USDE).(2015).FederalStudentAidHandbook.Retrievedfromhttps://ifap.ed.gov/fsahandbook/attachments/1516FSAHbkActiveIndexMaster.pdf
Vygotsky,L.S.(1978).Mindinsociety.Thedevelopmentofhigherpsychologicalprocesses.Cambridge,MA:HarvardUniversityPress.
WebAccessibilityInitiative.(2012).Webaccessibilityinitiativehomepage.Retrievedfromhttp://www.w3.org/WAI/
42 Quality Scorecard for Blended Learning Programs
1. Theblendedcourseincludesasyllabusoutliningcourseobjectives,learningoutcomes,evaluationmethods,booksandsupplies,technicalandproctoringrequirements,andotherrelatedcourseinformation,makingcourserequirementsandcoursescheduletransparent.*
2. Thecoursestructureensuresthatallstudents,regardlessoflocation,haveaccesstolibrary/learningresourcesthatadequatelysupporttheblendedcourse.*
3. Expectationsforstudentassignmentcompletion,gradepolicy,andfacultyresponseareclearlyprovidedinthecoursesyllabus.*
4. Linksorexplanationsoftechnicalsupportareavailableinthecourse(i.e.,eachcourseprovidessuggestedsolutionstopotentialtechnicalissuesand/orlinksfortechnicalassistance).
5. Rulesorstandardsforappropriatestudentbehavior,bothonlineandface‐to‐face,areprovidedwithinthecourse.
6. Instructionalmaterialsareeasilyaccessibletothestudent,easytouse,withanabilitytobeaccessedbymultipleoperatingsystemsandapplications.
7. Instructionalmaterialsareeasilyaccessedbystudentswithdisabilitiesviaalternativeinstructionalstrategiesand/orreferraltospecialinstitutionalresources.
8. Theblendedcourseisvisuallyappealingtothestudentandthecourseisnavigationallysound.
Course Structure
1. The blended course includes a syllabus outlining course objectives, learning outcomes, evaluation methods, books and supplies, technical and proctoring requirements, and other related course information, making course requirements and course schedule transparent.*
Thesoonerstudentsknowthecourserequirements,theeasiertheirtransitionwillbeintotheblendedclassroom.Themoredetailedthesyllabus,thefewerquestionsaninstructorwillhavetorespondtointheearlystageofparticipantsgettingtoknowandtrusteachotherandbecomecomfortableintheblendedcourseenvironment.
Ataminimum,adetailedcoursesyllabuswillincludecourseobjectives,learningoutcomes,evaluationmethods,informationaboutthetextbook(s)andotherrequiredmaterials,aswellastechnicalrequirements.Inaddition,thetextbookandrequiredcoursematerialinformationshouldbeavailablewellbeforeastudentenrollsinthecourse.Thisallowsadvancepreparationonthepartofthestudent,timeforaccessibilitysupportservicestohelpwithinput/outputissuesforstudentswithdisabilities,andcanalsoserveasanicerecruitingtoolfortheprogram.
Weeklymodules/unitsshouldindicateamoregranularsetoflearningoutcomesthatareclearlylinkedbacktothecourseobjectives.Toenhancetransparency,term‐specificdates(firstdayofclass,lastdaytodrop,finalsweek,gradesdue)andlocaldates(campus,state,andnationalholidays)shouldbeindicated.
Recommendations
Provide,ataminimum,thetextbookandrequiredmaterialsorresourcestostudentsinadvanceoftheircourseregistration.
Createauser‐friendlysyllabusthatiseasilyaccessible.Separatethesyllabusintosectionstoallowforeasydownloadingandprintingasneeded.
Developawell‐designedsyllabusincludinganswerstocommonquestions,providelinkstopopularcampuswebsites(e.g.,library)andincludeallinstitutionalpolicyrequirements.
Utilizeasetofconsistentstandardsforsyllabusdevelopmentsuchasabestpracticesrubric.
44 Quality Scorecard for Blended Learning Programs
2. The course structure ensures that all students, regardless of location, have access to library/ learning resources that adequately support the blended course.*
Allstudentsareentitledtothesamelearningresourcesprovidedbytheinstitutionregardlessofiftheyareattendingface‐to‐face,blended,orfullyonlinecourses.Thismeansanylearningresourcesupportthatisprovidedoncampusshouldalsobeprovidedoratleastofferedtoblendedstudents.Accessshouldincludetheinstitution’slibraryforbookcheckoutandonlineresearchcapabilitiesandallotherinstitutionallearningresourcessuchasawritingcenter,tutoringsupport,counseling,etc.
Inadditiontoalinktothelibrarywebsite,alinkspageforresourceswithinthecoursecanlistrecommendedjournalsobtainablethroughthecampuslibrary,linkstorelevanthowtouseonlineresources,informationaboutplagiarism,howtogethelp,andlinkstoexternalwebsitesthathaverequiredorrecommendedmaterials(whichmustbeaccessibleoravailableinalternateformats).Instantmessagingorchatprogramsmaybeusedtoprovidejust‐in‐timeservicethroughtheinstitutionalwebsitebecauseblendedstudentsmayneedadditional
Recommendations
Offerquicklinkstothelibrarywebsite,writingcenter,tutoring,andotherapplicablelearningresourceswithintheblendedcoursematerials.
Ensurealllibraryandacademicsupportservicesareavailabletoalltypesofstudents. Dedicateatleastonelibrariantoprovidededicatedsupportinawaythatisfeasiblegiven
theinstitution’sorganizationalstructureandresources. Communicatewithblendedstudentsusingavarietyofmethodstoremindthemofall
resourcesavailabletothem.
Course Structure 45
3. Expectations for student assignment completion, grade policy, and faculty response are clearly provided in the course syllabus.*
Clearlypostedexpectations,instructions,duedates,pointassignments/rubrics,andinstructorresponsetimesareveryimportantforstudentswhomightnototherwisebeorganizedenoughtokeepupovertime,andareparticularlyimportantforstudentswithexecutivefunctiondisorders.Gradingpoliciesshouldbedesignatedinthecoursesyllabus,includinglinkstoassignmentrubrics.
Instructorresponsetimescoulddifferentiateamongdiscussionboards,email,holidays,anddays“off”andshouldclearlyshowthattheinstructorwillnotbeavailable24x7.However,thereshouldbeguidelinesthathelpwithstudentexpectationsofwhentoanticipatearesponsefromtheinstructor.Onlineofficehourscanbehelpfulformoststudents,thoughsynchronousrequirementswillbeanadvantagetosome.Synchronousmethodsincludeinstantmessaging,webconferencing,face‐to‐facemeetings,andchat.Considerprovidingaone‐on‐onesessionwithastudentwhoneedsextratimetoinputtextorprocessquestions/answers.
“Head’sup”remindersofupcomingeventsareusefulforallstudents,andcangenerallybelinkedtotheemailsystemoftheLMSsothattheinformationwillsimultaneouslybecomeanannouncementwithinthecourseandanemailtoeachstudent.
Recommendations
Provideclearinstructorofficehoursaswellasexpectedresponsetimetoemails,discussionboardpostings,andtextmessages(ifencouragedbyinstructor)inthecoursesyllabus.
Designateinthecourseschedulewhencoursesessionsbegin,andincludetopics,readingassignments,andstudentdeliverableswithduedateandtime.
Ensureinstitutional,program,and/orspecificcoursegradingpoliciesareincludedinthesyllabus.
Encouragefacultytoprovidegradingrubricsforeachassignment. Developandprovidealatepolicythatclearlycommunicateswhatthepenaltywillbe(if
acceptedatall)whenassignmentsaresubmittedlate.
46 Quality Scorecard for Blended Learning Programs
4. Links or explanations of technical support are available in the course (i.e., each course provides suggested solutions to potential technical issues and/or links for technical assistance).
Allofthetechnicalaspectsofablendedcoursearenotalwaysobvioustostudents,whojustwanttoquicklygetintothecoursematerialsandstartthecourse.Instructorsshouldnothavetospendvaluableclasstimetryingtotroubleshoottechnicalissuesforstudentswhoneedhelp.Theinstitutionshould,ataminimum,provideahelppageortutorialforthelearningmanagementsystemandthecampusemailwebsite.Acampuspolicyon“safecomputing”isbeneficial.Students(andinstructors)areoftennotsureifaproblemiswiththeirInternetbrowser,theirblendedInternetprovider,thelearningmanagementsystem,theinstitution’sserver,ortheInternetitself.Instructorsshouldnotbetheprimaryresourceforresolvingtheseissues,especiallyfortheirstudents.
Accesstospecifictechnologysupportcanbelinkedwithinthecoursematerials.Forexample,ifanaudiofileistobeplayed,alinktothecorrespondingplayersoftwareshouldbeprovidedwithinstructionsonhowtodownloadit,installit,andplaythefile.
Recommendations
Provideacoursediscussionthreadfocusedontechnicalissueswherestudentsareencouragedtopostandanswerquestionstohelpeachother.
Provideafrequentlyaskedquestionsareainthecoursethataddressestypicaltechnicalsupportissues.
Includebasicsupportlinksinthecoursesyllabusandalsoincludealinktothestudentaccessibilitycenter/resources(alongwithafriendlystatementonrequestinghelpfordisabilityaccommodations).
Remindstudentstostorecopiesoftheirelectronicfilesinmorethanoneplaceandhaveanupdatedviruscheckerinstalledontheircomputer.
Ensurecampustechnicalresourcesandsupportsareavailableforstudentsinface‐to‐faceaspectsoftheblendedcourseorprogram.
Course Structure 47
5. Rules or standards for appropriate student behavior, both online and face‐to‐face, are provided within the course.
Becauseoftheubiquityoftechnology‐mediatedinteractionsthatblendedstudentsface,itisreallyimportanttobeawareof,andprovidesupportfor,eachaspectoftheseinteractions.Fromthetimeastudentfirstlogsintothecourseandfindsthesyllabus,throughoutalltheassignmentsubmissions,discussions,anduntilthelastexamorpaperishandedin,technologyiseverpresentandcanpossiblytocreatebarriersorbeasourceofstudentfrustration.Thisinturn,canleadtostudentmisbehaviorinthecourseandwhilefacultyaremoreaccustomedtodealingwiththatface‐to‐face,theonlineaspectoftheblendedclassroomcanbeintimidating.Facultyneedaframeworkofclassroomrulestoreferenceandreceiveguidance.
Someinstitutionsaredevelopingresourcesguidestoassistfacultyinmanagingstudentbehaviorblended.Examplesofsomeoftheseguidesorcodesofconductspecificallyalignedwithblendedcoursesinclude:
FoothillCollege(n.d.).http://www.foothill.edu/services/honorint.php VirginiaCommonwealthUniversity(2009).
https://www.vcu.edu/cte/resources/OTLRG/04_12_Behavior.html
Recommendations
Developrules,standards,orcodesofconductfocusingonappropriatestudentbehaviorforblendedstudents.
Trainfacultytobepreparedforpotentialstudentmisbehaviorwithrecommendationsforhowtoaddressit.Thiscouldincludepracticessuchasblendedroleplayingforhowaninstructorshouldrespondwhenastudentverbally(viatext)attacksanotherstudentinthediscussionforums.
48 Quality Scorecard for Blended Learning Programs
6. Instructional materials are easily accessible to the student, easy to use, with an ability to be accessed by multiple operating systems and applications.
Inclusivedesignshouldbethefoundationforblendedcoursedesignandcoursestructure.Whilestudentswithdocumenteddisabilitiespresentthemostobviousdesignchallenges,itisusefultoexpandthedefinitionof“at‐risk”beyondthat.Problemswithvision,mobility,hearing,andcognitioncanaffectanyoneatanytime(andnearlyeveryoneastheyage),ascanillnessandthesideeffectsofmedicationandfatigue.Otherbarriersincludehavingtoreadandwriteusinganunfamiliarlanguage,havingtodeciphernewlyencounteredacademicwritingandsubjectareaterminology,andbalancingmanydemandsontimeandenergy.
Usabledesignincludespresentingcoursematerialswith“writingfortheweb”rulessuchaschunking,plentyofwhitespace,simpleandobviousnavigation(withasfewclicksaspossible),clearlylabeledbuttonsandmenuitems,judicioususeofcolorwithanemphasisongoodcontrast,andnotusingcoloralonetoprovideinformationforthosewhoarecolor‐blind(forexample,adisplaywitharedbarshowingfemales,thegreenbarshowingmalesisindecipherabletosomestudents).
AccordingtotheUsability.gov(2014)website“UXisagrowingfieldthatisverymuchstillbeingdefined”(para.4).TheUserExperienceProfessionalAssociation(2014)definesuserexperiencedesignas“adisciplineconcernedwithalltheelementsthattogethermakeupthatinterface,includinglayout,visualdesign,text,brand,sound,andinteraction”(para.2).Gube(2010)assertedthatthe“userexperienceishowapersonfeelswheninterfacingwithasystem.Thesystemcouldbeawebsite,awebapplicationordesktopsoftwareand,inmoderncontexts,isgenerallydenotedbysomeformofhuman‐computerinteraction(HCI)”(para.4).Forblendedadministrators,focusingresourcesontheUXasitrelatestocoursestructureisincreasinglybecomingabestpractice.
Recommendations
Writethecoursesyllabus,instructions,andweeklyobjectivesineasytounderstandlanguage.
Usefileformatsthatstudentsaremostlikelytouse,i.e.pdforhtmlfiles. ProvidelinkswithintheblendedcoursetoInternetbrowserplug‐inswhenusingless
commonfileformats.Forexample,pdffilesneedtheAdobeReaderplug‐intoallowthefiletobereadwithinthebrowser.
ConsiderhowUXdesignprinciplesbecomeinherentlyembeddedwithinthecoursestructure.
Course Structure 49
7. Instructional materials are easily accessed by students with disabilities via alternative instructional strategies and/or referral to special institutional resources.
Adisabilitycanoccursuddenlyandunexpectedlytoanyone.Planningaheadbydesigningaccessiblecoursematerialsnotonlyservesallstudentsandinstructors,italsocreatespossibilitiesforclassescontinuingevenunderconditionsofnaturalorotherdisasters.Infact,therearemanypeoplewhodonotquitemeetthethresholdfor“disability”butwho,nonetheless,facechallenges.
Learningstyles,preferences,andabilitiesalsovarywidelyamongstudentsandinstructors,soawell‐designedcourseshouldhavemultiplemodesofcoursematerialformatsandalternativewaystosubmitwork.
Researchisclear—brainsandbodiesdiffergreatly,andaone‐size‐fits‐allapproachtocoursedesignbenefitsrelativelyfewstudents.If,ontheotherhand,thecourseisdesignedtobeasinclusiveaspossible,allstudents(andinstructors)benefit.
Recommendations
Provideaccessiblecoursedesignbyensuringthatrequiredcoursematerialscanbeusedwithassistivetechnologysuchasclosed‐captionedvideosandsoundfilesthathavetextscriptsprovided.
Ensurethatfaculty/professionaldevelopmentoffices,inconjunctionwithlibraryandotherrelevantunitsprovideongoing“bestpractices”incoursedesigntrainingandsupportservicestoaddressstudentswhorequireaccommodations.
Welcomestudentswithdisabilities;invitethemtodiscusstheirneedswithadvisors,andprovidelinkstoadditionalrelevantresources.
50 Quality Scorecard for Blended Learning Programs
8. The blended course is visually appealing to the student and the course is navigationally sound.
Oftenacourseshellismadeavailabletostudentpriortothefirstface‐to‐facesessionofablendedlearningcourse.Thiscourseshellcanbethefirstimpressionastudentmayhaveofacourseanditisimportantthatthecoursebebothwelcomingandinformative.
Acourseshellthatcontainsprimarilytextcanbeintimidatingtostudents.Therefore,thecourseshellshouldappropriatelyuseaudio,images,andvideoinameaningfulwaythatenhancesthestudentexperience(whileconsideringtheneedsofdiversestudents).Foranexample,whenthecourseinstructorprovidesanintroductionofthemselvestheyshouldincludeeitheravideointroduction(withtranscript)oratext‐basedonewithaphoto.
Astudentshouldnotbeoverwhelmedbytoomanymenuchoices.Menuitemsshouldbebothclearandsimpleandthiscanbeachievedbyreducingthenumberofchoicesonthemainmenu.Themainmenushouldrepresentthekeyareasofthecoursethatastudentmayneedtoaccess.
Thecourseshelldesignshouldalsoinformthestudentsthatthisisacoursethatutilizesboththeface‐to‐faceandonlinelearningenvironments.Indoingthis,theintegrationofthetwoenvironmentsshouldbehighlightedbyexplicitlyidentifyingwhatastudentistodopriortoaface‐to‐facesessionandhowtheface‐to‐facesessionconnectsbacktotheonlineportionofthecourse.Inaddition,thecourseshelldesignshouldgroupcontentbyweekormodulekeepingthecoursecontentandcomponentsgroupedbycontextwithinthecourseinaconsistentmanner.
Recommendations
Courseshouldutilizeasimilarcourseshelldesignwithintheprogram. Navigationmenurepresentsthehierarchyofthecourse. Coursecontentispresentedincontexttohowitwillbeused. Thecourseshellclearlydistinguisheswhatistobecompletedbythestudentpriorto
comingtoaface‐to‐facesession.
Course Structure 51
ReferencesforCourseStructure
Gube,J.(October5,2010).Whatisuserexperiencedesign?Overview,tools,andresources.SmashingMagazine.Retrievedfromhttps://www.smashingmagazine.com/2010/10/what‐is‐user‐experience‐design‐overview‐tools‐and‐resources/.
Usability.gov.(2014).Userexperiencebasics.Retrievedfromhttp://www.usability.gov/what‐and‐why/user‐experience.html
UserExperienceProfessionalsAssociation.(2014).Definitionofuserexperiencesandusability.Retrievedfromhttp://uxpa.org/resources/definitions‐user‐experience‐and‐usability
52 Quality Scorecard for Blended Learning Programs
1. Student‐to‐StudentandFaculty‐to‐Studentinteractionareessentialcharacteristicsandareencouragedandfacilitated.*
2. Instructorfeedbackonstudentassignmentsandquestionsisconstructiveandprovidedinatimelymanner.*
3. Instructorsusespecificstrategiestocreateanengaged,learning‐focusedpresenceinbothmodalitiesofthecourse.
4. Facultyteachthecourseasonecohesivewhole,with"presence"inboththeface‐to‐faceandtheonlineportionsofthecourse.
5. Resourcesareprovidedtoassiststudentsinconductingresearchonlineandassessingthevalidityofonlineresources.*
Teaching and Learning
1. Student‐to‐Student and Faculty‐to‐Student interaction are essential characteristics and are encouraged and facilitated.*
Bothstudent‐to‐studentandfaculty‐to‐studentinteractionsarerecommendedforaricher,moreengagingblendedenvironment.Infact,Boettcher(2006)recommendedbothasbestpracticesforteaching,inordertoalleviatestudentsfromfeelingaloneorabandonedandtocreateapositivelearningcommunity.Bothtypesofinteractionscanbeaccomplishedwithavarietyofasynchronousandsynchronousactivities.
Faculty‐to‐studentinteractionservestwoprimarypurposes:supportiveandinstructional.Facultyteachingblendedcoursescancreateasupportiveclassroomenvironmentbyensuringthattheyareactiveinthecourseearlyandoften.Inaddition,asupportiveenvironmentmaybecreatedthroughintroductory,welcoming,andcontinuedmessagesandannouncementstoindividualstudentsandthewholeclassthroughoutthecourseterm.Thesemessagescanbedeliveredthroughavarietyofforms,includingtext,socialmedia,audio,in‐person,andvideo.Providingmultipleoptionsforeachinteractionallowsstudentstoselecttheaccessibleand/orthemostpersonallevelofengagement.
Facultycanprovideanengaginginstructionalenvironmentbyprovidinginteractioninclassdiscussions,emails,face‐to‐facemeetings,andthroughdirectionswithintheclasscontentsuchaslectures,PowerPointpresentations,articles,podcasts,etc.Theseinteractionsmayalsobeofamoreindividualnaturesuchasprovidingfeedbackonindividualassignments,providingguidanceonthedevelopmentofapaper,providingfeedbacktostudents’postingsinthreadeddiscussions,etc.Coursecontentmaylenditselftogroupworkinwhichstudentscollaboratetocompleteassignments.Inthesesituations,facultyoversightandinputareoftenrequiredforameaningfuloutcomeforthestudentssuchashelpingtoformthegroups,facilitatingcollaboration,andrequiringindividualparticipation.
Thelevelofstudent‐to‐studentinteractionmaydependonthenatureandoutcomesofthecourse.Careshouldbetakentoensurethattheinteractionsareameanstoeitherprovideasupportive/socialenvironmentorallowstudentstoachievethelearningoutcomes.Facultycanrequirestudent‐to‐studentinteractionthroughthetypesofassignmentsandassessmentsdevelopedforthecourse.Additionally,facultycancreateassignmentsthatrequirepeerreviewandgroupwork.
Modernlearningmanagementsystemsprovideastructureforbothfaculty‐to‐studentandstudent‐to‐studentinteractions.Theseinclude,butarenotlimitedto,threadeddiscussions,chatrooms,grouptoolsandpages,andvideo.Astechnologiesaredeveloped,itisimportantforthefacultymemberstodecideifandhowthetechnologycanbeusedintheircourses.However,choosingtheappropriatetechnologyforaparticularactivityisnotasimportantasthedesignoftheactivityitself.
Recommendations
Createandimplementpolicyregardingfacultyinteractioninblendedclassessuchasparticipatinginthediscussionboards,offeringofficehours,andprovidingrobustfeedbackonassignments.
Monitorcourseevaluationsforstudentcommentsonlackoffacultyinteraction. Providefacultytipsforhowtoencouragestudent‐to‐studentinteractions. Providefacultybestpracticesforgroupworkandrecommendedactivitiesforblended
teachingandlearning. Providestudentswithrequirementsforinteractionandmonitorstudentinteraction. Providesupportforfacultytoevaluateandadoptnewtechnologiestoenhanceinteraction.
54 Quality Scorecard for Blended Learning Programs
2. Instructor feedback on student assignments and questions is constructive and provided in a timely manner.*
Forthisqualityindicator,thedefinitionoffeedbackistheconstructiveinformationtheinstructorprovidestostudentsregardingcourseassignmentsandactivitiesandshouldincludedirectinstructionsothatstudentscancorrectandimproveunderstanding.Oneoftheprimarywaysfacultyprovideinstructionintheonlineaspectsoftheblendedclassroomisthroughfeedback(Shaw,2014).Facultyplayacriticalroleinthisprocessas“effectivefeedbackmovesstudentsbeyondreflectiononwhattheyhaveaccomplished;itmovesthemforwardbyhelpingthemtoidentifygapsinknowledgeandgoalsandstrategiesforfuturelearning,bothinthecourseandinnon‐courseactivitiesintheirlives”(Getzlaf,Perry,Toffner,Lamarche,&Edward,2009,p.16).Infact,Getzlafetal.(2009)foundthemesforeffectiveinstructorfeedback:
● thereshouldbestudentinvolvementtocreateafeedbackprocesswithindividualizedfeedback;itshouldbegentleguidance;
● itshouldbepositiveandconstructive;● itshouldbetimely;and,● itshouldhelpstudentstoidentifygapsinknowledgeandgoalsandstrategiesfor
futurelearning.Theinstitutioncanplayanimportantpartinthisprocessbyprovidingtoolssuchasfeedbackformsandrubrics,aswellasprofessionaldevelopmentonstrategiesandprocessesforgoodfeedback.Thefeedbackprovidedbytheinstructormustbewithinarelevanttimeframeastoprovideconstructivecorrectionbecauseitiseasyforstudentstobecomefrustratedanddisengagediftheydonotreceivefeedbackintimetoapplyittothenextassignment.
Recommendations
Encourageorrequireifpossiblethatfacultyprovideconstructivefeedbackoncourseassignmentswithinastatedtimeframe.Forexample:includingthestatementthatallgradedassignmentswillbereturnedwithinoneweekaftersubmissionwillhelpwithstudentexpectationsandmaypreventmultiplestudentemailsaskingwhentoexpectthereturn.
Providefacultyresourcessuchasmodelsofgoodfeedback,feedbackforms,andrubricstohelpformulateconstructivefeedback.
Whentrainingfacultytoteachblendedcourses,emphasizetheimportanceanduseofconstructivefeedback.
Considerprovidingdiversefeedbacktypesoneachgradedassignmenttoencouragecontentmastery.
Teaching and Learning 55
3. Instructors use specific strategies to create an engaged, learning‐focused presence in both modalities of the course.
Thiscriterionrefersspecificallytoinstructorpresenceratherthanacollectivepresenceofinstructorandstudents.Studentsinblendedcoursesmustknowthattheirinstructorispayingattention,scaffolding,challenging,assessing,communicating,monitoring,andenhancingtheirlearningexperience.Theyneedtofeelaconnectiontotheirinstructorandknowthatheorsheisaccessibleandinterestedintheirsuccess.Byusingspecificstrategiestoestablishapresence,instructorscanprovidemotivation,improvelearningoutcomes,avoidmisunderstandingsandfeelingsofisolation,andmaximizestudentsatisfaction.Instructorscanhelpstudentsbecomeeffectiveblendedlearnersaswellascanalsoexperienceamorerewardingteachingexperience.
Withoutrobustinstructorpresenceinablendedcourse,studentsoftenfeelisolated,unsupported,unmotivated,unrecognized,andunsuccessful.Inaddition,theironlywayofknowingiftheyadequatelycomprehendthematerialandarecompletingassignmentsasexpectedisthegradestheyreceive,whenitistoolate.Formativefeedbackfromanactiveinstructorisacriticalsupportforthelearningprocess(Shaw,2014).Thelearningexperienceisenrichedfurtherwheninstructorsparticipateinandhelpguideblendeddiscussionsandcollaborativeprojects.Studentscanbechallengedtothinkmoredeeply,critically,andcreativelyiftheinstructorisobservant,responsive,andengaged.
InastudybyGaytonandMcEwen(2007),bothfacultyandstudentsperceivedthefollowingasimportantstrategiesforensuringinstructionalquality:maintainingopencommunicationwithstudents,requiringstudentstointeractwiththeinstructor(andeachother)tofostergroupcohesiveness,andbuildingastronglearningcommunity.Timelyanddetailedfeedbackregardingthequalityofstudentworkandbuildinggoodrapportandcollaborationwithstudentswerealsoconsideredimportant.
Instructorpresencecanbeaccomplishedinmanyways,andisbestaccomplishedbyusingacombinationofstrategies.Somearebuiltintothecourse,butmanyaredynamicactivitiesthatrequirethattheinstructorbefullyengagedwiththecoursedeliveryandstudentinvolvement.
Recommendations
Establishawelcomingclassenvironmentwithpersonaltouches,suchasthese:o Sendawelcomemessagebeforethecoursebeginso Postanintroductorytext,audio,orvideomessage,andinvitestudentstodothe
sameo Continuetouseaudio/videoinmessagestoindividualstoconveypresenceo Useaconversational,plainEnglishstyleo Addressstudentsbynameo Besensitivetocultures,genders,religions,nationalities,learnerconfidencelevels,
anddifferingabilitiesinallcommunicationso Calleachstudentduringtheclasstoconveypersonalinterest
Uselearningmanagementsystemfeaturesregularlytoconveyinstructorpresence,interest,andconsiderationtoannounceprogressandevents,topostofficehoursandcontactinformationincludingmethodsandtimes,andtoencourage,motivate,andchallengestudents.
Encourageinstructorstouse(butdon’trequire)socialnetworkingmediatocommunicatewithstudents.
Encouragefacultytoalwaysprovidestudentstimely,individualfeedbackonassignments.
56 Quality Scorecard for Blended Learning Programs
4. Faculty teach the course as one cohesive whole, with "presence" in both the face‐to‐face and the online portions of the course.
Facultypresencemeansmorethanjustprovidingcontenttostudentsinpersonoronline.Facultyneedtobeengagedwithstudentsinbothportionsofthecourseandrespondtostudents’questionsandconcerns.Whenteachingablendedcourse,therecanbeatendencytoutilizetheonlineportiontodelivercontentina“boltedon”componenttoaface‐to‐faceportion;therefore,itisimportantthatthefacultyispresentinthebothportionsofthecourse.
Asenseofcommunityshouldbeestablishedthatutilizesboththeonlineandface‐to‐faceportionsofthecourse.Facultyshouldestablishsocialpresenceinbothportionsoftheblendedcourseinordertocreateasenseofconnectedness.Thedialogshouldflowfromtheface‐to‐faceportiontotheonlineportionandbacktotheface‐to‐faceportion.
Recommendations
Setasideaportionoftheface‐to‐facesessiontoanswerstudentquestionsandparticipateinadiscussionboardwherestudentscanaskquestions.
Activelyencourageasenseofcommunitywithinthecourse.
Teaching and Learning 57
5. Resources are provided to assist students in conducting research online and assessing the validity of online resources.*
Studentsinblendedlearningcoursesareoftenaskedtoconductresearch,rangingfromastraightforwardundergraduateclassresearchprojecttoathesisordissertationforagraduateprogram.Becausestudentsinblendedcourseshavelessaccesstoaface‐to‐faceinstructor,resourcesmustbeprovidedwiththeonlinecoursematerialsforinformationonthisprocess,includingtheinstructor’sexpectationsregardingresearch.Studentsmustknowhowtofind,organize,evaluate,andciteinformationusedintheirresearch.
Studentsinblendedcoursesmustunderstandthattheyareaccountablefortheinformationtheyuseincompletingcourseassignments.Instructorsmustemphasizethatthestandardsfortheonlinecomponentofthecourseareasrigorousasthosefortheface‐to‐faceportion.Maintainingconsistencyandintegrityinresearchexpectationswillhelpmaintaincourseandprogramintegrity.Itwillalsopreparestudentstoeffectivelyuseonlineinformationbeyondthescopeofthecourse.Intheonlinecoursematerials,instructorscanembedorlinktoadequate,clearinstructiononhowtoconducteffectiveresearchoftheprofessionalliteraturethroughwebsearches,andlibraryholdingsordatabases.Onceinformationhasbeenlocated,studentsmustalsounderstandhowtoevaluatethatinformation.Withtheeaseofonlinesearchesandtheplethoraofinformationavailable,studentsmustlearntobecriticalconsumersofinformationandrecognizethecriteriaforreliablesources.Theymustknowhowtodiscernthedifferencebetweencredibleinformationfrompeer‐reviewedarticlesandbiasedinformationfromsubjectivesourcessuchasblogs,wikis,andcommercialsites(Naufel,Briley,Harackiewicz,Johnson,Marzec,&Nielsen,2010).
Recommendations
Stateexpectationsandresearchrequirements,clearlyandindetail,inthecoursesyllabusandonlinelearningmanagementsystem.
Providestudentsinformation:o OnhowtouseBooleansearchtermstofindabstracts,o Informationoncitation,andfull‐textarticlesinlibrarydatabaseso AboutthevariouslibrarydatabasesrelatedtothecoursetopicsaboutGoogle
Scholar(orothers)asausefulsourcesofinformationo Aboutthemostreliableandcrediblejournals(bydiscipline)o OnhowtocorrectlyciteInternet‐basedsources,includingtheDOIo Onhowtoevaluatetheinformationfoundonlinebystudentsforsource,accuracy,
credibility,focus,biasandrelevanceo Aboutmethodsandethicsofresearch,includingpropercitationand
acknowledgementofsources,plagiarism,andacademicintegrity. Considerusingaformtohelpstudentsevaluateonlinesources. Createanonlinetutorialonresearchtechniquesusingarecordedpresentationincluding
voiceandvideoifpossible,sothatstudentscanaccessandrevisitasneeded.
58 Quality Scorecard for Blended Learning Programs
ReferencesforTeachingandLearning
Boettcher,J.(2006).Tenbestpracticesforteachingonlinequickguidefornewonlinefaculty.Retrievedfromhttp://www.designingforlearning.info/services/writing/ecoach/tenbest.html
Getzlaf,B.,Perry,B.,Toffner,G.,Lamarche,K.,&Edward,M.(2009).Effectiveinstructorfeedback:Perceptionsofonlinegraduatestudents.TheJournalofEducatorsOnline,6(2).Retrievedfromhttp://www.thejeo.com/Archives/Volume6Number2/GetzlafetalPaper.pdf
Naufel,K.Z.,Briley,K.E.,Harackiewicz,A.S.,Johnson,K.P.,Marzec,K.P.,&Nielsen,M.E.(2010).Howdopsychologystudentsuseweb‐basedinformation:Trendsandimplicationsfromadescriptivestudy.NorthAmericanJournalofPsychology,12(1),1‐14.
Shaw,M.(2014).Teachingthroughfeedback.Retrievedfromhttp://olc.onlinelearningconsortium.org/effective_practices/teaching‐through‐feedback
Teaching and Learning 59
1. Technicalassistanceisprovidedforfacultybeforeandduringblendedcoursedevelopmentandteaching.*
2. Theinstitutionensuresfacultyreceivetraining,assistance,andsupporttopreparefacultyforcoursedevelopmentandeffectiveteachingwithtechnologyinavarietyofmodalities.*
3. FacultyreceivetrainingandmaterialsrelatedtoFairUse,plagiarism,andotherrelevantlegalandethicalconcepts.*
4. Facultyareprovidedon‐goingprofessionaldevelopmentrelatedtoblendedteachingandlearning.
5. Clearstandardsareestablishedforfacultyengagementandexpectationsaroundblendedteaching(e.g.responsetime,contactinformation,etc.).
6. Facultyareprovidedtraininginblendedteaching.
Faculty Support
1. Technical assistance is provided for faculty before and during blended course development and teaching.*
Asblendedlearningrequiresfacultytoadoptvariouslearningtechnologies,itismoreimportantthaneverthatinstitutionsprovidefacultywithtechnicalassistanceandongoingsupportforintegratingtechnologyintoboththeface‐to‐faceandonlineclassroom.Studentsexpecttoutilizethesetechnologiestointeractandcollaborateinbothoftheselearningenvironments.
Institutionsarechallengednotonlytoprovideanadequateleveloftechnicalsupportandassistanceforfaculty,butalsoappropriatesupportandassistance.Institutionsmustthinkstrategicallyabouthowtostaff,promote,structure,anddeliverthetechnicalassistancefacultyneedforteachinginablendedlearningcourse.Often,institutionsassociate“technical”with“informationtechnology,”andthetechnicalassistanceforcoursedevelopmentandblendedcoursesishousedwithinITdepartments.Technicalsupportshouldbeprovidedforthelearningmanagementsystem,webpagedevelopment,socialmediatools,audioandvideorecordings,graphicdevelopment,classroomwifi,classroomtechnologies,andnewemergingtechnologies.Supportmustalsobecontextual,pedagogy‐based,andeasilyaccessible,andfitwithintheoverallorganizationalstructuretomaximizeefficienciesandresources.Thesupportshouldcontinuethroughthetimeofteachingblendedcourses,andnotsolelyfocusoncoursedevelopment.
Recommendations
Embedtechnicalsupportpracticesforbothface‐to‐faceandonlineeducationwithintheinstitutionalmissionandculturetoencouragefacultyacceptance.Supportshouldbeallocatedtoassistfacultyinblendedcoursedevelopmentsotheywillbecomemorefamiliarwithonlineeducationanditsroleinthecontextoftheirowndepartment.
Carefullyconsiderwheretolocatetechnicalsupportservices.Thinkaboutexistingorganizationalstructuresanddecidewhethertocentralizeordecentralizeservices.Keepinmindaccessibility,approachability,consistency,maintainingeducationalcontext,andensuringexpertiseisavailable.
Technicalsupportwillbeneeded,attimes,forfacultyandtheirhomecomputers.Providearesourceinorderforfacultytobeabletotroubleshoothomecomputerswiththeonlineteachingenvironment.
Technicalsupportwillbeneeded,attimes,intheface‐to‐faceclassroom.Providearesourceforfacultytobeablerequestclassroomsupportduringaclasssothatthesessionisnotdisrupted.Thisisimportantduetothereducednumberofclasssessions.
Faculty Support 61
2. The institution ensures faculty receive training, assistance, and support to prepare faculty for course development and effective teaching with technology in a variety of modalities.*
Whenaninstitutionplacesvalueonblendedlearningbyprovidingpoliciesandsupportmechanismsthatsupportsfaculty,facultysatisfactionwithblendedlearningwillincrease.Thisincludesprovidingaccesstoeducationaltechnologies,instructionalteams,andguidanceastohowfacultycancommunicatethecourseformattostudents.
Theeducationaltechnologiesselectedbytheinstitutionshouldbewidelyadoptedwithconsiderationtohowitwillenhanceblendedlearning.Theselectionprocessofthesetechnologiesneedstoconsidertheprocessofteachingandlearning,aswellasfacultyandstudentusage.Asufficientstaffneedstobeavailabletosupportandmaintainthesetechnologies.
Aninstructionalteamofinstructionaldesigners,instructionaltechnologists,mediadevelopers,andotherspecialistsshouldjoinwiththefacultytodevelopblendedlearningcourses.By“unbundling”thevariousfunctionsofthefacultyanddistributingtheserolestotheinstructionalteam,thecoursecanbedevelopedandtaughtmoreefficientlyandeffectively.
Thissupportmayincludeinformationforfacultytocommunicatetostudentsabouttheblendedlearningformatandtipsforsuccess.
Theseresourcesneedtobeprovidedviadifferentmodalitiesinordertosupportbothpart‐andfull‐timefaculty.Thismayincludehelpguides,in‐personconsultation,andthroughonlinetechnologies.
Recommendations
Providecomprehensivecoursedevelopmentsupport,training,andassistanceforblendedlearninginstructors.Thefacultydevelopmentopportunitiesshouldbefrequentandongoing.
Examinedifferentmodelsofdelivery(virtualmodules,handouts,livetraining,self‐paced,etc.)andprovideavarietyofresourcesforprofessionaldevelopment.
Adoptavarietyoffaultytrainingandsupportstrategies,e.g.facultymentoringprograms,web‐basedknowledgebases,collaborativewikis,trainingworkshops,onlinesupportmaterials,andothers.
62 Quality Scorecard for Blended Learning Programs
3. Faculty receive training and materials related to Fair Use, plagiarism, and other relevant legal and ethical concepts.*
Theblendedenvironmentpresentseducatorswithconsiderablelegalandethicalchallenges.Understandingofissuesrelatedtoownership,privacy,identity,andcopyrightlawarecritical.CopyrightlawandFairUseareessentialconceptsforallfacultymemberstounderstand.Facultygenerallyapproachtheseissuesfromtwodistinctperspectives:asconsumerandcreator.Theirconcernsandperceptionsaboutcopyrightlawmaydifferdependingonthescenariobeingconsidered.Onecommoncopyrightissuerelatedtoblendedlearningisfacultyownershipofmaterialcreatedforblendedcourses.Eachinstitutionmustbeproactiveinsettingacoreintellectualpropertypolicy,andprovidingguidelinesforcompliance.
Fairuseisperhapstheleastunderstoodaspectofblendedcoursedevelopmentandteachingamongfaculty.Inarecentstudy,Sweeney(2006)foundthatfacultymembersweregenerallyunclearaboutcopyrightlawandwereuncertainofwhereandhowtolocateuniversitypolicies.Inaddition,facultynotedthattheywereconcernedabouttheircoursecontent,butcontinuedtousematerialforwhichtheydidnotseekcopyrightrelease.Theconsequencesofcopyrightinfringementcanbedireforanindividualandaninstitution,andcanincludedisciplinaryaction,termination,legalfees,andanassortmentofcriminalpenalties.Institutionsmustsupportfacultywiththeneededinformationandtrainingaboutcopyrightlawinordertoavoidabreach.Inadditiontotheriskofinfringement,alackofknowledgeaboutFairUsecanresultinfacultyavoidanceofincorporationofdiversecontentintoablendedcourse.
Anothermajorethicalissueforfacultyisplagiarism.Facultymustunderstandthenatureofplagiarismintheblendedenvironment,andlearntorecognizeitinordertodeterfrustratingandchallengingincidents.Facultymustunderstandtheirroleisnotto“catch”students;rather,facultyshouldhavetheknowledgeandtoolstoeducatestudentsaboutacademicintegrity,ethicalissues,andappropriateconduct.
Recommendations
Createchecklists,tipsheets,examples,workshops,andchunkedinformationtomakecopyrightcomplianceeasierforfacultytoapply.Seeforexample,CopyrightComplianceMadeSimple–SixRulesforCourseDesign(Enghagen,2001).
Ensurefacultyreceivecopyrightinformationwithinrelevantcontexts.Forexample,copyrightinformationinrelationtocoursedevelopmentshouldbeincludedinblendedcoursedevelopmentsupportmaterials.
Ensurethatcopyrightlawdoesnotbecomeonedepartment’sresponsibility,butrather,universalandintegratedthroughouttheinstitution.
Providefacultywithworkshops,discussions,roundtables,andotheropportunitiestocollaborativelydiscussethicalissuesinblendededucation.
Faculty Support 63
4. Faculty are provided on‐going professional development related to blended teaching and learning.
Blendededucationhasevolvedquicklyoverthepastdecade.Thefrequencywithwhichnewtechnologiesbecomeavailableisstaggering.Inaddition,asmoreandmoreresearchisconductedonelectronicallymediatededucationalpracticeandoutcomes,furtherevidenceemergesthatcompelsustochangeoureducationalmethodsandpedagogicalapproaches.
Anongoingprogramofprofessionaldevelopmentisanecessarypartofsupportingqualityinblendededucation.Stayingcurrentwiththebodyofresearchandtheavailabilityofnewtechnologiesiscriticaltosuccessintheblendedenvironment.Inaddition,focusonadditionalareasforfacultydevelopmentcanenhanceaninstructor’seffectiveness,suchasgenerationalstudies,learningtheory,timemanagement,andretentionstrategies.Professionaldevelopmentcanalsohelptoavoidinstructorburnout,andgiveinstructorsimportanttimeforreflection,learning,andpersonaldevelopment.
Thetraditionaldefinitionofprofessionaldevelopmentencompassesactivitiessuchassabbaticals,grantfunding,conferenceattendance,andactivitiesdirectlyrelatedtodevelopmentasaprofessionalorresearcher.However,amorestudent‐centereddefinitionisemerging,andthisisespeciallytrueofprofessionaldevelopmentforblendedfaculty.Professionaldevelopmentisnotsimplyaseriesofworkshopsorclassestohoneskills,thoughthosearecertainlyapartofprofessionaldevelopment.Rather,professionaldevelopmentisaholisticprocessthatshouldincludefaculty’slong‐termgoalsforimprovingteachingandlearning.
Itisamyththatprofessionaldevelopmentforfacultyiscost‐prohibitive.Althoughprofessionaldevelopmentrequiressomeinstitutionalinvestmentforstaff,materials,andprograms,theessenceofprofessionaldevelopmentthriveswithinacultureofcontinuousimprovement,studentlearning,andqualityfocus.
Recommendations
Defineandcommunicatetheimportanceofprofessionaldevelopmentforfaculty. Approachprofessionaldevelopmentasaprocessofindividualgoalsettingorself‐
improvementwithafocusonimprovingteachingforimprovinglearning. Ensurethatstaffselectedtoleadprofessionaldevelopmenteffortshavebuy‐inandsupport
fromfacultyandthatallinitiativesanddepartmentsworkcollaborativelyontheprograms. Proactivelyengagefacultybyensuringtheirawarenessofdevelopmentopportunities
throughvariouscommunicationchannels. Provideflexibleanddiverseprofessionaldevelopmentstrategiestomeetthedemandsof
thediversefacultypopulation.Engageadjunctfacultyintheteachingcultureoftheinstitution.
64 Quality Scorecard for Blended Learning Programs
5. Clear standards are established for faculty engagement and expectations around blended teaching (e.g. response time, contact information, etc.).
Inanageofaccountability,theestablishmentandimplementationofinstructionalstandardsprovidesinstitutionsawaytomeasureperformanceandeffectstudentlearningoutcomes.However,apotentialpitfallofstandardsisthattheysometimesinhibitinnovationandcreativityandbecomecodifiedasrules.
Institutionsshouldestablishstandardsthataredirectlyandpositivelytiedtostudentlearningoutcomes.Theonlywaytotrulyknowwhethercertaineducationalmethodscontributepositivelytostudentoutcomesisthroughthecollectionandanalysisofdata.Thus,individualswhoaredevelopingstandardsshouldaskthemselveswhattheywanttoaccomplishthrougheachstandardanduseevidencetojustifythestandard.
Acultureofqualityisbuiltonsharedvaluesaboutgoodpractice.Often,institutionsspendtoolittletimebuildingtheculture,andtoomuchtimecodifyingandenforcingstandards.Intheblendedenvironment,thisisespeciallytrue,asoftenthereisafocusonmonitoringfacultyperformanceforadherencetoresponsetimeandengagementindicators.
Oncestandardsareestablishedandunderstood,theremustbeaprocessandstaffinplacetocommunicatestandardsandenforcethem.Blendedfacultymustbemadeawareoftheperformanceexpectationsandstandardsofteachingperformance.Thecommunicationmustbeclear,consistent,andprecise.
Recommendations
Documentstandardsforfacultyexpectationsandalwaysincludetheminteachingcontractsandfacultypolicyhandbooks.Supportstandardsbyincludinginformalmessagesaboutspecificstandardsinadiverseselectionofmaterials(websites,collateralmaterial,meetingagendas,andsoforth).
Ensurethatappropriatepersonnelandsystemsareinplacetocommunicatestandardsandmonitorfacultyperformance.Inculcatethebenefitsofassessmentandperformanceevaluationintheinstitutionalculturesothatmethodsareembracedratherthancondemned.
Createandimplementblendedfacultycertificationcoursesthatimparttheessentialcomponentsofblendedteachingstandards,andprovidethenecessarytoolsandskillstoachievequalityblendedteaching.
Createaconsistentperformancereviewprocesswithdetaileddocumentation.Providefacultywithperformancefeedback,inwriting,aswellasspecificinformationonhowtorectifyanynotedissues.
Establishmeasuresforstandardsthatcomefromvarioussourcessothatperformanceassessmentscanbeimproved.Forexample,measurefacultyengagementinaclassbyadministrativeobservation,peerreview,studentreports,andloginactivity.
Assesscompetenceintheblendedclassroomthroughcombinedmeasures,includingwrittenskills,technicalskills,pedagogicalawareness,compliancewithadministrativedutiesandmore.
Faculty Support 65
6. Faculty are provided training in blended teaching.
Facultyneedtobepreparedtopurposefullyintegratetheonlineandface‐to‐faceportionsofablendedcourse.Thistrainingprogramshouldallowforfacultytoimmediatelytransfertheirnewknowledgetotheirpracticeandshouldincludetopicssuchascoursedesign,teachingstrategies,formativeandsummativeassessments,educationaltechnology,andtheintegrationofthesetopicsintooneanother.
Modelsforprovidingtrainingtofacultyincludeprogramsthataredeliveredface‐to‐face,online,orblended.Theseprogramsmaybeofferedasan“intensive”oneortwo‐weekprogramormaybespreadoutovermultipleweeks.Anotheroptionmayincludeprovidingmentoringtofacultynewtoblendedlearning.Theseprogramsshouldbeavailabletobothfull‐andpart‐timefaculty.
Recommendations
Provideafacultydevelopmentprogramtopreparefacultytoteachinablendedlearningenvironment.
Facultydevelopmentprogramshouldincludetopicssuchascoursedesign,teachingstrategies,formativeandsummativeassessments,andeducationaltechnologyasrelatedtoblendedlearning.
66 Quality Scorecard for Blended Learning Programs
ReferencesforFacultySupport
Enghagen,L.J.(2001).Copyrightcompliancemadesimple–sixrulesforcoursedesign.Needham,MA:TheOnlineLearningConsortium.
Sweeney,P.C.(2006,Spring).Faculty,copyrightlaw,andonlinecoursematerials.OnlineJournalofDistanceLearningAdministration,9(1).Retrievedfromhttp://www.westga.edu/%7Edistance/ojdla/spring91/sweeney91.htm
Faculty Support 67
1. Beforestartingablendedlearningprogram,studentscompleteanorientationorself‐assessmenttodetermineiftheypossesstheself‐motivationandcommitmenttolearn.*
2. Beforestartingablendedlearningprogram,studentsareadvisedabouttheprogramtodetermineiftheyhaveaccesstotheminimumtechnologyskillsandequipmentrequiredbythecoursedesign.*
3. Beforestartingablendedlearningprogram,studentsreceive(orhaveaccessto)informationaboutprograms,includingadmissionrequirements,tuitionandfees,booksandsupplies,technicalandproctoringrequirements,andstudentsupportservices.*
4. Throughoutthedurationofthecourse/program,studentshaveaccesstotrainingandinformationtheywillneedtosecurerequiredmaterialsthroughelectronicdatabases,interlibraryloans,governmentarchives,newsservices,andothersources.*
5. Throughoutthedurationofthecourse/program,studentshaveaccesstoappropriatetechnicalassistanceandtechnicalsupportstaff.*
6. Supportpersonnelareavailable(24/7)toaddressstudentquestionsandproblemsofatechnicalnature.*
7. Policy,processes,andresourcesareinplacetosupportstudentswithdisabilities.
8. Studentshaveaccesstoinformationregardingrequiredcoursematerialsinprintand/ordigitalformat,suchasISBNnumbersfortextbooks,booksuppliers,anddeliverymodespriortocourseenrollment.
9. Programdemonstratesastudent‐centeredfocusandintentionalityintheintegrationofonlineandface‐to‐faceresources.
10. Theinstitutionprovidesguidance/tutorialsforstudentsintheuseofallformsoftechnologiesusedforcoursedelivery.
11. Studentsareprovidedclearinformationforenlistinghelpfromtheinstitution.
Student Support
1. Before starting a blended learning program, students complete an orientation or self‐assessment to determine if they possess the self‐motivation and commitment to learn.*
Asonlineandblendedlearningmatures,studentsuccesshasbeenlinkedtocertainmotivationsandskillsets.Alem,Plaisent,Bernard,&Chitu(2014)foundinasystematicreviewthate‐learningreadinesswasrepresentedthroughattributessuchas:computerinternetself‐efficacy,self‐direction,motivation,interactionandattitude.Tosupportstudentretention,institutionsshouldtakeactiontoensurestudentsarebothcapableandequippedtosucceedinablendedlearningenvironment.AlthoughstudentsmaybefamiliarwithtechnologythatGarrisonandVaughan(2008)suggestprovidestheessentiallinkageinblendedlearning,andareabletofind,createandshareinformationviatheweb,TangandChaw(2015)foundthatdigitalliteracythatfacilitatesaccessing,understandingandapplyinginformationobtainedfromthewebisalsoimportant.Studentsshouldbeawarethat:
● Theyareenrolledinablendedlearningcoursethatfacilitateslearninginboththeonlineandfacetofacelearningenvironments
● Whileblendedcoursesgivemoreflexibilityinscheduling,theycanrequiremoreself‐disciplinethanatraditionalfacetofaceclass.
● Theywillhavetheopportunitytointeractwiththecourseinstructorinboththeonlineandfacetofacelearningenvironment.
● Onlinelearningmayrequirethestudenttoworkfromwrittendirectionsforcourseassignments.
● Itmaytakeaslittleasafewhoursorasmuchasseveraldaystogetfeedbackfrominstructors.
● Printmaterialsmaytheprimarysourceofdirectionsandinformationfortheonlineweeks.
● Studentsshouldfeelcomfortabletocontacttheinstructorforhelpassoonastheyneedhelpforthecourse.
Recommendations
Encouragestudentstoassesstheirmotivationandcommitmenttoblendedandonlinelearning(seebelow;MooreandShelton,2013).
o StudentOnlineReadinessTool(availablefromUniversitySystemofGeorgia)throughCreativeCommonslicenseandassesses:Technologyexperience,accesstotools,studyhabits,mylifestyle,goalsandpurposes,andlearningexperiences.
o SmarterMeasure™LearningReadinessIndicator(2016)isacommercialtoolthatassesses:Individualattributes,lifefactors,learningstyles,technicalcompetency,technicalknowledge,on‐screenreadingrateandrecall,andtypingspeedandaccuracy.
Developprocessestomeasurestudentsuccessandmotivation(i.e.,usinga“droppedstudent”surveytodeterminecommonalities).
Student Support 69
2. Before starting a blended learning program, students are advised about the program to determine if they have access to the minimum technology skills and equipment required by the course design.*
Onlinestudentsrelyontechnologytointeractwithinstructors,classmatesandinstitutionalsupportstaffinanonlineandblendedlearningenvironment.Therefore,itisimportanttoadvisestudentsoftheneedforbasiccomputerandtechnologyskillsandequipmentbeforetakingblendedclasses.Thisadvicecancomefromtheadmissionsoffice,viatheinstitutionalwebsiteandmarketingmedia.Manycollegeshaveimplementedonlineself‐assessmentactivitiesthatallowstudentstoascertainiftheypossessthetechnicalskillsneededtosucceedinanonlinecourse,asdiscussedinthepreviousindicator.Studentsuccessinablendedlearningenvironmentnecessitatesstudentshavethenecessaryknowledgeandabilitytouseinformationandcommunicationtechnology(Poon,2013),aswellasaccesstoanadequatecomputersystem.Toassessifastudenthasaccesstoadequatecomputersystem,institutionsneedtodisseminatetechnologyrequirements.s(Simonson,Smaldino,Albright,&Zvacek,2009,p.202).
Simonsonetal.(2009)remindsusthatthatthiscommunicationshouldstartbeforestudentsbeginthecourse.Technologyrequirementsshouldbeclearlydisplayedandarticulated,beforethestudentregistersforthecourse,includingrequirementsforbrowsers,browsersettingsandbrowseradd‐ons.Technologyrequirementsshouldalsoincludeanyspecialequipmentthatisspecifictothecourserequirements,suchascameras,microphones,CD‐ROM,DVDdrives,specialsoftware,ortheneedtodownloadadditionalsoftwareprograms.Allspecificonlinecoursetechnologyrequirementswillneedtobeincludedinthesourcesyllabuscourseschedule,andotherappropriateplaceswithinthecoursesiteaswellasbeeasilyaccessibleonthewebsitesoexpectationsareclearbeforeastudentsignsupforacourse.
Recommendations
Provideinformationontheprogramwebsite,throughemail,socialmedia,etc.concerningallrequiredonlinetechnologiesfortheblendedcoursebeforethecoursebegins.
Provideweblinkstohelpstudentslocatetheresourcestheyneed(i.e.,bookstore,vendors,websites,software,downloads).
Providealistoftroubleshootingtipsandfrequentlyaskedquestionsforstudentstoaccessonline.
Provideinformationontechnicalskillsandstandardsspecifictotheinstitution.Forexample,“Studentsshouldhavebasicskillsusingwordprocessingsoftware.”Disseminatetheinformationtostudentsthroughseveralchannelsofcommunication.
Addressspecifictechnologiesrequiredbytheinstitutionsuchasemailononlinelearningmanagementsystems.
Provideademoinvideooratutorialthatexplainsthetechnologiesneededforsuccess.
70 Quality Scorecard for Blended Learning Programs
3. Before starting a blended learning program, students receive (or have access to) information about programs, including admission requirements, tuition and fees, books and supplies, technical and proctoring requirements, and student support services.*
Studentsneedaccesstorelevantinformationregardingblendedprogramofferingsbeforemakingadecisiontoapplytoadegreeprogramorenrollinablendedcourse.Communicationofavailablesupportservicesandnecessaryinformationisanimportantelementineffectiveprogramadministration.
Themoreoptionsthestudenthastoaccessrelevantinformationandsupportservices,thebetter.Sincemanystudentswishtoconductbusinesswiththeinstitutionduringnon‐standardhours,providinganswerstofrequentlyaskedquestionsthatcanbeaccessedanytimehasprovenhelpfulforstudents.Furthermore,eachquestionclearlyansweredmayreducethenumberofresponsesneededfromsupportstafforinstructors.Studentservicessupportstaffandinstructorsshouldbeawareofalllocations(physicalandonline)todirectstudentsforanswerstoanyadditionalquestions.
Recommendations
Makeinformationrelatedtoin‐classandonlineattendanceexpectationsforblendedcoursesandprogramseasilyaccessibletoprospectiveandenrolledstudents.
Provideonlineinformationrelatedtoprogramadmission,enrollment,costandfinancialaid,books,policies,andstudentsupportservices.Keepthewebsiteupdated,organized,andeasilyaccessible.
Considerthefollowingquestionswhendevelopingtheinformationalwebsite:o Whatrequirementsmustbesatisfiedforstudentstogainadmittancetoacourse?o Howshouldstudentscommunicate(phone,email,fax,etc.)withtheschoolto
acquireinformation/support?o Whatproceduresshouldstudentsfollowtoaccessservices,suchasthelibrary
(identification,login,URL)?o Howdostudentscommunicatewithadvisorsandotherfacultymembers?o Whatkindsofbarriersmightstudentsencounterinsuccessfullycompletingtheir
programandhowwilltheinstitutionremovethesebarriers?
Student Support 71
4. Throughout the duration of the course/program, students have access to training and information they will need to secure required materials through electronic databases, interlibrary loans, government archives, news services, and other sources.*
Learningtouseonlinelibrarydatabasesforthefirsttimewithoutexperthelpcanbeintimidating.Andevenifstudentssuccessfullylocateappropriateresources,findingfull‐textversionsofjournalarticlesorarrangingfore‐documentdeliverycanoftenbeachallenge.Libraryprofessionalsshouldprovidetrainingandeducationalopportunitiesforstudentsandfaculty,andmaywanttoviewteachinginformationliteracyaspartoftheirjob.Instructorsshouldmakeuseoftheexperthelpthatlibraryprofessionalscanprovidesothattheycansupportandencourageuseoflibraryresources.
Recommendations
Provideaccesstoinformationonaccessinglibrarydatabases(i.e.,alinktolibraryresourcesandlibrarystaffcontactinformation).
Provideservicesthathelpstudentslocaterelevantinformationsuchasaself‐pacedtutorialorlibraryorientationmodule.
Createanonlinelibraryresearcharticlescavengerhuntforyourstudentsasaclassorsubjectorientationassignment.
Designatealibrarianforeachdisciplineareaandprovidestudentswiththedesignatedlibrarian’scontactinformation,sostudentshaveaprimarypointofcontactforquestionsandtrouble‐shooting.
72 Quality Scorecard for Blended Learning Programs
5. Throughout the duration of the course/program, students have access to appropriate technical assistance and technical support staff.*
Studentsmayencounterdifficultyworkingwithrequiredtechnologyinablendedcourseorprogram,andmayneedsupporttosuccessfullyfunctionusingsuchtechnology.However,theystillmustbeabletocompletecourselearningactivitiesandassignmentswithoutunduerelatedstress.
Bothstudentsandinstructorsworkinginablendedteachingandlearningenvironmentwillneedtechnicalsupportfromtimetotime.Ifastudentisunabletoresolvetechnicalproblemsinthecourseofstudy,frustrationandisolationwilloccur,which,inturn,mayimpactstudentretentionandlearning(Shelton&Saltsman,2005).
Facultyintroducinganewtechnologyorapplicationinablendedcoursemusttakeresponsibilityforworkingwithcampusinformationtechnologyservicestoensureappropriatedocumentationandtrainingisavailabletosupportthestudentsworkingwiththattechnologyorapplication.
However,“instructorscannotberesponsibleforotherthancursorytrainingoftheirstudentsintheuseoftechnology”(Miller&King,2003,p.292).Therefore,theinstitutionshouldprovideonlinesupportforthestudents.Ideally,thesupportshouldbe24x7whencombiningself‐helpandaccesstoapersonforsupport.Asmuchinformationaspossibleshouldbeavailableinaself‐helpformatsostudentscanaccesshelpatanytime.However,theremustbeaprocessforcontactingsupportapersonifadditionalhelpisneeded.Someinstitutionsprovideadirecttelephonenumber(oftentoll‐free);othershaveanonlineticketingsystemorchatroom.Thesupportshouldincludeeverythingfromhowtoresetapasswordtowhattodoifastudentgetslockedoutofanonlineexamination.
Recommendations
Provideaccessibleinformationonhowtoaccesstechnicalsupportonthewebsiteand/ortheLMS,atollfreephonenumber,and/orlivechat.
Providetechnicalsupportservicesascloseto24hours,7daysaweekaspossible. Clearlyspecifywhenvarioussupportservicesareavailableiftheyarenotoffered24x7. Createachannelthroughwhichfacultycanrequestthedevelopmentanddeliveryofnew
documentationandtraininginsupportofblendedcourseintegratedtechnologies. Createatechnicalsupportrequestformthatisweb‐basedwithdetailsaboutexpected
turnaroundtime,evening/weekendavailabilityandtelephoneand/orchatavailabilityfor“live”help.
Student Support 73
6. Support personnel are available (24/7) to address student questions and problems of a technical nature.*
Studentfrustrationwithtechnologyinlearningenvironmentsneedstobeavoided,andresearchsupportsthatfrustrationwithtechnologyhasbeencitedasakeyfactorinstudentfailure(Moore&Kearsley,1996).Inablendedteachingandlearningenvironment,asupportsystemneedstobeinplaceforaddressingtechnologyrelatedquestionsoutsideoftheclassroom.Inthelearningmanagementsystemorstudentportal,itshouldbeclearwhomtocontactifastudentforgetshis/herpassword,iftheLMSisnotworkingproperly,ifaninstructorisnotrespondingtothestudents,orifthecomplaintisnotgettingaddressedinatimelymanner.Students“needtounderstandwheretheresponsibilityoftheinstitutionliesregardingreliability.Policiesneedtobearticulatedsothatallpartiesareclearaboutresponsibilities”(Simonsonetal.,2009,p.201).
Recommendations
Providesupportstafftoaddressspecificquestionsaboutaccessingcourseandprogramcomponents.
Presentaclearprocessforstudentstoreportproblemsandmakecomplaints,preferablyonline.
Createasearchableonlineknowledgebasethataddressescurrentissuesthatmightaffectstudentsusingblendedlearningtechnologiesandapplications.
74 Quality Scorecard for Blended Learning Programs
7. Policy, processes, and resources are in place to support students with disabilities.
Althoughstudentswithdisabilitieshavealegalandfundamentalrighttobeabletoenrollinallclassesandprograms,theywillneedawaytoprocureassistancefromtheinstitutionwhennecessary.Theinstitutionshoulddesignatesupportpersonnel(oftenintheareaofstudentaffairs)whodocumentstudents’needs,provideguidanceinallocatingresources,andhelpdeterminehowbesttomeetthoseneeds.
Theinstitutionmusttakeusabilityandaccessibilityoftechnologiesandapplicationsintoconsiderationwhenofferingoptionsandsolutionstofacultycreatinganddeliveringblendedteachingandlearningcomponents.
Recommendations
Putinplaceprocessthatdeterminesthetypesofservicesthatshouldbeavailablefordisabledstudentsinteractingwithblendedlearningcomponents.
Putinplacepolicywhichsupportsstudentswithdisabilitiesacrossallmodesofdelivery,includingblendedteachingandlearning.
Assurethattheprocessfordocumentingadisabilityisaccessibletoalllearners. Providededicatedsupportstaffforservingtheneedsofdisabledstudents. Assurethatfacultyandstaffareawareofstudentsupportservicesandunderstandthebest
methodsforreferral.
Student Support 75
8. Students have access to information regarding required course materials in print and/or digital format, such as ISBN numbers for textbooks, book suppliers, and delivery modes prior to course enrollment.
Informationonthematerialsneededforeachcourseshouldbemadeavailabletostudentsatthetimeofregistrationorbeforesothatstudentscan“shop”forthecoursematerials.
TobeincompliancewithfederalHigherEducationOpportunityActof2008(HEOA)regulations:
Tothemaximumextentpracticable,aninstitutionmustincludeonitsInternetcoursescheduleforrequiredandrecommendedtextbooksandsupplementalmaterialtheInternationalStandardBookNumber(ISBN)andretailprice;iftheISBNisnotavailable,theauthor,title,publisher,andcopyrightdate;orifsuchdisclosureisnotpracticable,thedesignation“ToBeDetermined.”
Ifapplicable,theinstitutionmustincludeonitswrittencoursescheduleareferencetothetextbookinformationavailableonitsInternetscheduleandtheInternetaddressforthatschedule.
Institutionsdisclosingtheinformationtobeincludedontheircourseschedulesforrequiredandrecommendedtextbooks,andsupplementalmaterialareencouragedtoprovideinformationon:
• rentingtextbooks;• purchasingusedtextbooks;• textbookbuy‐backprograms;and• alternativecontentdeliveryformatssuchasebooks.
TheHEOAalsorequirestheGovernmentAccountabilityOffice(GAO)tostudytheimplementationofthissectionandreporttoCongress.
Recommendations
Developandcontinuouslyupdateonlineandprintmaterialsthatprovideinformationoncourseschedulesandallrelevanttextbooksandsupplementarymaterials.
Providestudentsandfacultywiththelocationofthecampusbookstore,includingdirections,hoursofoperation,andcontactinformation.
Provideeasyaccesstoonlinebookstoresthatstudentscansearchforthebestprices.(Manyinstitutionswillnotwanttodothissincetheyareoftenbusinesspartnerswiththecampusbookstore.Inthatcase,providelinkstothebooksonthecampusbookstorewebsite.)
76 Quality Scorecard for Blended Learning Programs
9. Program demonstrates a student‐centered focus and intentionality in the integration of online and face‐to‐face resources.
Blendedteachingandlearningapproachesaresteadilyevolvingalongsideexistingandemergingtechnologies.Toensureastudent‐centeredfocus,systems,services,resources,andinstructionmustbeintegratedtocreateaseamlessexperience,removingallobstaclestofocusthelearner'sattentiononknowledgetobeacquiredandapplied.
Theinstitutionshouldprovideadvertising,recruiting,andadmissionsinformationtostudentsthatadequatelyandaccuratelyrepresentstheprograms,requirements,andservicesavailable.Thereshouldbeasetofwrittenpolicies,procedures,andguidelinespertainingtodegree‐relatedcoursesandprogramswhichincludesinformationonadmissions,curriculum,requirementsforcompletionofthecoursesorprograms,costsandpaymentpolicies,financialaid,andanyotherpertinentinformation.
Institutionsshouldprovideinformationaboutthenatureofblendedofferings,detailingexpectationsrelatedtobothface‐to‐faceandblendedcomponents,includingthenatureoffaculty/studentinteraction,assumptionsaboutstudents’technologicalcompetenceandskills,technicalequipmentandsoftwarerequirements,andavailabilityofacademicandstudentsupportservices.
Recommendations
Provideequalaccesstostudentservicesappropriatetosupporttheonlineandface‐to‐facecomponentsofblendedcoursesprograms,includingfinancialaid,academicadvising,registration,accesstocoursematerials,proctoringrequirements,placementandcounseling,andcareersupport.
Committothecontinuationofaprogramorcertificateforitspublicizedtimeframetoallowalladmittedstudentstocomplete.
Ensurethatstudentservicepersonnelcanaccuratelyandquicklyanswerallstudentquestionsandhaveastructuredsystemforaddressingstudentcomplaints.
Provideprospectivestudentswitharealisticpreviewoftheblendedlearningexperiencethroughanorientationorcoursedemonstration.
Providetime‐managementresourcesforstudentstoeffectivelymanagethebalanceofonlineandface‐to‐faceworkloadwithintheblendedlearningframework.
Student Support 77
10. The institution provides guidance/tutorials for students in the use of all forms of technologies used for course delivery.
Blendedteachinglearningtakesadvantageoftechnologytoenhancetheclassroomexperience.Asstudentslearninablendedenvironment,theymayneedtrainingandguidancetomasterthetechnologiesusedforblendedcomponentdelivery.
Theinstitution'slearningmanagementsystemmaybeonecomponentoftheblendedcourseorprogram.“Thecomplexityofamodern[learning]managementsystem(LMS)andtherelatedinfrastructurecanbedaunting,yetprovidingastableandfunctionallearningenvironmentisabsolutelynecessary”(Shelton&Saltsman,2005,p.117).
AttheheartoftheLMSareclassroomadministrativefunctionslikestudentauthentication,discussionthreads,chat,gradebooks,andin‐classmessagingandannouncements.Allofthesystemsavailabletodaycontainthesecorecapabilities.However,additionalcontentdeliverymethods,interactioncomponents,andassessmentcapabilitiesmayalsobeutilizedaspartofablendedcourseorprogram.ThesetechnologiesmayresidewithintheLMS,orbeaccessiblefromtheopenWebormobiledevices.
Studentsmustbecomfortablewithusingthesetechnologiessothattheyarenotatalldistractedfromthelearningprocess.
Recommendations
Provideaccesstodocumentationandtrainingforalltechnologieswithinthescopeofblendedcourseandprogramofferings.
Developanddistributeminimumtechnologyrequirementsforstudentsinblendedcoursesorprograms.
Provideopportunitiestocrowdsourceknowledgebyhavingstudentstocontributetotrainingmaterialsrelatedtoblendedcourseandprogramtechnologies.
Enablefacultytocreatetheirowninstructionalvideosinsupportoftechnologiesusedinblendedcourses.
78 Quality Scorecard for Blended Learning Programs
11. Students are provided clear information for enlisting help from the institution.
Forinstitutionstobesuccessfulandcompetitive,theymustprovideafullrangeofacademicandadministrativesupportservices,alongwithclearcommunicationtostudentsonhowtoaccessthoseservices.
LinkstovariousacademicandadministrativesupportservicesoftheinstitutioncanbeplacedthroughouttheLMS,oninstitutionalwebpages,andadministrativeportals.Allrelevantcontactinformationshouldbelisted,includinghoursofoperation.Carefulattentionmustbemadetoensurethattheinformationlistedthroughoutthesevenuesisaccurateandup‐to‐date.
Recommendations
Providelinksincourseinstructions,schoolwebpages,andportalstocleardescriptionsofavailablesupportandtutorials/resourcesthatanswerbasicquestionsconcerningresearch,writing,technology,etc.
Remindstudentsviaemailorlearningmanagementsystemannouncementsthatthereisadistinctprocessforstudentstousetocontactadministrativeandacademicsupportservices.
Developaninstitution‐widestrategyforkeepingallpublishedadministrativeandacademicsupportserviceinformationup‐to‐dateandrelevant.
Student Support 79
ReferencesforStudentSupport
Alem,F.,Plaisent,M.,Bernard,P.,&Chitu,O.(2014).StudentOnlineReadinessAssessmentTools:ASystematicreviewapproach.TheElectronicJournalofE‐Learning,12(4),375‐382.Retrievedfromhttps://www.ejel.org
Garrison,D.R.,Vaughan,N.(2008).BlendedLearninginHigherEducation:Framework,Principles,andGuidelines.Jossey‐Bass
H.R.4137‐‐110thCongress.(2008).HigherEducationOpportunityAct.Retrievedfromhttp://www.govtrack.us/congress/bills/110/hr4137
Miller,T.,&King,F.(2003).Distanceeducation:Pedagogyandbestpracticesinthenewmillennium.InternationalJournalofLeadershipinEducation,6(3),283‐‐‐297.
Moore,J.,&Shelton,K.(2013).Socialandstudentengagementandsupport:TheSloan‐cQualityScorecardfortheadministrationofonlineprograms.TheJournalofAsynchronousNetworks,17(1),53‐72.Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ1011372.pdf
Moore,M.,&Kearsley,G.(1996).Distanceeducation:Asystemsview.Belmont,CA:WadsworthPublishing.
Poon,J.(2013).Blendedlearning:Aninstitutionalapproachforenhancingstudents’learningexperience.MERLOTJournalofOnlineLearningandTeaching,9(2).Retrievedfromhttp://jolt.merlot.org/vol9no2/poon_0613.htm
Shelton,K.,&Saltsman,G.(2005).Anadministrator’sguidetoonlineeducation.Greenwich,CT:InformationAgePublishing.
Simonson,M.,Smaldino,S.,Albright,M.,&Zvacek,S.(2009).Teachingandlearningatadistance:Foundationsofdistanceeducation(4thed.).Boston,MA:Pearson.
SmarterMeasure™(2016).Retrievedfromhttp://readi.info/about/
Tang,C.&Chaw,L.(2015).Digitalliteracyandeffectivelearninginblendedlearning.EuropeanConferenceone‐Learning:601‐XVIII.KidmoreEnd:AcademicConferencesInternationalLimited.Retrievedfromhttp://search.proquest.com/openview/
5e52b0f4a3bbbfcc4179c9f323e0a18e/1?pq‐origsite=gscholar
UniversitySystemofGeorgia.(2010).StudentOnlineReadinessTools.Retrievedfromhttp://ci.ed1stop.net/sort/html/tool.html
80 Quality Scorecard for Blended Learning Programs
1. Theprogramisassessedthroughanevaluationprocessthatappliesspecificestablishedstandards(i.e.,accreditorguidelinesand/orotherrecognizedagencysuchastheOLCScorecard).*
2. Avarietyofdata(academicandadministrativeinformation)areusedtoregularlyandfrequentlyevaluateprogrameffectivenessinordertoguidechangestowardcontinualimprovement.*
3. Intendedlearningoutcomesatthecourseandprogramlevelarereviewedregularlytoensurealignment,clarity,utility,appropriateness,andeffectiveness.*
4. Aprocessisinplaceandfollowedforthecomprehensiveassessmentofsupportservicesforfacultyandstudents.
5. Aprocessisinplaceandfollowedfortheassessmentofstudentretentioninblendedcoursesandprograms.
6. Programdemonstratescomplianceandreviewofaccessibilitystandards(Section508,etc.).
7. Courseevaluationscollectstudentfeedbackontheeffectivenessofinstructioninrelationtofacultyperformanceevaluations.
8. Courseevaluationscollectstudentfeedbackonqualityofblendedcoursematerials.
9. Aprocessisinplaceandfollowedfortheinstitutionalassessmentoffacultyblendedteachingperformance.
10. Aprocessisinplaceandfollowedfortheassessmentofstakeholder(e.g.,learners,faculty,staff)satisfactionwiththeblendedlearningprograms.
Evaluation and Assessment
1. The program is assessed through an evaluation process that applies specific established standards (i.e., accreditor guidelines and/or other recognized agency such as the OLC Scorecard).*
Theblendedprogramevaluationmaybedefinedas“themeansbywhichtheinstitutionsorproviderssettheirprogramgoalsandmeasureresultsagainstthosegoals”(CouncilforHigherEducationAccreditation,1998,p.1).Acommonperceptionhasbeenthatablendededucationprogramis“oftenpoorlydesignedand/orunderfunded;itismoreofanafterthoughtratherthananintegralpartofplanningandimplementation”(Thompson&Irele,2007,p.419).Butasblendededucationcontinuestogrow,asstudentsareattractedtothepromiseofconvenienceandflexibility,andaswelearnmoreaboutthepedagogicalandlearningadvantagesofblendededucation,blendedprogramsmustbeabletodemonstratethattheyareguidedbyandaccountabletothehigheststandards.Demonstratingevidenceofaccountabilitymeansimplementing“evaluationactivitiesthatassessalignmentofpedagogy,educationalactivities,anddesiredlearningoutcomes,plusaddressspecificissuesofusabilityandbenchmarkachievement,[which]providevaluableinformationforcontinualimprovement”(Balanko,2002,p.7).Thesesuggestedactivitiesforevaluationareagoodstartingpointforanyblendededucationprogram.Inaddition,anyregionalordisciplinespecificaccreditinggroupwillhavestandardsthatwillneedtobeadheredtoandcansupportanestablishedevaluationprocess.
Recommendations
Useanevaluationprocessthatisalignedtoestablishedstandardstoassessblendedprograms,anduseevaluationresultstoimproveprograms.Theevaluationshouldbecompletedwithmultipleassessmentsandavarietyofdatacollectedandanalyzed.
Documentevidenceofprogramevaluationandimprovementtouseforsupport.Thisshouldbedoneatleastannually.
Useapplicableaccreditingstandardsanddemonstrateadherencetothesestandardsforblendedprogramwithartifactsanddataanalysis.Forexample,usingtheinstitution’sregionalaccreditorstandards,showcomplianceforthosethatapplytotheblendededucationprogramandhowprogramimprovementsaredevelopedandimplementedafterdatacollectionandanalysis.
82 Quality Scorecard for Blended Learning Programs
2. A variety of data (academic and administrative information) are used to regularly and frequently evaluate program effectiveness in order to guide changes toward continual improvement.*
Asingleevaluationprocess(studentcoursesurveys,forexample)cannoteffectivelyorfullyevaluateablendededucationprogram.Qualityprogramevaluationwillrequiremultipleprocesses,instruments,evidenceandanalysis.Inaddition,inputsandoutputsshouldbeperiodicallyreviewed.Infact,“effectiveprogramevaluationisthebestwaytoachievecontinuousprogramimprovement”(Chapman,2006,para.33).
Aholistic,360‐degreeapproachtoprogramevaluationthatsupportscontinuousimprovementincludesdatacollectionandevidenceofanalysisinthefollowingareas:
•Perceptionofdepartmentcommitmenttotheprogram;
•Effectivenessoftheadministrator/coordinator;
•Extentandqualityofsupportservices;
•Qualityoftechnicalsupport;
•Perspectiveonstudentlearning;
•Qualityofstudentsandcomparisonwithtraditionalcampusstudents;
•Rigorofcoursesandprograms;
•Effectiveuseoffacultytime;
•Personalandprofessionalsatisfactionwithinvolvementintheprogram(Lesht,Montague,Page,Shaik,&Smith,2006,pp.98‐99).
Otherdatamaybeused;however,thesearetheprimaryareasthatshouldbefocusedupon.
Recommendations
Usemorethanasinglereviewprocesstoevaluatequality.Thefollowingareasshouldbecombinedforuseintheevaluationprocess:
o Studentsatisfactionandstudentcomplaints(programandsupportservices);o Studentsuccess/persistence/withdrawal/dropratesincludingfeedbackreceived;o Courseevaluations;o Facultysatisfaction;o Blendedcoursematerialsandmethodsreviews;o Learningoutcomesachievement(atthecourseandprogramlevel);o Evidenceofinstitutionalsupportforblendededucation.
Establishgoalsforeachoftheevaluationprocessesanddevelopimprovementplanswhenresultsfallshortofintendedgoals.Recognizeandsharepositiveresultswithallthoseinvolved.
Engageineffortstodetermineandcomparesuccessratesandqualitystandardswithotherprogramsoflikesizewithsimilarinstitutions.
Evaluation and Assessment 83
3. Intended learning outcomes at the course and program level are reviewed regularly to ensure alignment, clarity, utility, appropriateness, and effectiveness.*
Learningoutcomesshouldbethefoundationforbothcoursesandprograms.Alearningoutcomeis“ageneralstatementofwhatlearnerswillbeabletodoaftertheinstructioniscompleted”(Davidson‐Shivers&Rasmussen,2006,p.84).Likewise,learningoutcomesshoulddescribeexplicitstudentbehavior.Theyshouldbecurrent,relevant,appropriateforsupportingtheprogramgoals,andclearlystated—thesecharacteristicshelpwithdevelopingstudentassessmentactivities.Learningoutcomesshouldbereviewedforrelevanceandcourseactivitiesandassessmentsshouldbeupdatedaccordingly.
Effectiveassessmentdependsonstandardsandclearobjectives(Holtz&Radner,2006).Becauseclarityissoimportanttostudentunderstanding,thereshouldbeaperiodicrecurrentreviewtodetermineiflearningoutcomesareclearlydefinedandstillapplicableandrelevanttothelearninggoals.
Recommendations
Mapcourselearningoutcomestodemonstratehowtheysupportacademicprogramgoalsandobjectives.
Updatelearningoutcomesduringregularcurriculumreviewtoensuretheyareclear,current,andrelevant.
Regularlyreviewlearningoutcomesinblendedcoursesandprograms(includepeerreviewintheprocess)andensurelearningoutcomesineachcoursesupportoverallprogramgoals,objectives,andexpectedstudentproficiencies.
Demonstratealearningoutcomereviewprocesswithdocumentedchangesornotesofreviewprocess.
84 Quality Scorecard for Blended Learning Programs
4. A process is in place and followed for the comprehensive assessment of support services for faculty and students.
Evaluatingthequalityofablendededucationprogramshouldincludeanexaminationofallsupportservicesforbothstudentandfaculty.Thereviewprocessshouldincludeannualgoalsforimprovedservicesandsatisfactionlevels.Inaddition,theinstitutionshoulddemonstrateacommitmenttoaprocessforcontinuousimprovement.Forexample,agoalforthetechnologysupportservicescouldbetooffer24x7supporttostudentswith90%ofthosestudentsbelievingtheirneedsweremetafterthefirstphonecalloremail.Ifonly70%ofstudentsreportedthattheirneedsweremet,thenastrategyshouldbedevelopedtoimprovetheservicebeingprovidedtostudents.Allresultsofsurveysandimprovementstrategiesshouldbedocumentedforevidence.Supportserviceswouldincludelibraryservices,tutoring,bookstore,counseling,advising,blendedstudentorientations,financialaid,andcashierservices.Facultysupportserviceswouldincludetechnicalsupport,coursedevelopmentsupport,professionaldevelopmentactivities,andongoingsupportduringtheteachingprocess.
Recommendations
Surveyfacultyteachingblendedcoursesandannuallyassessallfacultysupportservicesforteachingandcreatingblendedcoursematerials,suchascoursedevelopmentsupport,training,andongoingteachingsupportforpossibleimprovementandfacultysatisfaction.Establishagoalforfacultysatisfaction.Ifthegoalisnotmet,developastrategyforimprovement.
Attheleast,annuallysurveyblendedstudentstoreviewstudentsatisfactionwithsupportservicestodeterminehowwellstudents’needsaremet,andusedatafromthereviewprocessforcontinuousimprovements.Forexample,findoutfromstudentsiftheyusethecampusbookstorefortextbookpurchasesandhowtheyfeelabouttheserviceprovided.
Evaluation and Assessment 85
5. A process is in place and followed for the assessment of student retention in blended courses and programs.
Programretentioncanbedefinedas“thenumberoflearnersorstudentswhoprogressfromonepartofaneducationalprogramtothenext.Inhighereducation,studentretentionisnormallymeasuredasenrollmentfromacademicyeartoacademicyear”(Martinez,2003,p.3).Retentionmaybemeasuredbyblendedcoursecompletionrateswhichshouldalsobecloselymonitored;however,retentioninvolvesreturningbackeachsemesterandbeingsuccessfullyenrolledincourses.
Whileretentionratesdovaryfromprogramtoprogram,therearemanyprogramsthatexperiencenormalorevenbetterretentionrates.Programleadersandinstructorscantakeproactivestepstoincreasestudentretention.
Itisclearthatdatamustbecollectedfromavarietyofareastoformulateastrategyforimprovingretention.Studentswhodropoutoftheprogramshouldbesurveyedandfeedbackcollectedontheirreasonsfornotfinishingacourseorprogram.
Recommendations
Examineanddocumentstudentretentionincoursesandprogramsfromsemestertosemesterandyeartoyear.Comparetheresultingdatatothefeedbackcollectedonpriorcourseevaluations.
Develop,implement,andassessstrategiestocontinuouslyimprovestudentretention. Providefacultyandadvisorswithstrategiesandtechniquesforimprovingstudent
retention. Surveystudentsforfeedbackregardingwhytheydroppedcoursesandreviewfeedbackfor
possibleimprovementinprocesses.
86 Quality Scorecard for Blended Learning Programs
6. Program demonstrates compliance and review of accessibility standards (Section 508, etc.).
Studentswithdisabilitieshavealegalandfundamentalrighttoaccesshighereducation,whichincludesblendedcoursesanddegreeprograms.IntheUnitedStates,legislationsuchastheAmericanswithDisabilitiesAct(ADA)of1990,Section504oftheRehabilitationAct(1973),andthe1998AmendmenttoSection508oftheRehabilitationActprovidefoundationrequirementsformakingblendededucationaccessibletoallstudents.However,inspiteoftechnologicalimprovementsinblendededucation,therearestillreportsthatstudentswithdisabilitiesarenotverywellserved(Kinash&Crichton,2007).Yet,thenumberofstudentswithdisabilitiesthatpursueapostsecondaryeducationcontinuestoincrease(Simoncelli&Hinson,2010).Becauseoflegislation,students’rights,andtheincreasingnumberofdisabledstudentsenrollinginblendedprograms,institutionsmustbeabletodemonstratecompliancewithaccessibilitystandards.Complianceincludesprovidingstudentsupportservicesanddevelopingwebsiteandblendedcoursematerialsthatareaccessiblebyadaptivetechnologiessuchasscreenreadersandothertechnologiesforassistingstudents.
Recommendations
Provideclearinformationonthewebsitetostudentswithdisabilitieswhoareinterestedinorenrolledinblendedclassesandlinkstospecialsupportpagesintheblendedcoursematerials.
DesignatesupportpersonnelthatareknowledgeableofADAcomplianceissuesinblendededucationprograms.
Providecontinuoustrainingforallthoseinvolvedwithservingstudentswithdisabilities. Publishcompliancewithaccessibilitystandardsontheblendededucationprogram’s
website. Applyuniversaldesignstandardsinblendedcoursedevelopmentandcoursematerials.
Evaluation and Assessment 87
7. Course evaluations collect student feedback on the effectiveness of instruction in relation to faculty performance evaluations.
Whilefacultyshouldneverbeevaluatedsolelyondatacollectedfromcourseevaluations,thefeedbackfromstudentsshouldbecarefullyexaminedbecauseitmayprovidespecificinformationthatcouldimprovepolicydevelopmentandpointtopotentialfacultybehaviorsthatwillneedcorrectionoradditionaltraining.Courseevaluationresultsmayalsosuggestlevelsofinstructoractivity,courseengagement,andtimelinessofresponsetostudents’questions.Infact,“byusingevaluationstoguaranteerapidturnaroundtimes,providecustomquestionsthatfocusoncurrentorevenplannedactivities,andaddresstheinnovativeteachingstrategiesandindividualcharacteristicsthatdistinguishdifferentcourses,facultyandstudentscancometoreflectmoreeasilyuponthoseuniqueaspectsofinstructionthatmattertothem”(Anderson,Brown,&Spaeth,2006,para.20).
Recommendations
Determineifcurrentcourseevaluationsforblendedcoursescollectstudentfeedbackoninstructorperformanceintheblendedclassroom.Ifnot,modifytheinstrumentsinordertocollectfeedbackspecifictoinstructorperformance.
Carefullyreviewblendedcourseevaluationresultsasapartoftheoverallperformanceevaluationprocessforfacultyteachingblended.Lookfortrendsorsimilarcommentsaboutindividualinstructors’blendedteachingperformance.
Baseduponthecourseevaluationresults,gooverthefeedbackwiththefacultymemberanddevelopaplantominimizesimilarnegativefeedbackthenexttimethecourseistaught.
88 Quality Scorecard for Blended Learning Programs
8. Course evaluations collect student feedback on quality of blended course materials.
Evaluationis“theformaldeterminationofthequality,effectivenessorvalueofaprogram,product,project,process,objective,orcurriculum”(Worthen&Sanders,1987,p.23).Blendedcourseevaluationshelpdeterminetheeffectivenessandvalueofacourseaswellasprovidefeedbackregardinginstructorperformance.Thus,Cavanaugh(2002)recommended,“studentsshouldbeaskedtoexplaintheirsatisfactionwiththeirexperience,includinglikesanddislikes”(p.184).Thedatacollectedfromcourseevaluations,ifreviewed,analyzed,andreflectedupon,supportsbetterawarenessoftheinstructionalpractice(Huba&Freed,2000).
Recommendations
Gatherinformationbeyondhowmuchthestudentlikedthecourse.Useevaluationstocollectinformationaboutthespecificblendedcoursematerialsandtextbooks,aswellashowtheinstructorperformedasthementorandfacilitatorofthecourse.
Periodicallyreviewthequestionsinthecourseevaluationsforrelevanceandeffectiveness. Inblendedcourseevaluations,askstudentshowtheyfeelabouttheeffectivenessofthe
blendedinstruction. Collectstudentfeedbackabouttechnologysupportingthecoursedeliveryandcourse
activities. Considerthefollowingquestionsforevaluations(Achtemeier,Morris,&Finnegan,2003,p.
8):o Werethecoursegoals,learningobjectivesandoutcomesmadecleartoyouatthe
beginningofthecourse?o Didyouhavethenecessarytechnologicalequipmentandskillsrequiredforthis
course?o Wasthereadequatetechnicalsupportifyouencountereddifficulties?o Wastheformatandpagedesignofthecourseeasytouse?o Weretheresufficientinstructionsgivenforyoutocompleteallassignments?o Didyoufeelhinderedinyourcourseexperienceinanyway?Pleasedescribe.o Werestandardsforevaluationofassignmentsmadeclear?o Didyoureceivepromptfeedbackonyourcompletedassignments?o Whatlearningactivitiesmostinfluencedyourlearninginthiscourse?
Evaluation and Assessment 89
9. A process is in place and followed for the institutional assessment of faculty blended teaching performance.
Thesuccessofablendededucationprogramcanoftendependlargelyuponfacultyinvolvedastheycandirectlyaffectstudentretention,studentsatisfaction,andstudentsuccess.Infact,Tinto(1999)foundinthetraditionalclassroom,thatthefrequencyandqualityoffacultyandstudentcontactisdirectlyrelatedtostudentpersistence.Thisalsoappliestotheblendedclassroom.Becauseoftheimpactfacultycanhaveontheblendedclassroomandstudents,facultyperformanceshouldbeassessedandtheresultsshouldbeprovidedtothefacultymemberaftereachcoursetheyteach.Thisshouldincludeavarietyofmeasurements,includingstudentevaluationsanddirectperformancereviewintheblendedclassroom.
Recommendations
Createpoliciestoguideexpectationsforfacultyperformanceinblendedclasses. Assessfacultyperformanceeachtimetheyteachablendedcourse.Thismaymeanadean
orprogramdirectorwillneedtoaccessthelearningmanagementsystemtoverifyinstructoractivity.
Provideperformancestandardsforfaculty.Developanimprovementplanandfollow‐upevaluationforthosewhodonotmeettheestablishedstandards.
90 Quality Scorecard for Blended Learning Programs
10. A process is in place and followed for the assessment of stakeholder (e.g., learners, faculty, staff) satisfaction with the blended learning programs.
Everyeducationprogramexiststomeettheneedsofexternalconstituentssuchasstudentsandfutureemployersandtoadvancesociety'sneedsoverall.Blendededucationprogramsuseinternalconstituenciessuchasadministrators,technologists,faculty,andstafftoachievethegoalsdefinedfortheprogram.Eachofthesegroupsofconstituentsisastakeholderoftheblendedprogram.
Establishingaprocessthatcollectsfeedbackfromstakeholdersregardingtheeffectivenessoftheblendedprogramgathersthedataneededtoestablishaprogramofcontinuousimprovement.Thesolicitationoffeedbackfrommultipleperspectivesallowstheprogramtoimplementmulti‐raterfeedback,or360‐degreeevaluation.Ideally,thisevaluationshouldincludeavarietyofmeasurementsandcovermostorallservicesofferedbytheprogram,inadditiontotheoverallsatisfaction.Oncefeedbackiscollected,correctiveactioncanbetakentoensuretheprogramcontinuestobeeffectiveinservingstakeholdersneeds.
Recommendations
Establisharegulartimeframeinwhichtoapplyquestionnaires/surveystodiscoverthestudents’andallexternalclients’levelofsatisfactionwiththeblendedprogram.Forexample,everyfallandspring,surveyblendedstudentstodeterminetheirsatisfactionlevelwiththeprogram.
Solicitinformalfeedbackfromallstakeholders.
Evaluation and Assessment 91
ReferencesforEvaluationandAssessment
Achtemeier,S.D.,Morris,L.V.,&Finnegan,C.L.(2003).Considerationsfordevelopingevaluationsofonlinecourses.JournalofAsynchronousLearningNetworks,7(1),1‐13.Retrievedfromonlinelearningconsortium.org/sites/default/files/v7n1_achtemeier_1.pdf
Anderson,J.,Brown,G.,&Spaeth,S.(2006).Onlinestudentevaluationsandresponseratesreconsidered.Innovate,2(6).Retrievedfromnsuworks.nova.edu/cgi/viewcontent.cgi?article=1124&context=innovate
Balanko,S.L.(2002).Reviewandresources:Onlineeducationimplementationandevaluation(Report02‐11).Seattle,WA:UniversityofWashingtonOfficeofEducationalAssessment.Retrievedfromhttp://www.washington.edu/oea/pdfs/reports/OEAReport0211.pdf
Cavanaugh,C.(2002).Distanceeducationquality:Successfactorsforresources,practicesandresults.InR.Discenza,C.D.Howard,&K.Schenk(Eds.),Thedesign&managementofeffectivedistancelearningprograms(pp.171‐189).Hershey,PA:InformationScience.
Chapman,D.D.(2006).Buildinganevaluationplanforfullyonlinedegreeprograms.OnlineJournalofDistanceLearningAdministration,9(1).Retrievedfromhttp://www.westga.edu/~distance/ojdla/spring91/chapman91.htm
CouncilforHigherEducationAccreditation.(1998).Assuringqualityindistancelearning:Apreliminaryview.WashingtonDC:Author.Retrievedfromhttp://www.chea.org/pdf/HED_Apr1998.pdf
Davidson‐Shivers,G.V.,&Rasmussen,K.L.(2006).Web‐basedlearning:Design,implementation,andevaluation.UpperSaddleRiver,NJ:PearsonMerrillPrenticeHall.
Diaz,D.,&Cartnal,R.(2006).Termlengthasanindicatorofattritioninonlinelearning.Innovate:JournalofOnlineEducation,2(6).
Holtz,J.K.,Radner,B.(2006).Assessmentandcollegeprogress:Capacitybuilding.InD.D.Williams,S.L.Howell,&M.Hricko(Eds.),Onlineassessment,measurement,andevaluation:Emergingpractices,(pp.275‐291).Hershey,PA:InformationScience.
Huba,M.E.,&Freed,J.E.(2000).Learner‐Centeredassessmentoncollegecampuses:Shiftingthefocusfromteachingtolearning.Boston,MA:Allyn&Bacon.
Kinash,S.,&Crichton,S.(2007).Supportingthedisabledstudent.InM.G.Moore(Ed.),Handbookofdistanceeducation,(2nded.,pp.193‐204).Mahwah,NJ:LawrenceErlbaumAssociates
Lesht,F.L.,Montague,R.A.,Page,V.J.,Shaik,N.,&Smith,L.C.(2006).Onlineprogramassessment:AcasestudyoftheUniversityofIllinoisatUrbana‐Champaignexperience.InD.D.Williams,S.L.Howell&M.Hricko(Eds.),Onlineassessment,measurementandevaluation:Emergingpractices(pp.92‐108).Hershey,PA:InformationScience.
Lynch,M.M.(2001).Effectivestudentpreparationforonlinelearning.TheTechnologySourceArchivesattheUniversityofNorthCarolina.Nov‐Dec.Retrievedfromhttp://technologysource.org/article/effective_student_preparation_for_online_learning/
Martinez,M.(2003).Highattritionratesine‐Learning:Challenges,predictors,andsolutions.TheeLearningDeveloper'sJournal,17(11),1‐9.Retrievedfromhttp://www.elearningguild.com
Rovai,A.P.(2003).Insearchofhigherpersistenceratesindistanceeducationblendedprograms.TheInternetandHigherEducation,6(1),1‐16.doi:10.1016/S1096‐7516(02)00158‐6
92 Quality Scorecard for Blended Learning Programs
Simoncelli,A.,&Hinson,J.(2010).Designingonlineinstructionforpostsecondarystudentswithlearningdisabilities.JournalofEducationalMultimediaandHypermedia,19(2),211‐220.Chesapeake,VA:AACE.Retrievedfromhttp://www.editlib.org/p/33205
Thompson,M.M.,&Irele,M.E.(2007).Evaluatingdistanceeducationprograms.InM.G.Moore(Ed.),Handbookofdistanceeducation(2nded.,pp.419‐432).Mahwah,NJ:LawrenceErlbaum&Associates.
Tinto,V.(1999,Fall).Takingstudentretentionseriously:Rethinkingthefirstyearofcollege.NationalAcademicAdvisingAssociationJournal,19(2),5‐9.
Willging,P.,&Johnson,S.D.(2009).Factorsthatinfluencestudents’decisiontodropoutofcourses.JournalofAsynchronousLearning,13(3).Retrievedfromhttp://files.eric.ed.gov/fulltext/EJ862360.pdf
Worthen,B.,&Sanders,J.R.(1987).Educationalevaluation:Alternativeapproachesandpracticalguidelines.WhitePlains,NY:Longman.
Evaluation and Assessment 93
Appendix
Blen
ded
Qua
lity
Scor
ecar
d Ru
bric
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
he in
stitu
tion
has
had
no
disc
ussi
ons
abou
t the
ble
nded
go
vern
ance
str
uctu
re.
Gov
erna
nce,
res
pons
ibili
ties,
de
cisi
on m
akin
g au
thor
ity a
nd
orga
niza
tion
of b
lend
ed o
pera
tions
is
haph
azar
d an
d in
hibi
ting
syst
emat
ic
and
cont
inuo
us im
prov
emen
t
Gov
erna
nce,
res
pons
ibili
ties,
de
cisi
on m
akin
g au
thor
ity a
nd
orga
niza
tion
of b
lend
ed o
pera
tions
is
som
ewha
t am
orph
ous
with
mul
tiple
un
its h
avin
g sp
ecifi
c re
spon
sibi
litie
s co
mpl
icat
ing
syst
emat
ic a
nd
cont
inuo
us im
prov
emen
t
Gov
erna
nce,
res
pons
ibili
ties,
de
cisi
on m
akin
g au
thor
ity a
nd
orga
niza
tion
of b
lend
ed o
pera
tions
is
delib
erat
e an
d cl
ear;
line
s of
aut
horit
y fo
r su
ppor
ting
units
are
del
inea
ted
(e.g
., in
a s
hare
d se
rvic
e ag
reem
ent)
.
2T
he in
stitu
tion
has
had
no
disc
ussi
ons
abou
t the
ble
nded
go
vern
ance
str
uctu
re a
nd d
ecis
ion
mak
ing
Gov
erna
nce,
res
pons
ibili
ties,
de
cisi
on m
akin
g au
thor
ity a
nd
orga
niza
tion
of b
lend
ed o
pera
tions
is
haph
azar
d an
d it
is n
ot a
lway
s cl
ear
whi
ch u
nit i
s ta
king
the
lead
Gov
erna
nce,
res
pons
ibili
ties,
de
cisi
on m
akin
g au
thor
ity a
n or
gani
zatio
n of
ble
nded
ope
ratio
ns is
so
mew
hat a
mor
phou
s w
ith m
ultip
le
units
hav
ing
spec
ific
resp
onsi
bilit
ies.
Gov
erna
nce,
res
pons
ibili
ties,
de
cisi
on m
akin
g au
thor
ity a
nd
orga
niza
tion
of b
lend
ed o
pera
tions
is
delib
erat
e an
d cl
ear;
line
s of
aut
horit
y fo
r su
ppor
ting
units
are
del
inea
ted
(e.g
., in
a s
hare
d se
rvic
e ag
reem
ent)
.
3N
o st
rate
gic
plan
exi
sts.
The
ble
nded
pro
gram
is in
the
proc
ess
of d
evel
opin
g its
ow
n st
rate
gic
plan
tied
to th
e in
stitu
tiona
l pl
an.
The
ble
nded
pro
gram
has
its
own
stra
tegi
c pl
an ti
ed to
the
inst
itutio
nal
plan
and
acc
redi
tatio
n st
anda
rds;
the
plan
is p
erio
dica
lly r
evie
wed
to
asce
rtai
n its
con
tinui
ng r
elev
ance
.
The
ble
nded
pro
gram
has
its
own
stra
tegi
c pl
an ti
ed to
the
inst
itutio
nal
plan
and
acc
redi
tatio
n st
anda
rds;
the
plan
is p
erio
dica
lly r
evie
wed
to
asce
rtai
n its
con
tinui
ng r
elev
ance
; ba
sed
on th
e re
view
, it i
s im
prov
ed
and
upda
ted.
4T
he in
stitu
tion
is e
ngag
ing
in
expl
orat
ory
conv
ersa
tions
abo
ut th
e po
tent
ial o
f ble
nded
lear
ning
.
The
inst
itutio
n is
wor
king
to b
uild
its
capa
bilit
ies
(e.g
., le
ader
ship
, bu
dget
ary
supp
ort,
supp
ort s
ervi
ces)
in
ord
er to
rea
lize
the
stra
tegi
c va
lue
of b
lend
ed le
arni
ng.
The
inst
itutio
n es
tabl
ishe
s se
nior
le
ader
ship
pos
ition
s to
ste
war
d th
e bl
ende
d in
itiat
ive
with
cle
ar b
udge
tary
su
ppor
t gui
delin
es; t
here
is a
co
mm
itmen
t to
deve
lop
the
nece
ssar
y se
rvic
es to
sup
port
st
uden
ts a
nd fa
culty
mem
bers
; but
bl
ende
d le
arni
ng h
as y
et to
bec
ome
a cl
ear
part
of t
he in
stitu
tion'
s m
issi
on/v
isio
n.
Ble
nded
lear
ning
is p
art o
f the
in
stitu
tion'
s m
issi
on/v
isio
n; th
e in
stitu
tion
has
esta
blis
hed
seni
or
lead
ersh
ip p
ositi
ons
to s
tew
ard
the
wor
k w
ith c
lear
bud
geta
ry s
uppo
rt
guid
elin
es; t
here
is a
com
mitm
ent t
o de
velo
p th
e ne
cess
ary
serv
ices
to
supp
ort s
tude
nts
and
facu
lty
mem
bers
.
5T
he in
stitu
tion
is e
ngag
ing
in
expl
orat
ory
conv
ersa
tions
abo
ut
blen
ded
lear
ning
.
The
inst
itutio
n is
in th
e be
ginn
ing
stag
es o
f rea
lizin
g ho
w a
ble
nded
le
arni
ng p
rogr
am s
uppo
rts
the
inst
itutio
n's
mis
sion
, val
ues,
and
st
rate
gic
plan
.
The
ble
nded
lear
ning
pro
gram
is
clea
rly ti
ed to
the
inst
itutio
n's
mis
sion
, val
ues,
and
str
ateg
ic p
lan.
The
ble
nded
lear
ning
pro
gram
is
posi
tione
d w
ithin
the
orga
niza
tiona
l st
ruct
ure
to e
nsur
e su
cces
s; it
is
clea
rly ti
ed to
the
inst
itutio
n's
mis
sion
, val
ues,
and
str
ateg
ic p
lan.
6T
he b
lend
ed p
rogr
am is
con
side
red
a 's
tart
-up'
and
is n
ot ti
ed to
the
inst
itutio
n's
stra
tegi
c pl
anni
ng
proc
ess
spec
ifica
lly w
ith r
egar
d to
re
sour
ce a
lloca
tions
.
The
ble
nded
pro
gram
str
uggl
es to
re
aliz
e st
rate
gic
impo
rtan
ce w
hen
it co
mes
to p
roce
sses
ass
ocia
ted
with
in
stitu
tiona
l pla
nnin
g an
d re
sour
ce
allo
catio
n
Pla
nnin
g an
d re
sour
ce a
lloca
tion
(e.g
., fin
anci
al r
esou
rces
) pr
oces
ses
asso
ciat
ed w
ith b
lend
ed le
arni
ng a
re
tied
to th
e in
stitu
tion'
s ov
eral
l st
rate
gic
stra
tegy
.
Ble
nded
lear
ning
is c
onsi
dere
d 'm
issi
on c
ritic
al';
plan
ning
and
re
sour
ce a
lloca
tion
(e.g
., fin
anci
al
reso
urce
s) p
roce
sses
ass
ocia
ted
with
onl
ine
lear
ning
are
tied
to th
e in
stitu
tion'
s ov
eral
l str
ateg
ic p
lan.
INS
TIT
UT
ION
AL
SU
PP
OR
T (
27 p
oint
s)
Th
e in
stit
uti
on
has
a g
ove
rnan
ce s
tru
ctu
re t
o
enab
le c
lear
, tim
ely,
eff
ecti
ve, a
nd
co
mp
reh
ensi
ve d
ecis
ion
mak
ing
rel
ated
to
b
len
ded
lear
nin
g c
ou
rses
/pro
gra
ms.
Th
e b
len
ded
lear
nin
g p
rog
ram
’s s
trat
egic
pla
n
is r
evie
wed
fo
r it
s co
nti
nu
ing
rel
evan
ce,
com
plia
nce
wit
h a
ccre
dit
atio
n o
bje
ctiv
es, a
nd
is
per
iod
ical
ly im
pro
ved
an
d u
pd
ated
.
Th
e in
stit
uti
on
has
def
ined
th
e st
rate
gic
val
ue
of
ble
nd
ed le
arn
ing
to
its
ente
rpri
se a
nd
st
akeh
old
ers
(stu
den
ts, f
acu
lty,
par
ents
, etc
.).
Qu
alit
y S
core
card
C
riter
ia fo
r E
xcel
lenc
e in
Ble
nded
Lea
rnin
g P
rogr
ams
Th
e o
rgan
izat
ion
al s
tru
ctu
re o
f th
e b
len
ded
le
arn
ing
pro
gra
m s
up
po
rts
the
inst
itu
tio
n's
m
issi
on
, val
ues
, an
d s
trat
egic
pla
n.
Th
e in
stit
uti
on
has
a p
roce
ss f
or
pla
nn
ing
an
d
reso
urc
e al
loca
tio
n f
or
the
ble
nd
ed le
arn
ing
p
rog
ram
, in
clu
din
g f
inan
cial
res
ou
rces
, in
ac
cord
ance
wit
h s
trat
egic
pla
nn
ing
.
Th
e in
stit
uti
on
has
a g
ove
rnan
ce s
tru
ctu
re t
o
enab
le s
yste
mat
ic a
nd
co
nti
nu
ou
s im
pro
vem
ent
rela
ted
to
th
e ad
min
istr
atio
n o
f b
len
ded
ed
uca
tio
n.
96 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
7T
he b
lend
ed p
rogr
am is
con
side
red
a 's
tart
-up'
and
is n
ot ti
ed to
the
inst
itutio
n's
stra
tegi
c pl
anni
ng
proc
esse
s sp
ecifi
cally
with
reg
ard
to
reso
urce
allo
catio
ns
The
ble
nded
pro
gram
con
sist
ently
st
rugg
les
to g
arne
r su
ffici
ent
reso
urce
allo
catio
n (e
.g.,
finan
cial
re
sour
ces)
to e
ffect
ivel
y su
stai
n th
e pr
ogra
m
The
inst
itutio
n de
mon
stra
tes
suffi
cien
t res
ourc
e al
loca
tion
to
effe
ctiv
ely
sust
ain
the
blen
ded
prog
ram
Ble
nded
lear
ning
is c
onsi
dere
d 'm
issi
on c
ritic
al';
the
inst
itutio
n de
mon
stra
tes
suffi
cien
t res
ourc
e al
loca
tion
to e
ffect
ivel
y su
stai
n an
d al
so g
row
the
blen
ded
prog
ram
.
8T
he in
stitu
tion
is e
ngag
ing
in
expl
orat
ory
conv
ersa
tions
abo
ut
blen
ded
lear
ning
.
The
inst
itutio
n is
in th
e be
ginn
ing
stag
es o
f rea
lizin
g ho
w a
ble
nded
le
arni
ng p
rogr
am s
uppo
rts
the
inst
itutio
n's
mis
sion
, val
ues,
and
st
rate
gic
plan
.
The
ble
nded
lear
ning
pro
gram
is
clea
rly ti
ed to
the
inst
itutio
n's
mis
sion
, val
ues,
and
str
ateg
ic p
lan.
The
ble
nded
lear
ning
pro
gram
is
posi
tione
d w
ithin
the
orga
niza
tiona
l st
ruct
ure
to e
nsur
e su
cces
s; it
is
clea
rly ti
ed to
the
inst
itutio
n's
mis
sion
, val
ues,
and
str
ateg
ic p
lan.
9N
o pr
oces
s is
est
ablis
hed
to e
nsur
e th
at p
erm
issi
ons
(Cre
ativ
e C
omm
ons,
Cop
yrig
ht, F
air
Use
, P
ublic
Dom
ain,
etc
.) a
re in
pla
ce fo
r ap
prop
riate
use
of a
ll co
urse
m
ater
ials
.
The
inst
itutio
n/ p
rogr
am is
in th
e be
ginn
ing
stag
es o
f dev
elop
ing
a co
nsis
tent
pro
cess
to e
nsur
e th
at
perm
issi
ons
(Cre
ativ
e C
omm
ons,
C
opyr
ight
, Fai
r U
se, P
ublic
Dom
ain,
et
c.)
are
in p
lace
for
appr
opria
te u
se
in a
ll co
urse
mat
eria
ls.
The
inst
itutio
n/ p
rogr
am h
as
deve
lope
d a
cons
iste
nt p
roce
ss to
en
sure
that
per
mis
sion
s (C
reat
ive
Com
mon
s, C
opyr
ight
, Fai
r U
se,
Pub
lic D
omai
n, e
tc.)
are
in p
lace
for
appr
opria
te u
se in
all
cour
se
mat
eria
ls; r
ando
m c
heck
s ar
e co
nduc
ted
to v
erify
adh
eren
ce.
The
inst
itutio
n/ p
rogr
am h
as
deve
lope
d a
cons
iste
nt p
roce
ss to
en
sure
that
per
mis
sion
s (C
reat
ive
Com
mon
s, C
opyr
ight
, Fai
r U
se,
Pub
lic D
omai
n, e
tc.)
are
in p
lace
for
appr
opria
te u
se in
all
cour
se
mat
eria
ls; c
ours
es a
re c
heck
ed fo
r ad
here
nce
to p
roce
ss a
nd g
uida
nce
prov
ided
whe
n is
sues
iden
tifie
d.
A p
roce
ss is
fo
llow
ed t
hat
en
sure
s th
at
per
mis
sio
ns
(Cre
ativ
e C
om
mo
ns,
Co
pyr
igh
t,
Fai
r U
se, P
ub
lic D
om
ain
, etc
.) a
re in
pla
ce f
or
app
rop
riat
e u
se o
f al
l co
urs
e m
ater
ials
.
Th
e in
stit
uti
on
dem
on
stra
tes
suff
icie
nt
reso
urc
e al
loca
tio
n, i
ncl
ud
ing
tec
hn
olo
gy
and
fi
nan
cial
res
ou
rces
, in
ord
er t
o e
ffec
tive
ly
sup
po
rt t
he
mis
sio
n o
f b
len
ded
ed
uca
tio
n.
Th
e o
rgan
izat
ion
al s
tru
ctu
re o
f th
e b
len
ded
le
arn
ing
pro
gra
m s
up
po
rts
the
inst
itu
tio
n's
m
issi
on
, val
ues
, an
d s
trat
egic
pla
n.
Blended Learning Scorecard Rubric 97
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
he b
lend
ed c
ours
e de
liver
y te
chno
logy
is n
ot c
onsi
dere
d im
port
ant;
it is
mor
e lik
e an
afte
r th
ough
t tha
n a
plan
ned
serv
ice.
The
ble
nded
cou
rse
deliv
ery
tech
nolo
gy is
con
side
red
impo
rtan
t, bu
t not
yet
vie
wed
as
‘mis
sion
crit
ical
’ an
d is
sup
port
ed a
s su
ch.
The
ble
nded
cou
rse
deliv
ery
tech
nolo
gy is
con
side
red
‘mis
sion
cr
itica
l’ an
d is
sup
port
ed a
s su
ch.
The
ble
nded
cou
rse
deliv
ery
tech
nolo
gy is
con
side
red
‘mis
sion
crit
ical
’ and
is s
uppo
rted
as
such
; the
pro
gram
/in
stitu
tion
uses
met
rics
and
benc
hmar
king
for
upgr
adin
g an
d im
prov
ing
tech
nolo
gies
.
2T
he in
stitu
tion
has
yet t
o es
tabl
ish
para
met
ers
or a
pla
n w
ith r
egar
d to
da
ta m
anag
emen
t sta
ndar
ds.
No
docu
men
tatio
n is
pro
vide
d to
kn
ow w
heth
er d
ata
cent
ers
(loca
l or
outs
ourc
ed, h
oste
d or
clo
ud
serv
ices
) ar
e ad
min
iste
red
in
com
plia
nce
with
est
ablis
hed
data
m
anag
emen
t pra
ctic
es s
uch
as IT
SM
st
anda
rds.
Dat
a ce
nter
s (lo
cal o
r ou
tsou
rced
, ho
sted
or
clou
d se
rvic
es)
are
adm
inis
tere
d in
com
plia
nce
with
es
tabl
ishe
d da
ta m
anag
emen
t pr
actic
es s
uch
as IT
SM
sta
ndar
ds.
Dat
a ce
nter
s (lo
cal o
r ou
tsou
rced
, ho
sted
or
clou
d se
rvic
es)
are
adm
inis
tere
d in
com
plia
nce
with
es
tabl
ishe
d da
ta m
anag
emen
t pr
actic
es s
uch
as IT
SM
sta
ndar
ds;
the
blen
ded
prog
ram
adm
inis
trat
ion
docu
men
ts c
ontin
ual c
ompl
ianc
e.
3T
o da
te, t
he te
chno
logy
sys
tem
pe
rfor
man
ce is
not
con
tinua
lly
mon
itore
d, tr
acke
d, a
nd r
epor
ted.
Tec
hnol
ogy
deliv
ery
syst
em
perf
orm
ance
is c
ontin
ually
mon
itore
d,
trac
ked,
and
rep
orte
d.
The
pro
gram
/ ins
titut
ion
view
s its
te
chno
logy
del
iver
y s
yste
ms
as
‘mis
sion
crit
ical
’; sy
stem
per
form
ance
is
con
tinua
lly m
onito
red,
trac
ked,
and
re
port
ed.
The
pro
gram
/inst
itutio
n v
iew
s its
te
chno
logy
del
iver
y sy
stem
s as
‘mis
sion
crit
ical
’; sy
stem
pe
rfor
man
ce
is c
ontin
ually
mon
itore
d, tr
acke
d, a
nd
repo
rted
; sys
tem
bac
kups
are
in
plac
e fo
r al
l nec
essa
ry te
chni
cal
syst
ems
(pre
fera
bly
with
off-
site
bac
kups
in
case
of a
clo
sure
due
to d
isas
ter)
.
4N
o ev
iden
ce is
pro
vide
d w
hich
ill
umin
ates
that
the
prog
ram
/in
stitu
tion
has
disc
usse
d or
is
enga
ging
in a
tech
nolo
gy p
lann
ing
proc
ess.
The
pro
gram
/ ins
titut
ion
is in
the
proc
ess
of d
evel
opin
g a
tech
nolo
gy p
lan.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d an
d di
ssem
inat
ed
a te
chno
logy
pla
n to
stu
dent
s an
d fa
culty
whi
ch in
clud
es e
lect
roni
c se
curit
y m
easu
res;
the
tech
nolo
gy p
lan
is
deve
lope
d in
acc
orda
nce
with
es
tabl
ishe
d st
anda
rds
and
reg
ulat
ory
requ
irem
ents
.
The
pro
gram
/inst
itutio
n ha
s de
velo
ped
and
diss
emin
ated
a te
chno
logy
pla
n to
stu
dent
s an
d fa
culty
whi
ch in
clud
es
elec
tron
ic s
ecur
ity m
easu
res;
the
tech
nolo
gy p
lan
is d
evel
oped
in
acco
rdan
ce w
ith e
stab
lishe
d st
anda
rds
and
reg
ulat
ory
requ
irem
ents
; fac
ulty
m
embe
rs a
re tr
aine
d on
str
ateg
ies
used
to e
nsur
e qu
ality
and
ad
here
nce
to s
tand
ards
.
5T
he in
stitu
tion
has
yet t
o ex
plor
e de
velo
ping
a c
ontin
genc
y pl
an in
the
even
t of a
pro
long
ed s
ervi
ce
disr
uptio
n.
The
inst
itutio
n is
in th
e pr
oces
s of
es
tabl
ishi
ng a
con
tinge
ncy
plan
in th
e ev
ent o
f a p
rolo
nged
ser
vice
di
srup
tion.
The
inst
itutio
n ha
s es
tabl
ishe
d a
cont
inge
ncy
plan
in th
e ev
ent o
f a p
rolo
nged
ser
vice
di
srup
tion.
The
inst
itutio
n ha
s es
tabl
ishe
d a
cont
inge
ncy
plan
in th
e ev
ent o
f a
prol
onge
d se
rvic
e di
srup
tion;
moc
k di
sast
er d
rills
are
con
duct
ed
perio
dica
lly to
ens
ure
back
-up
and
rest
orat
ion
proc
esse
s ar
e vi
able
and
m
aint
ain
data
inte
grity
; g
oals
hav
e be
en e
stab
lishe
d fo
r tim
ely
syst
em r
ecov
ery.
Th
e te
chn
olo
gy
syst
ems
rela
ted
to
th
e d
eliv
ery
of
ble
nd
ed le
arn
ing
pro
gra
ms
are
hig
hly
re
liab
le a
nd
op
erab
le w
ith
mea
sura
ble
st
and
ard
s b
ein
g u
tiliz
ed s
uch
as
syst
em
do
wn
tim
e tr
acki
ng
an
d t
ask
ben
chm
arki
ng
.*
Wh
eth
er t
he
inst
itu
tio
n m
ain
tain
s lo
cal d
ata
cen
ters
(se
rver
s), a
nd
/or
con
trac
ts f
or
ou
tso
urc
ed, h
ost
ed s
ervi
ces
or
clo
ud
ser
vice
s,
tho
se s
yste
ms
are
adm
inis
tere
d in
co
mp
lian
ce
wit
h e
stab
lish
ed d
ata
man
agem
ent
pra
ctic
es
such
as
the
Info
rmat
ion
Tec
hn
olo
gy
Ser
vice
M
anag
emen
t (I
TS
M)
stan
dar
ds,
wh
ich
incl
ud
e ap
pro
pri
ate
po
wer
pro
tect
ion
, bac
kup
so
luti
on
s, d
isas
ter
reco
very
pla
ns,
etc
.
Th
e co
urs
e d
eliv
ery
tech
no
log
y is
co
nsi
der
ed a
m
issi
on
cri
tica
l en
terp
rise
sys
tem
an
d
sup
po
rted
as
such
.
A d
ocu
men
ted
tec
hn
olo
gy
pla
n t
hat
incl
ud
es
elec
tro
nic
sec
uri
ty m
easu
res
(e.g
., p
assw
ord
p
rote
ctio
n, e
ncr
ypti
on
, sec
ure
on
line
or
pro
cto
red
exa
ms
if a
pp
licab
le, e
tc.)
is in
pla
ce
and
op
erat
ion
al t
o e
nsu
re q
ual
ity,
in
acco
rdan
ce w
ith
est
ablis
hed
sta
nd
ard
s an
d
reg
ula
tory
req
uir
emen
ts.*
Th
e in
stit
uti
on
has
an
est
ablis
hed
(u
pd
ated
an
d
con
tin
uo
usl
y re
view
ed)
con
tin
gen
cy p
lan
fo
r th
e co
nti
nu
ance
of
dat
a ce
nte
rs a
nd
su
pp
ort
se
rvic
es in
th
e ev
ent
of
pro
lon
ged
ser
vice
d
isru
pti
on
.
TE
CH
NO
LO
GY
SU
PP
OR
T (
21 p
oint
s)
98 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
6T
he b
lend
ed p
rogr
am is
in ‘s
tart
-up’
an
d ha
s ye
t to
deve
lop
a sy
stem
for
build
ing
and
supp
ortin
g th
e bl
ende
d in
fras
truc
ture
.
The
pro
gram
/ ins
titut
ion
has
a di
strib
uted
sys
tem
(e.
g., s
ever
al
func
tiona
l uni
ts a
re in
volv
ed)
for
build
ing
and
mai
ntai
ning
the
blen
ded
infr
astr
uctu
re.
The
pro
gram
/ ins
titut
ion
has
inve
sted
in
a c
entr
aliz
ed s
yste
m fo
r bu
ildin
g an
d m
aint
aini
ng th
e bl
ende
d in
fras
truc
ture
.
The
pro
gram
/inst
itutio
n ha
s in
vest
ed
in a
cen
tral
ized
sys
tem
for
build
ing
and
mai
ntai
ning
the
blen
ded
infr
astr
uctu
re;
it is
team
bas
ed,
colla
bora
tive,
com
preh
ensi
ve, a
ctio
n or
ient
ed a
nd n
onhi
erar
chic
al;
pers
pect
ives
of k
ey s
take
hold
ers
are
cont
inua
lly a
sses
sed
to im
prov
e th
e sy
stem
.
7N
o ev
iden
ce is
pro
vide
d th
at fa
culty
, st
aff o
r st
uden
t use
new
tech
nolo
gies
an
d ho
w th
ey a
re s
uppo
rted
.
Ent
repr
eneu
rial f
acul
ty m
embe
rs a
re
expl
orin
g ne
w te
chno
logi
es a
nd
deve
lopi
ng th
eir
indi
vidu
al s
kills
with
lim
ited
inst
itutio
nal s
uppo
rt.
The
pro
gram
/inst
itutio
n pr
ovid
es
suffi
cien
t ser
vice
s to
sup
port
facu
lty,
staf
f and
stu
dent
s in
the
use
of n
ew
tech
nolo
gies
.
The
pro
gram
/inst
itutio
n pr
ovid
es
suffi
cien
t ser
vice
s to
sup
port
facu
lty,
staf
f and
stu
dent
s in
the
use
of n
ew
tech
nolo
gies
; gui
des,
info
rmat
ion
shee
ts, o
r tu
toria
ls a
re d
evel
oped
; he
lp d
esk
supp
ort s
taff
are
know
ledg
eabl
e; fa
culty
wor
ksho
ps o
r ot
her
in-s
ervi
ce tr
aini
ng is
con
duct
ed
to e
nsur
e fa
culty
hav
e th
e ne
cess
ary
skill
s to
use
new
tech
nolo
gies
.
Fac
ult
y, s
taff
, an
d s
tud
ents
are
su
pp
ort
ed in
th
e d
evel
op
men
t an
d u
se o
f n
ew t
ech
no
log
ies
and
ski
lls a
pp
licab
le t
o b
len
ded
lear
nin
g.
A c
entr
aliz
ed s
yste
m p
rovi
des
su
pp
ort
fo
r b
uild
ing
an
d m
ain
tain
ing
th
e b
len
ded
ed
uca
tio
n in
fras
tru
ctu
re.*
Blended Learning Scorecard Rubric 99
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
here
is n
o in
dica
tion
that
cou
rses
ar
e de
sign
ed to
ens
ure
that
lear
ning
ou
tcom
es a
re m
et.
Ble
nded
cou
rses
are
des
igne
d to
m
eet l
earn
ing
outc
omes
, but
no
cons
iste
nt m
echa
nism
exi
sts
to
ensu
re c
ours
e an
d pr
ogra
m le
arni
ng
outc
omes
are
met
.
Ble
nded
cou
rses
are
des
igne
d to
m
eet l
earn
ing
outc
omes
and
a
mec
hani
sm e
xist
s to
ens
ure
cour
se
and
prog
ram
lear
ning
out
com
es a
re
met
; how
ever
, adh
eren
ce v
arie
s ac
ross
cou
rses
and
pro
gram
s.
All
blen
ded
cour
ses
are
desi
gned
to
ensu
re s
tude
nts
deve
lop
the
nece
ssar
y kn
owle
dge
and
skill
s at
bo
th th
e co
urse
and
pro
gram
leve
l; em
phas
is is
pla
ced
on b
oth
form
ativ
e an
d su
mm
ativ
e as
sess
men
t crit
eria
.
2N
o ev
iden
ce is
pro
vide
d w
hich
indi
cate
s gu
idel
ines
reg
ardi
ng m
inim
um s
tand
ards
ex
ist.
The
pro
gram
/inst
itutio
n ha
s be
gun
the
proc
ess
of d
evel
opin
g gu
idel
ines
co
ncer
ning
min
imum
sta
ndar
ds.
Gui
delin
es e
xist
, but
the
prog
ram
/inst
itutio
n ha
s ye
t to
fully
de
ploy
them
acr
oss
all c
ours
es.
Gui
delin
es e
xist
and
evi
denc
e (e
.g.,
exem
plar
y co
urse
s) is
pre
sent
ed
whi
ch s
how
case
s th
at s
tand
ards
are
fu
lly d
eplo
yed
acro
ss a
ll co
urse
s.
3T
he in
stitu
tion
has
yet t
o es
tabl
ish
cons
iste
ncy
in c
ours
e de
velo
pmen
t fo
r st
uden
t ret
entio
n, q
ualit
y or
hav
e a
cons
iste
nt n
avig
atio
nal s
truc
ture
.
The
inst
itutio
n is
exp
lorin
g m
etho
ds
to e
stab
lish
cons
iste
ncy
in c
ours
e de
velo
pmen
t for
stu
dent
ret
entio
n,
qual
ity o
r ha
ve a
con
sist
ent
navi
gatio
nal s
truc
ture
.
Inte
ntio
nal c
ours
e de
velo
pmen
t and
de
sign
add
ress
es c
once
rns
in
cons
iste
ncy
in c
ours
e rig
or,
achi
evem
ent o
f out
com
es, a
nd
cons
iste
nt n
avig
atio
nal s
truc
ture
. E
valu
atio
n of
cou
rse
outc
omes
in
cons
iste
nt w
ith S
yste
mat
ic E
valu
atio
n P
lan
that
incl
udes
a fe
edba
ck lo
op.
Inte
ntio
nal c
ours
e de
velo
pmen
t and
de
sign
add
ress
es c
once
rns
in
cons
iste
ncy
in c
ours
e rig
or,
achi
evem
ent o
f out
com
es, a
nd
cons
iste
nt n
avig
atio
nal s
truc
ture
w
hile
faci
litat
ing
lear
ner
inte
ract
ion
and
enga
gem
ent.
Eva
luat
ion
of
cour
se o
utco
mes
in c
onsi
sten
t with
S
yste
mat
ic E
valu
atio
n P
lan
that
in
clud
es a
feed
back
loop
. P
eer
revi
ew u
sing
a s
core
card
pro
vide
s di
rect
ion
for
stra
tegi
c pl
anni
ng a
s w
ell a
s fa
cilit
atin
g co
ntin
ual q
ualit
y im
prov
emen
t.
4T
he o
nlin
e co
mpo
nent
s of
the
blen
ded
cour
se a
re u
sed
for
to
deliv
er c
onte
nt th
at is
als
o re
peat
ed
in th
e fa
ce-t
o-fa
ce c
ours
e.
Onl
ine
cont
ent i
s ut
ilize
d to
pre
pare
st
uden
ts fo
r th
e fa
ce-t
o-fa
ce s
essi
on.
The
cou
rse
utili
zes
the
onlin
e co
mpo
nent
s th
at in
clud
e co
nten
t de
liver
y, fo
rmat
ive
asse
ssm
ents
, an
d/or
act
ive
lear
ning
to p
repa
re
stud
ents
for
the
face
-to-
face
se
ssio
ns.
The
cou
rse
utili
zes
the
onlin
e co
mpo
nent
s th
at in
clud
e co
nten
t de
liver
y, fo
rmat
ive
asse
ssm
ents
, an
d/or
act
ive
lear
ning
to p
repa
re
stud
ents
for
the
face
-to-
face
se
ssio
ns. I
n ad
ditio
n, th
e fa
ce-t
o-fa
ce
sess
ion
utili
zes
onlin
e en
viro
nmen
t in
a va
riety
of w
ays
both
dur
ing
and
afte
r th
e se
ssio
n en
ds.
5N
o pe
riodi
c re
view
pro
cess
of
inst
ruct
iona
l mat
eria
ls a
nd c
ours
e sy
llabi
exi
sts.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d a
plan
to p
erio
dica
lly r
evie
w c
ours
e sy
llabi
bu
t few
rev
iew
s ha
ve o
ccur
red.
Per
iodi
c re
view
of i
nstr
uctio
nal m
ater
ials
an
d co
urse
syl
labi
occ
urs,
but
on
an a
d ho
c ba
sis.
The
pro
gram
/inst
itutio
n ha
s a
cons
iste
nt
and
syst
em-w
ide
perio
dic
revi
ew p
roce
ss
of in
stru
ctio
nal m
ater
ials
and
cou
rse
sylla
bi to
det
erm
ine
curr
ency
, rel
evan
ce,
and
mea
sura
bilit
y.
6N
o pe
riodi
c re
view
pro
cess
of c
ours
e as
sign
men
ts a
nd a
ctiv
ities
exi
sts.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d a
plan
to p
erio
dica
lly r
evie
w c
ours
e as
sign
men
ts a
nd a
ctiv
ities
but
few
re
view
s ha
ve o
ccur
red.
Per
iodi
c re
view
of c
ours
e as
sign
men
ts
and
activ
ities
occ
urs,
but
on
an a
d ho
c ba
sis.
The
pro
gram
/inst
itutio
n ha
s a
cons
iste
nt
and
syst
em-w
ide
perio
dic
revi
ew p
roce
ss
of c
ours
e as
sign
men
ts a
nd a
ctiv
ities
; ru
bric
s or
sel
f-as
sess
men
t too
ls a
re
prov
ided
; iss
ues
of a
cade
mic
dis
hone
st
are
rais
ed.
Co
urs
e as
sig
nm
ents
an
d a
ctiv
itie
s ar
e re
view
ed p
erio
dic
ally
to
en
sure
th
ey m
eet
the
ble
nd
ed c
ou
rses
' an
d p
rog
ram
's le
arn
ing
o
utc
om
es.
Th
ere
is c
on
sist
ency
in c
ou
rse
dev
elo
pm
ent
for
stu
den
t re
ten
tio
n (
enro
llmen
t an
d c
ou
rse
com
ple
tio
n)
and
qu
alit
y (i
.e.,
cou
rses
in a
p
rog
ram
hav
e a
con
sist
ent
nav
igat
ion
al
stru
ctu
re).
A b
len
ded
co
urs
e sh
ou
ld b
e d
esig
ned
as
on
e co
hes
ive
wh
ole
, in
corp
ora
tin
g b
oth
fac
e-to
-fa
ce a
nd
on
line
exp
erie
nce
s in
co
mp
lem
enta
ry
way
s.
Inst
ruct
ion
al m
ater
ials
(b
oth
on
line
and
in-
clas
s) a
nd
co
urs
e sy
llab
i are
rev
iew
ed
per
iod
ical
ly t
o e
nsu
re t
hey
mee
t th
e b
len
ded
co
urs
e's
and
pro
gra
m's
lear
nin
g o
utc
om
es.*
Gu
idel
ines
reg
ard
ing
min
imu
m r
equ
irem
ents
fo
r co
urs
e d
evel
op
men
t, d
esig
n, a
nd
del
iver
y o
f b
len
ded
inst
ruct
ion
(su
ch a
s co
urs
e sy
llab
us
elem
ents
, co
urs
e m
ater
ials
, ass
essm
ent
stra
teg
ies,
fac
ult
y fe
edb
ack)
are
fo
llow
ed.*
A c
ou
rse
dev
elo
pm
ent
pro
cess
is f
ollo
wed
th
at
ensu
res
cou
rses
are
des
ign
ed w
ith
alig
nm
ent
bet
wee
n c
ou
rse
mat
eria
ls, a
sses
smen
ts a
nd
le
arn
ing
ob
ject
ives
so
th
at s
tud
ents
dev
elo
p
the
nec
essa
ry k
no
wle
dg
e an
d s
kills
to
mee
t m
easu
rab
le le
arn
ing
ou
tco
mes
at
the
cou
rse
and
pro
gra
m le
vel.*
CO
UR
SE
DE
VE
LO
PM
EN
T A
ND
INS
TR
UC
TIO
NA
L D
ES
IGN
(42
poi
nts)
100 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
7T
he in
stitu
tion
has
yet t
o de
mon
stra
te th
at c
ours
es a
re
deve
lope
d w
ith a
stu
dent
-cen
tere
d fo
cus.
No
form
al tr
aini
ng e
xist
s, b
ut s
ome
cour
ses
are
desi
gned
with
a s
tude
nt-
cent
ered
focu
s (u
se o
f stu
dent
-ce
nter
ed a
ssig
nmen
ts a
nd
cons
truc
tivis
t act
iviti
es).
Tra
inin
g is
pro
vide
d an
d th
e pr
ogra
m/in
stitu
tion
is m
akin
g an
ef
fort
to e
nsur
e co
urse
s ar
e de
velo
ped
with
a s
tude
nt-c
ente
red
focu
s (u
se o
f stu
dent
-cen
tere
d as
sign
men
ts a
nd c
onst
ruct
ivis
t ac
tiviti
es).
App
ropr
iate
trai
ning
is p
rovi
ded
and
the
prog
ram
/inst
itutio
n en
sure
s st
uden
t-ce
nter
ed in
stru
ctio
n (e
.g.,
assi
gnm
ents
con
nect
with
the
lear
ners
' life
, eng
age
stud
ents
in
lear
ning
how
they
lear
n) is
at t
he
hear
t of t
he c
ours
e de
velo
pmen
t pr
oces
s fo
r al
l cou
rses
.
8N
o ev
iden
ce e
xist
s th
at c
ours
es a
re
desi
gned
to p
rom
ote
facu
lty a
nd s
tude
nt
enga
gem
ent.
The
pro
gram
/ ins
titut
ion
incl
udes
facu
lty-
stud
ent e
ngag
emen
t in
the
cour
se d
esig
n pr
oces
s. F
acul
ty r
ecei
ve li
ttle
to n
o tr
aini
ng fo
r co
urse
des
ign.
The
pro
gram
/ ins
titut
ion
incl
udes
facu
lty-
stud
ent e
ngag
emen
t in
the
cour
se d
esig
n pr
oces
s T
rain
ing
is p
rovi
ded,
but
no
eval
uatio
n oc
curs
for
leve
ls o
f fac
ulty
and
st
uden
t eng
agem
ent.
App
ropr
iate
trai
ning
is p
rovi
ded
to
show
case
str
ateg
ies
that
pro
mot
e bo
th
facu
lty a
nd s
tude
nt e
ngag
emen
t and
a
proc
ess
is in
pla
ce to
eva
luat
e/as
sess
le
vels
, met
hods
and
freq
uenc
y of
en
gage
men
t.
9T
he in
stitu
tion
has
not i
dent
ified
a
stan
dard
pro
cess
for
defin
ing
cloc
k/cr
edit
hour
s in
con
grue
ncy
with
re
gula
tory
and
acc
redi
tatio
n st
anda
rds.
The
inst
itutio
n ha
s no
pr
oces
s to
ens
ure
that
cou
rse
wor
kloa
ds a
re r
evie
wed
to e
nsur
e ap
prop
riate
allo
catio
n of
cre
dit.
The
inst
itutio
n ha
s id
entif
ied
a st
anda
rd p
roce
ss fo
r de
finin
g cl
ock/
cred
it ho
urs
in c
ongr
uenc
y w
ith
regu
lato
ry a
nd a
ccre
dita
tion
stan
dard
s.
A s
tand
ard
proc
ess
for
defin
ing
cred
it/cl
ock
hour
s to
ens
ure
cons
iste
ncy
in a
lloca
tion
as w
ell a
s co
ngru
ency
with
reg
ulat
ory
and
accr
edita
tion
stan
dard
s as
def
ined
by
the
orga
niza
tion
and
US
DE
is
used
to
ens
ure
com
plia
nce
with
reg
ulat
ory
and
accr
editi
ng r
equi
rem
ents
. B
lend
ed c
ours
es a
re p
erio
dica
lly
revi
ewed
to e
nsur
e ap
prop
riate
al
loca
tion
of c
redi
t and
new
ble
nded
co
urse
s ar
e re
view
ed to
ens
ure
appr
opria
te a
lloca
tion
of c
redi
t.
A s
tand
ard
proc
ess
for
defin
ing
cred
it/cl
ock
hour
s to
ens
ure
cons
iste
ncy
in a
lloca
tion
as w
ell a
s co
ngru
ency
with
reg
ulat
ory
and
accr
edita
tion
stan
dard
s as
def
ined
by
the
orga
niza
tion
and
US
DE
is
used
to
ens
ure
com
plia
nce
with
reg
ulat
ory
and
accr
editi
ng r
equi
rem
ents
. B
lend
ed c
ours
es a
re p
erio
dica
lly
revi
ewed
to e
nsur
e ap
prop
riate
al
loca
tion
of c
redi
t and
new
ble
nded
co
urse
s ar
e re
view
ed to
ens
ure
appr
opria
te a
lloca
tion
of c
redi
t. A
cade
mic
org
aniz
atio
ns a
re a
ble
to
prov
ide
facu
lty s
uppo
rt to
ens
ure
the
equi
tabl
e di
strib
utio
n of
cre
dit a
mon
g bl
ende
d, o
nlin
e an
d fa
ce to
face
co
urse
s, w
ith a
n eq
uita
ble
dist
ribut
ion
of s
tude
nt w
ork.
10T
here
is n
o ev
iden
ce th
at th
e pr
ogra
m/
inst
itutio
n ha
s a
proc
ess
in p
lace
for
eval
uatin
g th
e ef
fect
iven
ess
of c
urre
nt
and
emer
ging
tech
nolo
gies
.
A p
lan
is in
pla
ce to
eva
luat
e th
e ef
fect
iven
ess
of c
urre
nt a
nd e
mer
ging
te
chno
logi
es, b
ut it
is y
et to
be
depl
oyed
.
A p
roce
ss is
in p
lace
to e
valu
ate
curr
ent
and
emer
ging
tech
nolo
gies
, but
it is
not
fo
llow
ed c
onsi
sten
tly a
cros
s al
l cou
rses
an
d pr
ogra
ms.
A c
onsi
sten
t pro
cess
is in
pla
ce a
nd
follo
wed
acr
oss
all c
ours
es a
nd p
rogr
ams
to e
valu
ate
the
effe
ctiv
enes
s of
cur
rent
an
d em
ergi
ng te
chno
logi
es to
sup
port
le
arni
ng o
utco
mes
and
enc
oura
ge c
onte
nt
expe
rts
to b
e fa
mili
ar w
ith in
nova
tive
te
chno
logi
es.
11T
he p
rogr
am/in
stitu
tion
does
not
pr
ovid
e gu
idan
ce r
egar
ding
ap
prop
riate
and
con
stru
ctiv
e us
e of
te
chno
logy
for
blen
ded
cour
ses.
The
pro
gram
/inst
itutio
n pr
ovid
es
limite
d gu
idan
ce o
n us
ing
tech
nolo
gy
cons
truc
tivel
y, a
nd/o
r th
ere
is
wid
espr
ead
use
of s
uper
fluou
s te
chno
logy
.
The
pro
gram
/inst
itutio
n is
mak
ing
prog
ress
tow
ard
follo
win
g be
st
prac
tices
for
the
use
of te
chno
logy
in
blen
ded
cour
ses
and
is li
miti
ng th
e su
perf
luou
s us
e of
tech
nolo
gy.
The
pro
gram
/inst
itutio
n ha
s gu
idel
ines
and
sta
ndar
ds fo
r in
vest
igat
ing
tech
nolo
gies
; cou
rse
embe
dded
tech
nolo
gies
act
ivel
y su
ppor
t the
ach
ieve
men
t of l
earn
ing
outc
omes
in a
ll co
urse
s an
d su
perf
luou
s us
e of
tech
nolo
gy is
m
inim
ized
.
Co
urs
e d
esig
n p
rom
ote
s b
oth
fac
ult
y an
d
stu
den
t en
gag
emen
t.
Co
urs
e em
bed
ded
tec
hn
olo
gy
acti
vely
su
pp
ort
s th
e ac
hie
vem
ent
of
lear
nin
g
ou
tco
mes
an
d d
eliv
erin
g c
ou
rse
con
ten
t an
d
un
nec
essa
ry u
se o
f te
chn
olo
gy
is m
inim
ized
.*
Co
urs
e w
ork
load
s ar
e re
view
ed t
o e
nsu
re it
is
app
rop
riat
e fo
r d
esig
nat
ed c
red
it a
lloca
tio
n.
A p
roce
ss is
est
ablis
hed
an
d f
ollo
wed
fo
r ev
alu
atin
g t
he
effe
ctiv
enes
s o
f cu
rren
t an
d
emer
gin
g t
ech
no
log
ies
to s
up
po
rt t
he
ach
ieve
men
t o
f le
arn
ing
ou
tco
mes
an
d
del
iver
ing
co
urs
e co
nte
nt.
Stu
den
t-ce
nte
red
inst
ruct
ion
is c
on
sid
ered
d
uri
ng
th
e co
urs
e d
evel
op
men
t p
roce
ss (
i.e.,
stu
den
t en
gag
emen
t, im
mer
sio
n, a
nd
per
son
al
resp
on
sib
ility
).
Blended Learning Scorecard Rubric 101
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
12N
o ev
iden
ce e
xist
s th
at u
sabi
lity
test
s ar
e be
ing
cons
ider
ed o
r co
nduc
ted.
Dis
cuss
ions
are
und
erw
ay a
nd p
lans
ar
e be
ing
deve
lope
d co
ncer
ning
how
to
con
duct
usa
bilit
y te
stin
g, b
ut n
o ac
tion
has
been
take
n to
dat
e.
The
pro
gram
/ ins
titut
ion
is c
ondu
ctin
g us
abili
ty te
sts,
but
the
reco
mm
enda
tions
hav
e ye
t to
be
incl
uded
in a
ll co
urse
s an
d pr
ogra
ms.
The
pro
gram
/inst
itutio
n ha
s a
form
al
and
cons
iste
ntly
app
lied
proc
ess
of
cond
uctin
g us
abili
ty te
sts
acro
ss a
ll co
urse
s an
d pr
ogra
ms;
rev
iew
re
com
men
datio
ns a
re in
corp
orat
ed
into
cou
rses
.
13F
acul
ty m
embe
rs h
ave
no c
onsi
sten
t in
volv
emen
t in
curr
icul
um
deve
lopm
ent.
Adm
inis
trat
ive
pers
onne
l hav
e an
ac
tive
role
in th
e de
velo
pmen
t and
de
cisi
on m
akin
g fo
r bl
ende
d cu
rric
ula,
but
facu
lty m
embe
rs
prov
ide
inpu
t.
Fac
ulty
mem
bers
hav
e an
act
ive
role
in
the
deve
lopm
ent a
nd d
ecis
ion
mak
ing
for
blen
ded
curr
icul
a, w
ith
adm
inis
trat
ive
over
sigh
t.
Fac
ulty
mem
bers
are
inte
gral
in b
oth
the
deve
lopm
ent a
nd d
ecis
ion
mak
ing
for
all b
lend
ed c
urric
ula.
14N
o ev
iden
ce e
xist
s th
at fa
culty
su
ppor
t and
res
ourc
es a
re p
rovi
ded
to p
rom
ote
the
best
use
of b
lend
ed
lear
ning
in c
ours
e de
velo
pmen
t and
in
stru
ctio
nal d
esig
n.
Som
e fa
culty
sup
port
and
res
ourc
es
are
in p
lace
and
bei
ng im
plem
ente
d an
d us
ed o
n an
ad
hoc
basi
s.
The
pro
gram
/inst
itutio
n pr
ovid
es
facu
lty s
uppo
rt a
nd r
esou
rces
on
a re
gula
r ba
sis
to s
uppo
rt s
peci
fic
cour
se o
r pr
ogra
m n
eeds
.
The
pro
gram
/inst
itutio
n pr
ovid
es
regu
lar,
com
preh
ensi
ve a
nd
cons
iste
nt fa
culty
sup
port
and
re
sour
ces
to s
uppo
rt th
e be
st u
se o
f bl
ende
d de
liver
y m
etho
d in
cou
rse
deve
lopm
ent a
nd in
stru
ctio
nal d
esig
n ac
ross
the
inst
itutio
n.
Cu
rric
ulu
m d
evel
op
men
t is
a c
ore
re
spo
nsi
bili
ty f
or
facu
lty
(i.e
., fa
cult
y sh
ou
ld b
e in
volv
ed in
eit
her
th
e d
evel
op
men
t o
r th
e d
ecis
ion
mak
ing
fo
r th
e b
len
ded
cu
rric
ulu
m
cho
ices
).F
acu
lty
sup
po
rt a
nd
res
ou
rces
are
pro
vid
ed t
o
pro
mo
te t
he
bes
t u
se o
f b
len
ded
del
iver
y m
eth
od
in c
ou
rse
dev
elo
pm
ent
and
in
stru
ctio
nal
des
ign
to
fac
ilita
te t
each
ing
an
d
lear
nin
g.
Usa
bili
ty t
ests
are
co
nd
uct
ed a
nd
ap
plie
d a
nd
re
com
men
dat
ion
s b
ased
up
on
Web
Co
nte
nt
Acc
essi
bili
ty G
uid
elin
es (
WC
AG
s) a
re
inco
rpo
rate
d in
th
e co
urs
e d
esig
n p
roce
ss.
102 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
he p
rogr
am/in
stitu
tion
does
not
ha
ve r
equi
rem
ents
or
stan
dard
s fo
r sy
llabi
.
Tex
tboo
k an
d re
quire
d le
arni
ng
mat
eria
ls a
re m
ade
avai
labl
e to
st
uden
ts in
adv
ance
of t
heir
cour
se
regi
stra
tion;
the
use
of s
ylla
bi
requ
irem
ents
or
stan
dard
s va
ries
acro
ss th
e co
urse
s or
pro
gram
.
Syl
labi
par
amet
ers/
stan
dard
s ar
e pr
ovid
ed to
all
facu
lty m
embe
rs a
nd
all b
lend
ed c
ours
es in
clud
e a
sylla
bus;
text
book
and
any
req
uire
d le
arni
ng m
ater
ials
are
mad
e av
aila
ble
to s
tude
nts
in a
dvan
ce o
f the
ir co
urse
re
gist
ratio
n.
Tra
inin
g an
d sy
llabi
st
anda
rds/
para
met
ers,
bas
ed o
n pr
ogra
m/in
stitu
tiona
l req
uire
men
ts)
are
prov
ided
to a
ll fa
culty
mem
bers
; al
l ble
nded
cou
rses
incl
ude
a sy
llabu
s; te
xtbo
ok a
nd r
equi
red
lear
ning
mat
eria
ls a
re m
ade
avai
labl
e to
stu
dent
s in
adv
ance
of t
heir
cour
se
regi
stra
tion;
the
prog
ram
/inst
itutio
n en
sure
con
sist
ency
in s
ylla
bi
plac
emen
t in
the
LMS
for
all b
lend
ed
cour
ses.
2T
he p
rogr
am/in
stitu
tion
has
no p
lan
to e
nsur
e th
at b
lend
ed s
tude
nts
have
ac
cess
to n
eces
sary
libr
ary/
lear
ning
re
sour
ces.
The
pro
gram
/ ins
titut
ion
has
no p
lan
to e
nsur
e bl
ende
d st
uden
ts b
lend
ed
stud
ents
hav
e ac
cess
to n
eces
sary
lib
rary
/ lea
rnin
g re
sour
ces
(e.g
., tu
torin
g, w
ritin
g ce
nter
).
The
pro
gram
/ ins
titut
ion
is b
uild
ing
out i
ts a
cces
s to
ens
ure
ble
nded
st
uden
ts h
ave
acce
ss to
nec
essa
ry
libra
ry/ l
earn
ing
reso
urce
s (e
.g.,
tuto
ring,
writ
ing
cent
er).
The
pro
gram
/ ins
titut
ion
ensu
re a
ll bl
ende
d st
uden
ts h
ave
acce
ss to
ne
cess
ary
libra
ry/ l
earn
ing
reso
urce
s (e
.g.,
tuto
ring,
writ
ing
cent
er)
rega
rdle
ss o
f geo
grap
hic
loca
tion.
3T
he p
rogr
am/in
stitu
tion
has
no p
lan
to e
nsur
e th
at r
equi
red
elem
ents
for
sylla
bi a
re in
clud
ed in
cou
rse
sylla
bi.
Info
rmat
ion
is p
rovi
ded
for
all f
acul
ty
mem
bers
, so
cour
se s
ylla
bi in
clud
e ke
y ex
pect
atio
ns a
nd g
radi
ng
polic
ies,
but
facu
lty r
espo
nse
time
is
not a
ddre
ssed
.
Fac
ulty
mem
bers
are
pro
vide
d in
form
atio
n co
ncer
ning
key
ex
pect
atio
ns fo
r sy
llabi
and
gra
ding
po
licie
s; th
e pr
ogra
m/in
stitu
tion
has
a re
com
men
ded
facu
lty r
espo
nse
time.
Info
rmat
ion/
trai
ning
is p
rovi
ded
for
all
facu
lty m
embe
rs s
o co
urse
syl
labi
in
clud
e ke
y ex
pect
atio
ns a
nd g
radi
ng
polic
ies;
the
prog
ram
/inst
itutio
n ha
s a
requ
ired
facu
lty r
espo
nse
time;
the
resp
onse
tim
e is
reg
ular
ly e
valu
ated
.
4N
o ev
iden
ce e
xist
s w
hich
illu
stra
tes
that
pro
vidi
ng li
nks
or e
xpla
natio
ns o
f te
chni
cal s
uppo
rt a
re u
sed
in b
lend
ed
clas
ses.
Som
e bl
ende
d co
urse
s pr
ovid
e lin
ks
or e
xpla
natio
ns o
f tec
hnic
al s
uppo
rt.
The
maj
ority
of b
lend
ed c
ours
es
prov
ide
links
or
expl
anat
ions
of
tech
nica
l sup
port
.
All
blen
ded
cour
ses
cons
iste
ntly
pr
ovid
e lin
ks, e
xpla
natio
ns o
r F
AQ
's
of te
chni
cal s
uppo
rt.
5N
o ev
iden
ce is
pro
vide
d w
hich
su
ppor
ts a
focu
s on
rul
es o
r st
anda
rds
of b
ehav
ior
with
in th
e pr
ogra
m/ i
nstit
utio
n.
Whi
le r
ules
or
stan
dard
s ex
ist w
ithin
th
e pr
ogra
m/ i
nstit
utio
n, th
ey a
re
broa
d w
ith li
mite
d ap
plic
atio
n to
bl
ende
d co
urse
s.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d ru
les,
sta
ndar
ds, o
r co
des
of c
ondu
ct fo
cusi
ng o
n ap
prop
riate
st
uden
t beh
avio
r fo
r bl
ende
d st
uden
ts.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d ru
les,
sta
ndar
ds, o
r co
des
of c
ondu
ct fo
cusi
ng o
n ap
prop
riate
st
uden
t beh
avio
r fo
r bl
ende
d st
uden
ts a
nd m
akes
thos
e co
nsis
tent
ly a
vaila
ble
(thr
ough
the
LMS
or
othe
r m
eans
) fo
r al
l cou
rses
.
6A
t the
pre
sent
tim
e, n
o at
tent
ion
has
been
giv
en to
the
acce
ssib
ility
of
inst
ruct
iona
l mat
eria
ls.
The
pro
gram
/ ins
titut
ion
has
a pl
an to
en
sure
inst
ruct
iona
l mat
eria
ls fo
r al
l co
urse
s ar
e ea
sily
acc
essi
ble;
eas
y to
use
; can
be
acce
ssed
by
mul
tiple
op
erat
ing
syst
ems.
The
pro
gram
/ ins
titut
ion
has
a pr
oces
s to
ens
ure
inst
ruct
iona
l m
ater
ials
for
all c
ours
es a
re e
asily
ac
cess
ible
; eas
y to
use
; can
be
acce
ssed
by
mul
tiple
ope
ratin
g sy
stem
s; h
owev
er, i
t is
not
cons
iste
ntly
dep
loye
d ac
ross
all
cour
ses.
The
pro
gram
/ ins
titut
ion
has
cons
iste
nt b
usin
ess
proc
esse
s to
en
sure
inst
ruct
iona
l mat
eria
ls fo
r al
l co
urse
s ar
e ea
sily
acc
essi
ble;
eas
y to
use
; can
be
acce
ssed
by
mul
tiple
op
erat
ing
syst
ems;
and
UX
des
ign
prin
cipl
es a
re in
here
nt to
cou
rse
desi
gn.
Th
e co
urs
e st
ruct
ure
en
sure
s th
at a
ll st
ud
ents
, re
gar
dle
ss o
f lo
cati
on
, hav
e ac
cess
to
lib
rary
/ le
arn
ing
res
ou
rces
th
at a
deq
uat
ely
sup
po
rt t
he
ble
nd
ed c
ou
rse.
*
Th
e b
len
ded
co
urs
e in
clu
des
a s
ylla
bu
s o
utl
inin
g c
ou
rse
ob
ject
ives
, lea
rnin
g
ou
tco
mes
, eva
luat
ion
met
ho
ds,
bo
oks
an
d
sup
plie
s, t
ech
nic
al a
nd
pro
cto
rin
g
req
uir
emen
ts, a
nd
oth
er r
elat
ed c
ou
rse
info
rmat
ion
, mak
ing
co
urs
e re
qu
irem
ents
an
d
cou
rse
sch
edu
le t
ran
spar
ent.
*
Exp
ecta
tio
ns
for
stu
den
t as
sig
nm
ent
com
ple
tio
n, g
rad
e p
olic
y, a
nd
fac
ult
y re
spo
nse
ar
e cl
earl
y p
rovi
ded
in t
he
cou
rse
sylla
bu
s.*
Lin
ks o
r ex
pla
nat
ion
s o
f te
chn
ical
su
pp
ort
are
av
aila
ble
in t
he
cou
rse
(i.e
., ea
ch c
ou
rse
pro
vid
es s
ug
ges
ted
so
luti
on
s to
po
ten
tial
te
chn
ical
issu
es a
nd
/or
links
fo
r te
chn
ical
R
ule
s o
r st
and
ard
s fo
r ap
pro
pri
ate
stu
den
t b
ehav
ior,
bo
th o
nlin
e an
d f
ace-
to-f
ace,
are
p
rovi
ded
wit
hin
th
e co
urs
e.
Inst
ruct
ion
al m
ater
ials
are
eas
ily a
cces
sib
le t
o
the
stu
den
t, e
asy
to u
se, w
ith
an
ab
ility
to
be
acce
ssed
by
mu
ltip
le o
per
atin
g s
yste
ms
and
ap
plic
atio
ns.
CO
UR
SE
ST
RU
CT
UR
E (
24 p
oint
s)
Blended Learning Scorecard Rubric 103
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
7N
o ev
iden
ce e
xist
s w
hich
illu
stra
tes
that
the
prog
ram
/ ins
titut
ion
is
prep
ared
to o
ffer
inst
ruct
iona
l m
ater
ials
that
are
eas
ily a
cces
sibl
e by
stu
dent
s w
ith d
isab
ilitie
s.
The
pro
gram
/ ins
titut
ion
has
ad h
oc
busi
ness
pro
cess
es to
ens
ure
inst
ruct
iona
l mat
eria
ls fo
r al
l cou
rses
ar
e ea
sily
acc
essi
ble
by s
tude
nts
with
di
sabi
litie
s.
The
pro
gram
/ ins
titut
ion
has
cons
iste
nt b
usin
ess
proc
ess
to
ensu
re in
stru
ctio
nal m
ater
ials
for
all
cour
ses
are
easi
ly a
cces
sibl
e by
st
uden
t with
dis
abili
ties.
Tra
inin
g is
pro
vide
d to
facu
lty a
nd th
e pr
ogra
m/ i
nstit
utio
n ha
s co
nsis
tent
bu
sine
ss p
roce
ss to
ens
ure
inst
ruct
iona
l mat
eria
ls fo
r al
l cou
rses
ar
e ea
sily
acc
essi
bly
by s
tude
nts
with
di
sabi
litie
s; p
rovi
de li
nks
to r
elev
ant
reso
urce
s.
8T
he p
rogr
am/in
stitu
tion
does
not
ove
rsee
th
e ae
sthe
tics
and
navi
gatio
n in
the
onlin
e co
mpo
nent
of b
lend
ed c
ours
es.
The
pro
gram
/inst
itutio
n pr
ovid
es
som
e tr
aini
ng a
bout
aes
thet
ics
and
navi
gatio
n fo
r fa
culty
des
igni
ng
blen
ded
cour
ses,
but
ther
e is
no
mon
itorin
g pr
oces
s to
ens
ure
qual
ity.
The
pro
gram
/inst
itutio
n pr
ovid
es
som
e tr
aini
ng a
bout
aes
thet
ics
and
navi
gatio
n fo
r fa
culty
des
igni
ng
blen
ded
cour
ses,
alo
ng w
ith
reco
mm
enda
tions
to m
eet
requ
irem
ents
.
The
pro
gram
/inst
itutio
n pr
ovid
es
trai
ning
with
spe
cific
gui
danc
e ab
out
aest
hetic
s an
d na
viga
tion
in th
e on
line
envi
ronm
ent,
incl
udin
g re
com
men
datio
ns to
mee
t re
quire
men
ts. T
he p
rogr
am/
inst
itutio
n re
gula
rly r
evie
ws
cour
ses.
Th
e b
len
ded
co
urs
e is
vis
ual
ly a
pp
ealin
g t
o t
he
stu
den
t an
d t
he
cou
rse
is n
avig
atio
nal
ly s
ou
nd
.
Inst
ruct
ion
al m
ater
ials
are
eas
ily a
cces
sed
by
stu
den
ts w
ith
dis
abili
ties
via
alt
ern
ativ
e in
stru
ctio
nal
str
ateg
ies
and
/or
refe
rral
to
sp
ecia
l in
stit
uti
on
al r
eso
urc
es.
104 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
he p
rogr
am/in
stitu
tion
does
not
ha
ve a
an
orga
nize
d m
eans
of
com
mun
icat
ing
expe
ctat
ions
re
gard
ing
inte
ract
ion
to fa
culty
.
Ent
repr
eneu
rial f
acul
ty m
embe
rs a
re
expl
orin
g va
rious
opp
ortu
nitie
s/to
ols
to e
ncou
rage
stu
dent
s an
d st
uden
t-fa
culty
inte
ract
ion
on a
cla
ss-b
y-cl
ass
basi
s.
Tra
inin
g is
pro
vide
d to
facu
lty to
sh
owca
se o
ppor
tuni
ties/
tool
s av
aila
ble
to e
ncou
rage
stu
dent
-st
uden
t and
stu
dent
-fac
ulty
in
tera
ctio
n an
d th
e pr
ogra
m/in
stitu
tion
enco
urag
es a
nd
faci
litat
es in
tera
ctio
n.
Tra
inin
g is
pro
vide
d to
facu
lty to
sh
owca
se o
ppor
tuni
ties/
tool
s av
aila
ble
to e
ncou
rage
stu
dent
-st
uden
t and
stu
dent
-fac
ulty
in
tera
ctio
n; th
e pr
ogra
m/in
stitu
tion
enco
urag
es a
nd fa
cilit
ates
in
tera
ctio
n; s
tude
nts
are
prov
ided
re
quire
men
ts o
r st
anda
rds
for
inte
ract
ion;
inte
ract
ion
is a
sses
sed;
su
ppor
t is
prov
ided
as
need
ed to
as
sist
facu
lty m
embe
rs in
eva
luat
ing
and
adop
ting
new
tech
nolo
gies
.
2T
he p
rogr
am/in
stitu
tion
prov
ides
no
polic
y or
rec
omm
enda
tion
to fa
culty
m
embe
rs c
once
rnin
g pr
ovid
ing
stud
ents
in b
lend
ed c
ours
es,
cons
truc
tive
and
timel
y fe
edba
ck.
Fac
ulty
mem
bers
wor
k in
depe
nden
tly
to e
nsur
e fe
edba
ck is
con
stru
ctiv
e an
d pr
ovid
ed in
a ti
mel
y m
anne
r.
Fac
ulty
mem
bers
rec
eive
trai
ning
ab
out p
rovi
ding
tim
ely
(wha
t are
the
reco
mm
ende
d tim
efra
mes
) an
d co
nstr
uctiv
e fe
edba
ck (
use
of r
ubric
s,
etc.
); m
ost c
ours
e sy
llabi
pro
vide
an
over
view
of f
eedb
ack
timel
ines
.
Fac
ulty
mem
bers
rec
eive
trai
ning
co
ncer
ning
pro
vidi
ng ti
mel
y (w
hat a
re
the
reco
mm
ende
d tim
efra
mes
) an
d co
nstr
uctiv
e fe
edba
ck (
use
of r
ubric
s,
etc.
); a
ll co
urse
syl
labi
pro
vide
an
over
view
of f
eedb
ack
timel
ines
.
3T
he p
rogr
am/in
stitu
tion
prov
ides
no
supp
ort,
trai
ning
, res
ourc
es, o
r po
licy
for
facu
lty m
embe
rs c
once
rnin
g st
rate
gies
for
crea
ting
cour
se
pres
ence
.
The
pro
gram
/inst
itutio
n ha
s co
mm
unic
ated
to fa
culty
the
impo
rtan
ce o
f ins
truc
tor
pres
ence
in
both
the
face
-to-
face
and
onl
ine
mod
aliti
es o
f the
cou
rse,
but
has
not
pr
ovid
ed tr
aini
ng, g
uide
lines
, or
reso
urce
s.
The
pro
gram
/inst
itutio
n ha
s co
mm
unic
ated
to fa
culty
the
impo
rtan
ce o
f ins
truc
tor
pres
ence
in
both
the
face
-to-
face
and
onl
ine
mod
aliti
es o
f the
cou
rse;
som
e re
sour
ces,
but
not
trai
ning
or
guid
elin
es, h
as b
een
prov
ided
.
The
pro
gram
/inst
itutio
n ha
s co
mm
unic
ated
to fa
culty
the
impo
rtan
ce o
f ins
truc
tor
pres
ence
in
both
cou
rse
mod
aliti
es, a
nd h
as
prov
ided
mod
els,
res
ourc
es, t
rain
ing,
an
d gu
idel
ines
. Ins
truc
tor
cour
se
pres
ence
is c
onsi
dere
d ve
ry
impo
rtan
t.
4T
he p
rogr
am/in
stitu
tion
prov
ides
no
supp
ort,
trai
ning
, res
ourc
es, o
r po
licy
for
facu
lty m
embe
rs c
once
rnin
g "c
ohes
ion"
in a
ble
nded
cou
rse.
The
pro
gram
/inst
itutio
n ha
s co
mm
unic
ated
to fa
culty
the
impo
rtan
ce o
f a c
ohes
ive
pres
ence
in
a bl
ende
d co
urse
, but
has
not
pr
ovid
ed tr
aini
ng, g
uide
lines
, or
reso
urce
s.
The
pro
gram
/inst
itutio
n ha
s co
mm
unic
ated
to fa
culty
the
impo
rtan
ce o
f coh
esiv
enes
s be
twee
n m
odal
ities
of t
he c
ours
e; s
ome
reso
urce
s, b
ut n
ot tr
aini
ng o
r gu
idel
ines
, has
bee
n pr
ovid
ed.
The
pro
gram
/inst
itutio
n ha
s co
mm
unic
ated
to fa
culty
the
impo
rtan
ce o
f coh
esiv
enes
s be
twee
n co
urse
mod
aliti
es, a
nd h
as p
rovi
ded
mod
els,
res
ourc
es, t
rain
ing,
and
gu
idel
ines
. Ins
truc
tor
cour
se
pres
ence
is c
onsi
dere
d ve
ry
impo
rtan
t.
5T
he p
rogr
am/in
stitu
tion
prov
ides
no
supp
ort t
o fa
culty
with
reg
ard
to
help
ing
stud
ents
lear
n ap
prop
riate
m
etho
ds fo
r ef
fect
ive
rese
arch
.
The
pro
gram
/inst
itutio
n ha
s a
plan
to
prov
ide
tuto
rials
, web
res
ourc
es, o
r ot
her
form
s to
hel
p st
uden
ts le
arn
appr
opria
te m
etho
ds fo
r ef
fect
ive
rese
arch
; how
ever
, the
y ha
ve y
et to
de
ploy
the
plan
.
The
pro
gram
/inst
itutio
n cr
eate
s va
rious
tuto
rials
, web
res
ourc
es, o
r fo
rms
to h
elp
stud
ents
lear
n ap
prop
riate
met
hods
for
effe
ctiv
e re
sear
ch a
nd fa
culty
are
offe
red
trai
ning
.
The
pro
gram
/inst
itutio
n cr
eate
s va
rious
tuto
rials
, web
res
ourc
es, o
r fo
rms
to te
ach
appr
opria
te m
etho
ds
for
effe
ctiv
e re
sear
ch; f
acul
ty
mem
bers
are
aw
are
of th
ese
reso
urce
s an
d co
nsis
tent
ly p
rovi
de
links
to th
ese
tool
s; fa
culty
als
o pr
ovid
e in
form
atio
n on
how
to a
sses
s th
e va
lidity
of r
esou
rces
. Fac
ulty
are
of
fere
d tr
aini
ng.
Stu
den
t-to
-Stu
den
t an
d F
acu
lty-
to-S
tud
ent
inte
ract
ion
are
ess
enti
al c
har
acte
rist
ics
and
ar
e en
cou
rag
ed a
nd
fac
ilita
ted
.*
Inst
ruct
or
feed
bac
k o
n s
tud
ent
assi
gn
men
ts
and
qu
esti
on
s is
co
nst
ruct
ive
and
pro
vid
ed in
a
tim
ely
man
ner
.*
Inst
ruct
ors
use
sp
ecif
ic s
trat
egie
s to
cre
ate
an
eng
aged
, lea
rnin
g-f
ocu
sed
pre
sen
ce in
bo
th
mo
dal
itie
s o
f th
e co
urs
e.
Res
ou
rces
are
pro
vid
ed t
o a
ssis
t st
ud
ents
in
con
du
ctin
g r
esea
rch
on
line
and
ass
essi
ng
th
e va
lidit
y o
f o
nlin
e re
sou
rces
.*
Fac
ult
y te
ach
th
e co
urs
e as
on
e co
hes
ive
wh
ole
, wit
h "
pre
sen
ce"
in b
oth
th
e fa
ce-t
o-f
ace
and
th
e o
nlin
e p
ort
ion
s o
f th
e co
urs
e.
TE
AC
HIN
G A
ND
LE
AR
NIN
G (
15 p
oint
s)
Blended Learning Scorecard Rubric 105
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
he p
rogr
am/in
stitu
tion
has
limite
d te
chni
cal r
esou
rces
and
can
pro
vide
lit
tle te
chni
cal s
uppo
rt.
The
pro
gram
/inst
itutio
n ha
s m
inim
al
tech
nica
l sup
port
res
ourc
es to
ass
ist
facu
lty m
embe
rs d
urin
g bo
th c
ours
e de
velo
pmen
t and
teac
hing
.
The
pro
gram
/ ins
titut
ion
has
adeq
uate
tech
nica
l sup
port
re
sour
ces
to a
ssis
t fac
ulty
mem
bers
du
ring
both
cou
rse
deve
lopm
ent a
nd
teac
hing
.
The
pro
gram
/inst
itutio
n ha
s ad
equa
te
tech
nica
l sup
port
res
ourc
es to
ass
ist
facu
lty m
embe
rs d
urin
g bo
th c
ours
e de
velo
pmen
t and
teac
hing
; tut
oria
ls
and
web
res
ourc
es h
ave
been
cr
eate
d to
aug
men
t fac
ulty
nee
ds;
the
tech
nica
l sup
port
res
ourc
es a
re
cons
ider
ed ‘m
issi
on c
ritic
al.’
2N
o ev
iden
ce is
pro
vide
d co
ncer
ning
fa
culty
trai
ning
.A
facu
lty d
evel
opm
ent p
lan
is in
pl
ace
and
som
e ad
hoc
trai
ning
oc
curs
.
The
pro
gram
/inst
itutio
n pr
ovid
es
regu
lar
trai
ning
bas
ed o
n sp
ecifi
c co
urse
or
prog
ram
nee
ds.
The
pro
gram
/inst
itutio
n pr
ovid
es
regu
lar,
com
preh
ensi
ve a
nd
cons
iste
nt c
ours
e de
velo
pmen
t su
ppor
t, tr
aini
ng a
nd o
ngoi
ng
assi
stan
ce (
e.g.
, men
torin
g pr
ogra
ms)
and
the
trai
ning
is
prov
ided
usi
ng d
iffer
ent m
odel
s of
de
liver
y (e
.g.,
virt
ual m
odul
es,
hand
outs
, liv
e tr
aini
ng).
3N
o ev
iden
ce e
xist
s th
at fa
culty
m
embe
rs a
re p
rovi
ded
trai
ning
or
othe
r m
ater
ials
rel
ated
to F
air
Use
, pl
agia
rism
, etc
.
Som
e lim
ited
trai
ning
exi
sts
for
facu
lty m
embe
rs.
Fac
ulty
mem
bers
are
pro
vide
d tr
aini
ng, c
heck
lists
, tip
she
ets
and
enga
ge in
dis
cuss
ions
con
cern
ing
Fai
r U
se, p
lagi
aris
m a
nd o
ther
re
leva
nt le
gal a
nd e
thic
al c
once
pts.
Fac
ulty
mem
bers
are
pro
vide
d tr
aini
ng, c
heck
lists
, tip
she
ets
and
enga
ge in
dis
cuss
ions
con
cern
ing
Fai
r U
se, p
lagi
aris
m a
nd o
ther
re
leva
nt le
gal a
nd e
thic
al c
once
pts;
th
e pr
ogra
m/ i
nstit
utio
n ha
s a
busi
ness
pro
cess
in p
lace
to a
sses
s co
mpl
ianc
e.
4N
o ev
iden
ce is
pro
vide
d w
hich
su
ppor
ts a
com
mitm
ent t
o on
goin
g pr
ofes
sion
al d
evel
opm
ent f
or fa
culty
m
embe
rs te
achi
ng b
lend
ed c
ours
es.
The
pro
gram
/inst
itutio
n pr
ovid
es
prof
essi
onal
dev
elop
men
t for
facu
lty
mem
bers
on
an a
d ho
c ba
sis.
The
pro
gram
/inst
itutio
n pr
ovid
es
ongo
ing
prof
essi
onal
dev
elop
men
t for
fa
culty
mem
bers
focu
sed
on b
lend
ed
teac
hing
and
lear
ning
.
The
pro
gram
/inst
itutio
n pr
ovid
es
ongo
ing
prof
essi
onal
dev
elop
men
t for
fa
culty
mem
bers
focu
sed
on b
lend
ed
teac
hing
and
lear
ning
; dev
elop
men
t op
port
uniti
es a
re p
rovi
ded
thro
ugh
vario
us d
eliv
ery
form
ats;
the
need
s of
fa
culty
mem
bers
are
con
side
red
whe
n de
velo
ping
a tr
aini
ng s
ched
ule
of to
pics
.
5T
o da
te, n
o di
scus
sion
s or
pla
nnin
g w
as e
vide
nt w
ith r
egar
d to
es
tabl
ishi
ng c
lear
ing
stan
dard
s.
The
pro
gram
/inst
itutio
n is
in th
e pr
oces
s of
est
ablis
hing
cle
ar
stan
dard
s fo
r fa
culty
eng
agem
ent
and
expe
ctat
ions
.
The
pro
gram
/inst
itutio
n ha
s es
tabl
ishe
d cl
ear
stan
dard
s fo
r fa
culty
eng
agem
ent a
nd
expe
ctat
ions
; ens
ure
appr
opria
te
pers
onne
l and
sys
tem
s ar
e in
pla
ce
to c
omm
unic
ate
stan
dard
s an
d m
onito
r fa
culty
per
form
ance
.
The
pro
gram
/inst
itutio
n ha
s es
tabl
ishe
d cl
ear
stan
dard
s fo
r fa
culty
eng
agem
ent a
nd
expe
ctat
ions
; ens
ure
appr
opria
te
pers
onne
l and
sys
tem
s ar
e in
pla
ce
to c
omm
unic
ate
stan
dard
s an
d m
onito
r fa
culty
per
form
ance
; cre
ate
and
impl
emen
t ble
nded
facu
lty
cert
ifica
tion
cour
ses;
eng
age
a co
nsis
tent
per
form
ance
rev
iew
pr
oces
s.
FA
CU
LT
Y S
UP
PO
RT
(18
poi
nts)
Cle
ar s
tan
dar
ds
are
esta
blis
hed
fo
r fa
cult
y en
gag
emen
t an
d e
xpec
tati
on
s ar
ou
nd
ble
nd
ed
teac
hin
g (
e.g
. res
po
nse
tim
e, c
on
tact
in
form
atio
n, e
tc.)
.
Fac
ult
y ar
e p
rovi
ded
on
-go
ing
pro
fess
ion
al
dev
elo
pm
ent
rela
ted
to
ble
nd
ed t
each
ing
an
d
lear
nin
g.
Th
e in
stit
uti
on
en
sure
s fa
cult
y re
ceiv
e tr
ain
ing
, as
sist
ance
, an
d s
up
po
rt t
o p
rep
are
facu
lty
for
cou
rse
dev
elo
pm
ent
and
eff
ecti
ve t
each
ing
w
ith
tec
hn
olo
gy
in a
var
iety
of
mo
dal
itie
s.*
Fac
ult
y re
ceiv
e tr
ain
ing
an
d m
ater
ials
rel
ated
to
Fai
r U
se, p
lag
iari
sm, a
nd
oth
er r
elev
ant
leg
al
and
eth
ical
co
nce
pts
.*
Tec
hn
ical
ass
ista
nce
is p
rovi
ded
fo
r fa
cult
y b
efo
re a
nd
du
rin
g b
len
ded
co
urs
e d
evel
op
men
t an
d t
each
ing
.*
106 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
6N
o ev
iden
ce is
pro
vide
d w
hich
su
ppor
ts a
com
mitm
ent t
o on
goin
g pr
ofes
sion
al d
evel
opm
ent f
or fa
culty
m
embe
rs te
achi
ng b
lend
ed c
ours
es.
The
pro
gram
/inst
itutio
n pr
ovid
es
prof
essi
onal
dev
elop
men
t for
facu
lty
mem
bers
on
an a
d ho
c ba
sis.
The
pro
gram
/inst
itutio
n pr
ovid
es
ongo
ing
prof
essi
onal
dev
elop
men
t for
fa
culty
mem
bers
focu
sed
on b
lend
ed
teac
hing
and
lear
ning
.
The
pro
gram
/inst
itutio
n pr
ovid
es
ongo
ing
prof
essi
onal
dev
elop
men
t for
fa
culty
mem
bers
focu
sed
on b
lend
ed
teac
hing
and
lear
ning
; dev
elop
men
t op
port
uniti
es a
re p
rovi
ded
thro
ugh
vario
us d
eliv
ery
form
ats;
the
need
s of
fa
culty
mem
bers
are
con
side
red
whe
n de
velo
ping
a tr
aini
ng s
ched
ule
of to
pics
.
Fac
ult
y ar
e p
rovi
ded
tra
inin
g in
ble
nd
ed
teac
hin
g.
Blended Learning Scorecard Rubric 107
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
here
is n
o ev
iden
ce to
sup
port
st
uden
t's a
re a
dvis
ed a
bout
ble
nded
pr
ogra
m r
eadi
ness
A d
efin
ed a
dvis
ing
proc
ess
is in
pl
ace
and
embe
dded
into
bus
ines
s pr
oces
ses
befo
re a
lear
ner
star
ts
blen
ded
clas
ses
A d
efin
ed a
dvis
ing
proc
ess
is in
pl
ace
and
embe
dded
into
bus
ines
s pr
oces
ses
befo
re a
lear
ner
star
ts
blen
ded
clas
ses;
dev
elop
s pr
oces
ses
for
lear
ners
to s
elf-
asse
ss th
eir
mot
ivat
ion
and
com
mitm
ent
A d
efin
ed a
dvis
ing
proc
ess
is in
pl
ace
and
embe
dded
into
bus
ines
s pr
oces
ses
befo
re a
lear
ner
star
ts
blen
ded
clas
ses;
pro
cess
es a
re
deve
lope
d fo
r le
arne
rs to
sel
f-as
sess
m
otiv
atio
n an
d co
mm
itmen
t; tu
toria
ls
and
chec
klis
ts a
re p
rovi
ded;
'tes
t dr
ives
' or
othe
r de
cisi
on s
uppo
rt to
ols
are
offe
red.
2T
here
is n
o ev
iden
ce th
at s
tude
nt's
ar
e ad
vise
d ab
out m
inim
um
tech
nolo
gy s
kills
or
equi
pmen
t
Info
rmat
ion
avai
labl
e to
pro
spec
tive
stud
ents
is in
com
plet
e an
d/or
not
av
aila
ble
befo
re e
nrol
ling
Info
rmat
ion
avai
labl
e to
pro
spec
tive
stud
ents
is c
ompl
ete
but n
ot
avai
labl
e pr
ior
to e
nrol
ling
and
star
ting
a bl
ende
d le
arni
ng p
rogr
am
Pro
spec
tive
stud
ents
are
pro
vide
d pr
ogra
m in
form
atio
n so
they
can
m
ake
info
rmed
dec
isio
ns b
efor
e en
rolli
ng; n
eces
sary
info
rmat
ion
is
easy
to fi
nd o
n th
e w
eb s
ite.
3T
here
is n
o ev
iden
ce th
at s
tude
nts
rece
ive
impo
rtan
t inf
orm
atio
n be
fore
st
artin
g a
blen
ded
prog
ram
.
Info
rmat
ion
avai
labl
e to
pro
spec
tive
stud
ents
is in
com
plet
e an
d/or
not
av
aila
ble
befo
re e
nrol
ling.
Info
rmat
ion
avai
labl
e to
pro
spec
tive
stud
ents
is c
ompl
ete,
but
not
alw
ays
avai
labl
e pr
ior
to e
nrol
ling
and
star
ting
a bl
ende
d pr
ogra
m.
Pro
spec
tive
stud
ents
are
pro
vide
d pr
ogra
m in
form
atio
n so
they
can
m
ake
info
rmed
dec
isio
ns b
efor
e en
rolli
ng; n
eces
sary
info
rmat
ion
is
easy
to fi
nd o
n th
e w
ebsi
te.
4T
he li
brar
y ha
s no
tuto
rials
, web
re
sour
ces
or fo
rms
to s
uppo
rt
stud
ents
enr
olle
d in
ble
nded
cou
rses
or
pro
gram
s.
Som
e bl
ende
d co
urse
s in
clud
e in
form
atio
n on
how
to a
cces
s lib
rary
pr
ofes
sion
als
whe
ther
they
are
ge
nera
lists
or
spec
ialis
ts; l
imite
d tu
toria
ls, w
eb r
esou
rces
or
form
s ar
e m
ade
avai
labl
e on
line
for
stud
ents
in
blen
ded
cour
ses
or p
rogr
ams.
The
libr
ary
has
vario
us tu
toria
ls, w
eb
reso
urce
s or
form
s to
hel
p st
uden
ts
lear
n ho
w to
use
all
of th
e lib
rary
re
sour
ces
(e.g
., in
terli
brar
y lo
an,
elec
tron
ic d
ocum
ent d
eliv
ery,
e-
rese
rve
syst
em);
mos
t ble
nded
co
urse
s in
clud
e in
form
atio
n on
how
to
acc
ess
libra
ry p
rofe
ssio
nals
w
heth
er th
ey a
re g
ener
alis
ts o
r sp
ecia
lists
.
The
libr
ary
has
vario
us tu
toria
ls, w
eb
reso
urce
s or
form
s to
hel
p st
uden
ts
lear
n ho
w to
use
all
of th
e lib
rary
re
sour
ces
(e.g
., in
terli
brar
y lo
an,
elec
tron
ic d
ocum
ent d
eliv
ery,
e-
rese
rve
syst
em)
embe
dded
in e
ach
cour
se; a
ll bl
ende
d co
urse
s in
clud
e in
form
atio
n on
how
to a
cces
s lib
rary
pr
ofes
sion
als
whe
ther
they
are
ge
nera
lists
or
spec
ialis
ts.
5T
here
is n
o ev
iden
ce th
at s
tude
nts
have
acc
ess
to te
chni
cal a
ssis
tanc
e an
d su
ppor
t.
Stu
dent
s ha
ve a
cces
s to
tech
nica
l su
ppor
t, bu
t sup
port
is in
cons
iste
nt
and/
or
not p
rovi
ded
durin
g pe
ak
hour
s ba
sed
on th
e st
uden
t’s ti
me
zone
.
Stu
dent
s ha
ve a
cces
s to
con
sist
ent
tech
nica
l ass
ista
nce
and
supp
ort b
ut
inst
ruct
ions
for
acce
ssin
g su
ppor
t are
un
clea
r an
d ho
urs
are
limite
d bu
t are
pr
ovid
ed d
urin
g pe
ak h
ours
.
Stu
dent
s ha
ve a
cces
s to
con
sist
ent
tech
nica
l sup
port
and
hav
e cl
ear
inst
ruct
ions
on
how
to a
cces
s;
tech
nica
l sup
port
ser
vice
s ar
e pr
ovid
ed 2
4X7X
365.
6N
o ev
iden
ce is
pro
vide
d of
how
st
uden
ts m
ay a
ddre
ss th
eir
ques
tions
an
d is
sues
.
It is
unc
lear
how
all
type
s of
stu
dent
qu
estio
ns, p
robl
ems,
and
bug
re
port
ing
is h
andl
ed a
cros
s co
urse
s an
d pr
ogra
ms.
Cle
ar a
nd c
onsi
sten
t sup
port
is
avai
labl
e an
d bu
sine
ss p
roce
sses
su
rrou
ndin
g th
e fu
ll ra
nge
of s
tude
nt
issu
es (
e.g.
, que
stio
ns, p
robl
ems,
bu
g re
port
s, c
ompl
aint
s) is
do
cum
ente
d.
Cle
ar a
nd c
onsi
sten
t sup
port
is
avai
labl
e an
d bu
sine
ss p
roce
sses
su
rrou
ndin
g th
e fu
ll ra
nge
of s
tude
nt
issu
es (
e.g.
, que
stio
ns, p
robl
ems,
bu
g re
port
s, c
ompl
aint
s) is
do
cum
ente
d; s
uppo
rt s
taff
mon
itor
reem
ergi
ng is
sues
to e
nsur
e m
itiga
tion
of a
ny s
yste
mic
pro
blem
s.
Bef
ore
sta
rtin
g a
ble
nd
ed le
arn
ing
pro
gra
m,
stu
den
ts r
ecei
ve (
or
hav
e ac
cess
to
) in
form
atio
n a
bo
ut
pro
gra
ms,
incl
ud
ing
ad
mis
sio
n r
equ
irem
ents
, tu
itio
n a
nd
fee
s,
bo
oks
an
d s
up
plie
s, t
ech
nic
al a
nd
pro
cto
rin
g
req
uir
emen
ts, a
nd
stu
den
t su
pp
ort
ser
vice
s.*
Bef
ore
sta
rtin
g a
ble
nd
ed le
arn
ing
pro
gra
m,
stu
den
ts a
re a
dvi
sed
ab
ou
t th
e p
rog
ram
to
d
eter
min
e if
th
ey h
ave
acce
ss t
o t
he
min
imu
m
tech
no
log
y sk
ills
and
eq
uip
men
t re
qu
ired
by
the
cou
rse
des
ign
.*
Th
rou
gh
ou
t th
e d
ura
tio
n o
f th
e co
urs
e/p
rog
ram
, stu
den
ts h
ave
acce
ss t
o
trai
nin
g a
nd
info
rmat
ion
th
ey w
ill n
eed
to
se
cure
req
uir
ed m
ater
ials
th
rou
gh
ele
ctro
nic
d
atab
ases
, in
terl
ibra
ry lo
ans,
go
vern
men
t ar
chiv
es, n
ews
serv
ices
, an
d o
ther
so
urc
es.*
Th
rou
gh
ou
t th
e d
ura
tio
n o
f th
e co
urs
e/p
rog
ram
, stu
den
ts h
ave
acce
ss t
o
app
rop
riat
e te
chn
ical
ass
ista
nce
an
d t
ech
nic
al
sup
po
rt s
taff
.*
Su
pp
ort
per
son
nel
are
ava
ilab
le (
24/7
) to
ad
dre
ss s
tud
ent
qu
esti
on
s an
d p
rob
lem
s o
f a
tech
nic
al n
atu
re.*
Bef
ore
sta
rtin
g a
ble
nd
ed le
arn
ing
pro
gra
m,
stu
den
ts c
om
ple
te a
n o
rien
tati
on
or
self
- as
sess
men
t to
det
erm
ine
if t
hey
po
sses
s th
e se
lf-m
oti
vati
on
an
d c
om
mit
men
t to
lear
n.*
ST
UD
EN
T S
UP
PO
RT
(33
poi
nts)
108 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
7T
here
is n
o ev
iden
ce th
at s
tude
nts
with
dis
abili
ties
are
supp
orte
d.T
here
is e
vide
nce
of s
uppo
rt fo
r st
uden
ts w
ith d
isab
ilitie
s, b
ut p
olic
ies
and
proc
esse
s ar
e no
t evi
dent
.
The
pro
gram
/ ins
titut
ion
has
polic
ies,
pr
oces
ses
and
reso
urce
s to
sup
port
st
uden
ts w
ith d
isab
ilitie
s; h
owev
er
stud
ents
with
dis
abili
ties
rece
ive
inco
nsis
tent
sup
port
or
inst
ruct
ions
fo
r ac
cess
are
unc
lear
.
The
pro
gram
/inst
itutio
n ha
s co
nsis
tent
pol
icie
s, p
roce
sses
, and
re
sour
ces
to s
uppo
rt s
tude
nts
with
di
sabi
litie
s; it
is c
lear
that
the
inst
itutio
n is
wor
king
to in
corp
orat
e th
e re
leva
nt s
tand
ards
and
/or
best
pr
actic
es; d
ocum
enta
tion
incl
udes
sp
ecifi
c co
urse
info
rmat
ion
as w
ell a
s ho
w s
tude
nts
acce
ss r
elev
ant
serv
ices
.
8T
here
is n
o ev
iden
ce th
at s
tude
nts
have
acc
ess
to r
equi
red
mat
eria
ls
prio
r to
enr
ollm
ent.
Acc
ess
to r
equi
red
cour
se m
ater
ials
va
ries
by c
ours
e an
d is
inco
nsis
tent
.A
cces
s to
req
uire
d co
urse
mat
eria
ls
is c
onsi
sten
t acr
oss
the
prog
ram
, but
st
uden
ts m
ust b
e en
rolle
d fo
r ac
cess
.
Acc
ess
to r
equi
red
cour
se m
ater
ials
is
con
sist
ent a
nd s
tude
nts
have
ac
cess
prio
r to
enr
ollm
ent.
9T
here
is n
o ev
iden
ce o
f a s
tude
nt-
cent
ered
focu
s or
inte
ntio
nal
inte
grat
ion
of r
esou
rces
.
Stu
dent
-cen
tere
d su
ppor
t ser
vice
s ar
e ta
rget
ed p
rimar
ily fo
r on
-cam
pus
stud
ents
.
Stu
dent
-cen
tere
d su
ppor
t ser
vice
s ar
e av
aila
ble
for
stud
ents
in b
lend
ed
cour
ses
or p
rogr
ams,
but
evi
denc
e is
in
com
plet
e an
d/or
har
d to
loca
te.
Stu
dent
-cen
tere
d su
ppor
t ser
vice
s ar
e av
aila
ble
for
stud
ents
in b
lend
ed
cour
ses
or p
rogr
ams.
The
inst
itutio
n ac
tivel
y in
tegr
ates
sup
port
ser
vice
s ac
ross
the
cont
inuu
m o
f lea
rnin
g m
odes
, inc
ludi
ng b
lend
ed c
ours
es
and
prog
ram
s. S
ervi
ces
are
cont
inuo
usly
mon
itore
d an
d up
date
d to
ser
ve th
e sh
ift in
ble
nded
lear
ning
ap
proa
ches
.
10T
here
is n
o ev
iden
ce th
at s
tude
nts
are
prov
ided
with
gui
danc
e co
ncer
ning
use
of t
echn
olog
y.
The
re is
evi
denc
e th
at g
uida
nce
is
avai
labl
e fo
r st
uden
ts b
ut g
uida
nce
prov
ided
is c
onfu
sing
or
uncl
ear.
In
stru
ctio
ns fo
r ac
cess
may
be
need
ed.
Gui
danc
e is
pro
vide
d fo
r on
ly th
e pr
imar
y te
chno
logy
use
d fo
r co
urse
de
liver
y. In
stru
ctio
ns fo
r ac
cess
are
cl
ear.
Gui
danc
e, fa
ct s
heet
s, in
fogr
aphi
cs
are
prov
ided
for
all t
ypes
of
tech
nolo
gies
use
d in
cou
rsew
ork
and
may
incl
ude
vide
o tu
toria
ls.
Inst
ruct
ions
for
acce
ss a
re c
lear
and
co
nsis
tent
acr
oss
all c
ours
es
rega
rdle
ss o
f del
iver
y m
ode.
11T
here
is n
o ev
iden
ce th
at s
tude
nts
are
prov
ided
inst
ruct
ions
for
enlis
ting
help
.
Stu
dent
s ar
e pr
ovid
ed in
form
atio
n to
en
list h
elp
but s
ervi
ces
are
limite
d,
acce
ss is
unc
lear
and
/ or
min
imal
ch
anne
ls a
re a
vaila
ble.
Stu
dent
s ha
ve a
cces
s to
bot
h on
e-tim
e se
rvic
es a
nd r
epea
ted
serv
ices
, bu
t inf
orm
atio
n is
not
cle
ar a
nd/o
r ch
anne
ls a
re li
mite
d.
Stu
dent
s ha
ve c
lear
info
rmat
ion
in
orde
r to
acc
ess
both
one
-tim
e an
d re
peat
ed s
ervi
ces;
mul
tiple
cha
nnel
s (e
.g.,
e-m
ail,
phon
e, c
hat,
web
co
nfer
enci
ng)
are
prov
ided
for
enlis
ting
assi
stan
ce; s
ervi
ce
stan
dard
s ar
e m
onito
red
over
tim
e an
d im
prov
emen
ts m
ade.
Stu
den
ts h
ave
acce
ss t
o in
form
atio
n r
egar
din
g
req
uir
ed c
ou
rse
mat
eria
ls in
pri
nt
and
/or
dig
ital
fo
rmat
, su
ch a
s IS
BN
nu
mb
ers
for
text
bo
oks
, b
oo
k su
pp
liers
, an
d d
eliv
ery
mo
des
pri
or
to
cou
rse
enro
llmen
t.
Th
e in
stit
uti
on
pro
vid
es g
uid
ance
/tu
tori
als
for
stu
den
ts in
th
e u
se o
f al
l fo
rms
of
tech
no
log
ies
use
d f
or
cou
rse
del
iver
y.
Pro
gra
m d
emo
nst
rate
s a
stu
den
t-ce
nte
red
fo
cus
and
inte
nti
on
alit
y in
th
e in
teg
rati
on
of
on
line
and
fac
e-to
-fac
e re
sou
rces
.
Stu
den
ts a
re p
rovi
ded
cle
ar in
form
atio
n f
or
enlis
tin
g h
elp
fro
m t
he
inst
itu
tio
n.
Po
licy,
pro
cess
es, a
nd
res
ou
rces
are
in p
lace
to
su
pp
ort
stu
den
ts w
ith
dis
abili
ties
.
Blended Learning Scorecard Rubric 109
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
1T
he p
rogr
am/ i
nstit
utio
n do
es n
ot
have
a p
lan
or p
roce
ss to
eva
luat
e th
e bl
ende
d pr
ogra
m
The
pro
gram
/ ins
titut
ion
has
deve
lope
d a
prog
ram
eva
luat
ion
plan
to
per
iodi
cally
ass
ess
the
prog
ram
. M
ostly
per
form
ed o
n ad
hoc
bas
is.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d or
follo
ws
spec
ific
stan
dard
s w
hich
are
con
sist
ently
and
pe
riodi
cally
use
d to
ass
ess/
eval
uate
th
e pr
ogra
m.
The
pro
gram
/inst
itutio
n ha
s de
velo
ped
or fo
llow
s sp
ecifi
c st
anda
rds
whi
ch a
re c
onsi
sten
tly a
nd
perio
dica
lly u
sed
to a
sses
s/ev
alua
te
the
prog
ram
; eva
luat
ion
resu
lts a
re
used
to im
prov
e bl
ende
d pr
ogra
m(s
).
2N
o ev
iden
ce e
xist
s th
at th
e pr
ogra
m/
inst
itutio
n ev
alua
tes
its b
lend
ed
prog
ram
usi
ng a
var
iety
of d
ata.
Ble
nded
pro
gram
eva
luat
ion
is ju
st
begi
nnin
g an
d lim
ited
acad
emic
or
adm
inis
trat
ive
mea
sure
s ar
e us
ed to
as
sess
pro
gram
effe
ctiv
enes
s.
Ble
nded
pro
gram
eva
luat
ion
incl
udes
so
me
acad
emic
(e.
g., c
ours
e ev
alua
tions
, lea
rnin
g ou
tcom
es
achi
evem
ent)
and
adm
inis
trat
ive
(e.g
., sa
tisfa
ctio
n su
rvey
s, s
tude
nt
succ
ess/
per
sist
ence
rat
es)
mea
sure
s to
ass
ess
effe
ctiv
enes
s.
Ble
nded
pro
gram
eva
luat
ion
incl
udes
a
wid
e va
riety
of a
cade
mic
(e.
g.,
cour
se e
valu
atio
ns, l
earn
ing
outc
omes
ach
ieve
men
t) a
nd
adm
inis
trat
ive
(e.g
., sa
tisfa
ctio
n su
rvey
s, s
tude
nt s
ucce
ss/
pers
iste
nce
rate
s) m
easu
res
to
asse
ss e
ffect
iven
ess;
the
eval
uatio
n oc
curs
reg
ular
ly a
nd fr
eque
ntly
; and
, re
sults
are
use
d to
gui
de c
hang
es.
3T
he p
rogr
am/in
stitu
tion
does
not
ha
ve a
rev
iew
pro
cess
to a
sses
s le
arni
ng o
utco
mes
.
Lear
ning
out
com
es a
re a
sses
sed
on
an a
d ho
c ba
sis
to e
nsur
e al
ignm
ent,
clar
ity, u
tility
, app
ropr
iate
ness
and
ef
fect
iven
ess.
Lear
ning
out
com
es a
re a
sses
sed
on
a re
gula
r ba
sis
to e
nsur
e al
ignm
ent,
clar
ity, u
tility
, app
ropr
iate
ness
and
ef
fect
iven
ess.
Lear
ning
out
com
es a
re a
sses
sed
on
a re
gula
r ba
sis
to e
nsur
e al
ignm
ent,
clar
ity, u
tility
, app
ropr
iate
ness
and
ef
fect
iven
ess;
a p
eer
revi
ew p
roce
ss
is u
sed;
out
com
es fr
om th
e pr
oces
s dr
ive
upda
tes.
4N
o pr
oces
s is
in p
lace
to a
sses
s su
ppor
t ser
vice
s fo
r fa
culty
and
st
uden
ts.
A p
roce
ss is
in p
lace
and
eng
aged
on
an
ad h
oc b
asis
to a
sses
s su
ppor
t se
rvic
es fo
r fa
culty
and
stu
dent
s.
A p
roce
ss is
in p
lace
and
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to
asse
ss s
uppo
rt s
ervi
ces
for
facu
lty
and
stud
ents
.
A p
roce
ss is
in p
lace
and
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to
asse
ss s
uppo
rt s
ervi
ces
for
facu
lty
and
stud
ents
; out
com
es s
erve
as
a fo
unda
tion
for
impr
ovem
ents
.
5N
o pr
oces
s is
in p
lace
to a
sses
s st
uden
t ret
entio
n.A
pro
cess
is in
pla
ce a
nd e
ngag
ed
on a
n ad
hoc
bas
is to
stu
dent
re
tent
ion.
A p
roce
ss is
in p
lace
and
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to
asse
ss s
tude
nt r
eten
tion.
A p
roce
ss is
in p
lace
and
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to
asse
ss s
tude
nt r
eten
tion;
out
com
es
serv
e as
a fo
unda
tion
for
impr
ovem
ents
.
6N
o pr
oces
s is
in p
lace
to r
egul
arly
re
view
acc
essi
bilit
y st
anda
rds/
Pro
gram
/ ins
titut
ion
has
desi
gnat
ed
pers
onne
l to
supp
ort a
cces
sibi
lity
need
s; a
pro
cess
is in
pla
ce a
nd
follo
wed
on
an a
d ho
c ba
sis
to
asse
ss a
cces
sibi
lity
stan
dard
s.
Pro
gram
/ ins
titut
ion
has
desi
gnat
ed
pers
onne
l to
supp
ort a
cces
sibi
lity
need
s; a
pro
cess
is in
pla
ce a
nd
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to a
sses
s ac
cess
ibili
ty
stan
dard
s.
Pro
gram
/ ins
titut
ion
has
desi
gnat
ed
pers
onne
l to
supp
ort a
cces
sibi
lity
need
s; a
pro
cess
is in
pla
ce a
nd
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to a
sses
s ac
cess
ibili
ty
stan
dard
s w
ith o
utco
mes
pub
lishe
d on
the
prog
ram
's w
ebsi
te; o
utco
mes
al
so s
erve
as
a fo
unda
tion
for
impr
ovem
ent.
Pro
gra
m d
emo
nst
rate
s co
mp
lian
ce a
nd
rev
iew
o
f ac
cess
ibili
ty s
tan
dar
ds
(Sec
tio
n 5
08, e
tc.)
.
Th
e p
rog
ram
is a
sses
sed
th
rou
gh
an
ev
alu
atio
n p
roce
ss t
hat
ap
plie
s sp
ecif
ic
esta
blis
hed
sta
nd
ard
s (i
.e.,
accr
edit
or
gu
idel
ines
an
d/o
r o
ther
rec
og
niz
ed a
gen
cy
such
as
the
OL
C S
core
card
).*
A p
roce
ss is
in p
lace
an
d f
ollo
wed
fo
r th
e co
mp
reh
ensi
ve a
sses
smen
t o
f su
pp
ort
se
rvic
es f
or
facu
lty
and
stu
den
ts.
A p
roce
ss is
in p
lace
an
d f
ollo
wed
fo
r th
e as
sess
men
t o
f st
ud
ent
rete
nti
on
in b
len
ded
co
urs
es a
nd
pro
gra
ms.
A v
arie
ty o
f d
ata
(aca
dem
ic a
nd
ad
min
istr
ativ
e in
form
atio
n)
are
use
d t
o r
egu
larl
y an
d
freq
uen
tly
eval
uat
e p
rog
ram
eff
ecti
ven
ess
in
ord
er t
o g
uid
e ch
ang
es t
ow
ard
co
nti
nu
al
imp
rove
men
t.*
Inte
nd
ed le
arn
ing
ou
tco
mes
at
the
cou
rse
and
p
rog
ram
leve
l are
rev
iew
ed r
egu
larl
y to
en
sure
al
ign
men
t, c
lari
ty, u
tilit
y, a
pp
rop
riat
enes
s, a
nd
ef
fect
iven
ess.
*
EV
AL
UA
TIO
N A
ND
AS
SE
SS
ME
NT
(30
poi
nts)
110 Quality Scorecard for Blended Learning Programs
0=D
efic
ient
1=D
evel
opin
g2=
Acc
ompl
ishe
d3=
Exe
mpl
ary
7C
ours
e ev
alua
tions
are
not
reg
ular
ly
revi
ewed
.C
ours
e ev
alua
tions
and
oth
er
feed
back
(e.
g., p
eer
revi
ew r
esul
ts)
are
revi
ewed
on
an a
d ho
c ba
sis
to
asse
ss e
ffect
iven
ess
of in
stru
ctio
n;
feed
back
gle
aned
from
the
revi
ew is
us
ed to
sha
pe fa
culty
per
form
ance
ev
alua
tions
.
Cou
rse
eval
uatio
ns a
nd o
ther
fe
edba
ck (
e.g.
, pee
r re
view
res
ults
) ar
e re
view
ed r
egul
arly
to a
sses
s ef
fect
iven
ess
of in
stru
ctio
n; fe
edba
ck
glea
ned
from
the
revi
ew is
use
d to
sh
ape
facu
lty p
erfo
rman
ce
eval
uatio
ns.
Cou
rse
eval
uatio
ns a
nd o
ther
fe
edba
ck (
e.g.
, pee
r re
view
res
ults
) ar
e re
view
ed r
egul
arly
to a
sses
s ef
fect
iven
ess
of in
stru
ctio
n; fe
edba
ck
glea
ned
from
the
revi
ew is
use
d to
sh
ape
facu
lty p
erfo
rman
ce
eval
uatio
ns; a
lso
used
as
outli
ne fo
r fu
ture
facu
lty d
evel
opm
ent t
opic
s or
ot
her
impr
ovem
ents
.
8C
ours
e ev
alua
tions
are
not
col
lect
ed
and
used
to a
sses
s th
e qu
ality
of
blen
ded
cour
se m
ater
ials
.
A c
ompr
ehen
sive
cou
rse
eval
uatio
n is
dep
loye
d pe
riodi
cally
to c
olle
ct
stud
ent f
eedb
ack;
out
com
es s
erve
as
a fo
unda
tion
for
impr
ovem
ents
.
A c
ompr
ehen
sive
cou
rse
eval
uatio
n is
dep
loye
d fo
r ea
ch b
lend
ed c
ours
e to
col
lect
stu
dent
feed
back
; ou
tcom
es s
erve
as
a fo
unda
tion
for
impr
ovem
ents
.
A c
ompr
ehen
sive
cou
rse
eval
uatio
n is
dep
loye
d fo
r ea
ch b
lend
ed c
ours
e to
col
lect
stu
dent
feed
back
; ou
tcom
es s
erve
as
a fo
unda
tion
for
impr
ovem
ents
to c
ours
e qu
ality
; the
ev
alua
tion
tool
is p
erio
dica
lly
revi
ewed
for
effe
ctiv
enes
s an
d re
leva
nce.
9N
o pr
ogra
m/ i
nstit
utio
nal p
olic
ies
have
bee
n de
velo
ped
to g
uide
ex
pect
atio
ns fo
r fa
culty
per
form
ance
.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d po
licie
s to
gui
de
expe
ctat
ions
for
facu
lty p
erfo
rman
ce
and
the
polic
y/ s
tand
ards
are
sha
red
with
facu
lty m
embe
rs.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d po
licie
s to
gui
de
expe
ctat
ions
for
facu
lty p
erfo
rman
ce;
polic
y/ s
tand
ards
are
sha
red
with
fa
culty
; sta
ndar
ds a
re u
sed
perio
dica
lly to
eva
luat
e a
facu
lty
mem
ber's
per
form
ance
.
The
pro
gram
/ ins
titut
ion
has
deve
lope
d po
licie
s to
gui
de
expe
ctat
ions
for
facu
lty p
erfo
rman
ce;
polic
y/ s
tand
ards
are
sha
red
with
fa
culty
; the
se s
tand
ards
are
use
d to
ev
alua
te a
facu
lty m
embe
r's
perf
orm
ance
eac
h tim
e th
ey te
ach
in
a bl
ende
d m
odal
ity.
10N
o pr
oces
s is
in p
lace
to a
sses
s st
akeh
olde
r sa
tisfa
ctio
n.A
pro
cess
is in
pla
ce a
nd fo
llow
ed o
n an
ad
hoc
basi
s to
ass
ess
stak
ehol
der
satis
fact
ion.
A p
roce
ss is
in p
lace
and
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to
asse
ss s
take
hold
er s
atis
fact
ion.
A p
roce
ss is
in p
lace
and
follo
wed
on
a re
gula
r ba
sis
(e.g
., an
nual
ly)
to
asse
ss s
take
hold
er s
atis
fact
ion;
ou
tcom
es s
erve
as
a fo
unda
tion
for
impr
ovem
ents
.
Co
urs
e ev
alu
atio
ns
colle
ct s
tud
ent
feed
bac
k o
n
qu
alit
y o
f b
len
ded
co
urs
e m
ater
ials
.
A p
roce
ss is
in p
lace
an
d f
ollo
wed
fo
r th
e in
stit
uti
on
al a
sses
smen
t o
f fa
cult
y b
len
ded
te
ach
ing
per
form
ance
.
A p
roce
ss is
in p
lace
an
d f
ollo
wed
fo
r th
e as
sess
men
t o
f st
akeh
old
er (
e.g
., le
arn
ers,
fa
cult
y, s
taff
) sa
tisf
acti
on
wit
h t
he
ble
nd
ed
lear
nin
g p
rog
ram
s.
Co
urs
e ev
alu
atio
ns
colle
ct s
tud
ent
feed
bac
k o
n
the
effe
ctiv
enes
s o
f in
stru
ctio
n in
rel
atio
n t
o
facu
lty
per
form
ance
eva
luat
ion
s.
* A
dapt
ed fr
om In
stitu
te fo
r H
ighe
r E
duca
tion
Pol
icy'
s Q
ualit
y on
the
Line
: Ben
chm
ark
for S
ucce
ss in
Inte
rnet
-bas
ed D
ista
nce
Edu
catio
n (2
000)
.
Blended Learning Scorecard Rubric 111