How do you get there from here?
Overcoming Barriers in Implementing a Quality
Assurance Process
WCET 21st Annual ConferenceDeb Adair, Quality Matters
Evelyn Everett Knowles, Park University
Li Wang, University of Northern Colorado
Heidi Ashbaugh, Texas Woman’s University
10/23/2009Denver, Colorado
Presentation Agenda•QM Program Introduction
•Deb Adair
Challenges/Solutions in Implementation:
Course Development & Review Process
•Evelyn KnowlesFaculty Development Training
Li Wang A Multi-Year Plan for Implementation
Heidi Ashbaugh
Sharing Your Questions, Issues and Solutions
Deb Adair, Ph.D.Director of Quality MattersAdjunct Associate Professor, University of Maryland University College
What is Quality Matters?
“Quality Matters: Inter-Institutional Quality Assurance in
Online Learning” Quality Matters is a not-for-profit
subscription service providing tools and training for quality assurance of online courses
Initially developed by MarylandOnline with funding from FIPSE, it was designed by faculty for faculty
Adopted by a large and broad user base, QM represents a shared understanding of quality in online course design
A set of standards (rubric) for the design of online and hybrid courses
A peer review process (faculty to faculty) for reviewing and improving online and hybrid courses
A faculty support tool used by instructional development staff
A professional development opportunity
Quality Matters is …
QM as a National Standard
370 + current subscribers(http://www.qualitymatters.org/Documents/Subscriber%20List%20for%20Publication.pdf
)
41 states represented QM has trained 5000+ faculty and
instructional design staff Recognized by
- Sloan C Excellence in Online Teaching and Learning Award
-2008 USDLA Outstanding Leadership in the field of Distance Learning
MONTANA
WYOMING
IDAHO
WASHINGTON
OREGON
NEVADA
UTAH
CALIFORNIA
ARIZONA
NORTH DAKOTA
SOUTH DAKOTA
NEBRASKA
COLORADO
NEW MEXICO
TEXAS
OKLAHOMA
KANSAS
ARKANSAS
LOUISIANA
MISSOURI
IOWA
MINNESOTA
WISCONSIN
ILLINOISINDIANA
KENTUCKY
TENNESSEE
MISS
ALABAMA GEORGIA
FLORIDA
SOUTHCAROLINA
NORTH CAROLINA
VIRGINIA
WV
OHIO
MICHIGANNEW YORK
PENN
MARYLAND
DELAWARE
NEWJERSEY
CONNRI
MASS
MAINE
VTNH
ALASKA
HAWAII
PUERTO RICO
VIRGIN ISLANDS
Current Subscribers
Statewide Subscribers
Quality Matters Subscribers
The QM Approach
The QM toolset and process A faculty-driven, peer review process
that is…CollaborativeCollegialContinuous Centered - in academic foundation - around student learning
A shared goal: Quality online courses
Improved courses Engaged faculty Reduced course
development time Ongoing faculty
development Quality
benchmarking Awareness &
support for online learning
Institutional improvement
85 %
A course development process leading to QM-recognized courses
Professional development (training) for faculty and staff
Institutional self-sufficiency in conducting official QM peer course reviews
Getting there with QM can include…
Evelyn Everett Knowles, Ph. D.Coordinator of Program Quality DevelopmentPark University
Course Development Challenges
A Model for Enhancing Online Course Development :The Park University Continuous Quality Development Model
Continuous Quality Continuous Quality Development ModelDevelopment Model
Rapid Growth in Online Learning
Rapid Growth in Online Learning
Culture of Assessment
Culture of Assessment
Platform Change / Content Manager
Platform Change / Content Manager
Continuous Quality Development Model
Continuous Quality Development Model
Collaboration between Faculty and Instructional Designer
Collaboration between Faculty and Instructional Designer
Quality Matters Quality Matters
Development Contract Course Developer Requirements Payment for Development New Online Course Development Initiate Course Development
Maintenance ContractCourse Maintenance Requirements Payment for Maintenance
Online Course Development Policies
Limits on the Timing and Number of Courses Developments per Developer
Policies Directly Impacting Online Course Development
Accessibility and ADA standards and requirements
Academic FreedomFull-Time Faculty Contract
Online Adjunct Faculty Contract Faculty Course Load
Faculty Course Preparation Faculty Evaluation
Student Academic Issues Online Attendance and GradesProctoring for Online Programs
Online Course Student Training
Academic Policies
Operational PoliciesInformation Security
Bookstore and LibraryInstitutional Policies
Continuous Quality Continuous Quality Development ModelDevelopment Model
Literature Research
Research Standards
Best Practices
Course DevelopmentQuality Matters
StandardsQuality Matters
Standards
Online Course Development
CD Requirements & Checklist
Quality Matters Rubric
Quality Matters Rubric
Park Online Quality Standards Review QSR
University and CDL Policies
Academic Department
Requirements
Park University Requirements
Park University Requirements
Content ApprovalContent Approval
eCollege capabilities
Best Practices
Common Course Structure Common Course Structure ::
– Where things are located – What content to develop– Accountability
Checklist mapped to :Checklist mapped to :– Quality Matters standards– University requirements– Best Practices– OIES
Quality Review Quality Standards Review
– Quality Matters RubricQuality Matters Rubric•www.qualitymatters.org•Report returned to Instructional Designer•Recommendations and changes are made before the courses are duplicated for the next term
– Academic Department ReviewAcademic Department Review•Required changes are made by the instructional designer with the Course Developer
Recognized Courses
2009 Quality Matters Recognized Courses
The following courses were recently recognized by Quality Matters in 2009.CourseTitleFaculty DeveloperInstructional Designer AC435Ethics for AccountantsJolene LamptonChris Slupianek BI210 The Human Body Carol Sanders Jim Whelan CJ332 Institutional, Industrial and Commercial Security Joe Angelo
Jackie Artmayer,David Curtis EN234Intro to FictionJane WoodJackie Artmayer FI363Financial Institutions and MarketsJoe KubecCasey Johnson GO205Intro to MeteorologyLeAnn AllisonChris SlupianekPS424Industrial and Organizational PsychologyWilliam NastJim Whelan
2008 Quality Matters Recognized Courses
The following courses were recently recognized by Quality Matters in 2008.CourseTitleFaculty DeveloperInstructional DesignerCJ232Introduction to CorrectionsMichael Eskey CJ400 Constitutional Law in Criminal JusticeSamantha BurkeJackie Artmayer CJ430 Criminal Justice Research MethodsMike EskeyJim Whelan CS322Web Programming IIKathleen Kalata EC300 Intermediate MicroeconomicsGuerman KornilovTeri Wright EC301Intermediate MacroeconomicsGuerman Kornilov Teri Wright EN201 Introduction to Literature Jane WoodJackie Artmayer EN221 Afro-American Literature Cynthia WilliamsJackie Artmayer EN306A Scientific and Technical Writing Dees Stallings EN306B Business Communications Dees Stallings EN306CProfessional Writing in the Disciplines Emily Donnelli HR434 Compensation Management Gina HellerKristy Sailors IS205 Managing Information Systems Frank UzinskyKathleen Kalata LG201 Systems Engineering and Analysis Ernest ReidJackie Artmayer LG415 Quality Control Scott SowderJackie Artmayer MG261 Business Law II Cathy TaylorDavid Curtis MG401 Senior Seminar in Business Paul FordKristy Sailors MG495 Business Policy Dennis HermerdingJim Whelan PS221Adolescent PsychologyTeresa MasonKristy Sailors PS361 Cross Cultural Psychology Dennis KerkmanJim Whelan PS410 Social Influence and Persuasion Jean MandernachJim Whelan SO208 Social Inequalities David Curtis SO210 Social Institutions Pat McGintyTeri Wright SO306Writing for Social SciencesJean MandernachJim Whelan
2007 Quality Matters Recognized Courses
The following courses were recognized by Quality Matters in 2007. CourseTitleFaculty DeveloperInstructional Designer BI101Biological ConceptsCarol SandersJim Whelan CS321
Web Programming 1Katie KalataPS315 Theories of PersonalityJean Mandernach SO141 Introduction to SociologyDavid Curtis
Li Wang, Ph.D.Instructional DesignerCenter for the Enhancement of Teaching and LearningUniversity of Northern Colorado
Faculty Training Challenges
ImplementingImplementing
IntroducingIntroducing
PlanningPlanning
GoalsGoals
CETLCETL OESOES
FutureFuture UltimateUltimate
facultyfaculty
Center for the Enhancement of Teaching & Learning (CETL)
Models Conversations Funds
Office of Extended Studies (OES)
Implementing
Introducing
PlanningPlanning
Goals
Needs Assessment (meetings in fall 2008)
Faculty Training (December 2008: onsite and three-day training)
QM project key staff training and training customization (ongoing)
Offering Workshops: group and one on one
Implementing
IntroducingIntroducing
Planning
Goals
ImplementingImplementing
Introducing
Planning
Goals
CETLCETL OESOESfacultyfaculty
Next : CETL
Implementing: CETL
Goals: 4 C Principles
Incentives: Travel award, training fees, & recognition
Timeline: 1. Training (Spring 09) 2. Self review (summer 09)
3. Peer review (fall 09)
ImplementingImplementing CETLCETL OESfaculty
Next : CETL Cont.
Implementing: CETL Cont.
Focus: Improve the quality of course design
Collaboration: Support, Share ideas across disciplines
Important Considerations: Time, Organization, collaboration
ImplementingImplementing CETLCETL OESfaculty
Next : FacultyBack: CETL
ImplementingImplementing CETL OESfacultyfaculty
Next : OES
Cohort Spring 2009 Non Cohort Summer 2009 Individuals
Comments:
“ This workshop was highly beneficial in the following ways: encourages quality in online teaching, fosters faculty collaboration, gives support for faculty”
Back: CETL Cont.
• Spanish• Philosophy• Nursing• Criminal Justice• Special Education• Sociology• Library• Bilingual ESL• Educational Technology.
ImplementingImplementing CETL OESOESfaculty
Next : Main Presentation
OES: New Programs and Courses
Goals: Quality improvement, teaching, collaboration All new programs developed using QM
Incentives: RFP - $1000 to $2000 stipend for faculty developing new courses with QM
Progress: 3 new programs, 4 major revisions2 sets professional development modules
1 program - QM Program Level Certification
New: MA Criminal Justice, MA Spec. Ed. Gifted/Talented, Interventionist Spec. Ed. Professional Development Series, ASL Interpreting Series
Revised: MS and Ph.D. in Nursing Ed, RN to BSN, Nursing Ed. Cert
Back: Faculty Cont.
Implementing
Introducing
Planning
GoalsGoals FutureFuture UltimateUltimate
Next : Future
GoalsGoals FutureFuture Ultimate
Future Goals
CETL: Cohort and Non Cohort Conducting Peer Review Recruiting participants
OES: Continuing use of RFP Courses informally reviewed prior to delivery One certified faculty in each participating program
Others: Continuing collaboration between CETL & OES Continuing support for QM related scholarship of teaching and
learning Involving more faculty as non cohort (offering workshops) Conducting research
Next : Ultimate
GoalsGoals Future UltimateUltimate
Ultimate Goals
QM certified Representatives from each Program/department/college
Ongoing Official Course Reviews Ongoing Official Program Reviews Research dissemination
4 C Principles
Back: Future
End
1. Awareness2. Time3. Incentives4. Retention5. Collaboration
Heidi AshbaughSenior Instructional Design SpecialistTexas Woman’s University
Multi-Year Plan Challenges
Implementing the QM Process at Texas Woman’s University
Presented by - Heidi AshbaughSr. Instructional Design Specialist
Texas Woman’s Universitywith much input from - Allison Peterson
Sr. Instructional Design SpecialistTexas Woman’s University
Implementation
• Phase 1 (Started Spring 2007)– Goal: To train 70 faculty as QM Peer Reviewers– Completed on schedule (2 ½ years)
• Phase II (In process)– Goal: To initiate a Peer Review process at TWU– Approximately 3 years
• First 2 years focus on set up and trial of an internal process
• Third year goal is to be able to manage the formal QM process internally
Concerns
• Misunderstandings by faculty about instructors being “required” to participate– We have never considered making this a
“required” process.– Both phases of the project are voluntary.
• Concerns by faculty about “requiring” courses to meet QM standards– We do not have any control or reason to “require”
that courses meet QM.
Governance & Resources• Currently Distance Education/Lifelong Learning
are the governing authorities for QM– This is more managing the logistics of the program
than really “governing”.• Funding
– Fortunately DE has a budget that supports our continuation and expansion of this project.
– Faculty participation in QM training is considered faculty development.
– Some of our annual budget is dedicated to training and a limited number of national reviews.
Instructional Design Concerns
• Time commitments - QM is not a full time job for anyone– The IR has to devote time from an already busy
schedule to manage training and reviews on both the national and campus level.
– The Instructional Designers have to make time for multiple informal reviews.
– Our Administrative Assistant must make time to organize finances for the annual subscription, as well as for purchasing materials, additional trainings, and official reviews.
Current Status • Working to move forward and introduce a Peer Review
process at TWU– We have enough faculty who have been through peer
reviewer training to do this.• Continuing concerns with having the program getting
too big too fast– Time constraints are the main concern here.
• Looking at new challenges - considering putting up some complete programs for national review– There are many details that need to be addressed before
this becomes a reality.– Time constraints will be a concern with this, as well.
The TWU Team
Keith Restine, Associate Director of Distance Education
Alli Peterson, Senior Instructional Design Specialist, Denton & QM Institutional Representative
Valerie Shapko, Senior Instructional Design Specialist, Houston
Heidi Ashbaugh, Senior Instructional Design Specialist, Dallas
Tracey Mac Gowan, Instructional Design Specialist
Jake McBee, Instructional Design Specialist
What are your quality assurance initiatives?What challenges do you face?How are you addressing them?
What is are your Challenges and Solutions in QA Implementation?