When Upgrades go Bad(Terribly, Horribly Wrong)

Jean Mankoff, M.S. Director, Instructional Technology Delivery Systems

Vicky Johnson Instructional Operations Systems Administrator Certified Blackboard System Administrator

Catherine Dutton, Ph.D. Manager, Instructional Technology Software Applications

Texas Woman's University

• Approximately 14,000 students• 3 campuses • Denton (Main Campus)• Dallas• Houston

• Non- traditional student enrollment

Demographics

Blackboard at TWU• 2,500 live courses a semester• Current version 9.1 – Service Pack 5• Started with Course Info 1 in 1999• Vendor Building Blocks• Wimba (Collaborate)• Turnitin• Learning Objects• McGraw Hill• Tegrity

Overview• Why We Made the Move• Project Plan for the Upgrade• The Big Delay• Communication with Faculty and Support Staff• Working with Blackboard to prepare for Upgrade• Upgrading to 9.0• The Decision to Move to 9.1• Upgrade Results• Consequences and Repercussions• The Clean-up• Current Situation• Future Upgrades

Upgrade Project Plan• March ‘09 Started Project Plan for Spring ’10

Upgrade• April ‘09 Managed Hosting Notification• Support on 7.3 would end in October• All managed hosting clients needed to upgrade as

soon as possible• Moved upgrade date to August ’09• May ‘09 Upgraded Staging Server to 9.0

Upgrade Project Plan (cont…)• June ‘09 Through listserv information and local

testing, postponed upgrade to December ’09• Informed upgrade must occur by December since

support ended in October• Training for 9 would occur during Fall Semester on

staging server• December• Bring Production Down After Finals• Upgrade on Current Production Server

• Go Live in January ‘10 with 9

The Big Delay• So Many Bugs – So Little Time

• Testing Showed Platform Very Unstable• Listserv Ripe With Issues

• Trip to DC in October – My 3 Demands• Would Support Current Version Until Dec. ’10• Would Provide New 9.0 Production Server

– Run Concurrently with 7.3 Server– No Cost to University

• Would Maintain our 7.3 Server Running Concurrently until November ’10

Communication with Faculty & Support Staff

• Campus Wide Announcement of Delay• Met with Every Department on Campus During Scheduled

Fall Faculty Meetings• Shared Delay Reasons• Introduced Upgrade Campaign – Change the Control

Panel – Change the World• Provided Schedule of Training

Working with Blackboard to Prepare for Upgrade

• Consultations• CHM• Managed Hosting• Global Services (Snapshots)

• Testing on staging• Testing on new production server

Upgrade to 9.0• Began live training in March• All courses moved from 7.3 server to new server• Began Snapshots April 1 to create shells for

Summer and Fall• All Tips of the Week centered on upgrade• Attended faculty meetings to conduct training• Went live for summer sessions on 9.0• Monitored listservs for current issues

Decision to Move to 9.1• Although 9.0 seemed fairly stable - issues reported

appeared fixed in 9.1• Initial results showed it to be more stable• Recommendations from Managed Hosting • Major concerns of stability on 9.0 with more load• Limited testing time line for 9.1

Upgrade Results• Major issues with downtime• Bugs we didn’t see on Staging, appeared on Production• Course copy didn’t work• Banners gone• System slowdowns• Changes to course files from 9.0 to 9.1 had a significant

impact on course copy

Consequences & Repercussions• Faculty revolt – demanded to go back to 7.3• Faculty removed content and used other

resources• MoodleRooms• Google Docs• PBWiki

• Faculty meetings turned into heated accusations

• Credibility questioned• Demands of no more upgrades• Demands to look at other LMS options

The Clean-up• Increased training options• Continued faculty meetings• Applied as many patches as possible• Helped faculty with work-arounds

Current Situation &Future of Upgrades

• Stayed on 9.1 Service Pack 1 until August ‘11• Upgraded to Service Pack 5 August ‘11• Evaluating Service Pack 7• Will upgrade once a year on odd Service Packs• Will stay at least 2 Service Packs behind• Will apply patches as needed

Conclusion• Shared responsibility – Blackboard and our campus• No matter how well you think you planned…• Consider staying 2 Service Packs behind• No matter how it feels – It’s not personal!!!