Change Management: Planning for the changes ahead, a University of Florida perspective
Narasi Ramachandran, Learning Support ServicesMartin Smith, Computing and Networking Services
Mike Stoufer, Instructional Computing Services
12th Sakai Conference – Los Angeles, California – June 14-16
Overview
• Adopted Sakai in 2010 after review by a faculty review committee
• Pilot was run in Summer 2010 with a select few courses
• Fall 2010 went live with 3100 sections and 39,000 students
• Adoption grew to 4100 section in Spring 2011• Old CMS system was retired in May 2011!!!
Sakai Deployment overview
• 11 roles configured for UF clients (Student, CE Student, Auditor, Observer,Instructor, Guest Lecturer, Facilitator, Designer, TA, Non Grading TA, Grader
• Course and project sites deployed with slight variations in roles and purposes
• Course sites setup by LSS and enrollment managed by ICS programmed CM Synchronizer
• Project sites open for all any valid UF user can create his/her site
Input Mechanisms
• Gap analysis by CMS review committee• Features and bugs reported by clients through LSS• CMS Governance committee for features recommended by
faculty• Infrastructure group for performance and security
recommendations• ICS bridges gap between all groups
12th Sakai Conference – Los Angeles, California – June 14-16
The Players
• Three main groups that support various parts of Sakai at UF:o Instructional Computing Services (Developers) o Learning Support Services (Training, Q/A & User
Support) o Open Systems Group (Infrastructure)
Technology resources (platforms, tools, etc)
o Oracle databases, Application servers, Load balancer o CFEngine, Autogenerated configuration files (Apache,
sakai.properties, security.properties, etc)
The Players
e-Learning Support Services
• Supports front end clients faculty and students with several e-learning technologies
• Provides feedback and input for developers and infrastructure unit on bugs, performance etc
• Q/A on new builds at various stages• Request new features and report bugs• Communicate with several stakeholders on changes
The PlayersInstructional Computing Services (ICS)
Developers • Tool Installation/Upgrade/Patching • Feature Programming • Integration with UFL SIS• Integration with HR and Continuing Education Registration• Bug/Issue Resolution • Build Creation and Scheduling
The Players
Open Systems Group (OSG)Hardware/Software Infrastructure
• Clustered Oracle support (configuration, support, DBA)• Hardware Support (Storage tier, OS tier, tomcat tier)• Build Promotion (SQL scripts, Sakai binary builds)• 24/7 monitoring and support (system-wide to user)• Resource management and growth (utilization of each tier)
12th Sakai Conference – Los Angeles, California – June 14-16
Change management processAnd now here’s a…
Open Systems (Infrastructure group)
• Promotes build to a test cluster (identical to prod)o Does not address software updates below Sakai o Could involve updates to automated configuration
management (new properties, changes to httpd or tomcat configuration)
o Could involve SQL scripts that we hope are repeatable!o Often involves scouring logs for any differences
• Performs basic functional testing such as:
o Can I login using Shibboleth (SSO)?o Can I still see Sakai sites?
Open Systems (Infrastructure group)
• Performs load testing based on LSS scriptso Up to 2,500 concurrent userso Tasks like tests/quizzes, submit an assignment, run a site
stats report, and more...o Looks for responses per second, as well as functional
success or failureo Examines tomcat load, thread counts, AJP
communication, connection counts o This approach is still limiting...
• Participates in the "Go? No go?" decision
'Trac'king Build ProgressTrac, an Open Source Project (http://trac.edgewall.org)
• Web-Based Software Project Management• Interfaces to Subversion or Git• Provides a Method of Communication Between Groups• Allows Developers to Link a Subversion Commit to a Ticket
'Trac'king Build Progress
Wiki: Developer Schedule and 'How To' Information
'Trac'king Build Progress
Timeline: Follow Changes in Trac (Subversion, Tickets, etc.)
'Trac'king Build Progress
Roadmap: Provides Graphical View of Tickets Status
'Trac'king Build Progress
Browse Repository: Easy Way to View Change History and See Diffs in Code.
'Trac'king Build Progress
View Tickets: Easily Configurable Way to View Tickets by Various Criteria
'Trac'king Build Progress
Build Flow
Promotion ScheduleFeature Freeze (approx. 6 1/2 weeks prior to promotion) Last opportunity for LSS to request a tool or feature to be added to the upcoming build.
Code Freeze (approx. 4 1/2 weeks prior to promotion)No new code may be added after this date. Existing code may be modified to fix issues found during Q/A on dev instance.
Move to Test Instance (approx. 2 1/2 weeks prior to promotion)Full Q/A testing by LSS.Load testing by OSG.
Promotion Schedule
Go/No Go Decision (approx. 4 days prior to promotion)All three groups (LSS, ICS, OSG) must agree that the promotion to production is a 'Go'. OSG schedules maintenanceLSS announces planned maintenance to community
Promotion Day!OSG performs promotionLSS and ICS staff are available in case of issues
LSS testing matrix
Primarily 2 types of matrix used for testing documentation• Basic template - used for testing upgrades to existing
versions• Advanced template - used for testing new version
LSS Testing Process
• All LSS staff engage in the process• Tools broken down by individuals• Realms configuration managed by only a select few• Certain testing features rolled out to other campus e-
learning admins• Bugs/Features/Tasks assigned to ICS developers with
documentation
Input Mechanisms
• From trouble tickets• Enhancements from the Sakai community• Faculty requests channeled thorough the CMS governance
committee • Admin staff for minor features/settings
LSS Testing
Sample Matrix -
12th Sakai Conference – Los Angeles, California – June 14-16
Things that don't fit
... and how to handle those changes outside the process
12th Sakai Conference – Los Angeles, California – June 14-16
The big items
• Web services / middle-ware • Course creation, roster loading, enrollment and Helpdesk
tools• Sakai settings - ICS and LSS roles• Other things that can be changed on the fly from the
database • Maintain dev instance and individual dev individual dev
VM's for developers • Username changes using JWS• Off-cycle maintenance• Wholesale changes to the cluster layout
Questions?
Contact Information
Narasi [email protected]
Mike [email protected]
Martin [email protected]
Thank you for joining us
