Organizational Changes and Outcomes Monitoring
Presented by:
Kay Chapa and Kelly Gonzalez
� Background� Methods� Tools and Measures� Results� Conclusions� Questions
Kay Chapa Kelly Gonzalez
The University of Texas Southwestern Medical Center - Mission Statement and Goals:� To improve health care in our communitycommunitycommunitycommunitycommunitycommunitycommunitycommunity, Texas,
our nation, and the world through innovation and education
� To educateeducateeducateeducateeducateeducateeducateeducate the next generation of leaders in patient care, biomedical science and disease prevention
� To conduct high-impact, internationally recognized researchresearchresearchresearchresearchresearchresearchresearch
� To deliver patient care patient care patient care patient care patient care patient care patient care patient care that brings UT Southwestern’s scientific advances to the bedside – focusing on quality, safety and service
Kay Chapa Kelly Gonzalez
� UT Southwestern Medical Center Library� 50+ staff members
� 10 operational units
� Office of Assistant Vice-President for Library Services (OAVP)
� Library Director
� Deputy Director
� Assistant Director for Strategic Planning
� Program Evaluation and Assessment Specialists (2)
Kay Chapa Kelly Gonzalez
� Evolutionary process
� Growing competencies
� Organizational building blocks� strategic planning
� project and portfolio management
� assessment
Kay Chapa Kelly Gonzalez
� 2009 Annual Staff Survey� Connecting with our clients
� Library operations
� Internal communication
� Changes not adequately monitored
� Needed an instrument� Evaluate organizational changes
� Evaluate and monitor departmental and strategic plan progress
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
“Credible Evidence”
Source: Donaldson, Stewart I., Christina A. Christie, and Melvin M. Mark. 2009. What counts as credible evidence in applied research and evaluation practice? Los Angeles: SAGE.
He rebuttedHe refuted
He said some moreHe said something else
He talked about his recordHe talked about his record
He has some ideasHe has some other ideas
“The cognitive justification for democracy lies not in prevention of bad policies, but in the fact that the bad ideas promulgated in the form of policies and programs can be detected.”
Source: Donaldson, Stewart I., Christina A. Christie, and Melvin M. Mark. 2009. What counts as credible evidence in applied research and evaluation practice? Los Angeles: SAGE.
Kay Chapa Kelly Gonzalez
“In other words, modern democracies need credible assessments of the causal impacts of public policies and the programs developed to carry out the policies.”
Source: Donaldson, Stewart I., Christina A. Christie, and Melvin M. Mark. 2009. What counts as credible evidence in applied research and evaluation practice? Los Angeles: SAGE.
Kay Chapa Kelly Gonzalez
� Annual
� Organization, Unit-level, Individual
� Strategic Plan
� Performance Measurement
� Outcomes Evaluation and Monitoring
Kay Chapa Kelly Gonzalez
• What happened after the changes?• Did we get the results we were expecting?• What did we get that we were not expecting? • Did performance at multiple levels –organization, unit, individual – improve?
• Did performance decline?• Do adjustments need to occur?
� Measuring and Monitoring Change
� Instruments Evolve
� Instrument 1: scoring system, based on LibQUAL+™
Kay Chapa Kelly Gonzalez
� Performance measurement
� Strategic planning
� Project planning
� Quality improvement
� Program evaluation
� Outcomes monitoring
� Impact evaluation
Kay Chapa Kelly Gonzalez
Moving Our Library Forward IntelligentlyMoving Our Library Forward IntelligentlyMoving Our Library Forward IntelligentlyMoving Our Library Forward Intelligently
Courtesy of: Richard Wayne, Assistant Director of Strategic Planning
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
OEC and Organizational Changes: An Evaluation Plan FY2008/2009
Reviewer: OEC
Outcomes (short-term)
Project Information
Inputs
Expected Outcomes (short term and long term)
Maj
or Iss
ues
Impact (Institutional Goals, long-term)Outcomes (short-term)
Changes: Organization Level
Inputs
Tier 1
Maj
or Iss
ues
Impact (Institutional Goals, long-term)
Connecting
with our
Clients
Library
Operations
Internal
Communica-
tion
Teaching Learning Research Patient Care
Start End
OAVP: +1 AL, +1 SP = 2 FTEs x x x 1-Sep 1-Jan
DSTP created: DIRD + DA x x 1-Sep 1-Jan
CD: +1 FTE x 1-Sep 1-Jan
RSES: CORE - LIMO = 5 FTEs x x x 1-Sep 1-Jan
LIMO: CORE - RSES = 3 FTEs x x x 1-Dec 1-Jan
ID: HR, mgr x x 1-Sep 1-Jan
Archives & Special Collections unit
created x x 1-Sep 1-Nov
IS: mgr and server maintenance
activities merged x 1-Aug
Admin: Pam will assist with
assessment x 1-Sep
DATA: team transitions to task force x 1-Sep
Liaison Team: dissolved after Phase 2
implemented x x 1-Jan
Event Planning: becomes permanent
team x x 1-Sep 1-Nov
Marketing Team: will reconsider after
new hire in LIMO x x 1-Aug
Interest groups dissolved x 1-Sep 1-Nov
"Lunch with Laurie" implemented x 1-Sep 1-Nov
OEC reps nominated by OEC and
elected by staff x 1-Sep 1-Nov
Changes: Organization Level
Tier 1
Maj
or Iss
ues
Reviewer: AVP and DD
1)Needed to get staff out of the physical library by increasing outreach and liaison work. 2)Two units without managers, ID and CORE. 3)Issues with staffing/scheduling in ID. 4)Interpersonal issues in CORE. 5)Decreasing need for clerical tasks in Tech Svcs. 6)CD needs addt'l staff. 7)DA and DIRD relationship. 8)Lack of success in CATMAN. 9)Ongoing
communication issues not addressed by interest groups. 10)Lack of coordination among multiple units involved w/collection mgt. 11)Srategic planning needs to be a constant process.
Changes: Unit/Individual Level
Deliverable Name/Task Name
Connecting
with our
Clients
Library
Operations
Internal
Communica-
tion
Teaching Learning Research Patient Care
Start End
ADMIN
PI1: Increase % of time that Pam
Thompson works with Program
Evaluation and Assessment
Specialists on the Library Data Farm. x 1-Sep
PI2: Integrate Library's statistical
snapshot into the Library Data Farm. x x
Archives
PI1: Develop Archives & History of
Medicine Plan to expand/provide
access to stored collection via display
and/or digitization. x x
CD
PI1: Increase % of time devoted to new
activities such as
a. Active participants in ERM
implementation
b. Ejournal subject browsing issue
revealed as a result of Sharon
Cardwell’s LibQual+ project; Collection
Development invested in helping with
resolution
x 1-Sep
Outcomes (short-term) Impact (Institutional Goals, long-term)
1)Needed to get staff out of the physical library by increasing outreach and liaison work. 2)Two units without managers, ID and CORE. 3)Issues with staffing/scheduling in ID. 4)Interpersonal issues in CORE. 5)Decreasing need for clerical tasks in Tech Svcs. 6)CD needs addt'l staff. 7)DA and DIRD relationship. 8)Lack of success in CATMAN. 9)Ongoing
communication issues not addressed by interest groups. 10)Lack of coordination among multiple units involved w/collection mgt. 11)Srategic planning needs to be a constant process.
Source: University of Wisconsin – Extension http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
University of Wisconsin-Extension, Program Development and Evaluation (pg. 18)
INPUTS OUTPUTS OUTCOMES
Program
investments
Activities Participation Short Medium
What
we
invest
What
we do
Who
we
reach
What results
Long-
term
Feedback loops and multiFeedback loops and multi--dimensionsdimensions
� Performance Indicators(PIs)� Created collaboratively� Library Director (AVP), Deputy Director, Departmental Manager
� Mapped to Short-Term Outcomes� 2009� 16 organizational changes
� 12 PIs
� 4 changes without PIs related to “internal communication”� creation or dissolution of teams or task forces
� 2010� 11 organizational changes and PIs
Kay Chapa Kelly Gonzalez
� Additional Staff for Collection Development� 2 consecutive years added 1 staff member
� Total FTE for CD = 4
� Performance Indicators� Includes metrics, targets and intended outcomesoutcomes
� Unit to effectively change workflow and accommodate new activities and responsibilities
� Specifically related to implementation of ERMS
� Findability and access issues to electronic journals
Kay Chapa Kelly Gonzalez
Performance
Indicators (PIs)
Kay Chapa Kelly Gonzalez
� 2009� PI1: “Increase time devoted to new activities in CD”
� Active participants in ERMS implementation
� Full Success
� 2010� PI2: “Integrate electronic journal access management into unit activities and responsibilities”
� Specified job responsibilities for “Electronic Resources Specialist”
� Full Success
Kay Chapa Kelly Gonzalez
� Library Strategic Goal� “Provide clients with a more integrated, easier-to-use findability tool for the array of Library electronic resources”
� 2009 Organizational Change� Merge two departments to create Digital Services & Technology Planning (DSTP)
� Developing, implementing and managing new products for library’s web site
Kay Chapa Kelly Gonzalez
� Performance Indicators� 2009
� “Seamless integration of services, initiatives and resources with project and maintenance activities”
� ERMS, Client Contact Database, Digital Access Management System
� Full Success
� “Blur the lines between serials, web, research & development components of unit organization”
� Tasks remained disparate
� PI not appropriate
� Failure
Kay Chapa Kelly Gonzalez
� 2010 Organizational Change� Removed serials responsibilities from DSTP
� -3, +2, 1 vacancy = 7 FTE
� Focus unit on supporting the virtual library
� Performance Indicators� 2010
� “Build unit cohesion by organizing responsibilities, cross training and cross collaboration within the unit in order to provide enhanced support for virtual library”
� “Develop and show progress on Strategic Objectives in support of Goal 1 (findability solutions)”
Kay Chapa Kelly Gonzalez
� 2010� PI1: Build Unit Cohesion
� Full Success
� Each unit member was clearly responsible for specific projects and application support
� PI2: Findability Solutions
� Six active objectives
� Considered successful
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
Kay Chapa Kelly Gonzalez
“By consciously structuring our efforts to identify inputs for decisions we set the stage to achieve meaningful outcomes.”
Source:Busby, Lorraine. 2008. “Reality Through Evidence,” Evidence Based Library and Information Practice 3(1:82).
University of Wisconsin-Extension, Program Development and Evaluation (pg. 29)
Source: University of Wisconsin – Extension http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Kay Chapa Kelly Gonzalez
� Richard Wayne, Assistant Director of Strategic Planning
� Jane Scott, Design and Promotion Specialist
Kay Chapa Kelly Gonzalez