North Dakota Virtual Learning

InstituteDay 2

August 12, 2021

Which flavor of ice cream best represents how you’re feeling today?

https://www.eatyourfeelingsicecream.com/flavors

Agenda

3

Welcome & Overview

Finding Your Path Forward

Evidence-Based Strategies for Building and Sustaining Momentum

Quality Improvement-Testing and Learning for Change

Leading Change

Finding Your Path Forward

Learning Objectives

• Discuss opportunities to strengthen evidence and evidence-based decision-making processes.

• Review key implementation activities in the active implementation stages framework.

• Determine which implementation stage(s) your work is in.

Miracles Required?

Is this you?

What Does it Take?

It Takes Teams

It Takes Evidence

It Takes Support

It Takes Intentional

Learning

It Takes Time

It Takes Evidence

Community-Defined Evidence/ Vison of Success

Based on Community Centered Evidence-Based Practice Approach from the National Latin@ Network and Jacobs et al. 2012

Documented Evidence/

Information

Evidence-Based Decision Making

Community Practitioners’

Expertise

What Works

Environment and Organizational

Context

Environment and Organizational

Context

Community-defined evidence is a set of practices that communities have used and found to yield positive results as determined bycommunity consensus over time. These practices may or may not have been measured empirically but have reached a level of acceptance by the community.

Community-Defined Evidence

The Community-Defined Evidence Project, National Latino Behavioral Health Association

Identifying What “Works”

• What changes might you make to the way you identify and select innovations moving forward?

• What opportunities are there to strengthen your evidence and evidence-based decision making processes by applying an equity lens?

Group Discussion

Evidence ≠ Outcomes

And yet…

Implementation science refers to the “methods or techniques used to enhance the adoption, implementation, and sustainability” of an intervention (Powell et al., 2015)

What is Implementation Science?

In other words, implementation science addresses HOW we do our work.

Stretch Break!

What Does it Take?

It Takes Teams

It Takes Evidence

It Takes Support

It Takes Intentional

Learning

It Takes Time

It Takes Time

Change Takes Time…

Block Grant Strategy

DHS/DSS Integration

Work as One/One Dakota

3-5 Years

Exploration Installation Initial Implementation

Full Implementation

• Identify innovations• Examine innovation

core components• Assess need & fit • Consider

implementation supports

• Choose innovation(s)

• Develop Implementation Team

• Acquire resources• Equip staff• Prepare organization• Build implementation

supports

• Measure performance• Initiate improvement

cycles• Strengthen

Implementation supports

• Communicate progress

• Sustain implementation supports

• Consider scale up• Continuous

assessment & improvement

• Manage turnover & drift

• Document & share

Engage System Stakeholders and People with Lived Experience

Consider a project you’re currently working on.

• Individually reflect on this work and identify which stage(s) you think it’s currently in. (2 min)

• As a small group, share which stage(s) your projects are in and why. (8 min)

10 minutes

Reflect and Share

3-5 Years

Exploration Installation Initial Implementation

Full Implementation

• Identify innovations• Examine innovation

core components• Assess need & fit • Consider

implementation supports

• Choose innovation(s)

• Develop Implementation Team

• Acquire resources• Equip staff• Prepare organization• Build implementation

supports

• Measure performance• Initiate improvement

cycles• Strengthen

Implementation supports

• Communicate progress

• Sustain implementation supports

• Consider scale up• Continuous

assessment & improvement

• Manage turnover & drift

• Document & share

Engage System Stakeholders and People with Lived Experience

Tool: Implementation Stages Planning Tool

• The Community-Defined Evidence Project, National Latino Behavioral Health Association

• Brownson, Ross, etc. Evidence-Based Public Health, Oxford University Press. 2010.

References

Evidence-Based Strategies for Building and Sustaining

Momentum

Learning Objectives

• Describe the three main types of implementation supports

• Apply the Implementation Support Checklist to your project to identify areas of strength and opportunities for growth

Miracles Required?

Do you have a role here?

What Does it Take?

It Takes Teams

It Takes Evidence

It Takes Support

It Takes Intentional

Learning

It Takes Time

It Takes Support

3-5 Years

Exploration Installation Initial Implementation

Full Implementation

• Identify innovations• Examine innovation

core components• Assess need & fit • Consider

implementation supports

• Choose innovation(s)

• Develop Implementation Team

• Acquire resources• Equip staff• Prepare organization• Build implementation

supports

• Measure performance• Initiate improvement

cycles• Strengthen

implementationsupports

• Communicate progress

• Sustain implementation supports

• Consider scale up• Continuous

assessment & improvement

• Manage turnover & drift

• Document & share

Engage System Stakeholders and People with Lived Experience

Teams Build the Supports

Asha represents someone who plays a key role in carrying out a change/activity/project within your context.

Who is Asha in your context?

Say hello to Asha!

Asha Activity

Individually consider what Asha needs to support their work. (5 min)

Add one idea per sticky in Jamboard.

Implementation Supports

•Competency Supports

•Organizational Supports

•Leadership Supports

]

Competency Supports

Staff Selection Training

Coaching Fidelity/Performance Assessment

From the National MCH Workforce Development Center.

Organizational Supports

]

Decision-Support Data System

Facilitative Administration

Systems Intervention

From the National MCH Workforce Development Center.

Ash

Leadership Supports

]

Enabling Context

Continuous Communication

Multilevel Engagement

From the National MCH Workforce Development Center.

Stretch Break!

Implementation Supports Checklist

Implementation Supports ChecklistTraining – How does training build staff competency to implement the practice?

Program leaders and managers… Rating Action Notes

1. Skill-based training is secured for relevant staff (e.g., training integrate opportunities for practice and feedback)

2. Training data are used to develop competency and improve training

Activity: Implementation Supports Checklist

• Individually reflect on the checklist questions within the context of one of your projects. (8 min)

• As a group, discuss what resonated with you an any ahas. (17 min)

Debrief

How might you apply this checklist to your own work moving forward?

Quality Improvement -Testing and Learning

for Change

What Does it Take?

It Takes Teams

It Takes Evidence

It Takes Support

It Takes Intentional

Learning

It Takes Time

It Takes Intentional Learning

• Understand the importance of testing and learning

• Identify opportunities for testing and learning

• Understand how the PDSA cycle facilitates testing & learning

Learning Objectives

Learning Pathways• Leading change is almost never a

smooth road

• A Learning Mindset can help you navigate and address the potholes

• None of you have all the answers. This is not a weakness, but an opportunity.

Testing and Learning in Everyday Life

What’s an example of something you have

tested in your personal life?

Testing and Learning in Everyday Life

“Mom, I LOVE these cupcakes.

Can you make them for my birthday next

month?”

“Sure, I’ve never made them before, but I’ll make them for your

birthday!”

The Day of the Party…

• Have you been in this situation?• What could the mom have done to

possibly avoid this situation?

You Are Here

Consider a Current or Recent Project…

How did you feel when you first began implementing this project?

• Five Whys, Fishbone

• Impact Matrix, Prioritization Matrix

• PDSA Cycles, Model for Improvement

• Lean/Six Sigma

All Those Tools

All That JargonPurpose Output

Testing/Pilot Testing

Get a “good enough draft” into real life to learn if and how it works in actual practice

• Understanding of needs• Improved idea with fewer

“bugs”

Scale-up/Spread

Take what worked in a pilot test and implement it with more sites

• Sustainable, systematic change• Infrastructure to support change

What to Test1. Implementation processesSelection of staff Recruitment of sitesEngagement of stakeholdersCommunication with sitesTraining/Coaching of sites

2. Components of intervention/programTraining/Technical assistanceCommunication (brochures, labeling, messaging, etc.)Policy changes Environmental changes

3. Data collection processesAssessment toolsFrequency of reportingTypes of reports

Source: Akin, Bryson, Testa, et. al. Usability testing, initial implementation, and formative evaluation of an evidence-based intervention: Lessons from a demonstration project to reduce long-term foster care. Evaluation and Program Planning 41 (2013) 19–30.

.

When to Test

Exploration Installation Initial Implementation

Full Implementation

3-5 Years

How to Test: PDSA CycleAct Plan

Study Do

• Objective of cycle • Questions/predictions• Plan to carry out the cycle

(who, what, where, when)

• Carry out the plan• Document problems/

unexpected observations• Begin analysis of data

• Complete the analysis of data

• Compare data to predictions

• Summarize what was learned

• What changes are to be made?

• Adapt? Or Abandon?• Next cycle?

Tips for Success

1 Test

5 Tests

Learn and Get Better

Which option will provide you with the best

learning opportunity?

Stretch Break!

• Identify root causes

Making Quality Improvement Effective

• Prepare for the change• Collect baseline data before you start

• Identify QI champions to support the QI process and any subsequent changes implemented

• Thinking with the end in mind • Long-term process • Ensure overall performance is improving

Making Quality Improvement Effective

• Document, learn from, and share your process• Learn from what doesn’t work• Document your process• Celebrate successes!

Making Quality Improvement Effective

Peer Sharing

• What are you currently testing (or planning to test) in your own work?

• What else would you like to test that you’re not currently? How might you plan for this?

Key Points to Remember

• Be intentional and create a plan• Start small and test for a short time• Start with an informal “friendly”

audience• Learn and get better with each new test• Test under different conditions

Resources

MCH Navigator• PDSA Overview• PDSA Cycle: Scenario and Example

References

• Top 10 Problems Encountered By Quality Improvement Teams. (2010, December 7). Retrieved from http://www.phf.org/phfpulse/Pages/Top_Problems_Encountered_By_QI_Teams.aspx

• Barriers to Quality Improvement. Retrieved from http://www.cqeacademy.com/cqe-body-of-knowledge/management-and-leadership/barriers-to-quality-improvement/

• HRSA. (2011, April). Quality Improvement. Retrieved from https://www.hrsa.gov/sites/default/files/quality/toolbox/508pdfs/qualityimprovement.pdf

• Quality Improvement Tool Summary Sheet. Retrieved from https://www.mchnavigator.org/transformation/documents/QI/Summary%20of%20QI%20Tools.pdf

63

LEADING CHANGE

Stephen OrtonSenior Fellow, Public Health Leadership

NC Institute for Public Health

MUTUAL LEARNING

Smart Leaders Smarter TeamsRoger Schwarz

Goal: DEVELOPMENT, learning

Mutual Learning

© 2013 Roger Schwarz & Associates. All rights reserved.

Mut

ualL

earn

ing

Appr

oach

© 2013 Roger Schw

arz & Associates.

All rights reserved.

Unilateral Control Approach

Learning Objectives• Share best practices to build teams for

transformation

What Does it Take?

It Takes Teams

It Takes Evidence

It Takes Support

It Takes Intentional

Learning

It Takes Time

It Takes Teams

Creative, Proactive Problem-Solving Units

Transformational Teams

Candor & Curiosity

YOUR VIEW

MY VIEW

What leads me to think this is Y.

I came to this conclusion after seeing Y.

I’m intrigued by the way you see this; can you give me an example?

Help me see this through your lens.

I think we should do X.

I believe that X is thee best choice at this time.

What might I be missing?Push back on me here, what do you think?I’ve said what I think; now I’m interested in what others think, especially those who see it differently.

• Supports mutual learning & co-design

• Documents aims, goals, tasks, membership, roles, norms

• Developed collaboratively with all team members

• Revised as needed

Tool: Team Charter

Phot

o by

Han

nah

Busi

ngon

Uns

plas

h

You have the power to build the team you need

Take time to build a transformational team...it will pay off

Building the Team(s) You Need!Ph

oto

by Je

rem

y Bi

shop

on U

nspl

ash

• What’s already working/going well with current teams/partners you work with?

• What opportunities to strengthen teams and multi-level collaboration can you identify?

• What support might potential partners/ individuals need to participate?

• What are possible next steps?

Group Discussion

Reflections and Wrap-Up