KTE Community of PracticeCase study: Using an educational influential network to disseminate an evidence based tool
Team: Christie Brenchley, Mary Egan, Jane Gibson, Rosemary Lysaght, Kathy MacDonald, Susan Rappolt, Rhoda Reardon, Lynn Shaw, Cynthia Neilson
Overview
1. Project overview
Creating the network
Creating the evidence based tool
2. Dissemination Goals
Target Audience
Key messages
Sources
3. Dissemination activities
4. Evaluation
Overview: Creating the network
• Network of informal opinion leaders (OTs) created based on adopted Hiss methodology
• Partners – College of Occupational Therapists of Ontario (COTO), Ontario Society of Occupational Therapists (OSOT)
• Surveyed – 643 with ‘Work-based practice”, 41% response rate
• Identified – 66
• Inaugural meeting April 7/06
• Formed a steering committee
Overview: Creating the Tool Working Together: Successful Strategies for Return to Work
• Purpose: To provide OTs with a content specific tool to be used with employers, based on the Seven Principles of RTW
• Seven Principles of RTW an evidence tool from a systematic review completed by IWH scientists on the effectiveness of RTW
• Draft tool was prepared by a working team of OT EIs and tested in several focus groups
Dissemination Goals and Target Audience
1. Utilize a tool format to share evidence about effective RTW principles for OTs in work practice
2. Tool to be disseminated to OT EIs first before full profession
3. Opportunity to evaluate dissemination via OT EI network
Working Together
Key Messages
Messages based on Seven Principles of RTW
• Early contacts
• Planning for the Workers’ Return
• Implementing a successful RTW Program
• Creating a RTW-friendly workplace
Dissemination Activities
1. Researchers and KTE staff prepared an evaluation plan
a) Survey
b) In depth interviews
2. Received ethics approval from U of T
3. Invited OT EIs to be part of the evaluation plan
a) Received individual identifiers
b) Only a subset agreed to interviews
4. Tool introduced at an invitational OT EI workshop
5. Three months after select introduction to OT EIs, held webinar
6. Tool available on-line at IWH and OSOT
7. Articles written in IWH and OSOT Newsletters
8. Tool used in OT course at UWO
9. Poster at OSOT conference
Evaluating the Tool Dissemination
• Researchers volunteered their time
(Susan R, Lynn S, Rosemary L, Mary Egan and Christie B (OSOT))
• 2 Parts to Evaluation:
1. Evaluating effectiveness of OT EI network
(survey, in-depth interviews)
2. Evaluating change in knowledge and practices
(survey)
• Data is currently being analyzed
Survey Design
• Time 1: 53 Questions Total
12 questions to test baseline knowledge
Remaining questions asked about OT EI network and their practices
enabling individuals to RTW
• Time 2: 19 Questions Total
12 questions tested knowledge after using the tool
Remaining questions asked about their 3 commitments to change
• Time 3: 23 Questions Total
12 questions tested knowledge retention
Remaining questions asked about the progress of their 3 commitments
to change and changes in their practice
Evaluation planT1Survey Knowledge
(1 month beforeWorkshop/webinar)
T2Survey Knowledge and Commitment to Change
(After receiving tool)
T3Survey knowledgeActing on Commitment to change
(1 month after T2)
OT EIs Workshop
N=8 N=8 N=4
OT EIs No workshop
N=6 N=3 N=3
OTsWebinar
N=49 N=30 N=28
Qualitative responses
Time 1: Question #17- I value the participation in the OT EI network: (Group 1 & 2)
Participants felt that the OT EI network offered them the opportunity to network and discuss with others whom are dedicated to professional issues, personal growth and promoting the value of research.
“I like connecting with others practicing in [the] same practice area and access to research findings” (Group 2 participant)
Qualitative Responses
Time 1: Question #18- Participating in the OT EI Network has had an influence on the ways in which I practice: (Group 1 and 2)
Participants felt that there has been good practice applications presented to date, however it was mentioned by some of the participants that they felt they haven’t been exposed to the group long enough to determine this.
“[It has] increased my awareness of practice guidelines and increased my resources to access this type of information and to forward the information on to other OTs” (Group 1 participant)
Qualitative responses
Time 3: Question #18- In thinking about yourself and your practice, describe the overall effects of the Working Together: Successful Strategies for Return to Work practice guideline on your practice.
Participants felt that the tool was great and looked forward to using it to facilitate return to work. It also helped them better understand how underlying processes are important in how they approach facilitating return to work. Participants also mentioned that it provides an evidence based structured guideline that is easy to use and ensures that no steps are missed.
“This will be helpful in future with RTW clients. It has refined my thinking in this area of practice, for future use” (Group 2 participant)
Lessons Learned
1. Survey Design:
• Qualitative questions difficult to capture in a survey
• Format of survey could have been designed to compare between the 3 groups, allowing us to determine effects of delivery of information
• Pilot testing survey questions could have eliminated poorly worded/unclear questions
• Larger sample size could have decreased the impact of drop out rate
Lessons Learned (2)
2. Delivery of Survey
• Providing participants with their commitments to change after the second survey would have allowed more participants to be able to answer questions in the third survey
• Sending a reminder e-mail to the participants with their identification number and the survey would eliminate incorrect identifiers
Questions for You
• Are there other ways for us to determine if EI groups are effective?
• What are realistic outcome measures that we should be using to evaluate our tools?
• Once we have the results of the evaluation, what should our next steps be?
• What is the appropriate time frame to evaluate a tools usage?
