Evaluation of Interprofessional Competency (Practice ... · PDF fileEvaluation of...

Evaluation of Interprofessional Competency (Practice) tools for Internationally Educated Health Professionals

May 06, 2016

Prepared by:

Mubashir Arain, Shelanne Hepp, Esther Suter*, Shyama Nanayakkara, Siegrid Deutschlander*, Liz

Harrison*, Grace Mickelson*, Lesley Bainbridge* and Ruby Grymonpre*

*Western Canadian Interprofessional Health Collaborative (WCIHC) core team members

Copyright © 2016. Province of British Columbia. All rights reserved. This material is owned by the Government of British Columbia and protected by copyright law. It may not be reproduced or redistributed without the prior written permission of the Province of British Columbia. Production of this document has been undertaken with the support and collaboration of the Western and Northern Health Human Resources Planning Forum and its member jurisdictions, and has been made possible through a financial contribution from Health Canada. Reference to the material contained in this document should be accompanied by appropriate acknowledgement to the source document and its sponsors. The views expressed herein do not necessarily represent the views of Health Canada. Please cite this report as Arain M, Hepp S, Suter E, Deutschlander S, Harrison L, Mickelson G, Bainbridge L, Nanayakkara S, and Grymonpre R. 2016. Evaluation of Interprofessional Competency (Practice) tools for Internationally Educated Health Professionals. This report was commissioned by the Government of British Columbia with a grant from Health Canada.

i | P a g e

Contents ACRONYMS ................................................................................................................................................... ii SUMMARY ..................................................................................................................................................... 1 BACKGROUND ............................................................................................................................................... 3 METHODOLOGY ............................................................................................................................................ 3 RESULTS ........................................................................................................................................................ 5

1. Knowledge improvement of the participants 8

Pre and Post assessments ..................................................................................................................... 8 Reflections ........................................................................................................................................... 12

2. Feedback on the modules 14

INTRODUCTORY MODULE................................................................................................................... 16 COMPETENCY-BASED MODULES ........................................................................................................ 17

Interprofessional Conflict Management......................................................................................... 17

Team Functioning ............................................................................................................................ 19

Interprofessional Communication .................................................................................................. 20

Patient-Centred Care ....................................................................................................................... 21

Collaborative Leadership ................................................................................................................ 22

Role Clarification ............................................................................................................................. 23

TIME TAKEN TO REVIEW ICT FOR IEHPS ............................................................................................. 24 LANGUAGE .......................................................................................................................................... 25 CULTURAL ASPECTS ............................................................................................................................. 25 SCENARIOS .......................................................................................................................................... 26 ASSESSMENTS ..................................................................................................................................... 27 LICENSING EXAM ................................................................................................................................. 28 TECHNICAL ISSUES ............................................................................................................................... 28

CONCLUSIONS ............................................................................................................................................. 29 LIMITATIONS ............................................................................................................................................... 29 RECOMMENDATIONS ................................................................................................................................. 30 APPENDIX I. IEHP PRE AND POST ASSESSMENT TOOL ................................................................................ 32 APPENDIX II. SURVEY FOR IEHPS ................................................................................................................. 35 APPENDIX III. INTERVIEW GUIDES ............................................................................................................... 46 APPENDIX IV: REFLECTIVE QUESTIONS ....................................................................................................... 48 APPENDIX V: ETHICS EXEMPTION FROM UNIVERSITY OF SASKATCHEWAN .............................................. 49

ii | P a g e

ACRONYMS LPN = Licensed Practical Nurse RN = Registered Nurse RPN = Registered Psychiatric Nurse OT = Occupational Therapist PT = Physical Therapist ICT = Interprofessional (Practice) Competency Toolkit IEHP = Internationally Educated Health Professional IP = Interprofessional IMG= International Medical Graduates OSCEs – Objective Structured Clinical Skills Examinations CIHC– Canadian Interprofessional Health Collaborative

1 | P a g e

SUMMARY

Purpose

The Interprofessional Competency (Practice) Toolkit for Internationally Educated Health

Professionals (ICT for IEHPs) consists of several collaborative training modules designed specially for

IEHPs. The WCIHC study team has a diverse experience and expertise to direct and oversee curriculum

development for health care providers trained outside of Canada (WCIHC website:

http://www.inspirenet.ca/resources/western-canadian-interprofessional-health-collaborative-wcihc). We

pilot tested the modules with a cohort of IEHPs and their educators to evaluate whether the these tools

are useful and appropriate for leaners who were trained outside of Canada. We tested the online format

and any knowledge and attitude changes among IEHPs toward IP collaboration. Their feedback will be

used to improve the quality and relevance of the materials.

Methods

We conducted surveys and follow-up interviews with IEHPs and educators from four provinces

(British Columbia, Alberta, Saskatchewan, and Manitoba). We aimed to include seven different

healthcare professionals: Licensed Practical Nurses (LPN), Registered Nurses (RN), Registered Psychiatric

Nurses (RPN), Pharmacists, Physicians, Occupational Therapists (OT), and Physical Therapists (PT). We

also used a pre-post assessment tool and reflective questions to measure the effectiveness of the ICT for

IEHPs.

Findings

Thirty IEHPs and five educators reviewed the ICT for IEHPs modules and provided their feedback

in this evaluation.

Two educators were from Alberta, two from Manitoba and one from Saskatchewan. IEHP

participants were 48% from British Columbia, 33% from Alberta, 10% from Manitoba, and 7% from

Saskatchewan. The majority of IEHPs were RNs (36%), followed by pharmacists (27%); other IEHPs were

LPNs (13%), OTs (10%), PTs (7%), and Physicians (7%). At the time of the survey, 60% of participants

(n=18) were preparing for their professional licensing exam and 40% (n=12) had completed their

licensure requirements (post-licensure).

We compared the scores of the participants who completed assessment before and after

reviewing the competency-based modules using Wilcoxon signed-rank test for pairwise comparison.

Pre- versus post- intervention confidence scores for all items under the role clarification and patient-

centred care domains increased significantly (p value <0.01). The scores increased for most questions in

http://www.inspirenet.ca/resources/western-canadian-interprofessional-health-collaborative-wcihc

2 | P a g e

Team Functioning, Collaborative Leadership, and Communication domains. In contrast, the post

assessments score only increased for half of the questions in Conflict Management domains.

Most of the participants found the content of the competency-based mini modules very useful.

IEHPs also found the introductory module a good overview of IP collaboration. However, educators

found the introductory module less useful than the competency-based modules. Both IEHPs and

educators reported that the introductory module was lengthy and suggested moving some of its content

to the competency-based mini modules.

The ICT for IEHPs covered a range of cultural aspects in the modules and most IEHPs were able

to relate to the scenarios presented. Pharmacy IEHPs recommended adding some scenarios of IP

collaboration in a community setting.

Participants found the pre and post assessments useful for measuring changes in their

confidence levels for each IP competency after reviewing the ICT for IEHPs modules. Participants

suggested adding more multiple choice style questions.

Nine participants (7 IEHPs and two educators) reported one or more technical issues with

accessing the modules online.

Conclusions

Overall, the ICT for IEHPs is a very useful resource and these modules significantly improve the

knowledge of IEHPs about IP collaborative practice. Some minor amendments are required before

launching the final ICT for IEHPs modules.

Key recommendations

Based on the evaluation, we recommend the following:

Reduce the length of the introductory module by moving some of its content to the

competency-based mini modules

Add more objective assessment methods such as multiple choice style questions

Address all technical issues before the final launch of ICT for IEHPs

3 | P a g e

BACKGROUND The ICT for IEHPs is a learning resource for building collaborative practice competencies among IEHPs. In

collaboration with the project curriculum working group, the curriculum designer Fluid Learning Design

was contracted to develop the modules for this project. The module content is based on the Canadian

Interprofessional Health Collaborative (CIHC) National Competency Framework (1). Fluid Learning

Design has previously developed a similar set of modules (IPC on the RUN) for Canadian healthcare

professionals (2).

Although the project targeted seven different health professions for logistical reasons, the content of

the content of ICT for IEHPs is not specific to any particular healthcare discipline. IEHPs from any

healthcare discipline can benefit from these modules. The ICT for IEHPs comprise an introductory

module followed by six competency-based mini modules; each mini module speaks to one of the six

competency domains: conflict management, team functioning, interprofessional communication,

patient-centred care, collaborative leadership, and role clarification.

This evaluation aimed to ascertain

1) What aspects of the ICT for IEHPs (e.g. format, content, language, appropriateness, and

cultural aspects) do IEHPs and educators find most helpful?

2) What do they find least helpful? And

3) Did the ICT for IEHPs improve knowledge and attitudes towards IP collaboration?

We will use the feedback from educators and IEHPs to improve the quality and relevance of the

modules.

METHODOLOGY

We conducted this evaluation from March 2016 to May 2016 in the four Western Canadian Provinces

(Alberta, Manitoba, Saskatchewan, and British Columbia). We aimed to test the ICT for IEHPs modules

with IEHPs from seven health professions: OTs, PTs, RNs, RPNs, LPNs, pharmacists, and physicians. Our

aim was to recruit seven IEHPs and two educators from each province (36 total). We defined ‘educator’

as an individual who is responsible for educating and assessing IEHPs in preparation for the IEHPs

profession specific licensing exam. The WCIHC core team approached stakeholders to connect with

educators (bridging program instructors) and asked to assist with IEHP recruitment. We used our

‘Stakeholder Directory’ to identify regulators and educators who could approach potential participants

and send them an invitation to participant in this evaluation. The ‘Stakeholder Directory’ is a list of

4 | P a g e

relevant stakeholders from each jurisdiction and was developed specifically for this project through an

environmental scan, a web search, and word-of-mouth recommendations.

Both IEHPs and educators were asked to complete the following activities:

• Review the ICTs for IEHP modules and complete the assessment toolkit (pre- and post-

assessment questionnaire; Appendix I)

• Complete a survey (Appendix II) about the appropriateness and relevance of the content for

IEHPs

• Participate in a follow-up telephone interview (interview guide; Appendix III)

We developed a survey tool and assessment questionnaires for this study. For the face validity of the

tools, we circulated it to all core team members and received feedback.

One research technician and two research consultants approached the participants to complete

questionnaires and conduct the follow-up interviews across four provinces. We also offered gift cards of

a nominal amount to IEHP participants only. IEHP participants who successfully completed all three

evaluation components (reviewing the module, complete the survey and participate in the follow-up

interview) received a $50 gift card. Participants only reviewing the module and completing the survey

received $25 gift card.

We also asked IEHPs to send their reflections on each module to get a better sense whether the content

improved their understanding about interprofessional collaboration (Appendix IV). The introductory

module had eight reflective questions including three multiple choice questions. All competency-based

modules had two reflective questions: one in the beginning after a scenario to get reflections on the

scenario and a second question near the end of the module to get reflections on the overall module

content. The Patient-Centred Care module had an additional question asking IEHPs to reflect on their

healthcare practice back home.

The pre- and post-assessment questionnaire had the same set of questions that measured confidence

level on a Likert scale from ’not confident”, “somewhat confident”, “confident”, to “very confident”. We

scored these confidence levels from 1-4. We compared the average pre-assessment scores with post-

assessment scores separately for each question within the six competency domains. We embedded the

pre-assessment questionnaire in the introductory module while the post-assessment questionnaire was

embedded after the last competency-based mini module. We also graphically compared the percentage

of participants who reported ‘strongly agree’ in pre- and post-assessments.

In the questionnaire, we collected responses on the format and content of the ICTs for IEHPs, time spent

to review each module, cultural appropriateness, and language. We also collected demographic

5 | P a g e

information of IEHPs such as age, gender, country of origin, professional background, and the province

of residence in Canada. In the follow-up interviews, we asked questions about the usefulness of the

content, method of delivery, and relevance to IEHPs.

We analyzed the questionnaires and pre-post assessments in SPSS using descriptive and inferential

statistics. We reported median with inter-quartile range (IQR) where data had extreme values. For the

Likert scale questions in the survey, we dichotomized responses into ‘agree’ or ‘strongly agree’ versus all

other responses. Due to the small sample, we used non-parametric statistical tests for comparing

module scores between educators and IEHPs (Fisher's Exact Test) and pre-post assessments (Wilcoxon

signed-rank test). We performed content analysis for interviews and open-ended responses in the

survey.

We obtained ethical approval for the study from the Research Ethics Board, University of Manitoba and

the Conjoint Health Research Ethics Board, University of Calgary. Ethics boards from University of

British Columbia and University of Saskatchewan both considered this project an evaluation and waived

the need for HREB approval (Appendix V).

RESULTS

Thirty out of 55 IEHPs (response rate 55%) and five out of 13 educators completed the questionnaire

(response rate 38%). Nineteen IEHPs and three educators also participated in the follow-up interviews.

Of the IEHP participants, 18 provided their reflections and 16 also completed pre and post-assessment

questionnaires.

Demographics of educators

The five educators resided in Alberta (n= 2), Saskatchewan (n=1), and Manitoba (n=2); two of them were

involved in teaching internationally educated physicians, the others were associated with programs for

internationally educated pharmacists (n=1), nurses (n=1), and physical therapist (n=1). The age of

educators ranged from 51 to 58 years. All educators were female.

Demographics of IEHPs

The age of the IEHPs ranged from 23 years to 48 years (Figure 1). Most participants (63%) were female

(n=19). Of the total sample, 50% participants were from British Columbia (n=15), 33% from Alberta

(n=10), 10% from Manitoba (n=3), and 7% from Saskatchewan (n=2) (Figure 2).

6 | P a g e

Figure 1: Age distribution of the IEHP survey participants (n=30)

Figure 2: Provinces of the IEHPs (n=30)

Most IEHP survey respondents (n=10) received their professional degree from the Philippines

(34%). Seventeen percent (n=5) of participants received their professional degree from Nigeria. Other

common countries were Egypt, India, and South Africa, with 10% each (n=3). The remaining countries

were distributed equally at 3% (n=1) including Iran, Australia, Singapore, Taiwan, USA, and South Africa

(Figure 3).

0

5

10

15

20-25 26-30 31-35 36-40 41-45 46-50

Age in years

Age Distribution

7 | P a g e

Figure 3: Home countries of the IEHPs where they completed their professional degrees (n=30)

The majority of the participants were RNs (36%; n=11) and pharmacists (27%; n=8); the others were

LPNs (13%; n=4), OTs (10%; n=3), PTs (7%; n=2), and Physicians (7%; n=2) (Figure 4). We did not have any

representation from RPNs.

Figure 4: Healthcare profession of the IEHPs (n=30)

8 | P a g e

At the time of the survey, 60% of participants (n=18) were preparing for their professional licensing

exam and 40% (n=12) had completed their licensure requirements (post-licensure). The majority (67%)

had attended a bridging program. Of the 12 participants who completed their licensure, only six had

started practicing in Canada: three had two years of experience and the other three had one year of

experience. The number of years participants practised in their home country varied from none to 18

years (mean=6.47; SD=4.68) (Figure 5).

Figure 5: Experience of healthcare professions in home country (n=30)

MODULE EVALUATION

The evaluation of ICT for IEHPs modules provided information on two main aspects:

1. Usefulness of these modules for improving knowledge and understanding of IEHPs about

interprofessional practice (knowledge improvement)

2. Improvement of the quality and relevance of modules to IEHPs (feedback on the content)

We only measured the knowledge improvement of IEHPs. We obtained feedback on modules from both

IEHPs and educators.

1. Knowledge improvement of the participants

We measured the knowledge and understanding of the IEHPs with a set of pre- and post-assessments

and reflective questions embedded within the ICT for IEHPs modules.

Pre- and post-assessments

Tables 1a and 1b show the scores of the participants who completed assessments before and after

reviewing the competency-based modules for the six interprofessional competency domains.

The confidence scores increased significantly for all questions in Role Clarification and Patient-Centred

Care domains after reviewing the module. The scores increased for most questions in Team Functioning,

1

13

7

9

0

2

4

6

8

10

12

14

No experience <1-4 years >5-9 years >10 or more years

9 | P a g e

Collaborative Leadership, and Communication domains [Table 1a & 1b]. In contrast, the post

assessments score only increased for half of the questions in Conflict Management domains; there was

no significant improvement in the confidence scores for other questions in this domain. This may

indicate that more intense education and coaching may be required to increase IEHP’s confidence in

their conflict management skills.

We also graphically compared the percentage of participants who reported their ratings as ‘very

confident’ in the pre and post assessments (Figure 6). We averaged the number of participants providing

ratings across survey items within each domain and calculated the percentage of people providing

different ratings. Each bar represented the average percentage of people providing the ratings within

each domain. There was a shift towards higher confidence level after reviewing ICT for IEHPs.

10 | P a g e

Table 1a Pre- and post-assessment scores of participants (n=16); Likert scale (1=Not Confident to 4=Very Confident)

Questions Before Mean (SD)

After Mean (SD)

p-value

Role Clarification 1. Described own role and that of others 2. Recognized and respected the diversity of other health and social care roles, responsibilities, and competencies 3. Performed own roles in a culturally respectful way 4. Communicated roles, knowledge, skills, and attitudes using appropriate language 5. Considered the roles of others in determining own professional and interprofessional roles 6. Accessed others’ skills and knowledge appropriately through consultation 7. Integrated competencies/roles seamlessly into models of service delivery

2.87 (0.9) 3.00 (0.7)

2.88 (1.0) 3.00 (0.9) 2.63 (0.8) 2.50 (0.9) 2.63 (1.0)

3.47 (0.5) 3.81 (0.4)

3.50 (0.6) 3.53 (0.6) 3.63 (0.5) 3.31 (0.7) 3.31 (0.6)

0.02*

<0.01*

0.01* 0.02*

<0.01* 0.02*

<0.01*

Patient-Centred Care 1. Supported the participation of patients/clients, their families 2. Shared information with patients/clients in a respectful manner 3. Ensured that appropriate education and support is provided to patients/family members 4. Listened respectively to the expressed needs of all parties in delivering care

2.94 (0.9) 2.94 (0.9) 2.88 (1.0) 2.80 (0.9)

3.81 (0.4) 3.69 (0.6) 3.56 (0.5) 3.80 (0.4)

0.01*

<0.01* 0.02*

<0.01*

Team Functioning 1. Understood the process of team development 2. Developed principles for working together 3. Effectively facilitated discussions and interactions among team members 4. Participate, and be respectful of all members in collaborative decision-making 5. Regularly reflected on functioning with team learners/practitioners 6. Established and maintained effective and healthy working relationships with learners/practitioners, patients/clients, and families 7. Respected team ethics

2.47 (0.9) 2.73 (0.8) 2.47 (1.0) 3.00 (0.9) 2.47 (0.9) 2.79 (0.9)

3.20 (1.0)

3.27 (0.5) 3.20 (0.8) 3.33 (0.6) 3.67 (0.4) 3.47 (0.5) 3.50 (0.5)

3.67 (0.4)

<0.01*

0.14 0.01* 0.03*

<0.01* 0.05

0.08

* Significant differences at 95% confidence level; p values were calculated using Wilcoxon signed-rank test for pairwise comparison

11 | P a g e

Table 1b Pre- and post-assessment scores of participants (n=16); Likert scale (1=Not Confident to 4=Very Confident)

Questions Before Mean (SD)

After Mean (SD)

p-value

Collaborative Leadership 1.Worked with others to enable effective patient/client outcomes 2. Advanced interdependent working relationships among all participants 3. Facilitated effective team processes 4. Facilitated effective decision-making 5. Established a climate for collaborative practice among all participants 6. Co-created a climate for shared leadership and collaborative practice 7. Applied collaborative decision-making principles 8. Integrated the principles of continuous quality improvement to work processes

2.80 (0.6) 2.40 (0.9) 2.13 (0.9) 2.33 (0.9) 2.47 (1.0) 2.40 (1.1) 2.47 (0.9) 2.60 (0.9)

3.40 (0.5) 3.07 (0.7) 3.33 (0.6) 3.33 (0.6) 3.40 (0.6) 3.47 (0.5) 3.47 (0.5) 3.40 (0.6)

<0.01*

0.06 <0.01* <0.01* 0.01* 0.01*

<0.01* 0.02*

Interprofessional Communication 1. Established team work communication principles 2. Actively listened to other team members including patients/clients/families 3. Communicated to ensure common understanding of care decisions 4. Developed trusting relationships with patients/families and other team members 5. Effectively used information and communication technology

2.47 (0.9) 3.13 (0.8) 2.67 (0.9) 3.27 (0.7) 2.67 (0.8)

3.27 (0.7) 3.87 (0.3) 3.53 (0.5) 3.53 (0.5) 3.33 (0.6)

0.01*

<0.01* <0.01*

0.23 0.02*

Interprofessional Conflict Management 1. Valued the potential positive nature of conflict 2. Recognized the potential for conflict to occur; taking constructive steps to address 3. Identified common situations that are likely to lead to disagreements or conflicts 4. Knew and understood strategies to deal with conflict 5. Set guidelines for addressing disagreements 6. Effectively worked to address and resolve disagreements 7. Established a safe environment in which to express diverse opinions 8. Developed a level of consensus among those with differing views; allowing all members to feel their viewpoints have been heard no matter what the outcome

2.62 (1.0) 2.77 (1.0) 2.67 (1.0) 2.46 (1.1) 2.46 (1.1 2.69 (1.0) 2.83 (1.1) 2.46 (1.0)

3.31 (0.7) 3.38 (0.6) 3.42 (0.6) 3.31 (0.6) 3.23 (0.7) 3.46 (0.6) 3.42 (0.7) 3.31 (0.6)

0.03* 0.07 0.06 0.06

0.02* 0.03* 0.16

<0.01*

* Significant differences at 95% confidence level; p values were calculated using Wilcoxon signed-rank test for pairwise comparison

12 | P a g e

Figure 6: Pre and post comparison of participants reporting “Very Confident” in each IP competency domain

Reflections

Reflections of IEHPs showed the improvement in the understanding of IEHPs about IP collaboration in all

six competency domains.

Introductory Module

In the introductory module, IEHPs expressed their understanding of effective team meetings, sharing

common goals, effective communication between healthcare professionals, respecting other team

members, negotiating responsibilities, and providing the best approach to patient care.

My understanding of interprofessional collaborative practice has changed a bit in terms of having additional knowledge of how it works. Interprofessional collaborative practice is working together with a team of health care professionals, including the patient and

0%

10%

20%

30%

40%

50%

60%

70%

80%

Average percentage of respondents reporting 'Very Confident'

Pre

Post

13 | P a g e

his or her family to create a common goal and having an effective communication in order to achieve them.

Interprofessional Conflict Management

IEHPs reflected on factors contributing to conflicts and how a conflict could be constructive or

destructive. The constructive conflict was described as using effective conflict management to build

strong teams, listening to all team members and coming to a compromise, sharing the leadership role,

and respecting team members opinions prior to making final decisions. The destructive conflict was

described as trying to win an argument instead of reaching consensus and lack of understanding among

team members. Some IEHPs identified strategies on how they would deal with conflict after reviewing

this module such as being understanding, compromising, providing alternative solutions, listening,

sharing leadership, and focusing on best patient care.

Now I understood that not all conflicts are destructive and conflicts can be beneficial towards a patient centred goal if we know about the role of other professionals and respect other members’ opinion.

Collaborative Leadership

IEHPs had two main reflections at the end of this module. First, IEHPs reported their understanding that

patients and families are the experts and leaders in decision making about care. Second, participants

reported that team leadership should be based on who has the most knowledge of the patient and

advocate more for the patient.

I will be more comfortable to share my knowledge because I know that it will be respected by my colleagues. I will also keep in mind that I need to give a chance for other team members to express and share their expertise and come up with a better treatment goal for the patient.

Interprofessional Communication

Participants reflected on how this module improved their understanding of communicating with other

healthcare professionals by recognizing communication styles, recognizing cultural and professional

diversity, and understanding needs and preferences of others.

Through this module I realize that I am from collectivist high context and multicultural communicative culture and now fully aware of intercultural communication challenges.

Patient-Centred Care

Participants reflected on the scenario in the module and provided their thoughts about the meaning of

patient-centred care. The module helped participants realize that other health care professionals,

patients, and families should collaborate to find common goals.

The values and principles (described in this module) will have a great impact as every client is different and the health practitioner should know the client and their family that is involved with the care.

14 | P a g e

Team Functioning

Participants reflected on their understanding that how teams of different healthcare professionals

function effectively.

Understanding how the interprofessional collaborative team functions in a different context will help me become a better part of teams. In this way, I will be able to clarify how they communicate and work given the differences in our professions.

2. Feedback on the modules

We collected feedback on ICT for IEHPs through surveys and interviews. The survey had a similar set of

questions for each module including some Likert scale questions. We compared scores of Likert scale

questions between the modules and also between educators and IEHPs.

The high average scores of Likert scale questions for each of the ICT for IEHPs modules indicated

that the participants strongly agreed with language, the flow of the modules, assessment tools, and

reflective questions in the modules. The comparison of the scores of each Likert scale question between

all ICT for IEHPs modules showed no significant difference between these modules [Table 2]. We

analyzed their verbal and written feedback to generate key themes and subthemes. We categorized

participants by their home country and profession.

15 | P a g e

Table 2 Comparing Likert scores (1=Strongly Disagree to 5=Strongly Agree) among all ICT for IEHPs modules for each question (n=35)

Modules

Introductory Conflict Management

Team Functioning


Patient-Centred care


Role Clarification

P values

Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD)

Language was appropriate

Easy to follow Assessment tools

Improved IP understanding

Improved IP knowledge

Improved IP attitudes

4.56 (0.78)

4.52 (0.66) 4.31 (0.75) 4.26 (0.82)

4.40 (0.77)

4.40 (0.84)

4.52 (0.66)

4.64 (0.48) 4.30 (0.63) 4.27 (0.76)

4.55 (0.56)

4.52 (0.61)

4.52 (0.56)

4.55 (0.56) 4.31 (0.64) 4.33 (0.73)

4.48 (0.50)

4.55 (0.56)

4.55 (0.56)

4.58 (0.50) 4.33 (0.64) 4.33 (0.81)

4.56 (0.56)

4.55 (0.61)

4.58 (0.50)

4.58 (0.50) 4.41 (0.56) 4.30 (0.81)

4.52 (0.66)

4.48 (0.71)

4.55 (0.50)

4.61 (0.55) 4.33 (0.64) 4.34 (0.82)

4.45 (0.71)

4.39 (0.74)

4.55 (0.50)

4.59 (0.49) 4.41 (0.61) 4.45 (0.61)

4.58 (0.50)

4.55 (0.56)

0.99

0.98 0.97 0.96

0.88

0.88

* p values were calculated using one-way ANOVA

16 | P a g e

INTRODUCTORY MODULE We found variations in responses from educators and IEHPs on the content in the introductory module.

IEHPs were significantly more likely to agree that the language in the module was appropriate than

educators. IEHPs were also significantly more likely to agree that the introductory module improved

their understanding and attitude towards IP practice than the educators [Table 3].

IEHPs reported that the introductory module was very important because it provided the context of the

competency modules. The introduction helped IEHPs to understand why collaborative practice is

important and anticipate learning expectations in the competency-based modules.

I liked the introduction provided baseline knowledge and know what to expect from the other module. It was a good overview. (LPN, Nigeria)

I think I felt that the intro module set a good context and touched on many of the things that were brought up in the mini modules. The ideas that are presented. If I just did the mini modules I wouldn’t have understood why this is important and how to apply in the workplace. (USA, RN)

Educators found the scenario slightly confusing and suggested more clarity to demonstrate

interprofessional aspects.

I am not sure all people would naturally make the case or relationship in the presented case of Esther story. It is a case with a poor outcome. If you were going to use the case, there should be a bit more about Esther and what went wrong. We didn’t know what went wrong. I think it is an ok example, but went off into patient safety and not necessarily IP collaboration (Educator, PT Program)

Both IEHPs and educators perceived that the introductory module was lengthy and could be shortened

to allow more time on the competencies.

The first module was the introduction – it went too long and could be more precise. I think the length should be the same, but I think you should dedicate more time to the six modules. (Pharmacist, Egypt)

The first introductory module was too long. Overall, the introduction is too long and I lost the direction of where that was going or what you are trying to do. I am afraid that you might lose people in the introduction and do not go on to the modules. I thought the modules were better learning than the initial introduction. So if that could be shorter and not lose people and put more context in the modules instead of them just listening would be better. (Educator, IMG Program)

17 | P a g e

Table 3. Comparing responses between IEHPs and educators to Likert scale questions in the introductory module

Modules

Strongly Disagree

Disagree Neither Agree nor Disagree

Agree Strongly Agree

Difference

% (n) % (n) % (n) % (n) % (n) p-values

Language was appropriate IEHP

Educators

Easy to follow IEHP

Educators

Assessment tools were relevant IEHP

Educators

Understanding my scores IEHP

Educators

Improved my IP knowledge IEHP

Educators

Improved my IP attitudes IEHP

Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0)

40 (2)

0 (0) 20.0 (1)

3.3 (1) 0 (0)

3.4 (1) 0 (0)

3.3 (1) 0 (0)

0 (0) 20.0 (1)

0 (0) 0 (0)

0 (0) 0 (0)

6.7 (2) 20.0 (1)

6.9 (2) 60.0 (3)

6.7 (2) 20.0 (1)

10.0 (3) 40.0 (2)

27.6 (8) 20.0 (1)

39.3 (11) 40.0 (2)

43.3 (13) 40.0 (2)

37.9 (11) 20.0 (1)

33.3 (10) 40.0 (2)

26.7 (8) 0 (0)

72.4 (21) 40.0 (2)

60.7 (17) 40.0 (2)

46.7 (14) 40.0 (2)

51.7 (15) 20.0 (1)

56.7 (17) 40.0 (2)

63.3 (19) 40.0 (2)

0.01*

0.15

0.47

0.03*

0.47

0.02*

* Significant differences at 95% confidence level; p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed

COMPETENCY-BASED MODULES Participants found the competency-based mini-modules very useful to learn the overview of each IP

competency. However, participants reported that some information was repetitive from the first

module.

In the mini-modules, some of the content there is already shown in the introductory one. (RN, Philippines)

We have summarized participants’ feedback on each competency-based module.

Interprofessional Conflict Management Most of the participants strongly agreed or agreed with most of the questions about the usefulness of

the modules without any significant difference between educators and IEHPs [Table 4].

18 | P a g e

Many IEHPs identified that managing and dealing with conflicts is important when working in a team-

based environment. Health care team members must trust, respect, and listen to input from each other

and resolve conflicts constructively. The module provided adequate knowledge on resolving and

avoiding conflicts among team members.

I learned how to resolve the conflict by listening to them and collectively resolving. (PT, Nigeria)

Overall, educators also agreed that the module content was very useful. One educator suggested adding

videos to test IEHPs’ understanding.

There could be more videos and within the videos the students were asked to comment on the situation. There could be some things on negative or positive conflict. Maybe add a module that has more videos of real encounters and scenarios and questions like what kind of conflict was the nurse portraying here? (Educator, PT Program)

Table 4. Comparing responses between IEHPs and educators to Likert scale questions in conflict management module

Modules

Strongly Disagree



Difference

% (n) % (n) % (n) % (n) % (n) p-values


Educators

Easy to follow IEHP

Educators


Educators


Educators


Educators


Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0)

20.0 (1)

0 (0) 0 (0)

0 (0) 0 (0)

3.6 (1) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

7.1 (2) 20.0 (1)

3.6 (1) 40.0 (2)

3.6 (1) 0 (0)

3.6 (1) 20.0 (1)

39.3 (11) 40.0 (2)

32.1 (9) 60.0 (3)

53.6 (15) 40.0 (2)

46.4 (13) 40.0 (2)

35.7 (10) 60.0 (3)

35.7 (10) 40.0 (2)

60.7 (17) 40.0 (2)

67.9 (19) 40.0 (2)

39.3 (11) 40.0 (2)

46.4 (13) 20.0 (1)

60.7 (17) 40.0 (2)

60.7 (17) 40.0 (2)

0.15

NA

0.40

0.09

0.85

0.28

*p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed

19 | P a g e

Team Functioning Three IEHPs were unsure whether the assessments related to the module were relevant [Table 5]. Both

IEHPs and educators strongly agreed with all other aspects of the module. IEHPs learned about the

variations in the functioning of different teams and what to consider when entering a new team.

Team functioning and role clarification were two interesting areas that were lacking or not properly addressed during my training or practice due to the different practice system in Nigeria. This is a huge opportunity to learn them. (Pharmacist, Nigeria)

Educators believed that many IEHPs worked in hierarchical health systems and transitioning to a new

system may require mentorship at the front line to help them with the process.

I think it is so different from the systems they may be used to. Many of my students worked in private health care centres in the Philippines where the physician hires and fires the RNs. So the nurses do not question the physician. It is different here where there is an expectation to speak up. (Educator, Nursing Program)

Table 5. Comparing responses between IEHPs and educators to Likert scale questions in Team Functioning module

Modules

Strongly Disagree



Difference

% (n) % (n) % (n) % (n) % (n) p-values


Educators

Easy to follow IEHP

Educators


Educators


Educators


Educators


Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

3.6 (1) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0)

20.0 (1)

0 (0) 20.0 (1)

11.1 (3) 0 (0)

3.6 (1) 20.0 (1)

0 (0) 0 (0)

0 (0) 20.0 (1)

42.9 (12) 40.0 (2)

39.3 (11) 40.0 (2)

48.1 (13) 60.0 (3)

46.4 (13) 40.0 (2)

50.0 (14) 60.0 (3)

39.3 (11) 40.0 (2)

57.1 (16) 40.0 (2)

60.7. (17) 40.0 (2)

40.7 (11) 40.0 (2)

46.4 (13) 40.0 (2)

50.0 (14) 40.0 (2)

60.7 (17) 40.0 (2)

0.15

0.15

0.59

0.40

NA

0.15

Note: p values were calculated using Fisher's Exact Test;NA where all participants agreed or strongly agreed

20 | P a g e

Interprofessional Communication IEHPs were more likely to agree that the IP communication module improved their attitude towards IP

practice than educators [Table 6]. We found no significant difference between educators and IEHPs for

the other aspects of the IP communication module.

This module helped me to understand that different people from different cultures communicate in different ways. Therefore I ought to exercise patience and respect especially when working with multi generations. It also helped to learn about SBAR and its use to unify and simplify communication. (RN, Nigeria)

Participants identified an error in this module; the low context and high context pictures in the scenario

were interchanged.

Table 6. Comparing responses between IEHPs and educators to Likert scale questions in Interprofessional Communication module

Modules

Strongly Disagree



Difference

% (n) % (n) % (n) % (n) % (n) p-values


Educators

Easy to follow IEHP

Educators


Educators


Educators


Educators


Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

3.6 (1) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

7.1 (2) 20.0 (1)

10.7 (3) 20.0 (1)

0 (0) 20.0 (1)

0 (0) 40.0 (2)

42.9 (12) 60.0 (3)

42.3 (11) 40.0 (2)

50.0 (14) 40.0 (2)

32.1 (9) 40.0 (2)

35.0 (10) 40.0 (2)

35.7 (10) 20.0 (1)

57.1 (16) 40.0 (2)

57.9 (15) 60.0 (3)

42.9 (12) 40.0 (2)

53.6 (15) 40.0 (2)

63.0 (17) 40.0 (2)

64.3 (18) 40.0 (2)

NA

NA

0.40

0.58

0.16

0.01*


21 | P a g e

Patient-Centred Care The module highlighted the need for collecting and sharing information among the team for a better

patient care. Participants found the module very helpful in understanding patients’ needs, such as,

including patients in setting up care goals and advocating for patients.

I liked the module on patient centred care. I came from a medical system where we make goals for patients and don’t include the patients in making goals; we just assess and provide goals. (OT, Philippines)

Table 7 shows that all participants, both IEHPs and educators, either agreed or strongly agreed that the

language and flow of the module were appropriate.

Table 7. Comparing responses between IEHPs and educators to Likert scale questions in Patient-Centred Care module

Modules

Strongly Disagree



Difference

% (n) % (n) % (n) % (n) % (n) p-values


Educators

Easy to follow IEHP

Educators


Educators


Educators


Educators


Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

3.6 (1) 0 (0)

3.6 (1) 0 (0)

3.6 (1) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

3.7 (1) 0 (0)

7.1 (2) 40.0 (2)

0 (0) 0 (0)

0 (0) 20.0 (1)

39.3 (11) 60.0 (3)

39.3 (11) 60.0 (3)

51.9 (14) 60.0 (3)

39.3 (11) 20.0 (1)

39.3 (11) 40.0 (2)

35.7 (10) 40.0 (2)

60.7 (17) 40.0 (2)

60.7 (17) 40.0 (2)

44.4 (12) 40.0 (2)

50.0 (14) 40.0 (2)

57.1 (16) 60.0 (3)

60.7 (17) 40.0 (2)

NA

NA

0.84

0.15

0.85

0.28

Note: p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed

22 | P a g e

Collaborative Leadership This was the most interesting module as this was a new concept for many IEHPs. They learned that

everyone on the team could take the leadership role depending on the circumstances.

The strategies to be an effective leader were exceptionally helpful in preparation to be a potential leader of the interprofessional team. Because many times, an individual may lack the confidence or unsure how to lead the team effectively. The leader or 'expert' will change according to the present needs and goals of the patient. (OT, Singapore)

Many IEHPs reported that they worked in a hierarchical system, so it was useful to learn the concept of

shared leadership.

The most interesting part is the collaborative leadership. I found it fascinating that the true leaders are actually the patient and any health care professional who has the best relationship with the patient. This removes hierarchies between health care professions. The new aspect for me is the discussion on conflict management. (Pharmacist, Nigeria)

Physician IEHPs also found it useful to know that leadership in healthcare is not limited to physicians

only in the Canadian health care system.

The module explains that in some areas you can be a leader but in another area, other professional can take a leadership role. This module changed my mind. (Physician, Iran)

IEHPs also said that the module offered practical strategies, and this would allow them to more openly

share and communicate.

This module helped me understand and look at all health care professionals equally; thus help me become more proactive and confident in sharing what I know that will make the team goals successful. (OT, Philippines)

Both IEHPs and educators strongly agreed or agreed with all aspects of module content [Table 8].

Educators also believed that the content would help IEHPs become familiar with the Canadian

healthcare system.

This module will help to guide IEHP's in the direction of shared leadership and provide them with a better idea of what direction the Canadian Health Care system is going (Educator, Pharmacy Program)

One educator suggested presenting a difficult situation to support better learning of IEHPs.

A less confident IEHP can be shown who may have difficulty understanding local jargon or phrases, from a high context system and professional hierarchies. (Educator, IMG Program)

23 | P a g e

Table 8. Comparing responses between IEHPs and educators to Likert scale questions in Collaborative Leadership module

Modules

Strongly Disagree



Difference

% (n) % (n) % (n) % (n) % (n) p-values


Educators

Easy to follow IEHP

Educators


Educators


Educators


Educators


Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

3.6 (1) 0 (0)

3.6 (1) 0 (0)

3.6 (1) 0 (0)

0 (0) 0 (0)

0 (0) 20.0 (1)

7.1 (2) 20.0 (1)

7.1 (2) 40.0 (2)

3.6 (1) 0 (0)

3.6 (1) 20.0 (1)

42.9 (12) 60.0 (3)

32.1 (9) 40.0 (2)

50.0 (14) 40.0 (2)

33.3 (9) 20.0 (1)

39.3 (11) 40.0 (2)

39.3 (11) 40.0 (2)

57.1 (16) 40.0 (2)

67.9 (19) 40.0 (2)

42.9 (12) 40.0 (2)

55.6 (15) 40.0 (2)

53.6 (15) 60.0 (3)

53.6 (15) 40.0 (2)

NA

0.15

0.40

0.16

0.72

0.40

Note p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed

Role Clarification The participants learned the importance of understanding the roles, scopes of practice, and

competencies of other professionals in the team.

I learned how to approach other health care professionals and asking their scope because others could misinterpret it when asked about their scope/responsibility. (RN, Philippines)

I will be able to better work in a team knowing my role and their role hence avoiding any duplication. (PT, India)

Educators also reported that the module was helpful for IEHPs to learn more about the role of health

care professionals working in a team.

This module will help them to understand that there are a number of people involved in patient care and hopefully encourage them to determine the roles of those they will work with (Educator, Pharmacy Program)

24 | P a g e

All IEHPs as well as educators agreed or strongly agreed with the language appropriateness and flow of

the role clarification module [Table 9]. All participants also agreed or strongly agreed that the content

improved their knowledge about IP collaborative practice. However, educators were significantly less

likely to agree that understanding assessment scores in the module would improve how an IEHP would

work with other professionals.

Table 9. Comparing responses between IEHPs and educators to Likert scale questions in Role Clarification Care module

Modules

Strongly Disagree



Difference

% (n) % (n) % (n) % (n) % (n) p-values


Educators

Easy to follow IEHP

Educators


Educators


Educators


Educators


Educators

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

0 (0) 0 (0)

3.7 (1) 20.0 (1)

0 (0) 40.0 (2)

0 (0) 0 (0)

0 (0) 20.0 (1)

42.9 (12) 60.0 (3)

37.0 (10) 60.0 (3)

48.1 (13) 40.0 (2)

46.4 (13) 20.0 (1)

42.9 (12) 20.0 (2)

42.9 (12) 20.0 (1)

57.1 (16) 40.0 (2)

63.0 (17) 40.0 (2)

48.1 (13) 40.0 (2)

53.6 (15) 40.0 (2)

57.1 (16) 60.0 (3)

57.1 (16) 60.0 (3)

NA

NA

0.29

0.02*

NA

0.15


TIME TAKEN TO REVIEW ICT FOR IEHPS The median time to complete each competency-based mini-module was 15-20 minutes without any

significant difference between IEHPs and educators, which aligns with our estimates in the module

development process [Table 10]. The average time was not an appropriate measure due to some

extreme values in the data set.

25 | P a g e

The median time to complete the introductory module was 1 hour for IEHPs and 2 hours for educators,

which was longer than what we aimed for during the module development.

LANGUAGE The majority of IEHPs said that the language in the modules was fairly easy to understand.

I understand it perfectly fine. It was just right. I find I learned it faster because they were to the point. (RN, Philippines)

Some IEHP participants had difficulty in following videos.

When the person is trying to explain something, for example they are speaking about function of a team, somehow it is very fast, and I need to come back or maybe need to concentrate more. (Physician, Iran)

The language is appropriate but the tone of speaking is a little bit fast. (Pharmacist, Egypt)

Educators also perceived that the language in the modules is the appropriate level for internationally

educated providers to understand. However, educators were less sure about whether or not the written

text would be too difficult.

I think the verbal language the people can understand okay. Some of the written text, I am not sure about. There will be some examples where text was required to type some of the concepts. I saw that was complicated. (Educator, IMG Program)

CULTURAL ASPECTS IEHPs reported that the content captured a range of cultural variation to understand and be respectful

of multiculturalism. IEHPs were aware that Canada has many diverse cultures so examples from each

Table 10. Comparing time spent to complete each ICT for IEHPs module among IEHPs (n=28) and educators (n=4)

Modules

Reported time to complete each module

(in minutes)

IEHPs

Reported time to complete each module

(in minutes)

Educators

Median (IQR) Median (IQR)

Introductory module 60.0 (30 – 180) 120.0 (98 – 143)

Conflict management 20.0 (15 – 30) 15.0 (13 – 50)

Team functioning 20.0 (15 – 30) 15.0 (15 – 26)

Communication 21.0 (15 – 30) 17.5 (11 – 28)

Patient-centred care 21.5 (15 – 30) 17.5 (11 – 28)

Collaborative leadership 20.0 (16 – 30) 15.0 (15 – 26)

Role clarification 20.0 (15 – 30) 15.0 (15 – 15)

IQR=interquartile range

26 | P a g e

could not be included. However, some IEHPs had suggestions to improve the cultural aspect of ICT for

IEHPs.

You need to capture more about aboriginal culture. Sometimes you need to know more about their culture, the way they are thinking, the way we need to address their health issues. (Physician, Iran)

The module did not address when is the time to quit, when is the time to withdraw, when is the time to say ok this is against my religion. (Pharmacist, Nigeria)

Working in the Middle East for many years, there are also many cultural nuances that need to be mentioned (e.g., limiting physical and eye contact, handshakes with those of the opposite sex). (RN, South Africa)

Educators reported that the modules described various health care providers from other

countries, which should resonate with IEHPs. Educators also mentioned that IEHPs often are not aware

of diversity and inclusiveness in care and how to interact with patients or providers from various

cultures. When dealing with different cultures, a shift in communication and practice may be required to

effectively care for certain populations. For many, being culturally sensitive is not something they have

been asked to do in previous health care environments.

I think being aware is the first part. I think the modules made them more aware of other cultures. Some of the audio described different healthcare providers from other countries. (Educator, Nursing Program)

One educator believed that transitioning to a collaborative patient-centred health system versus a

hierarchical physician driven system may cause uncertainty for some IEHPs. IEHPs may have a lot of

questions when trying to navigate this new environment, which may cause them stress. Recognizing or

acknowledging that cultural conflicts may be difficult to resolve was not addressed in the modules.

My experience has been that the international physicians… causes a lot of stress and uncertainty and I am not sure that that is addressed that it is okay and normal. The piece missing is that the two characters were confident and resolved their problems easily, but in reality the cultural conflicts transitioned here aren’t that easily addressed and it causes a lot of people stress. (Educator, IMG Program)

SCENARIOS Overall, participants perceived that the scenarios presented in the modules were very useful and most

of the IEHPs were able to relate their on situations to them. The scenarios made the contextual

information more practical.

I can relate well with some of them because being IEHP; I can compare where I worked before coming to Canada. (RN, Nigeria)

The scenarios are eye openers and relevant for my practice. I believe the colleges of each discipline can present specific case studies relevant to their jurisdiction's practice. (RN, Philippines)

27 | P a g e

However, some IEHPs, particularly pharmacy IEHPs, mentioned that the module should have some

scenarios of health care professionals working in a community setting.

Pharmacists are mostly working outside the hospital and all scenarios in the modules are hospital-based. 99% of internationally educated pharmacists are working in community pharmacy. (Pharmacist, Egypt)

Most of these scenarios are hospital-based while we work in community pharmacy. (Russia, Pharmacist)

Some of the scenarios can be based on a community-based setting type of health care team, whereby there is a good mix of regulated as well as non-regulated professions (e.g. housing workers, program planners) in the team. (PT, Nigeria)

Another participant mentioned that labeling the health care professional in scenarios would help them

understanding these scenarios.

During the scenarios, it was not clear which professional was talking. (RN, Nigeria)

ASSESSMENTS Most IEHPs found that the assessments were helpful in their understanding of the modules. Many

appreciated an opportunity to think and reflect about experiences. The reflections offered an

opportunity for IEHPs to think about applying the competencies to practical situations. However, some

IEHPs said that the assessments could be improved and offered the following suggestions:

Objectively measure competency learnings instead of subjective assessment

Prompt IEHPs to identify errors or less satisfactory collaboration practices

Measure learning only through multiple choice options as some IEHPs find writing a challenge

Use a different response rating as it may not be accurate for IEHPs to say they “strongly agree”

May be it is not accurate to say I strongly agree. For some parts you can give 1 – 10 rating instead. (Physician, Iran)

May be some of the questions you can add like if IEHPs really understood it (module content). Some sort of question that make you aware what they learned in the module. (OT, Philippines)

It will be nice to have some short quizzes, not just reflections. (LPN, Philippines)

Educators also suggested adding some objective assessment to assess competencies.

I would have them do assignments that are marked in the course or activities with which they are given feedback and maybe a quiz to see how they demonstrate it. (Educator, PT Program)

Participants identified that one question appeared twice in both pre- and post-assessment tools under

the role clarification domain.

28 | P a g e

LICENSING EXAM IEHPs had mixed opinions whether the content provided in the modules was relevant to the licensing

exams for their professions. Some believed that the content is useful for objective structured clinical

examination (OSCE) exams while others believed that the content in these modules may not be directly

relevant to the licensing exam.

It will particularly help in OSCE exam, we have real scenarios. So this module is very important for people going for OSCE exam to know how to collaborate with other health professionals. It will not help much in MCQs. (Pharmacist, Egypt)

If I have to take the exam on OT then I will just focus on subject. This module is good for practice but not relevant to licensing exam. (OT, Philippines)

Our license exam is more about subject knowledge. I don’t know if this is relevant to licensing exam. (RN, Nigeria)

TECHNICAL ISSUES The majority of participants did not report technical issues with the online modules. Nine participants

(seven IEHPs and two educators) reported one or more technical issues when reviewing the modules.

Four participants reported having trouble with sign on after creating the login account. The modules will

also need some adjustments to address other issues such as confusing forward arrows on slides and

sound issues with some videos. The summary of other key technical issues is provided in Box 1.

Box 1 Key technical issues mentioned by the survey participants

1. Slow buffering and it won’t move forward sometimes 2. Figuring out how to do the 'pop ups'. Arrows on the screen shot were confusing. 3. Sometimes there was not forward arrow which meant I had to go back to the mouse. Is it

possible to have either all forward arrow buttons on each slide or be able to use the arrow keys on the keyboard rather than the mouse

4. Kept getting a pop up stating that the internet connection was not secure and kept restarting

5. I had to use internet explorer because the video were not uploading using google chrome 6. I cannot use my IPAD because it does not have a Java software to run the program 7. My system was slowed down by the scorn software package even when I used the

recommended browser. Some of the videos did not have sound and the software sometimes refreshed by itself and I lost some of my answers to the quiz. These were very stressful to me and time consuming. It took me about 12 hours to complete the entire course.

29 | P a g e

CONCLUSIONS

The ICT for IEHPs is a very useful resource and IEHPs across different health care professions in

Canada may benefit from this resource. The ICT for IEHPs modules enhance the knowledge and

understanding of IEHPs of interprofessional practice. The content of these modules is also useful for

some professional licensing exams that involve an OSCE, such as pharmacy and medicine.

The use of terminology, contexts to practice, and varied cultural backgrounds in scenarios made

the modules relatable to IEHPs. The language in the modules seems to be appropriate for the most part.

The content captures a range of cultural variation to understand and be respectful of multiculturalism.

Further scenarios can be added to demonstrate what is acceptable and what is not acceptable in a

multicultural environment. IEHPs may require additional training on cultural competencies to apply to

both patients (e.g., First Nation’s populations) and healthcare providers.

The pre- and post-assessment tools and reflective questions are embedded in the introductory

and competency-based modules to measure their effectiveness. Although reflection exercises are

subjective and have no right or wrong answers, the questions prompt IEHPs to apply their learnings to

similar situations.

Overall, the competency-based mini modules were considered very useful and appropriate and

do not need any major change. The introductory module needs to be shortened.

Currently, the modules are available on a temporary host website. We also need to address all technical

issues (e.g., slow buffering) prior to launching them on a large-scale.

LIMITATIONS

We identified the following limitations of this evaluation:

Respondent burden was high so the response rate was low, particular from educators. There

were also variations in the response rate among provinces and professional groups.

We were only able to measure knowledge and attitude of the participants; we could not

measure the actual impact on clinical practice which would need a long-term follow up.

Another important limitation was the small number of participants, particularly the number of

educators. Our sample was limited so our findings may not have captured the true

representation of IEHPs in Western Canada.

Some potential participants who initially showed interest in the study were not able to

participate due to very tight timelines.

30 | P a g e

RECOMMENDATIONS

Based on our evaluation, we identified some areas for further improvements of the ICT for IEHPs

modules prior to the final launch.

Reduce the length of the introductory module

Clarify the patient safety scenario in the introductory module to ensure the scenario is linked to

IP collaborative practice

Add more objective assessment methods such as quizzes and multiple choice questions

Delete repeated questions in pre and post assessment tool under role clarification domain

Label the profession of healthcare providers in scenarios

Consider adding more scenarios where IEHPs communicate respectfully when dealing with

cultural issues and sensitive patient information or situations

Consider adding some scenarios of interprofessional teams working outside the hospital

environment

Consider adding videos on the examples of negative and positive conflicts

Make correction in low context and high context pictures in the communication module; the

pictures are interchanged

Address all technical issues before the final launch of ICT for IEHP

We also recommend future evaluation of ICT for IEHPs with a larger sample and increased

representation from different healthcare professions.

31 | P a g e

REFERENCES

1. A National Interprofessional Competency Framework. (2010). Canadian Interprofessional Health

Collaborative, College of Health Disciplines, University of British Columbia. URL

http://www.cihc.ca/files/CIHC_IPCompetencies_Feb1210.pdf (accessed on 02 Sep 2015)

2. IPC on the Run modules, College of Health Disciplines, University of British Columbia. URL

http://www.ipcontherun.ca (accessed on 02 Sep 2015).

http://www.cihc.ca/files/CIHC_IPCompetencies_Feb1210.pdf

http://www.ipcontherun.ca/

32 | P a g e

APPENDIX I. IEHP PRE AND POST ASSESSMENT TOOL

After completing the interprofessional competency-based modules for internationally educated health care providers, think about how confident you feel NOW to enact the competencies for collaborative practice in your future practice. Enter your first name here This will be used to track your responses, which will be anonymized prior to analysis.

I. Role Clarification Understanding your own role and the roles of those in other professions, and using this knowledge appropriately to establish and achieve patient/client/family and community goals. Not

Confident Somewhat Confident

Confident Very Confident

Describe own role and that of others Recognize and respect the diversity of other health and social care roles, responsibilities, and competencies

Perform own roles in a culturally respectful way Communicate roles, knowledge, skills, and attitudes using appropriate language

Consider the roles of others in determining own professional and interprofessional roles

Access others’ skills and knowledge appropriately through consultation

Consider the roles of other in determining own professional and interprofessional roles

Integrate competencies/roles seamlessly into models of service delivery

II. Patient/Client/Family/Community-Centred Care Seek out, integrate and value, as a partner, the input, and the engagement of the patient/client/family/community in designing and implementing care/services. Not



Support the participation of patients/clients, their families, and/or community representatives as integral partners alongside with health care personnel

Share information with patients/clients, (or family and community)in a respectful manner and in such a way that it is understandable, encourages discussion, and enhances participation in decision-making

Ensure that appropriate education and support is

33 | P a g e

provided to patients/clients, family members and others involved with care or service Listen respectively to the expressed needs of all parties in shaping and delivering care or services

III. Team Functioning Understand the principles of team work dynamics and group/team processes to enable effective interprofessional collaboration. Not



Understand the process of team development Develop a set of principles for working together that respects the ethical values of members

Effectively facilitate discussions and interactions among team members

Participate, and be respectful of all members’ participation, in collaborative decision-making

Regularly reflect on functioning with team learners/practitioners and patients/clients/families

Establish and maintain effective and healthy working relationships with learners/practitioners, patients/clients, and families, whether or not a formalized team exists

Respect team ethics, including confidentiality, resource allocation, and professionalism

IV. Collaborative Leadership Understand and can apply leadership principles that support a collaborative practice model. Not



Work with others to enable effective patient/client outcomes

Advance interdependent working relationships among all participants

Facilitate effective team processes Facilitate effective decision-making Establish a climate for collaborative practice among all participants

Co-create a climate for shared leadership and collaborative practice

Apply collaborative decision-making principles Integrate the principles of continuous quality improvement to work processes and outcomes

34 | P a g e

V. Interprofessional Communication Communicate with others in a collaborative, responsive and responsible manner. Not



Establish team work communication principles Actively listen to other team members including patients/clients/families

Communicate to ensure common understanding of care decisions

Develop trusting relationships with patients/clients/families and other team members

Effectively use information and communication technology to improve interprofessional patient/client/community-centred care

VI. Interprofessional Conflict Management Actively engage self and others, including the client/patient/family, in positively and constructively addressing disagreements as they arise. Not



Value the potential positive nature of conflict Recognize the potential for conflict to occur and taking constructive steps to address it

Identify common situations that are likely to lead to disagreements or conflicts, including role ambiguity, power gradients, and differences in goals

Know and understand strategies to deal with conflict

Set guidelines for addressing disagreements Effectively work to address and resolve disagreements, including analyzing the causes of conflict and working to reach an acceptable solution

Establish a safe environment in which to express diverse opinions

Develop a level of consensus among those with differing views; allowing all members to feel their viewpoints have been heard no matter what the outcome

Thank You! Hopefully this survey served as a valuable reflection on where you currently stand with regards to your collaborative competencies and how you will practice in the future.

35 | P a g e

APPENDIX II. SURVEY FOR IEHPS

36 | P a g e

37 | P a g e

38 | P a g e

39 | P a g e

40 | P a g e

41 | P a g e

42 | P a g e

43 | P a g e

44 | P a g e

45 | P a g e

46 | P a g e

APPENDIX III. INTERVIEW GUIDES IEHPs

Project title: Interprofessional (Practice) Competency Tools for Internationally Educated Health Professionals

This interview will take about 30 minutes

1. How useful were these modules in improving your knowledge about working with

health care professionals from other disciplines?

2. How well did you relate to the scenarios/situations shown in the modules?

3. How easy to understand was the language in the modules?

4. What content in the modules is most important or meaningful for you and why?

5. What content in the module is not important to you and why?

6. Of the two modules that you reviewed which is more important and why?

7. Of the six mini-modules in the Competency-Based Module which are more important and why?

8. How useful did you find the assessments in the module?

9. How can the pre and post assessments in these modules be improved?

10. What content appears to be missing from each module?

11. What cultural aspects were not covered in each module?

12. How can we make the components of the module culturally respectful and sensitive?

13. How useful do you think the module is useful for you in your preparation for licensure?

14. Do you like the current format/style of teaching the module or do you want to see a different format/content style?

If you are interested of receiving a report of evaluation results, may I have your email address please?

Thank you for your participation

47 | P a g e

Educators

Project title: Interprofessional (Practice) Competency Tools for Internationally Educated Health Professionals

This interview will take about 30 minutes

1. How useful do you believe these modules will be in improving the IEHPs

knowledge about interprofessional collaboration?

2. How well do you believe the IEHPs will relate to the scenarios/situations illustrated in the modules?

3. How easy do you think the language in the modules will be for IEHPs to understand?

4. What content in the modules do you believe is most important or meaningful for IEHPs and why?

5. What content in the module do you believe is not important for IEHPs and why?

6. Of the two modules that you reviewed which is more important and why?

7. Of the six mini-modules in the Competency-Based Module that you reviewed, which are

more important and why?

8. How useful did you find the assessments in the module?

9. How can the pre and post assessments in these modules be improved?

10. What content appears to be missing from each module?

11. What cultural aspects were not covered in each module?

12. How can we make the components of the module culturally respectful and sensitive?

13. Do you think the modules will be useful to IEHPs in their preparation for licensure?

14. Do you like the current format/style of teaching the module or do you want to see a different format/content style?

If you are interested of receiving a report of evaluation results, may I have your email address please?

Thank you for your participation

48 | P a g e

APPENDIX IV: REFLECTIVE QUESTIONS Introductory module

Interprofessional Collaboration

1. Describe your current understanding of interprofessional collaborative practice. You will have

the opportunity to revisit your description at the end of the module to see how your

understanding has changed.

2. What percentage of medical errors are due to communication and teamwork issues?

3. Communication failures are the leading cause of inadvertent patient harm.

4. Ineffective communication across professions results in medical errors.

5. What made this team meeting effective?

6. What might effective collaboration look like in the context in which you work?

7. What are some of the challenges to interprofessional collaboration?

8. Your goal for future learning.

Competency-based module

Conflict Management/Resolution

9. What happened?

10. How does an appreciation of other personalities cultures, generations, and, professions impact

the way you will collaborate?

Team Functioning

11. What happened?

12. How does an understanding of how interprofessional collaborative teams function in different

contexts impact the way you will collaborate?


13. What happened?

14. How does an appreciation of other personalities, cultures, generations, and professions impact

the way you will collaborate?

Patient Centred Care

15. What happened?

16. In previous practice, how have you involved patients and their families in their care?

17. How does appreciation of the values and principles of interprofessional patient- centred care

within the Canadian context impact the way you will collaborate?


18. What happened?

19. How does your knowledge about collaborative leadership impact the way you will collaborate?

Role Clarification

20. What happened?

21. How does respect for the diversity of other health and social care roles responsibilities and

Competencies impact the way you will collaborate?

49 | P a g e

APPENDIX V: ETHICS EXEMPTION FROM UNIVERSITY OF SASKATCHEWAN

Date post:	01-Feb-2018
Category:	Documents
Upload:	vuongthuan
View:	225 times
Download:	0 times

Evaluation of Interprofessional Competency (Practice ... · PDF fileEvaluation of...

Documents