Applying Best Practices in Residency Selection :

A Quality Improvement Project for the Dalhousie Internal Medicine Program

Authors: Alison Rodger, Marko Balan, Ian Epstein

Date: September 30, 2016

Delete this box before presenting: Four title slide options have been provided with different feature

images, select one for use in your presentation.

I do not have an affiliation (financial or otherwise) with a

pharmaceutical, medical device or communications organization.

Je n’ai aucune affiliation (financière ou autre) avec une

entreprise pharmaceutique, un fabricant d’appareils

médicaux ou un cabinet de communication.

2

Delete this box before presenting: Please keep only the applicable statement, shown above is Option 1

with Option 2 to follow on next slide.

Note: Speakers who have no involvement with industry should inform the audience that they cannot identify any conflict of interest.

Supprimez cette boîte avant de faire votre présentation : Veuillez conserver uniquement l’énoncé pertinent; ci-dessus se trouve l’option 1, l’option 2 paraissant à la diapo suivante. Note: Les conférenciers qui n’ont aucun lien avec l’industrie doivent aviser leur auditoire qu’ils n’ont aucun conflit d’intérêts à déclarer.

Summary

• Transparency, knowledge translation and applicant ranking were the

main areas of weakness identified in this review.

• This project lead to the development of program-specific documents

and protocols for the applicant selection process.

• Controversy exists regarding how to best manage conflict of interests

and whether to use information other than that contained in application

files.

3

Background

• Significant variation exists between programs for applicant selection

for postgraduate residency programs

• Ensuring efficacy and standardization in resident selection is

challenging

• Increasing emphasis has been placed on the responsibility of medical

faculties in training a diverse group of physicians, with the right

composition of generalists and specialists who are willing to serve in

underserviced areas1,2

• Best practice recommendations have been developed by a working

group at the University of Toronto and have been proposed to be used

by programs across Canada1

4

Purpose and Methods

• The purpose of this study was to evaluate the residency selection

process based on the twenty-four best practice recommendations as

outlined by the Best Practices in Application and Selection Final

Report (BPAS) 1

• Two of the study authors conducted document review and interviews

with the program administrators and the Program Director

• Adherence to each best practice recommendation was determined by

consensus agreement between both authors

• The authors of the study identified and completed action items specific

to the program’s deficiencies

5

Dalhousie Internal Medicine Application and Selection Process

6

Results

7

Transparency Adherence

1. Define selection goals Partially

2. Define where relevant attributes assessed No

3. State processes and metrics to candidates Partially

4. Maintain records of adherence Yes

5. If using information outside application, must be consistent, fair, transparent

No

6. If using such information, must have process to investigate and validate

No

7. Specific practice for retention and protection of records Yes *Table developed using Best Practices in Applications and Selection Final Report 1

Results

8

Fairness Adherence

8. Assess each component independently Partially

9. Abide by guidelines for management of conflict of interest

Partially

Selection Criteria

10. Program specific criteria for assessment Yes

11. Criteria specific and validated for specialty Partially

Process

12. Assessment systems must be standardized Partially

13. Assessments based on skills or previous behaviours Yes

14. Assessments based on multiple independent samples Yes

15. Regularly assess outcomes of process Partially *Table developed using Best Practices in Applications and Selection Final Report 1

Results

9

Assessors Adherence

16. Selection team with breadth of perspectives Yes

17. Assessors trained in all aspects of the process Partially

Assessment Instruments

18. Strive to incorporate objective assessments Yes

Knowledge Translation

19. Share best practices among programs No

20. Innovations in process done in scholarly manner No

Ranking

21. Process to receive information alleging improper behaviours

Partially

22. Clear criteria for ‘do not rank’ No

23. Rank candidates based on assessment Partially

24. Rank using pre-defined, transparent process Partially *Table developed using Best Practices in Applications and Selection Final Report 1

Discussion

• Areas of weakness were in transparency, applicant ranking and

knowledge translation

• Applicant selection practices specific for the Dalhousie Internal

Medicine Program were updated and documented reflecting best

practices as per BPAS 1

Program goals were revised and selection process goals were developed

File review and interview templates were updated to better reflect desired

candidate attributes and selection goals

File reviewer and interviewer guides created

• Changes to the current interviewer structure were not deemed

necessary

10

Discussion

• Goals and templates approved by Residency Program Committee

• Dalhousie Legal concerns regarding wording of proposed conflict of

interest policy

• Dalhousie Legal suggests omitting information obtained outside of an

applicant’s file

• CaRMS website for 2017 application cycle updated to reflect new

Dalhousie IM selection practices as per BPAS1

11

Summary

• Transparency, knowledge translation and applicant ranking were the

main areas of weakness identified in this review.

• Undergoing this project led to the development of program-specific

documents and protocols for the applicant selection process that will

improve the overall process.

• Controversy exists regarding how to best manage conflict of interests

and whether to use information other than that contained in application

files.

12

References

• 1. Bandiera et al. 2013. Postgraduate Medical Education University of

Toronto, 2013.

• 2. The Future of Medical Education in Canada: A Collective Vision for

Postgraduate Medical Education in Canada. 2012.

• 3. Maksidi G et al. 2011. Journal of Surgical Education 2011; 68(1):

67-72.

• 4. Provan JL et al 1995. Canadian Medical Association Journal. 1995;

153(7): 919-923.

• 5. Stephenson-Famy A et al 2015. Journal of Graduate Medical

Education. 2015; 7(4): 539-48.

13

• Download the ICRE App,

• Visit the evaluation area in the Main Lobby, near Registration, or

• Go to: http://www.royalcollege.ca/icre-

evaluations to complete the session evaluation.

Help us improve. Your input matters.

• Téléchargez l’application de la CIFR

• Visitez la zone d’évaluation dans le hall principal, près du comptoir d’inscription, ou

• Visitez le http://www.collegeroyal.ca/evaluations-

cifr afin de remplir une évaluation de la séance.

Aidez-nous à nous améliorer. Votre opinion compte!

You could be entered to win 1 of 3 $100 gift cards.

Vous courrez la chance de gagner l’un des trois chèques-cadeaux d’une valeur de 100.

14