Residency Director’s Forum: Applicant Pools - Resident Selection ProcessR. Tim Yoho, DPM, MS, FACFASProfessor and Dean, College of Podiatric Medicine & Surgery Des Moines University
The Education & Training for Health Care Transformation Conference
The Ten Pillars Shaping the Future of Health Science Education:
External Forces, Ongoing Evolution and Future Trends
Jeffrey P. Gold, MDChancellor & Executive VP Health Affairs
Executive Dean, College of Medicine & Life Sciences
The University of Toledo
The Future of Health Science Education
Curriculums will become more learner and learning focused.
Students will be assessed by individual competencies and the outcomes of their learning.
Inclusion of inter-professional components.
Integration of simulation and virtual immersive reality.
Academic health centers as learning and teaching hubs.
Standardization of knowledge, skill and professionalism competencies.
Heightened emphasis on team-based care assessments.
Increased accountability for healthcare outcomes, need for lifelong learning skills.
A more diverse and international healthcare workforce.
Focus on healthcare delivery to ensure the sustainability of the field..
Competency and outcome based assessment strategies will not link the educational programs to time of study, but will link them to their individual competencies and to the outcome of their learning.
The ability to perform in the domains of knowledge and skill, as well as professionalism and communication, will be increasingly measured on an individual basis and the chronological passage through the curriculum well be adjusted accordingly.
Just a individuals learn at different paces, the best way to swing a golf club or acquire a foreign language , others learn the health professions at different paces.
680 680 680 680 680 680 680 680 690
1.31.4
1.5 1.5 1.5
1.8
1.3 1.3
0
0.5
1
1.5
2
2.5
3
3.5
4
0
200
400
600
800
1000
1200
1400
2014 2015 2016 2017 2018 2019 2020 2021 2022
Total Apps Total Enr Seats Ratio A/S
Applicant/Enrollment Data for Podiatric Medical Schools
2.8
2.9
3.0
3.1
3.2
3.3
3.4
0.0
5.0
10.0
15.0
20.0
25.0
2014 2015 2016 2017 2018 2019 2020 2021
APP MCAT Mat MCAT App SGPA Mat SGPA
APMLE 1 First Time Pass Rate 2014-2019: 85%
Four-Year Graduation Rate 2014-2018: 80-85%
MCAT Score 2019-2021 Scaled to Old System
Applicant/Enrollment Academic Data for Podiatric Medical Schools
Residency Application Data 2014-2018
• Number of applicants per program• Mean: 79.5 Range: 72.5 - 85.4
• Number of programs per applicant• Mean: 28 Range: 27 – 30
• What does this tell us?
Importance Ratings by Program DirectorsHarfmann KL, Zirwas MJ. Can performance in medical school predict performance in residency? A compilation of correlative studies. J Am Acad Dermatol, 2011;65: 1010-22.
• Top Five• Interpersonal skills during the
interview• Interactions with faculty
during the interview• Professional attributes
exhibited during the interview• Perceived commitment to
specialty• Clerkship performance
• Bottom Five• Personal statement• Graduate of highly regarded
medical school• Honors membership• Demonstrated involvement in
research• Interest in academic career
Resident Selection CriteriaShofler D. Selecting residents in podiatric medicine and surgery. Jr Foot and Ankle Surgery, 2015;54:565-76.
• Top Five RD’s• Previous disciplinary action• Number of classes failed• Performance during clerkship
or visit• Opinions of other attending
physicians and residents• Number of Part I attempts
• Top 5 Students• Performance during clerkship
or visit• Opinions of other attending
physicians and residents• Performance during social
portion of the interview• Previous disciplinary actions• Performance during skills
portion of the interview
Resident Selection Criteria(Shofler)
• Bottom Five• Reputation of podiatric medical
school• Extracurricular non-academic
activities• Student undergraduate
experiences• Letters of recommendation• Communications with dean of
the podiatric medical school
• Bottom Five-Students• Student undergraduate
experience• Communication with the dean
of the podiatric medical school• Reputation of podiatric medical
school• Involvement in research• Extracurricular non-academic
activities
What Directors Value
• Social appropriateness/emotional maturity• Self and situational awareness
• Teachable• Motivated/Growth Mindset/Work Ethic• Team Player• Integrity• Humility• Likeability• Balance of Knowledge and Social Skills
Performance on USMLE Steps 1 and 2(Harfmann)
• High performance on examinations at one level likely predicts high performance on examinations at future levels.
• Correlation of USMLE to noncognitive performance measures are unclear and unpredictable.
Performance in Preclinical Coursework(Harfmann study)
• Lower performance on any academic marker was associated with an increased risk of poor knowledge and professionalism ratings.
• Correlations between resident performance and preclinical GPAs from different institutions are unlikely to be useful.
• Preclinical performance from a single institution is consistently measured and is more likely to be a valid tool for comparison of students from that institution.
Clinical Performance(Harfmann study)
• Clerkship specific performance failed to find a strong correlation between clinical performance and future noncognitive performance measures.
• Identified a positive correlation between overall medical student performance and future noncognitive performance.
• Can be used to predict future cognitive performance.
Clinical PerformanceYoho RM, Tallerico V, Vardaxis V. Relationship between student academic and clinical performance in podiatric medical education at Des Moines University. 2012; 102:4, 314-18
• Single institution study• N=202• Strong correlation between academic performance in the first-year to
the second-year• Third-year clinical training consistent for all students• Professionalism objective scored (noncognitive dominant)• A significant positive correlation between academic achievement and
clinical performance for pooled data.
Letters of Recommendation
• Can be used to predict future noncognitive performance if evaluation is standardized and objectified. Strong correlation. (Harfmann)
• The development of a standardized letter of recommendation in podiatric medicine and surgery might be appropriate to improve this aspect of the residency application. (Shofler)
Interview (Harfmann)
• Interviewers blinded to cognitive application data to more accurately assess noncognitive traits. Very strong.
• Can be used to predict future noncognitive performance. Very strong
• Orthopedic programs characterized the residency interview as “getting to know the applicant.”
• Appears to be a key factor in selecting the best fit. • Interview should be “primarily social.” (Shofler)
Performance Rating of Current Residents
• 8% different levels of dissatisfied.• 12% Somewhat satisfied• 33% Satisfied• 47% Very satisfied
• 52% identified a joint decision of the residency director, attending physicians and resident physicians as their selection process.
APMLE 1
Pass Fail
6Academic deficiencies
Academic deficiencies
No deficiencies
ProfessionalismAssessment
3GPA/Rank
1GPA/Rank
Grades
4Lower quartile
5Lower quartile
Grades
No Deficiencies
2Lower Quartile
Residency selection funnel
Clinical Competence (Knowledge of Medicine)
Communication Skills
Ethical and Legal Understanding
Excellence
Humanism
Accountability
Arnold and Stern, 2006
Altruism
Professionalism“Professionalism is demonstrated through a foundation of clinical competence, communication skills, and ethical and legal understanding upon which is built the aspiration to and wise application of the principles of: excellence, humanism, accountability, and altruism.”
Constructing a clerkship and Interview evaluation process/tool that’s assesses the Principles of Professionalism • Look at what others are doing for interviews:
• Interviewers should be blinded• Applicants should be aware of blinding and rationale• Interviewers should have specific questions addressed to each applicant intended to
investigate a specific quality or characteristic deemed critical by the program.• Applicants should be scored numerically on the basis of answer content and delivery
• Clerkship performance• Consider task-based interview techniques (standardized patient interviews, short
teaching presentations…) • Develop own evaluation tool based on what is important to the program.
Summary• You can compare academic performance of candidates from the same institution but unpredictable to
compare to other programs. • Lower performance on any academic marker (APMLE or course failure) is associated with an increased
risk of poor knowledge and possibly lower professionalism attributes.**• A common set of professionalism/noncognitive competencies incorporated into the fourth-year clerkship
evaluation for all colleges may level the playing field. • Clerkship performance matters but don’t immediately eliminate applicants that did not rotate at your
site. • Task based clerkships with grading rubrics.• Standardized letters of recommendation may have merit. • Value the interview process. Focus on the values your program in seeking a candidate. Multiple sets of
eyes. • REMEMBER: Students will be assessed by individual competencies and the outcomes of their learning.
Competency and outcome based assessment will not link the educational program to time of study, but will link them to their individual competencies and to the outcome of their learning.
Building an Applicant Pool
• The podiatric profession has to have the will to address this threat.• Pride in the profession.• Active outreach to high school and college students.• Mentoring/shadowing opportunities, including major scientific
events (http://www.dpmnetwork.org/membership/find-a-mentor) • Podiatric medical school based programs (http://www.aacpm.org) • Post-bac programs (https://apps.aamc.org/postbac/#/index) • MCAT/DAT/GRE cohorts• Tour for Diversity• Marketing a third type of MEDICAL SCHOOL