Choosing our Residents

Michael J. Ruckenstein M.D.

Otorhinolaryngology: Head and Neck Surgery at PENN Excellence in Patient Care, Education and Research

since 1870

Residency Selection2 Anecdotes

The ‘Superstar’The ‘Sub’

David Kennedy quoting ?Mike Johns“If you are picking 4 residents, 1 will be a

superstar, 2 will be average, and 1 will be an underperformer”

Shouldn’t we be doing better?Entering medical school is a highly selective

process

Otolaryngology – HNS is a highly competitive match

Penn is highly coveted, ‘top 10’ residency

Shouldn’t we be able to select 4 ‘superstars’?

How we are choosing - all

How we are choosing - OTO

Cognitive AbilitiesUSMLEs

Clerkship grades

AOA status

Correlate with performance on inservice scores

Do not correlate with resident performance during residency

Letters of RecNot standardized

Written in ‘code’ that isn’t always shared

Recapitulate ‘cognitive’ skills

May or may not correlate on all relevant factors

Vary according to who is writing them

Vary on who is reading them!

Dean’s Letter (MSPE)Include the positive, not the negative

Fantasyland – ‘Jewish Day School phenomenon’

90% of the applicants are above average!

The InterviewUnstructured

Individual

No set criteria to evaluate

Designed to ‘get the candidate talking’

The unstructured, casual conversation interview is the MOST invalid method of accessing candidates!

Most valid interviewIndividual or group

Should assess a specific area

Structured

Situational > Job-Related> Psychological

How can we improve?Define characteristics (Dimensions) of an ideal

resident

Evaluate for factors that may predict for these qualities

DimensionsCognitive AbilitiesPersonalityTechnical AbilitiesResearch Abilities

Cognitive AbilitiesThese can be readily assessed using existing

dataUSMLEsClerkship grades (highly variable between

institutions)AOA

What do they mean in terms of performance?We may wish to look for a ‘minimum’ as opposed

to selecting for the highest scores

PersonalityConscientiousness

DependabilityAttention to detailFollow through

Collegiality/agreeablenessAbility to work well with othersAbility to take directionBeing nice

EmpathyAbility to read in emotions and take their

perspectives

Personality - 2Communication Skills

Ability to communicate with colleaguesAbility to communicate with patients

GRITPerseverance and passion for long term goals

ClutchAbility to function and act under pressure

Moral and Ethical Standards

Personality - 3Some data to indicate this Dimension may be

the most predictive of success

How do you assess for all these FacetsSelf reporting – prone to confabulationScalesCan be included in standardized LOR Interview – structured to assess one or more of

these facets

ResearchResearch productivity prior to entering residency

not correlated with performance during residency

May be desirable for a program to include research skills in their residency ‘mix’

Some evidence that research may be misrepresented in application

Likely good topic for structured interviewAssess performed researchPresentation of a ‘research problem’

Technical SkillsInnate ability to perform surgical tasks

Age, sex, gender, grades, manual dexterity do not correlate with operative competence

Visual spatial abilities – the skill of mentally manipulating an object in 3-D correlates with initial acquisition of a surgical skillKnot tyingZ-plasty

Mental Rotation Test

Surgical AbilitiesTechnical ability not the sole factor in making a

‘good surgeon’

Surgical decision makingPreoperative determination of indications for

surgery Intraoperative decisionsPost-operative management

Multicenter Studies Retrospective

GraduatesProspective

Incoming residents/applicants

ProtocolCognitive Assessment

USMLE Part 1Clerkship marksAOA

ResearchNumeric Scale

Graduate degreePublicationsOTO-HNS relatedIndependent research (creativity)

Protocol 2Personality

NEO PI-RBest test of 5 Factor Model of Personality

MSCEITEmotional Intelligence

TechnicalMental Rotation TestSIM (e.g. M&T)

Protocol 3For prospective study, a standardized letter of

recommendation would be very advantageous

Conclusions

Most of the criteria we currently rely on to select resident candidates are not valid

We are proposing retrospective and prospective studies to quantify factors that may predict an applicant’s future success as a resident