2013

2013; 35: 352–358

ORIGINAL ARTICLE

Laying the foundation: Teaching policy andadvocacy to medical trainees

DANIELLE MARTIN, SUSAN HUM, MARGARET HAN & CYNTHIA WHITEHEAD

University of Toronto, USA

Abstract

Background: A novel and comprehensive two-year health policy curriculum was developed and implemented for family

medicine residents at two University of Toronto-affiliated teaching sites.

Aim: To evaluate the impact of the curriculum on residents’ knowledge of health policy issues, and its usefulness to their learning.

Method: The evaluation included a pre-post delivery assessment of residents’ content-based knowledge of issues in the Canadian

healthcare system. Residents were also asked to evaluate the content, process and usefulness of the health policy curriculum.

Results: At the end, more than two-thirds of residents had a better understanding of the Canadian healthcare system. The overall

pre–post scores showed that residents retained content-based facts in some detail. However, more importantly, residents’ positive

evaluations of the curriculum indicated they were engaged, enthusiastic and recognized its importance for their learning.

Conclusion: Despite residents’ positive evaluations, questions remain as to how best to assess the success of health policy

curricula. Moving beyond the popular pre–post test, less traditional approaches might complement standard program evaluation

methods in future. As educators increasingly develop curricula aimed at non-biomedical expertise, we must consider how we can

most meaningfully evaluate long-term impact on graduates’ approach to clinical practice and their engagement in health system

advocacy.

Background

Internationally, healthcare systems grapple with issues such as

wait times, human resources shortages and the public-private

mix of health services (Health Council of Canada 2008). In

Canada, as elsewhere, high-level Commissions have attempted

to address these challenges (Kirby 2002; Romanow 2002;

Castonguay 2008). Such societal deliberations undoubtedly

will result in changes affecting the practice of medicine, and

physicians must be prepared to respond and participate in

these debates. Medical and residency training must therefore

equip learners with a basic understanding of the structure of

their healthcare systems, as well as tools to analyze health

policy issues as they arise in public forums. Health policy

education is believed to help physicians fulfill the role of

Health Advocate by providing them with the tools to promote

the interests of patients at the societal level (Clancy 1995).

Health Advocate is one of seven essential Canadian

CanMEDS competencies or Roles (Frank 2005) that was

formally adopted by the Royal College of Physicians and

Surgeons of Canada in 1996, and by the Canadian College of

Family Physicians (2009; CANMEDS-FM) for their residency

training programs. The other competencies entrenched in

accreditation standards, training objectives and evaluations

during and after residency training include: Medical Expert

(central role), Communicator, Collaborator, Manager, Scholar

and Professional. It is within the Health Advocate domain that

residents are prepared for their future role as patient advocates

at the societal level and to engage in policy development.

Before a physician can competently intervene at a system

level in service of the health of her practice population, she

must first have an understanding of the basic structure of that

system, a notion of the avenues that exist for intervention, and

a sense of the critical issues being debated in our time. A

knowledge gap of such health policy issues and structures

Practice points

. Medical students and residents are eager for training in

health policy issues.

. A novel and comprehensive two-year health policy

curriculum was implemented and evaluated for its

impact on family medicine residents’ knowledge of

health policy issues, and its usefulness to their learning.

. A pre–post survey design showed that residents retained

content-based facts in some detail. More importantly,

residents’ positive evaluations of the curriculum indi-

cated they were engaged, enthusiastic and recognized

its importance for their learning.

. It remains to be determined how best to assess the

success of health policy curricula. Innovative techni-

ques, such as reflective or narrative writing may be used

to determine both the short- and long-term impact of this

curriculum on trainees’/graduates’ approach to clinical

practice and their engagement in health system

advocacy.

Correspondence: Danielle Martin, Women’s College Hospital Family Practice Health Centre, 76 Grenville St., Toronto, ON, USA. Tel: 416-323-

6400x5006; fax: 416-323-6335; email: docdaniellemartin@gmail.com

352 ISSN 0142–159X print/ISSN 1466–187X online/13/050352–7 � 2013 Informa UK Ltd.

DOI: 10.3109/0142159X.2013.770453

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.

exists among medical trainees. In fact, a need for advocacy

training in medical curricula was identified by medical students

and residents at least as early as the 1990s (Clancy 1995; Shortt

1997). Recently, a call was made for the adoption of a

standardized core health policy curriculum by medical schools

throughout the United States and Canada, with content tailored

for regional and local needs (Finkel 2009; Mou 2011).

Unfortunately, there are still very few North American medical

programs that formally teach students and residents about

healthcare systems and other health policy issues.

In 2007, faculty members in Canada’s largest family

medicine training program developed a novel and ambitious

health policy curriculum that was piloted with approximately

40 residents at two University of Toronto-affiliated academic

training sites. Advocacy at the individual patient level was

generally considered to be fairly well-taught through role

modeling by preceptors. Residents see their mentors pick up

the phone to get a quicker appointment for a patient with an

urgent problem; write letters in support of social housing

applications; or battle it out with public and private insurers for

coverage. It was at the broader system level where we

identified an opportunity to improve our current practice as

educators.

For this reason, our approach to enhancing teaching of the

Health Advocate Role was to design a health policy curriculum

as outlined in Box 1. Our goal was for trainees to enter into

practice with a solid understanding of how the healthcare

system works, the challenges it faces, and a commitment to

their role in its ongoing renewal. The approach included some

didactic teaching, but mainly focused on small and large group

discussions, critical analysis of media coverage in both the lay

and medical press of health system issues, and ongoing

discussion of the physicians’ role in the system, both

individually and collectively.

The purpose of this curriculum evaluation project was to

quantify any impact on family medicine residents’ content-

based knowledge of issues in the Canadian healthcare system.

Residents were also asked to evaluate the content, process and

usefulness of each health policy session.

Methods

Survey

Three cohorts of residents (2008–2010, 2009–2011 and 2010–

2012) were administered a three-page survey prior to

exposure to the curriculum (pre-curriculum) and at the end

of their two-year residency program (post-curriculum). The

survey included: (1) Ten multiple choice questions to evaluate

fact-based knowledge of the Canadian healthcare system and

(2) four open-ended questions about challenges facing the

current healthcare system, wait time issues and family

physicians’ role in healthcare reform. A copy of the survey is

included in Appendix 1.

Resident evaluations

Residents were also asked to evaluate each of the five sessions

of the health policy curriculum, using a three-section, one-

page questionnaire. In the first two sections, residents rated the

Box 1. Curriculum Description.

Objectives:

At the end of the 2-year curriculum, residents will be able to:

1. Define the structure of financing and delivery of Canada’s healthcare system

2. Identify similarities and differences between Canada’s healthcare system and other industrialized nations

3. Analyze issues relating to cost, sustainability, efficiency, and access in Canada’s healthcare system

4. Dissect and examine proposed solutions to system challenges

5. Assess and critically evaluate media coverage of health policy issues

Each of the following sessions took place over 2.5 hours on a designated mandatory academic half-day teaching session for all family practice residents at two

teaching sites:

Session 1: Introduction to Canada’s healthcare system

� Basics of healthcare financing and delivery

� Role of federal and provincial governments

� Evidence on single-payer vs multiple-payer systems and not-for-profit vs for-profit delivery

� Cost, health outcomes and access in Canada vs. other nations

� Analysis of media articles relating to content

Session 2: ‘‘Hot topics in Health Policy’’

� Challenges faced by Canada’s healthcare system, including wait times, aging population, pharmaceutical costs, health human resources shortages, etc

� Proposed solutions, both public and private

� Evidence-based analysis of solutions

� Analysis of media articles relating to challenges and solutions

Session 3: Healthcare reform in other nations

� Proposed healthcare reforms in the United States and their potential impact on Canada

� Brief overview of healthcare systems in other nations – ‘‘walk around the globe’’

� Group analysis of strengths and weaknesses of Canada’s healthcare system in relation to others

Session 4: The Physician Role in Health System Reform

� MD reactions to Medicare

� MD remuneration in Canada and other nations

� Analysis of health policy coverage in medical press (e.g. National Review of Medicine, Medical Post) and medical literature (e.g. CMAJ, NEJM, JAMA)

� MD organizations and their role in primary care reform and advocacy on policy issues

Session 5: Summary Session and Feedback

Teaching health policy to medical trainees

353

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.

content and process of each session; and their overall

expectations using an 11-point Likert scale. The last section

asked residents what they liked best and any suggestions for

change.

Data analysis

Descriptive statistics were computed for all variables, including

frequency counts and percentages for categorical variables, or

means and standard deviations for continuous variables.

Fisher’s Exact Test was used to analyze the proportion of

residents answering content-based questions correctly, and

unpaired t-test was used to compare overall scores pre- and

post-curriculum. p values5 0.05 were considered significant.

Evaluations were analyzed quantitatively and responses to

open-ended questions were compiled and themes analyzed.

This study received ethics approval from Women’s College

Hospital Research Ethics Board, Toronto, ON.

Results

Residents were mainly female and most were in their mid-late

20 s. At the beginning of residency, only one-quarter of

residents had ever been exposed to health policy teaching.

By the end of residency, more than two-thirds of residents felt

they had a better understanding of the Canadian healthcare

system (Table 1).

Overall, pre–post curriculum scores of content-based

questions about the Canadian healthcare system suggested

that residents were able to retain most facts. However, even

after exposure to the two-year curriculum, there was a range of

scores (0–8 out of 10) (Table 2).

Residents’ evaluation of the content and process of the first

and final sessions were highly rated. More importantly,

residents’ extremely positive evaluations indicated they were

engaged and enthusiastic about the curriculum. Their com-

ments also suggested that they recognized the importance of

the curriculum to their future practice (Table 3).

‘‘Making us aware about the issues of our healthcare

and how we need to be active. Thought it was very

good at providing stats and info about our healthcare

and really thought-provoking.’’

‘‘Gave us a more complete picture of what’s going

on in the healthcare system. Clarified some important

notions.’’

‘‘Encouraged us to ponder important questions.’’

‘‘ This series of talks should be made university-

wide; maybe at academic core days.’’

‘‘Good breakdown of the different healthcare

models and I liked the discussions we had around

specific countries. Lots to talk about here and not

enough time . . .’’

‘‘ This should be a required course.’’

Discussion

Medical students and residents are eager for training in health

policy issues (Shorrt 1997; Agrawal 2005; Greysen 2009; Patel

2009, 2011). A program similar to ours was delivered to

Queen’s University family medicine residents in Kingston, ON

in 1995, and it was also rated as highly informative and

extremely relevant to their training (Shorrt 1997). Likewise, an

intensive 3 week George Washington University multi-speci-

alty elective course was rated as very/extremely useful by 84%

of residents, and their overall post-course self-reported

Table 2. Frequency of correct responses and overall scores to content-based questions about the Canadian healthcare system.

Pre-Curriculum(n¼ 65) n (%)

Post-Curriculum(n¼32) n (%) p Value*

1. Funding/delivery mechanisms 36 (55.4) 29 (90.6) 0.0001

2. Proportion spending publicly funded 45 (69.2) 21 (65.6) 0.45

3. Spending compared to other industrialized nations 3 (4.6) 10 (31.3) 0.001

4. Services covered under the Canada Health Act (CHA) 34 (52.3) 25 (78.1) 0.01

5. Spending as a proportion of GDP 34 (52.3) 29 (90.6) 0.0001

6. Results of private, for-profit delivery of healthcare services 55 (84.6) 29 (90.6) 0.32

7. Pharmaceutical costs are rising most rapidly in Canada 41 (63.1) 27 (84.4) 0.03

8. Taiwan’s healthcare system most resembles Canada’s 8 (12.3) 3 (9.7) 0.50

9. Extra billing not allowed under CHA 4 (6.2) 11 (35.5) 0.0001

10. Increased use of healthcare system by healthy seniors driving up costs 9 (13.8) 9 (30) 0.06

Overall score (out of 10); mean�SD 4.14� 1.56 (range: 2–7) 6.03�1.77 (range: 0–8) 0.0001**

*Fisher’s Exact Test; ** unpaired t-test.

Table 1. Demographic characteristics of family medicineresidents.

Characteristic

Pre-Curriculum(n¼65)n (%)

Post-Curriculum(n¼ 32)n (%) p Value

Gender

Female 51 (78.5) 24 (75) NS

Male 14 (21.5) 8 (25)

Age (years)

20–24 2 (3.1) –

25–29 49 (75.4) 16 (50)

430 13 (20) 14 (43.8)

440 1 (1.5) 2 (6.3)

Ever attended health

policy sessions? Yes 16 (24.6) 27 (84.4) 50.001

Understand healthcare system

Yes 28 (43.1) 22 (68.8) 50.001

D. Martin et al.

354

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.

knowledge and interest in pursuing further health policy

training increased from 3% to 83%; and from 37% to 82%,

respectively (Greysen 2009).

We evaluated the impact of our curriculum with a

traditional quantitative pre-post survey. At the beginning of

residency, only one-quarter of trainees had any previous

exposure to health policy teaching, and by the end of their

postgraduate training, more than two-thirds of residents had a

better understanding of the Canadian healthcare system. When

residents’ knowledge about the Canadian healthcare system

was assessed, their overall pre-post scores showed that most

were able to retain content-based facts in some detail.

However, there was a range of scores (0–8 out of 10), and a

minority of residents had forgotten most of the knowledge-

based content in the two years between the administration of

the surveys. This is not surprising, since prior studies of

medical students have shown that up to 45% of fact-based

content may be lost depending upon the basic course (D’Eon

2006). Nonetheless, residents’ positive evaluation showed they

were engaged, enthusiastic and recognized the importance of

the health policy curriculum to their learning.

Beyond positive reactions, the question remains how best

to evaluate the success of health policy curricula. Are measures

of knowledge and enthusiasm sufficient? The Health Advocate

role is considered one of the most difficult CanMEDS

competencies to teach and evaluate (Verma 2005). Some

curricular guidelines have been developed (Flynn 2008;

Dharamsi 2011), and it has been suggested that innovative

techniques such as journaling be used to complement standard

curriculum evaluation methods (Shield 2012). Another

approach would be to conduct focus groups with participating

residents, to explore any shifts in their attitudes toward system-

level issues over the course of the two-year curriculum.

Further follow-up assessments would be necessary to

determine the long-term impact of this curriculum on

graduates’ approach to clinical practice, and their engagement

in health system advocacy. This could be done in a variety of

ways. For instance, a comparative study assessing the under-

standing of and engagement in system-level issues between

family physicians who completed the health policy curriculum

and a control group could be carried out. Alternatively,

graduates of such a curriculum could be followed over time

and their involvement in policy-decision making within

professional organizations or Local Health Integration

Networks tracked.

Limitations and future directions

There were several limitations to our evaluation project. Our

sample size was small, and the cohorts at the beginning and

end of residency were not necessarily the same, due to

maternity or sick leaves, and some residents did not attend all

sessions. Since some residents did not attend the final session

when the post-curriculum survey was implemented, we were

constrained to analyze unpaired pre–post curriculum scores.

Furthermore, this curriculum was only delivered to two out of

12 university-affiliated teaching sites. Future directions might

involve university-wide implementation, using innovative

techniques such as reflective or narrative writing to assess

any impact on residents’ personal and professional develop-

ment and any attitudinal changes towards health policy issues

and advocacy.

Conclusion

Family medicine residents’ highly positive evaluations of a

novel and comprehensive two-year health policy curriculum at

the University of Toronto suggested that they were engaged

and enthusiastic about the curriculum, and they recognized its

importance to their learning. Given that residents often dismiss

the quality and utility of formal academic curricula to teach

about roles other than Medical Expert, their unbridled

enthusiasm for this curriculum is, in and of itself, noteworthy.

Moreover, it is encouraging that the pre�post test showed

knowledge gain about health policy issues. However, as this

health policy curriculum rolled out, we realized that common

program evaluation measures of knowledge and learner

satisfaction did not capture what we hoped to accomplish.

Our overarching goal is to produce a generation of engaged,

committed health advocates. Given this, what are appropriate

metrics for success? As residency programs worldwide struggle

to teach and assess various CanMEDS Roles, we need to think

together as a community of medical educators about the best

Table 3. Quantitative assessment of the health policy curriculum.

Section A

Session Evaluation: scale of 0–10;

(where 0¼ strongly disagree/unacceptable; 10¼ strongly agree/excellent)

After 1st session

(n¼ 30)

After final session

(n¼ 13)

Content

The seminar/round was relevant, well-researched, summarized major points, increased

knowledge/awareness of issues

9.73þ0.52 9.77þ0.44

(range: 8–10) (range: 9–10)

A/V & Handouts

Material was clear, well-organized, interesting, concise, original, informative 9.79þ0.41 9.69þ0.48

(range: 9–10) (range: 9–10)

Process

Presenter was clear, spoke well, encouraged interest, thinking discussion and questions,

facilitated training

9.90þ0.31 9.77þ0.44

(range: 9–10) (range: 9–10)

Section B

Overall Expectations: scale of 0–10 (where 0¼Below; 10¼Exceeds) 9.77þ0.51 9.77þ0.44

(range: 8–10) (range: 9–10)

Teaching health policy to medical trainees

355

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.

ways to assess our learners and evaluate our curricula in these

essential areas of physician expertise.

Notes on contributors

DANIELLE MARTIN is a family physician and Board Chair of Canadian

Doctors for Medicare. She is clinical staff at Women’s College Hospital and

lecturer in the Department of Family and Community Medicine at the

University of Toronto.

SUSAN HUM is a Research Associate at Women’s College Hospital

Department of Family and Community Medicine.

MARGARET HAN has a Master degree in Public Health from the University

of Toronto.

CYNTHIA WHITEHEAD is an Associate Professor and Vice Chair Education

Department of Family and Community Medicine at the University of

Toronto.

Acknowledgements

This project was funded by the inaugural Louise Naismith

Award and the Harrison Waddington Fellowship, both

awarded to Dr Danielle Martin by the Department of Family

and Community Medicine, University of Toronto. We thank the

writing group of academic family physicians at Women’s

College Hospital for their feedback in the preparation of this

manuscript.

Declaration of interest: The authors report no conflicts

of interest.

References

Agrawal JR, Huebner J, Hedgecock J, Sehgal AR, Jung P, Simon SR. 2005.

Medical students’ knowledge of the U.S. health care system and their

preferences for curricular change: A national survey. Acad Med

80(5):484–488.

Castonguay C. 2008. Rapport du groupe de travail sur le financement du

systeme de sante. En avoir pour notre argent. Bibliotheque et Archives

nationales du Quebec. Available from http://www.anq.qc.ca/

Documents/RapportFR_FinancementSante.pdf

Clancy TE, Fiks AG, Gelfand JM, Grayzel DS, Marci CD, McDonough CG,

Peppercorn JM, Roberts TG, Smith AL, Winickoff JP. 1995. A call for

health policy education in the medical school curriculum. J Am Med

Assoc 274(13):1084–1085.

D’Eon MF. 2006. Knowledge loss of medical students on first year basic

science courses at the University of Saskatchewan. BMC Med Educ 6:5.

Available from http://www.biomedcentral.com/1472-6920/6/5

Dharamsi S, Osei-Twum J, Shroff F, Mu L, Woollard R. 2011. The health

advocate role: Preparing future physicians for socially responsive

practice. Vancouver, BC: University of British Columbia, Department of

Medicine.

Finkel M. 2009. Incorporating health policy in the medical school and

residency curriculum. Clin Teach 6(4):237–240.

Flynn L, Verma, S. 2008. Fundamental components of a curriculum for

residents in health advocacy. Med Teach 30:e178–e183.

Frank JR. editor. 2005. The CanMEDS 2005 physician competency

framework. Better standards. Better physicians. Better care. Ottawa,

ON: The Royal College of Physicians and Surgeons of Canada.

Greysen SR, Wassermann T, Payne P, Mullan F. 2009. Teaching health

policy to residents – Three-year experience with a multi-specialty

curriculum. J Gen Intern Med 24(12):1322–1326.

Health Council of Canada. 2008. Rekindling reform: Healthcare renewal in

Canada, 2003–2008. Ottawa, ON: Health Council of Canada.

Kirby, MJL. 2002. The health of Canadians � The federal role final report

volume six: Recommendations for reform. Ottawa, ON: Standing Senate

Committee on Social Affairs, Science and Technology. Available

from http://www.parl.gc.ca/Content/SEN/Committee/372/soci/rep/

repoct02vol6-e.htm

Mou D, Sarma A, Sethi R, Merryman R. 2011. The state of health

policy education in U.S. medical schools. N Engl J Med 364:

e19(1)–e19(3).

Patel MS, Davis MM, Lypson ML. 2011. Advancing medical education by

teaching health policy. N Engl J Med 364(8):695–697.

Patel MS, Lypson ML, Davis MM. 2009. Medical student perceptions of

education in health care systems. Acad Med 84(9):1301–1306.

Romanow RJ. 2002. Building on values: The report of the royal commission

on the future of healthcare in Canada. Ottawa, ON. Available from

http://www.cbc.ca/healthcare/final_report.pdf

Shield RR, Farrell TW, Nanda A, Campbell SE, Wetle T. 2012. Integrating

geriatrics into medical school: Student journaling as an innovative

strategy for evaluating curriculum. Gerontologist 52(1):98–110.

Shortt SE, Hodgetts PG. 1997. A curriculum for the times: An experiment in

teaching health policy to residents in family medicine. Can Med Assoc J

157(11):1567–1569.

The College of Family Physicians of Canada. 2009. CanMEDS – Family

Medicine Working Group on Curriculum Review October 2009:

Mississauga, ON: The College of Family Physicians of Canada.

Available from http://cfpc.ca/uploadedFiles/Education/

CanMeds%20FM%20Eng.pdf

Verma S, Flynn L, Seguin R. 2005. Faculty’s and residents’ perceptions of

teaching and evaluating the role of health advocate: A study at one

Canadian university. Acad Med 80(1):103–108.

D. Martin et al.

356

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.

Appendix

Department of Family & Community Medicine

University of Toronto

LEARNING NEEDS ASSESSMENT AND HEALTH POLICY

CURRICULUM EVALUATION

Teaching health policy to medical trainees

357

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.

D. Martin et al.

358

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y E

ast C

arol

ina

Uni

vers

ity o

n 09

/07/

13Fo

r pe

rson

al u

se o

nly.