Medical Educator's Portfolios: How To Be Recognized and Promoted For Your Technology-

Enhanced Medical Education Scholarship"

Donna M. D'Alessandro, M.D."donna-dalessandro@uiowa.edu"

Michael P. D'Alessandro, M.D."michael-dalessandro@uiowa.edu"

Educational Informatics"www.educationalinformatics.org"

www.slideshare.net/michaeldalessandro"

Departments of Pediatrics and Radiology"University of Iowa College of Medicine / University of Iowa Children's Hospital"

Disclosures"

• Nothing to disclose"

About Us"

•  Educational Informatics Laboratory's vision since 1989"•  To improve patients' care, outcomes and lives;"•  By changing physician's knowledge, attitudes, and behaviors;"•  Through the creation and evaluation of tools, techniques, and procedures that shift

learning from the classroom and lecture hall to the point-of-care"•  Strong record of innovation in medical education"

•  Resulting in grants, peer-reviewed publications, awards"•  Serve 2 million learners / year ~ highest impact factors on Internet"

•  Experience on promotion + tenure committees at Department + College levels"

Workshop Outline"

• What is a medical educator's portfolio?"• What do you put in a medical educator's

portfolio?"• How do you include your technology-

enhanced educational scholarship in a medical educator's portfolio?"

• How do you interpret what is in the medical educator's portfolio? "

Introduction"

"The worst thing that can happen for a Promotion and Tenure committee is to not have data, and the second worst is to have a disorganized dossier.""

- Interview comment from Simpson et. al. Acad Medicine, 2004 "

•  Our goal"•  Help you to gather and organize your"

"Evidence of educational scholarly achievements""…and thus help you to manage your career"

Uses of the Medical Educator's Portfolio"

•  For self-reflection and learning"•  In the promotion and tenure process"•  To assist in advising and mentoring others"•  To empower diversity in scholarship"•  For "overcoming universal entropy""

Some Questions"

• Where are you from and what do you do?"

• What is a question you want answered in this workshop?"

Some Definitions"•  Scholarship definition"

•  In 1990, Ernest Boyer defined scholarship in four domains:"•  1. Discovery - i.e. traditional research; "•  2. Integration - collaborative connections across disciplines; "•  3. Application - vital interaction between research and practice; "•  4. Teaching (educational scholarship) - the creation of new knowledge

about teaching and learning in the presence of learners "

•  Educational Scholarship definition ""Educational scholarship refers to any material, product or resource

originally developed to fulfill a specific educational purpose that has been successfully peer-reviewed and is subsequently made public through appropriate dissemination for use by others." - AAMC Web site 2005"

•  Medical Educator's Portfolio definition""Portfolio is a systematic collection of information documenting expertise in

an area, usually incorporating multiple sources of information collected over time to demonstrate excellence." - Seldin P. 1997. "

Example Medical Educator's Portfolio"

http://www.mededportal.org/publication/626"

Medical Educator's Portfolio Contents"•  Teaching"

•  Volume - detailed list of venues taught (e.g. classroom, clinic, laboratory), number / type of learners (e.g. medical students, nurses, patients), number of hours taught, teaching modality used (e.g. lecture, discussion, case-based presentation, etc.) "

•  Honors and awards - teacher of the year, unsolicited letters "•  Assessment of learnersʼ performance - in-training / board

examinations, OSCE results, pre / post assessments"•  Learner assessments of oneʼs teaching - demonstration of basic

teaching skills or as an excellent teacher - learner ratings, unsolicited letters"

•  Peer assessment of teaching - demonstration of basic teaching skills or as an excellent teacher, recognition as a national leader in education - peer evaluations, videotape evaluations, honors and awards, invitations for dissemination of teaching "

•  Curriculum development / innovation / evaluation - examples and documentation of one's best work with discussion of the philosophy, application and evaluation / outcomes of the curriculum or other educational project"

•  Continuing education for oneself - participation at local or national meetings to improve own skills "

Medical Educator's Portfolio Contents"•  Advising / Mentoring "

•  Volume of learner advising - list of venues of advising, number / types of advisees, number of hours or length of time advising, advising modality (e.g. role model, formal advisor, etc.) "

•  Learner assessment of advising - advisor ratings, unsolicited letters"•  Other evaluation of advising - advisees honors, publications,

leadership positions etc. "•  Mentorship of other faculty locally and nationally - national faculty

development program"•  Research - supervision, mentoring, thesis / dissertation committees"

•  Educational Administration / Leadership"•  Education committees - department, medical school, national including

oneʼs role, service dates, special projects "•  Accreditation committees - including resident review committee,

ACGME, GME, etc. "•  Leadership positions - clerkship director, residency program director,

dean, national organization's educational activities "•  Curriculum development including role, dates of service, special

projects (see also Teaching section)"

Medical Educator's Portfolio Contents"

•  Dissemination / Scholarship of Teaching "•  Peer-reviewed publications with contributions listed: conception

and design, data acquisition, data analysis and interpretation, manuscript drafting, manuscript critical review, statistical expertise, senior author or other special designation "

•  Books / chapters "•  Editorial services - reviewer, editorial board "•  Invited / keynote presentations - locally, nationally "•  National meeting presentations on educational topics "•  Grants and contracts "•  Peer recognition "

•  Personal Statement (at beginning or end)"•  Philosophy of teaching"•  Long term goals as an educator"

What You Need From Your Medical Education Web Site / App / Ebook for Your Medical

Educator's Portfolio"•  Why create a medical education Web site / Ebook / App?"

•  To teach + enhance your reputation as medical educator"•  Is valid form of educational scholarship ~ Is a scholarly work"

•  Counts towards promotion / tenure if documented "•  Documentation of scholarship"

•  Design ~ Build ~ Utilize ~ Evaluate ~ Refine"•  Traditional metrics for scholarly works"

•  Grants obtained"•  Peer-reviewed articles published"

•  Evolving metrics for scholarly works ~ Web 1.0 metrics"•  Peer reviews conducted by organizations"

•  AAMC MedEdPORTAL, ACGME, Merlot"•  Reviews published in print journals"•  Awards"•  Bibliometric analysis / Impact factor"•  Web statistics ~ overall usage: pages read / visitors "•  Users comments (fan mail = true peer review) "

Three Step Approach To Data Gathering"

•  1. Instrumenting your Web site / Ebook / App to gather all data"•  Installing a Web statistics package for Web site"

•  SiteMeter - www.sitemeter.com"•  Google Analytics - www.google.com/analytics"

•  Utilizing preexisting App Store statistics packages for Ebook / App"•  Amazon Kindle / Apple iTunes / Google Play"

•  Installing a comment / feedback form"

Three Step Approach To Data Gathering"•  2. Web publicity and marketing"

•  Submit Web site to "•  (Consider creating marketing Web site for Ebook / App)"•  Search engines - google.com, www.bing.com"•  Internet Archive - www.archive.org"•  Internet directories - www.dmoz.org"•  Health sciences directories - www.martindalecenter.com"•  Health on the Net - www.hon.ch"

•  Add links to related articles in Wikis"•  www.wikipedia.org, www.wikibooks.org"

•  Index in Open Education Resources repositories"•  AAMC MedPortal, www.curriki.org, www.e-booksdirectory.com, www.lemill.net,

www.merlot.org, www.oercommons.org, textbookrevolution.org, wlh.webhost.utexas.edu"•  Send announcement"

•  groups.google.com (misc.education.medical), groups.yahoo.com, www.medscape.com, www.sermo.com, Facebook groups, Twitter, society mailing lists, friends / colleagues"

•  Check search engines for top 100 hits for related terms / keywords"•  Ask these sites to trade links"

•  Send e-mails to journal reviewers"•  Professional meetings ~ hand out flyers ~ teach workshops"•  News Alerts to track your progress - www.google.com/alerts"

Three Step Approach To Data Gathering"•  3. The regular maintenance, evaluation and continuous

quality improvement of Web site / Ebook / App"•  Usually most neglected step ~ Why most fail"•  Evaluation is key - listen to your users"•  Take the long view - commit to 10 years"•  Keep it simple stupid! - in terms of design + technology"•  Always have a backup of the Web site / Ebook / App"•  Keep a diary from day one"•  Yearly curation / scholarship harvest"

•  Reconfirm Step 1 - your mission / audience / goals / metrics"•  Check - links / HTML / accessibility"•  Preserve - archive, print, screen shot"•  Measure evolving metrics - inbound links, Web statistics, bibliometric

analysis, downloads"•  Measure Altmetrics - ImpactStory.org"•  Search for reviews - LexisNexis, Google News Archive, Google

Scholar, Amazon 'Search Inside Books', Google Books"

Results of Evolving Metrics = Peer Review By Quantitative Acclimation(What You Need From Your Medical Educator's Portfolio For Your Medical Education Web Site / App / Ebook)"

www.pediatriceducation.org/reviews.html"

Altmetrics Web 2.0 metrics"

•  Bibliometrics = traditional techniques for measuring scholarly impact"•  Citations, h-index, i10-index"

•  Altmetrics = alternative metrics of measuring scholarly impact"•  Creation and study of new metrics based on the social Web for

analyzing and informing scholarship ~ Altmetrics.org"•  Measures Web-driven scholarly interactions"

•  How often are research products cited, downloaded, shared, viewed, mentioned, blogged, tweeted, bookmarked, commented on, liked, etc."

•  Example research products being measured"•  For published articles in PubMed, et.al"•  For software in GitHub"•  For dataset in Dryad or figshare"•  For slides in SlideShare"•  For any Web site or Web resource"

Your First Step Towards Altmetrics ImpactStory.org"

5 minutes to set up if you have Google Scholar Profile"

Results of Altmetrics = Peer Review By Quantitative Acclimation"

Promotion Lessons Learned"•  Understand the criteria for promotion"

•  Get on promotions committee to see criteria applied and to see how others are viewed"

•  The arc of progress is crucial"•  Your work should not be all over the place; Dr. X = Y"

•  It is a personality contest, it is about who you are"•  A nice personality is given much more leeway than a malignant one"

•  Writing the essay"•  Tell me a story"•  You are writing a legal document"

•  You are making your case for promotion"•  Don't be shy about your accomplishments"•  Back up your claimed accomplishments with facts"

•  There are evil forces out there"•  The only way to fight evil is with facts and data that is well

organized and presented"

The Essay"•  Teaching"

•  Local - formal - lectures: undergraduate, graduate, continuing"•  Local - informal: at conferences, in labs, in clinics / on wards"•  National / International: lectures - continuing, visiting professor"

•  Service"•  Clinical"

•  Responsibilities, % time, RVUs compared to others, clinical revenue"•  Administrative"

•  Local - committees: departmental, hospital, college"•  Local - courses, curricula"•  National / International - committees, editorial boards"

•  Scholarship (only place to describe your scholarship)"•  Papers"•  Grants"•  For each medical education Web site / App / Ebook"

•  Traditional and evolving metrics / altmetrics"•  Have someone read it"•  Make sure contents meet all requirements"•  Have supplementary materials organized logically"

The Essay - Explaining New Metrics"Digital library scholarship can be measured using traditional metrics such as grants obtained for it and

peer reviewed articles published about it as well as evolving metrics such as peer reviews conducted by organizations such as the AAMC and ACGME, reviews published in journals, bibliometric analysis, overall usage of it, and comments from users."

In traditional print publishing, through bibliometric analysis performed by Thomson-Scientific, each journal article and each journal can be assigned an impact factor, which is a measure of authority, based upon the number of citations each receives from other journal articles. Web publishing has quickly evolved in an identical manner, and through bibliometric analysis each Web page and each Web site can be assigned an impact factor, which is a measure of authority, based upon the number of links each receives from other Web pages. Additionally, it has been shown that the impact factor of electronic editions of print journals on the Web correlates significantly to the impact factor of the print edition of the journal."

The top three search engines (Google, Baidu, Bing) account for 81% of all searches on the Internet, each using a unique and proprietary algorithm to perform their bibliometric analyses. The end result is that the Web pages and Web sites with the highest impact factors are listed first in the results returned by a search engine for a given term."

Today, the Internet is rapidly becoming the most common place that answers are sought to questions in many domains, including medicine. Most users (60%) doing a search never look beyond the first page of search engine results (first 10 hits) and an additional 30% of users never look beyond the first three pages of results (first 30 hits). Therefore, being on the first page of results for a given search term is a highly influential place to be."

For the last several years, for the search engine terms "pediatrics," "general pediatrics," and "pediatric education" my digital libraries are consistently displayed on the first page of search results / first 10 hits for each of the top 3 search engines. This means that for these search engine terms, my digital libraries have the highest impact factors on the Internet and are therefore amongst the top 10 authoritative Web sites on these subjects."

Questions on Construction"• Who has a medical

educator's portfolio / teaching portfolio? "• How did you construct your

portfolio?"• How do you maintain your

portfolio?"•  What is in your medical

educator's portfolio?"• What else do you think

should be in a medical educator's portfolio?"

Questions on Use"

•  Have you or anyone you know been promoted for computer-based medical education scholarship?"•  If so, how did they do it?"

•  What problems do you already anticipate from your institution?"•  Who can you turn to for help?"

•  What problems have you already encountered with your technology-enhanced scholarship and how have you fixed them?"

•  For the already promoted, what problems did you face and what words of wisdom can you give others?"

•  Other comments? "•  Tips and tricks? Words of wisdom?"

Conclusion / Call For Action"

•  You are now armed with the knowledge necessary to successfully create a medical educator's portfolio"

•  Your plan of action"• Will you create or refine a medical educator's

portfolio?"• What are / is your first steps?"• When will have your first step done?"• Put this by your computer…"

References For Medical Educator's Portfolio"•  A medical educator's portfolio"

•  Educator Portfolio Template of the Academic Pediatric Association's Educational Scholars Program. "•  Gusic ME, Chandran L, Baldwin CD, D'Alessandro DM, Balmer DF.

MedEdPORTAL; 2007. "•  Available from: http://www.mededportal.org/publication/626"•  Also at: http://www.ambpeds.org/education/educator_portfolio_template.cfm"

•  How to use the medical educator's portfolio"•  The Educator Portfolio: A Tool For Career Development."

•  Baldwin CD, Gusic ME, Chandran L. AAMC Faculty Vitae 2008."•  Available from: http://www.aamc.org/members/gfa/faculty_vitae/148574/

educator_portfolio.html"•  Evaluation tool of the medical educator's portfolio"

•  Evaluating the performance of medical educators: a novel analysis tool to demonstrate the quality and impact of educational activities. "•  Chandran L, Gusic M, Baldwin C, Turner T, Zenni E, Lane JL, Balmer D, Bar-On

M, Rauch DA, Indyk D, Gruppen LD. Acad Med. 2009 Jan;84(1):58-66."

References"•  American Association of Medical Colleges MedEdPortal. Educational

Scholarship for Promotion and Tenure. Available from the Internet at: http://www.mededportal.org/download/190392/data/educational_scholarship_guide.pdf (rev. 2009, cited 4/2013)."

•  Boyer EL. Scholarship Reconsidered: Priorities for the Professoriate. 1990; The Carnegie Foundation the Advancement of Teaching: Princeton, NJ. "

•  Glassick CE, Huber MR, Maeroff GI. Scholarship Assessed: Evaluation of the Professoriate. 1997; San Francisco, CA: Jossey-Bass. "

•  Manion Fleming V, Schindler N, Martin GJ, DaRosa DA. Separate and Equitable Promotion Tracts for Clinician-Educators. JAMA. 2005; 294(9)1101-1104. "

•  Ruiz JG, Candler CS, Qadri SS, Roos BA. E-Learning as Evidence of Educational Scholarship: A Survey of Chairs of Promotion and Tenure Committees at US Medical Schools. Acad Med. 2009; 84(1): 47-56."

•  Seldin P. The Teaching Portfolio: A Practical Guide to Improved Performance and Promotion/Tenure Decisions. Second Edition. Bolton, MA: Anker Publishing Company, 1997. "

•  Simpson DE, Hafler J, Brown D, Wilkerson L. Documentation Systems for Educators Seeking Academic Promotion in U.S. Medical Schools. Acad Med. 2004; 79(8): 783-90. "

To see a variety of medical education Web sites and Ebooks, visit www.educationalinformatics.org"To stay up-to-date on digital scholarship, visit ProfHacker at chronicle.com/blogs/profhacker/"