Designing a system for performanceappraisal: balancing physicians’accountability and professionaldevelopmentElisa Bindels1,2,3* , Benjamin Boerebach4† , Renée Scheepers5† , Annemiek Nooteboom6,7, Albert Scherpbier8 ,Sylvia Heeneman9 and Kiki Lombarts1
Abstract
Background: In many healthcare systems, physicians are accustomed to periodically participate in individualperformance appraisals to guide their professional development. For the purpose of revalidation, or maintenance ofcertification, they need to demonstrate that they have engaged with the outcomes of these appraisals. Thecombination of taking ownership in professional development and meeting accountability requirements may causeundesirable interference of purposes. To support physicians in their professional development, new Dutchlegislation requires that they discuss their performance data with a non-hierarchical (peer)coach and draft apersonal development plan. In this study, we report on the design of this system for performance appraisal in aDutch academic medical center.
Methods: Using a design-based research approach, a hospital-based research group had the lead in drafting andimplementing a performance appraisal protocol, selecting a multisource feedback tool, co-developing and pilotinga coaching approach, implementing a planning tool, recruiting peer coaches and facilitating their training and peergroup debriefings.
* Correspondence: [email protected]†Benjamin Boerebach and Renée Scheepers equal contribution.1Department of Medical Psychology, Amsterdam Center for ProfessionalPerformance and Compassionate Care, Amsterdam University MedicalCenters, Amsterdam, the Netherlands2Department of Educational Development and Research, Faculty of Health,Medicine and Life Sciences, Maastricht University, Maastricht, the NetherlandsFull list of author information is available at the end of the article
Bindels et al. BMC Health Services Research (2021) 21:800 https://doi.org/10.1186/s12913-021-06818-1
Results: The system consisted of a two-hour peer-to-peer conversation based on the principles of appreciativeinquiry and solution-focused coaching. Sessions were rated as highly motivating, development-oriented, concreteand valuable. Peer coaches were considered suitable, although occasionally physicians preferred a professionalcoach because of their expertise. The system honored both accountability and professional development purposes.By integrating the performance appraisal system with an already existing internal performancesystem, physicians were enabled to openly and safely discuss their professional development with a peer, whilealso being supported by their superior in their self-defined developmental goals. Although the peer-to-peerconversation was mandatory and participation in the process was documented, it was up to the physician whetheror not to share its results with others, including their superior.
Conclusions: In the context of mandatory revalidation, professional development can be supported when theappraisal process involves three characteristics: the appraisal process is appreciative and explores developmentalopportunities; coaches are trustworthy and skilled; and the physician has control over the disclosure of the appraisaloutput. Although the peer-to-peer conversations were positively evaluated, the effects on physicians’ professionaldevelopment have yet to be investigated in longitudinal research designs.
Keywords: Continuing professional development (CPD), Revalidation, Maintenance of certification (MoC), Re-registration,Performance appraisal and assessment, Multisource feedback (MSF), Coaching, Design-based research (DBR)
BackgroundFor the purpose of both continuing professional devel-opment (CPD) and accountability to the public, in manywestern countries physicians periodically participate inperformance appraisals. Performance appraisal is aneducational intervention, which involves a formativeconversation between two professionals. Originally, theappraisal process was intended to provide a physicianwith feedback on his or her performance, map aphysician’s progress and identify areas for further devel-opment. Performance appraisal was not designed as anassessment of competence which a physician wouldeither pass or fail [1]. In response to society’s call fortransparency however, performance appraisal has beenincorporated into procedures for revalidation (UK),recertification (USA, Canada) and re-registration(Netherlands). In this context, participation in perform-ance appraisal is assumed to provide more objective as-surance that a physician is up to date and fit to practicemedicine [2–4]. The interlinking of appraisal and reval-idation presents itself as a delicate matter, as this maycause undesirable interference of two types of goals: pro-fessional development versus accountability. Physiciansmay perceive these goals as conflicting, framing theappraisal process as summative (i.e. to detect and weedout bad apples) instead of formative (i.e. to raise stan-dards and support professional development) [5, 6]. Theliterature indicates that this interference may jeopardizephysicians’ engagement in the appraisal process, sincemandatory accountability and transparency require-ments seem principally incompatible with the needs forpsychological safety and intrinsic motivation. Care mustbe taken that the appraisal process is experienced andused as an opportunity for learning and development
and is not turned into a tick-box-exercise, associatedwith a loss of personal investment and a disposition ofcompliance.Up till now, no consensus exists on the appropriate in-
corporation of performance appraisal into revalidationprocedures. Some countries have no formal process inplace, while others rely heavily on the collection ofcredits for continuing learning and development activ-ities [7]. Nevertheless, there is a growing interest in theuse of multisource feedback (MSF) for performance ap-praisal purposes, and regulatory bodies in the USA,Canada, the UK and the Netherlands use MSF as part ofrevalidation and evaluation programs for practicingphysicians [8–10]. Feedback from peers, co-workers andresidents is found to be necessary to inform physicians’self-assessment by providing a more realistic view ofhow they perform [11]. Research on how physicians useMSF, however, has demonstrated that this feedback doesnot self-evidently find its way into performance change.Feedback can be perceived as disconfirming or disap-pointing, it may evoke an emotional reaction that caninterfere with the ability to assimilate and learn from it,or the feedback may not be specific enough to catalyzecertain performance changes [10]. Also, one’s self-efficacy and motivation and the collegial culture can in-fluence the process of using feedback for learning andchange. Reflective discussions guided by another personwithin a respected, engaged relationship (a peer, coachor mentor) can foster feedback reconciliation with one’sown self-assessment and promote growth throughfacilitating feedback acceptance and use and offeringappropriate challenge. Coaching strategies can supportpersonal and professional development by guiding thefeedback recipient in identifying their own needs and
Bindels et al. BMC Health Services Research (2021) 21:800 Page 2 of 12
goals and developing a realistic action plan [12, 13]. Theaddition of ‘coaching’ to the feedback lexicon is significant,because it places feedback in a different light. Coaching isdevelopmental, and philosophically it moves feedbackaway from its historic tie to assessment and moves it to-wards learning [13]. Feedback, in combination with coach-ing, can be a powerful strategy for fostering physicians’continued development and growth. More broadly, it canserve to support physicians in their complex professionalroles and their capacity to be self-directed yet functioncapably as team member, to make decisions on their ownyet know when to ask for help, and to maintain their ownhealth and sense of well-being in the presence of multiplesystem and other demands [14]. Reports of alarming levelsof stress and burnout among physicians direct our atten-tion to ways of supporting physicians for these compli-cated roles and nurture their resilience [15].In the Netherlands, the requirements for re-registration
have recently changed; new legislation concerning physi-cians’ individual performance appraisal has been adopted.As per 2020, practicing physicians are required to five-yearly collect performance feedback, discuss their feed-back with a trained, non-hierarchical facilitator and defineand follow-up on a personal development plan (PDP) [16].The aforementioned concern over the two types of poten-tially conflicting goals of re-registration and performanceappraisal, however, may jeopardize this appraisal process.As mentioned earlier, mandatory accountability and trans-parency requirements seem principally incompatible withthe needs for psychological safety and intrinsic motivation.In this study, we report on the design of an individual per-formance appraisal system in a Dutch academic medicalcenter, in which this incompatibility was taken into ac-count. The design problem was formulated as follows:how to design a system that will successfully comply withboth revalidation requirements and the need for profes-sional development? By using a design-based research(DBR) approach, our aim was to define, build and imple-ment a system around characteristics that are crucial forthe facilitation of professional development in the contextof mandatory re-registration. Since MSF programs do nottraditionally involve coaching elements, the design pre-sented in this study is among the first attempts to com-bine MSF and coaching. We aim for this design to serveas an inspiring example for other institutions or healthcare systems dealing with the same challenge. Before wewill discuss the method however, we will first providerelevant background information on the re-registrationprocedure and appraisal systemin the Netherlands.
Policy backgroundRequirements for re-registration in the NetherlandsAfter completion of residency training, physicians in theNetherlands are registered with the Registration
Committee of Medical Specialists. In order to maintaintheir license, physicians need to meet a number of re-registration requirements, which is assessed once everyfive years. These involve (1) working in clinical practicefor a minimum of 16 h per week, (2) participating inaccredited expertise-promoting activities equaling aminimum of 40 hours per year, and (3) participating inthe external peer review (called visitatie) program of aphysician’s professional body. The latter national pro-gram focuses on physicians’ group performance, assessedin highly protocolled peer site-visits. Participation in thevisitatie program is mandatory since 1995. Since the per-formance of individual physicians is not the primaryfocus of the visitatie program, the Dutch medical regula-tory and registration authorities recently added a fourthre-registration requirement. As per 2020, physicians (4)must demonstrably work on their individual professionaldevelopment [16]. This new requirement is referred toas the IFMS (Individual Functioning of Medical Special-ist) requirement [4].
Performance appraisal: the IFMS systemThe IFMS system is based on research and is built onprevious designs of appraisal systems used in theNetherlands. In the period 2005–2007, the first IFMSsystem was developed in a collaborative project with theDutch National Organization of Medical Specialists(Orde van Medisch Specialisten or OMS), the DutchInstitute of Quality Improvement (CBO), 8 medical pro-fessional societies and 8 pilot (non-academic) hospitals.One of the authors (KL) was the national advisor andco-project leader. The development, implementation andevaluation have been accounted for in a PhD thesis [17]and policy recommendations for the future of IFMSwere published by the national IFMS committee of theOMS [18]. Three instruments, which were developed inCanada, the US and the UK, were examined for suitabil-ity in the Dutch context [19]. The first two instruments,the Canadian method of multisource feedback (MSF)[20] and the American method of Peer Associate Rating(PAR) [21] both used structured questionnaires to col-lect information about a physician’s individual perform-ance. These questionnaires were presented to personswith whom the physician under evaluation has a closeworking relationship, that is physician-colleagues, staff,residents and patients for the Canadian method andphysician-colleagues for the American method. Thethird instrument, the British method of Appraisal &Assessment (A&A) [22], concerns a qualitative perform-ance assessment posing 3 open-ended questions to alimited number of the physician’s peers and coworkers.For the final Dutch IFMS system, the MSF methodologyand the A&A methodology were combined by usingquantitative questionnaires with room for qualitative
Bindels et al. BMC Health Services Research (2021) 21:800 Page 3 of 12
feedback [23]. The IFMS requirement consists of a five-yearly evaluation of individual performance and dictatesthe use of multisource feedback (MSF). The procedureentails three consecutive steps: (1) take part in thecollection of MSF, (2) discuss the feedback with atrained facilitator with whom there is no hierarchical orotherwise dependent relationship, and (3) define andyearly evaluate/adjust a personal development plan(PDP). Every physician must maintain a portfolio inwhich he or she shows evidence of his or her profes-sional performance activities and outcomes covering allprofessional (medical and generic) competencies [24].The objective of the IFMS requirement is to maintain orenhance the quality of physicians´ individual perform-ance; its purpose is not to identify malfunctioning physi-cians. Other systems are available to manage (suspected)malfunctioning or poor performance [4].
Performance appraisal: the IFMS system in academicmedical centersIn the Netherlands, physicians carry out their profes-sional duties in various organizational and legal contexts.Physicians can be independent entrepreneurs organizedin ‘medical specialist companies’ through which special-ist care is contracted by hospitals, or physicians areemployed by hospitals. At this point, an importantdifference between physicians working in academic andnon-academic settings must be noted. Physicians work-ing in academic medical centers are employed by thehospital, and therefore they are obliged to demonstrateaccountability for their performance to the hospital.Within the hierarchical structure of academic medicalcenters, there has long been an existing procedure in-volving an annual review of the physician’s individualperformance. The annual review is executed by thephysician´s hierarchical superior, i.e. head of department(HoD), and provides a means to review the physician’sindividual career development in alignment with thestrategic objectives of the hospital. This annual reviewallows for customization of individual development, butdoes not take into account actual performance feedbackfrom peers and coworkers [7]. The initial IFMS system,as described in the previous section, was not developedfor use in academic medical centers and was notmandatory for individual re-registration purposes. Thenew context in which participation in IFMS is mandatedfor re-registration posed a brand new situation, requiringa new design. The new to be designed and implementedIFMS system must be embedded within the existingstructure of annual reviews. Redundancy between theIFMS procedure and the procedure of annual reviewshad to be prevented or at least minimized. On the onehand, the physician needs to be able to safely discuss hisor her performance with a trained, non-hierarchical
facilitator; on the other hand, it must be assured that thephysician can be supported by his or her superior inintended developmental activities. The present study isabout the design of the IFMS system in one specific hos-pital, the Academic Medical Center (AMC). After thedevelopment and implementation of this IFMS systemwas completed, the AMC was merged with VU UniversityMedical Center and continued as one organization underthe new name Amsterdam University Medical Centers(Amsterdam UMC).
MethodStudy designTo design, build, and implement the IFMS system in theAcademic Medical Center that would honor both ac-countability and professional development purposes, weadopted a design-based research (DBR) approach. DBRis a fruitful approach for the (re)design of work-basedlearning environments and assessment programs. Thefive important characteristics of DBR are the following:(1) it takes place in continuous cycles of design, evalu-ation and redesign; (2) it takes place in an authenticpractice context; (3) it is aimed both at testing and refin-ing theories and also advancing practice; (4) it is a meth-odologically diverse and operation-oriented process; (5)designers, researchers and practitioners with differentexpertise interact frequently and share their ideas [25].Within design-based research, four phases can bedistinguished: (1) analysis and exploration, (2) designand construction, (3) evaluation and reflection, and (4)implementation and spread [26]. In the case of the IFMSsystem in the Academic Medical Center, the researchteam first began to clarify the design problem and set upa fruitful collaboration with the main stakeholders.Second, the research team worked together with thestakeholders to articulate the design principles or criteriato be taken into account. Third, the research team em-pirically investigated design ideas on a small scale andreflected on the findings to adapt and strengthen theoverall design. Fourth, the research team involved vari-ous stakeholders to tailor the new system to the specifichospital context and scale up the implementationprocess. In the following two sections, we will elaborateon the role and composition of the research team, thestakeholders involved and the different steps in the de-sign, evaluation and redesign of the IFMS system.
Research team and stakeholdersThe Executive Board of the hospital assigned the designof the IFMS system, from protocol development throughevaluation and implementation, to a hospital-based re-search group. This research group, named ProfessionalPerformance and Compassionate Care (PP&CC), has ex-tensive experience in physicians’ performance evaluation
Bindels et al. BMC Health Services Research (2021) 21:800 Page 4 of 12
and improvement (www.professionalperformance-amsterdam.com). Four members of PP&CC (KL, RS, BBand EB) formed the project team, all also co-authoringthis paper. The other two authors (SH, AS) have exten-sive experience in educational research and developmentand acted as a sounding board in the conception anddrafting of this article. In 2016 the project team soughtthe opinions of both internal and external experts onmedical leadership, hospital administration and repre-sentatives of the Dutch medical regulatory and registra-tion authorities. This was done to assure a good positionof the IFMS system within the organizational context ofthe hospital. Expertise from a psychologist (AN) wassought to develop an approach to discuss the perform-ance feedback, which would facilitate physicians’ profes-sional development. The hospital’s physician group of(17) neurologists was found willing to pilot thisapproach. Involvement of various stakeholders wasessential to tailor the new system to the specific hospitalcontext. For example, coordination with departmentheads was needed to let the IFMS system fit in with theexisting procedure of annual reviews.
Design, evaluation and redesignA cyclic process of defining underpinning design princi-ples, evaluation activities and refinement was the basisfor the development of the IFMS procedure in theAMC. The process consisted of five steps:
1. Development of an IFMS protocol;2. Selection of an MSF tool;3. Development, piloting and evaluation of a coaching
approach;4. Recruitment, training and evaluation of coaches;5. Hospital-wide implementation.
.At the beginning of the design process, the project
team together with the hospital board, set out a numberof design principles or criteria to be taken into accountin the system’s design:
� First, the IFMS protocol would have to be developedin consultation with the various stakeholders andshould align with the ambitions of the AMCregarding the integration of the IFMS procedurewith the existing procedure of annual reviews.
� Second, the MSF tool should cover the broad rangeof competencies as described in the competencyframework used by the Dutch Central College ofMedical Specialists (CCMS) [27]. Also, the toolshould be scientifically sound, easy to use and avoidadministrative burden for both the physician beingevaluated as well as his/her assessors. Other
feedback tools known to the hospital’s physicianswere to be included as well.
� Third, the coaching approach to discuss thefeedback report had to optimally facilitatephysicians’ professional development. In practice,this meant: facilitating interpretation of feedbackdata, supporting physicians in setting developmentalgoals in the context of their own practice, anddevelop a route to improvement and change.
� Fourth, the coaches needed to be skilled,trustworthy and have close affinity with the medicalworkplace. Also, as a means to increase physicians’involvement in the IFMS system, the system neededto be based on the principle ‘for physicians, byphysicians’. For these reasons, coaches should berecruited among the physician workforce (peercoaches).
� Fifth, the redesign process would be subjected toevaluation and research. It was thus decided topilot-test the IFMS procedure in one department,the neurology department. Evaluation results wouldserve as input for redesign or adjustment of theprocedure. After piloting the procedure andrecruiting and training peer coaches, the procedurewould be evaluated in two other departments at thestart of hospital-wide implementation. Thereafter,the implementation process would be scaled up toinclude all physicians. Last, the handling of the IFMSprocedure would ultimately have to be transferredfrom the (temporary) project group to theorganization.
ResultsIn this section, we will subsequently describe the fivesteps of developing the IFMS system in the AcademicMedical Center (i.e. IFMS protocol, MSF tool, coachingapproach, coaches, and hospital-wide implementation).
IFMS protocolIn consultation with the various stakeholders and basedon incremental evaluation (see further), the system waswritten down in detail in the IFMS protocol. Since theAMC was the first Dutch academic medical center topresent its solution to the new IFMS requirement, theAMC offered its protocol for review to the (national)medical registration authority (RGS). The RGS sanc-tioned the protocol. Next, the protocol was approved bythe hospital’s highest medical leadership and the WorksCouncil, and finally ratified by the hospital’s ExecutiveBoard. Upon request, the research team generouslyshared information about the IFMS protocol withindividual hospitals and through various professionalnetworks, i.e. by presenting the protocol in multiplemeetings of the Dutch Federation of Medical Specialists,
Bindels et al. BMC Health Services Research (2021) 21:800 Page 5 of 12
in particular an invitational meeting for UniversityMedical Centers medical staff representatives in November2018.The protocol describes the process of collecting MSF
data, the (planning of the) coaching session, the docu-mentation of the session, drafting the personal develop-ment plan (PDP), and the reporting and discussion ofthe PDP with the (hierarchical) peer superior (see Fig. 1).It contains specific articles reflecting ‘the middle ground’between what is mandated by law and what is preferredin terms of prioritizing and promoting professionalgrowth. Notably, the protocol contains the ‘negotiatedcompromises’ related to confidentiality of the coachingsession and (non-)disclosure of its findings. Specifically,the protocol determines the face-to-face (peer) coachingsession as strictly confidential and the physician is in-vited to bring any (performance) topic to the table fordiscussion. Also, the protocol prescribes that the result-ing navigation document – a preliminary PDP -(seeFig. 1) remains confidential; it is up to the physicianwhether or not to share this document with others, in-cluding his or her superior. Following the protocol, theonly thing the coach is allowed to report to the physi-cian’s superior is that the coaching session has takenplace. Physicians may craft their navigation documentinto a final PDP. In contrast to the navigation document,the final PDP which is modified and approved by thephysician, is sent to the superior for discussion duringthe annual review. The superior will confirm and note inthe physician’s HR file that the specialist in question hasmet the IFMS requirement; this documentation serves as
proof of compliance to be forwarded to the authorities,the RGS.
Some articles in the protocol show how the AMCchose for exceeding the minimum legal requirements infavor of facilitating professional development.
� First, instead of once every five years as required bylaw, it was decided to facilitate participation in theIFMS procedure twice every five years.
� Second, for intervening years, the protocolrecommends to additionally discuss the progress ofthe PDP with peers during a peer group meeting,under supervision of an external coach.
� Third, in addition to the required MSF data (seefurther) all specialists are expected to also submitother available systematically collected performanceinformation. In particular, physicians should submittheir feedback reports on their supervisory andteaching skills as collected through the well-validated System for Evaluation of TeachingQualities (SETQ) [28].
MSF tool: INCEPTThe INCEPT (INviting Co-workers to Evaluate PhysiciansTool) questionnaire was selected to collect multisourcefeedback [23]. The INCEPT had been previously developedby the research group PP&CC and had been investigatedfor its psychometric qualities and feasibility. The INCEPTcovers three domains of professional performance: profes-sional attitude, patient-centeredness and organization and
Fig. 1 Timeline IFMS trajectory in the Amsterdam Academic Medical Center (AMC)
Bindels et al. BMC Health Services Research (2021) 21:800 Page 6 of 12
(self-)management. It is a 21 item questionnaire andfacilitates respondents’ narrative comments. Collection offeedback takes place in a web-based environment, throughwhich physicians invite 8 peers, 8 co-workers and 8residents to evaluate their performance and also fill in aself-evaluation. Once the feedback is gathered, on averageafter four weeks, an individualized MSF report is generated.Physicians can download this report through their personalonline profile and send the report to the coach. They areencouraged to also include other sources of feedback suchas residents’ evaluations regarding training activities.Optionally, physicians can fill in a reflection form in thesame web-based environment in preparation for the coach-ing session.
Coaching approach: DANADevelopment of the DANAA coaching approach was developed by a psychologist(AN), who had approximately 20 years of experience incoaching physicians. The newly developed approach wasbased on Appreciative Inquiry practices and solution-focused coaching and was named the DevelopmentalAppreciative Navigational Approach (DANA). The aimof DANA was to optimize the use of each individualphysician’s talent, to identify visible and hidden personalqualities and to formulate an overarching professionaldevelopment goal. To enable the physician to expand onintrinsic motivation and individual qualities and to pre-vent the coaching session from becoming a tick-box-exercise in which the coach would skim through thefeedback, the duration of the coaching session was set attwo hours. The session was preferably held in the priv-acy of the physician’s office, where confidentiality wasassured at the start of the session. The session resultedin the formulation of a (short) navigation documentsummarizing the physician’s qualities and ambitions, adescription of one or more identified developmentalgoals and multiple concrete actions, preferable set in
time and place to achieve those goals. An overview ofthe substantive elements of the DANA is displayed inTable 1.
Evaluation of the DANA - methodsThe newly developed DANA was pilot-tested in the de-partment of neurology; all neurologists participated inan individual coaching session with the experiencedpsychologist. Within two weeks after the session, a re-searcher (KL) conducted semi-structured interviews toevaluate how they had experienced the DANA session inthe light of their professional development. The inter-view contained questions about how the session hadcontributed to their insight into personal qualities anddevelopmental opportunities. In addition, several gradingscales were used, such as “I have experienced this ses-sion as ‘empowering’ vs. ‘discouraging’/‘appreciative’ vs.‘judgmental’/‘safe’ vs. ‘unsafe’. Furthermore, specialistswere asked how they had experienced the role of thecoach and whether they had any objection to discuss thedocumentation of the coaching session with their super-ior during their annual review. Lastly, they were askedabout the appropriateness of the five-yearly frequency ofthis coaching session. Interviews lasted for 30–50 mi-nutes and were audio-taped. The content of the inter-views was summarized and used as input for adjustmentof the procedure. The interview protocol is available assupplementary material.
Evaluation of the DANA - resultsThe pilot test in the neurology department took placefrom January till March 2017. All 17 neurologists ratedthe session as highly motivating, appreciative,development-oriented, concrete and valuable. Theycharacterized the session as open and safe. For most ofthem, the session did not reveal new insights about theirperformance. Rather, it confirmed existing ideas, or wasinstrumental in rendering, accepting or internalizing the
Table 1 Steps of the Developmental Appreciative Navigational Approach (DANA)
Step Content
1. Introduction Build a foundation for a trusting relationship.
2. Confidentiality Discuss the purpose of the coaching session and explain the procedure.
3. ‘In your element’ Explore the ideal work situation, when the physician is able to work in a way that gives energy and brings out the bestin him/her.
4. Desired futuredevelopment
Explore ambitions and wishes for the future and satisfaction with the current range of duties.
5. Performance feedbackdata
Recognize a pattern in the personal feedback data.
6. Personal qualities Name qualities and connect with feedback data.
7. Improvement goal Identify and develop a plan for improvement; use scale questions to gain insight into motivation and self-efficacy.
8. Document Jointly reflect on the session and provide a written summary of the content of the session and the development plandiscussed.
Bindels et al. BMC Health Services Research (2021) 21:800 Page 7 of 12
feedback received. The session did however deepen theirinsights in their developmental opportunities andimprovement goals. For all neurologists, at least oneimprovement goal was formulated. They reported to behighly committed to and motivated for this goal. Therole of the coach was greatly appreciated and unani-mously rated with a minimum score of 8 out of 10.Some expressed to be interested in additional coachingsessions to follow up on the defined development goals.Most neurologists reported to consider peer coaches topotentially also be good coaches, although someexpressed their preference for a professional coach. Thispreference was related to both the confidentiality of thecoaching session and the expertise of a professionalcoach. Criticism was expressed in terms of the disclosureof the coaching document to their superior, i.e. the Headof Department, as this could hamper the openness dur-ing the coaching session. The neurologists supported theproposal to schedule a coach session every two years, in-stead of once every five years as required by law. Basedon the evaluation of the pilot test the protocol wasadjusted.
CoachesRecruitment and training of coachesGiven the importance for coaches to be trustworthy, andbearing in mind the requirement of budget neutrality,coaches were recruited among the hospital’s own phys-ician workforce. The Executive Board sent out a letter toall senior medical leadership – i.e. chairs of theorganizational divisions- in which they were requestedto nominate a number of potential coaches in propor-tion to the number of physician staff working in the div-ision. Qualities such as good communication andempathic skills were emphasized as crucial selection cri-teria. The coaching job would take approximately 30–40 hours per year for 8-10 coaching sessions, includingpreparation and drafting the navigation report. At leastonce a year, coaches would participate in a two-hourmeeting with other coaches to exchange experiences andimprove skills. The appointment as a coach was set forthe duration of three years. Participation in a trainingguided by the experienced psychologist was mandatory.The training took 8 hours and consisted of informationtransfer, direct teaching, experiential personal develop-ment exercises and coaching skills practice, using roleplay. For participation in the training, 8 accredit-ation points were awarded.
Evaluation of coachesPeer coaches were recruited among the physician work-force in July 2017. In October 2017, 10 peer coacheswere trained by the psychologist (AN). Subsequently, theproject team planned coaching sessions for them within
two departments, neonatology and rehabilitation medi-cine. Within two weeks after the coaching sessions inthese departments, two researchers (BB, RS) conductedsemi-structured interviews with physicians to evaluatethe session and the role of the coach, using a shortenedversion of the pilot interview protocol. Interviews lastedfor 15–20 min and were audio-taped. The content of theinterviews was again summarized and used as input foradjustment of the procedure.In total, 15 physicians were interviewed; their experi-
ences were comparable to those of the neurologists inthe pilot phase. However, a number of physicians notedthat the emphasis in the coaching session had been verymuch on the appreciation of positive qualities, leavingless room and attention for the formulation of develop-mental goals and drafting a plan for attaining thesegoals. In response to this, more attention was paid tostrengthening the developmental orientation of thecoaching conversation in the DANA training and themeetings with the coaches, by paying special consider-ation to the factors which may both impede and enableprogress and success, setting timelines and consideringhow to measure success. Once again, the evaluation re-sults showed that physicians sometimes had a preferencefor a professional non-peer coach. Based on theseresults, the inclusion criteria for the recruitment of add-itional coaches were extended to also include psycholo-gists and professionals who otherwise had insight intothe physician workplace. A list of external coaches wascompiled to be able to refer physicians to a professionalcoach for additional sessions if desirable or necessary,which could be funded from their personal budget.
Hospital-wide implementationGradual scale-upAfter (initial) implementation in the departments ofneurology (pilot) and neonatology and rehabilitationmedicine in 2017, the IFMS procedure was implementedin another 6 departments in 2018. The project teamcontacted the heads of these departments and informedthe medical staff about the IFMS procedure. Practicalmatters such as the management of MSF through theweb-based environment were entrusted to secretary staff,who were provided with manuals and templates. Uponrequest, heads of department were provided with a train-ing on how to follow-up on the coaching session duringthe annual review in a positive and development-oriented manner. In October 2018, the group of coacheswas expanded with 9 coaches. In the same period, anonline planning tool for efficient scheduling of sessionswas introduced. Coaches registered their availability andprovided a profile text about themselves, so that physi-cians could make an informed decision for a coach. Tosupport coaches in their development as a coach, coach
Bindels et al. BMC Health Services Research (2021) 21:800 Page 8 of 12
meetings were organized which were supervised by thepsychologist. By the end of 2018, 96 specialists had takenpart in the IFMS procedure.
Transfer to HR departmentIn May 2019, the design, evaluation and implementationof the IFMS system was completed and the procedurecould be embedded into the organization. The oper-ational responsibility of the IFMS procedure was trans-ferred from the research group PP&CC to the hospital’sHuman Resources department. The research group con-tinued to support and advise the HR staff on contentmatters, and also continues investigations into thefollow-up on coaching sessions. The HR departmentsupported continuous recruitment and training of coa-ches, as well as the scheduling of coach meetings andthe monitoring of the degree of participation and leadtime on department level. In 2019, another 12 depart-ments took part in the new system and the group ofcoaches was brought to final strength with the additionof 11 coaches, resulting in a total number of 30 coaches.By the end of 2019, 162 out of 624 physicians in thehospital had completed the IFMS procedure.
DiscussionIn this paper we reported on a design-based research(DBR) development of a new performance appraisal sys-tem that serves both physicians’ professional developmentneeds and mandatory accountability, or re-registration re-quirements. The new system reflects the compromisesmade in dealing with the perceived conflicting nature ofdevelopment and accountability purposes. Characteristicsthat appeared crucial and non-negotiable for the facilita-tion of professional development were an appreciative anddevelopment-oriented performance appraisal process, thetrustworthiness and skills of the peer coaches and a non-disclosed appraisal output. The DBR approach contrib-uted to the successful implementation and positivereception by physicians, organizational leadership andregulators. We will discuss our findings in more detailbelow by elaborating on (1) the combination of MSF andcoaching in performance appraisal, and (2) professionalregulation and performance appraisal in the organizationalsphere.
The combination of MSF and coaching in performanceappraisalIn many western healthcare systems (e.g. the USA,Canada, the UK, the Netherlands), initiatives have beendeveloped to facilitate reflection on MSF results [12, 29,30]. In this paper we reported on the development of anew coaching approach (DANA) to guide peer-to-peerconversations following MSF. To our knowledge, thisapproach is among the first positive psychology coaching
interventions that have been specifically developed forpracticing physicians participating in an MSF programin the context of revalidation. The evaluation results ofthe pilot and the initial implementation showed thatphysicians highly appreciated the session and the coa-ches. Peer coaches were considered trustworthy giventheir familiarity with the working context. Nevertheless,some physicians expressed a preference for an externalprofessional coach given their specific skills, amplecoaching experience and relative independence of “thesystem”. The latter seemed particularly true for the moresenior physicians in leadership positions. Both expertiseand confidentiality were considered of special import-ance because of the potential sensitivity of the topicsthat were discussed during the coaching session.Although we did not systematically collect these data,anecdotal evidence showed that physicians struggledwith high workloads, organizational changes, well-beingproblems or issues related to interpersonal dynamicswith colleagues and managers. This is in line with re-search on the relationship among physicians’ workload,social support and well-being [31–33], highlighting theneed for research that strengthens the evidence-basedunderpinnings of the positive psychology approach inorganizations [34].
Professional regulation and performance appraisal in theorganizational sphereThe results as reported in this paper should be consid-ered within the dynamics of the organizational sphere ofthe hospital setting. With the advent of revalidation,organizations have become intermediaries in the rela-tionship between physicians and regulatory authorities,enacting regulatory processes on their behalf and ex-tending regulatory surveillance and oversight at locallevel [35]. Within this organizational sphere, however,complex relational and governance issues already exist.In this paper, the challenge of effectively combining theformative coaching session with the existing hierarchicalannual review with the peer superior deserves special at-tention. The initial proposal of offering physicians a per-sonal coaching session which outcomes were then to bepresented to their superior for the purpose of supportingimplementation of the PDP in practice, was not deemedfeasible. Understandably, knowing that the results of apersonal coaching session will be shared with superiors,likely inhibits the agenda setting, openness and depth ofa conversation. In practice, this could lead to suboptimalsessions when areas in need of improvement or personalambitions that conflict with the strategic objectives ofthe organization, would remain underexposed. This po-tential adverse effect should always be considered inlight of the formal requirements which do not mandatedisclosing results to any authority. In our case, it was
Bindels et al. BMC Health Services Research (2021) 21:800 Page 9 of 12
decided that safety and confidentiality should take prece-dence over involvement of organizational leadership.This may indicate physicians’ lack of trust in theorganization. Although in the literature it has beenpointed out that the traditional professional-manager di-chotomy is no longer a valid reflection of contemporaryprofessions and organizations [36–38], the experience ofsuch a dichotomy may still be present, especially withinhierarchical academic medical centers. The current sys-tem’s design, where physicians are encouraged to sharetheir performance results and coaching session’s reportbut may choose not to, can be considered a compromise.The future will have to show how this turns out in prac-tice. We believe the negotiated compromise is an im-portant contribution to meaningful conversations.
Strengths and limitationsIn accordance with the principles of DBR, the designand implementation of a new system for performanceappraisal was carried out as a cyclic process of design,evaluation and redesign, in an authentic real-life setting,in which the researchers, designers and practitionersinteracted frequently to share their ideas. The DBR ap-proach allowed us to propose, test and amend solutionsfor the challenging goal of satisfactorily serving the dif-ferent purposes of accountability and professional devel-opment. The resulting system, as reported in this paper,reflects the various compromises that were reachedthrough extensive deliberations. For the success of thesystem, during the design process we constantly kept inmind the importance of the quality of the feedback con-versation. We did this by developing a new coaching ap-proach and ensuring a thorough evaluation, first whenthe conversation was conducted by a psychologist andlater again when the conversation was conducted by apeer coach. The main limitation of this DBR project re-lates to tracking the actual impact of the performanceappraisal process on physicians’ subsequent professionalperformance and development. Given the size and leadtime of the project and our focus on a solid design solu-tion, an extensive and complete impact analysis is notyet available. Efforts to secure reflexive monitoring arestill underway. Another consideration is the fact that thisproject took place in the local setting of one Dutch aca-demic medical center, so the proposed design solutionmay not fit other contexts. However, several other (aca-demic) medical centers have shared with the authorsthat they copied our protocol or used it as a basis tobuild their own. This suggests that the design is at leastto some extent transferable to other settings. Neverthe-less, we did undertake steps to stimulate a broader rele-vance than the local situation by including a detaileddescription of the context of the project and formulatingclear design principles. Similar to many other DBR
projects, the roles of researcher and designer were ful-filled by the same persons. In order to prevent the find-ings of the study to be influenced by the researchers’biases, two researchers from another institution (AS andSH) were involved in the research process.
Recommendations for practiceThere are a number of practical matters that should betaken into account when designing and implementing aperformance appraisal system in the hospitalorganization. Efforts at different times in the design andimplementation process and at different organizationallevels are necessary. First, it is crucial that the systemwill not add too much administrative burden to physi-cians and the organization in general, for example bymaking use of existing and for physicians familiar infra-structures to collect feedback data or plan coaching ses-sions. Second, the timing of the appraisal process withina department needs to fit in with other ongoing projects,so that physicians are supported in their commitment toprovide meaningful feedback to their fellow physicians.Third, it is important to ensure that peer coaches buildcoaching expertise by conducting sufficient sessions peryear. The challenge for organizations is to find workablearrangements to combine physicians’ time commitmentsto coaching and their clinical responsibilities. A combin-ation of peer coaches, non-peer coaches and externalcoaches may be one way to deal with this. Fourth, afterthe system’s initial implementation, the organizationneeds to secure reflexive monitoring. This may be con-sidered a typical responsibility of the HR department,showing the organization’s commitment to a high per-forming medical staff.
Future researchTo ensure that the performance appraisal system will in-deed contribute to physicians’ professional performanceand development, it is crucial that the system becomesroutinely embedded in physicians’ organizational andprofessional contexts. A theory that may serve as a sensi-tizing tool for longitudinal monitoring efforts is thenormalization process theory (NPT) [39, 40]. The frame-work originating from this theory facilitates systematicexploration of why some processes lead to a practice be-coming successfully (or not) embedded (i.e. normalized)and sustained, by attempting to understand the interven-tion in relation to the everyday practice of those in-volved. There are four main components to NPT whichmay be the foci of monitoring efforts: coherence (i.e.physicians’ sense-making of the performance appraisalprocess in the context of revalidation); cognitive partici-pation (i.e. commitment and engagement of physicians);collective action (i.e. the work physicians have done tomake the performance appraisal process function); and
Bindels et al. BMC Health Services Research (2021) 21:800 Page 10 of 12
reflexive monitoring (i.e. the evaluative work researchersand stakeholders do to assess and understand the bene-fits and costs of the performance appraisal process). Forthe last component, it is important to bear in mind thenature of the topics discussed during coaching sessions.Monitoring efforts could include person-centeredmeasures related to work engagement, well-being andinterpersonal communication, complemented with moredistal outcomes of coaching related to the widerorganization, such as team climate and overall hospitalculture. In this regard, the proposed group meetings todiscuss the progress of the PDP with peers, led by anexternal coach, may offer additional opportunity forgaining insight into how the new system can contributeto performance change. In the event that the appraisalprocedure and the annual review procedure remain sep-arated, hospitals need to be – in some way – informedabout the content or results of the coaching sessions.This is imperative, as hospitals and physicians bear ajoint responsibility for the quality of care.
ConclusionsIn this study, we designed a new performance appraisalprocedure for physicians undergoing re-registration thatwould successfully comply with both accountabilityrequirements as well as fulfill the need for professionaldevelopment. The study took place in the local contextof a large academic medical center in the Netherlandsand the findings maybe translated into design guidelinesbroadly applicable in other hospital settings, and evenhealthcare systems. In the context of revalidation,professional performance and development can besupported when the appraisal process is appreciative anddevelopment-oriented, when the coach is skilled andtrustworthy and when the physician has control over thedisclosure of the output of the process to others.
AbbreviationsCPD: Continuing Professional Development; MoC: Maintenance ofCertification; MSF: Multisource Feedback; DBR: Design-Based Research;IFMS: Individual Functioning Medical Specialists; INCEPT: INviting Co-workersto Evaluate Physicians Tool; SETQ: System for Evaluation of TeachingQualities; DANA: Developmental Appreciative Navigational Approach
Supplementary InformationThe online version contains supplementary material available at https://doi.org/10.1186/s12913-021-06818-1.
Additional file 1. Interview protocol for the evaluation of the DANA /coaching session
AcknowledgementsThe authors would like to thank the interviewed physicians for their timeand participation.
Authors’ contributionsEB, KL, AS and SH made substantial contributions to the conception anddesign of the article. BB, RS and AN made substantial contributions to theacquisition of data. All authors made substantial contributions to the analysisand interpretation of data. EB drafted the article; all authors revised thearticle. All authors have approved the submitted version and agree to bepersonally accountable for their contributions. All authors ensure thatquestions related to the accuracy or integrity of any part of the work, evenones in which they were not personally involved, are appropriatelyinvestigated, resolved, and the resolution documented in the literature.
Authors' informationElisa Bindels was a PhD candidate, Amsterdam Center for ProfessionalPerformance and Compassionate Care, Department of Medical Psychology,Amsterdam University Medical Centers, Amsterdam, the Netherlands, andDepartment of Educational Development and Research, Faculty of Health,Medicine and Life Sciences, Maastricht University, Maastricht, theNetherlands, at the time of the study. The author is now a lecturer,Department of Clinical Psychology, Faculty of Social and Behavioral Sciences,Utrecht University, Utrecht, the Netherlands; ORCID: 0000-0001-6096-4950. Benjamin Boerebach is staff advisor, Department of Value-Based Health-care, St. Antonius Hospital, Nieuwegein, the Netherlands; ORCID: 0000-0002-5931-9783. Renée Scheepers is assistant professor, Department of Socio-Medical Sciences, Erasmus School of Health Policy and Management, Eras-mus University of Rotterdam, Rotterdam, the Netherlands;ORCID:0000-0001-5750-3686. Annemiek Nooteboom is psychologist, Department of IntensiveCare, Amsterdam University Medical Centers, Amsterdam, the Netherlands,and independent consultant, Nooteboom Consult, Amsterdam, theNetherlands. Albert Scherpbier is professor, Department of Educational De-velopment and Research, Faculty of Health, Medicine and Life Sciences,Maastricht University, Maastricht, the Netherlands; ORCID:0000-0001-9652-0163. Sylvia Heeneman is professor, Department of Pathology, Faculty ofHealth, Medicine and Life Sciences, Maastricht University, Maastricht, theNetherlands; ORCID: 0000-0002-6103-8075. Kiki Lombarts is professor,Amsterdam Center for Professional Performance and Compassionate Care,Department of Medical Psychology, Amsterdam University Medical Centers,Amsterdam, the Netherlands;. ORCID: 0000-0001-6167-0620.
FundingNot applicable.
Availability of data and materialsThe dataset analyzed during the current study is available from thecorresponding author on reasonable request.
Declarations
Ethics approval and consent to participateThis study received an ethical waiver from the medical ethics reviewcommittee of the Amsterdam Academic Medical Center, University ofAmsterdam, the Netherlands, under dossier number W20_176.All participantsconsented to participation in the data collection by providing oral informedconsent. Ethical approval was waived for this study, given that participantswere not subjected to any interventions and/or were forced into any specificbehaviors or modes of operation. A formal audit by the committee wastherefore deemed unnecessary; no other considerations were provided tothe researchers. However, as researchers we are to comply with the rulingresearch codes and general research integrity practices. That is why weundertook several actions to protect the anonymity of our respondents, forexample by not using the names of interviewed specialists, but assigningeach specialist a number and storing the audio-taped information (includingthe oral informed consent) in a secure place.
Consent for publicationNot applicable.
Competing interestsThe authors declare that they have no competing interests.
Bindels et al. BMC Health Services Research (2021) 21:800 Page 11 of 12
Author details1Department of Medical Psychology, Amsterdam Center for ProfessionalPerformance and Compassionate Care, Amsterdam University MedicalCenters, Amsterdam, the Netherlands. 2Department of EducationalDevelopment and Research, Faculty of Health, Medicine and Life Sciences,Maastricht University, Maastricht, the Netherlands. 3Department of ClinicalPsychology, Faculty of Social and Behavioral Sciences, Utrecht University,Heidelberglaan 1, H1, 54, 3584 CS Utrecht, the Netherlands. 4Department ofValue-Based Healthcare, St. Antonius Hospital, Nieuwegein, the Netherlands.5Department of Socio-Medical Sciences, Erasmus School of Health Policy andManagement, Erasmus University of Rotterdam, Rotterdam, the Netherlands.6Department of Intensive Care, Amsterdam University Medical Centers,Amsterdam, the Netherlands. 7Nooteboom Consult, Amsterdam, theNetherlands. 8Department of Educational Development and Research,Faculty of Health, Medicine and Life Sciences, Maastricht University,Maastricht, the Netherlands. 9Department of Pathology, Faculty of Health,Medicine and Life Sciences, Maastricht University, Maastricht, theNetherlands.
Received: 5 June 2020 Accepted: 23 July 2021
References1. Brennan N, Bryce M, Pearson M, Wong G, Cooper C, Archer J.
Understanding how appraisal of doctors produces its effects: a realist reviewprotocol. BMJ Open. 2014;4(6):e005466.
2. General Medical Council. Ready for revalidation; the Good medical practiceframework for appraisal and revalidation. https://www.gmc-uk.org/-/media/documents/The_Good_medical_practice_framework_for_appraisal_and_revalidation___DC5707.pdf_56235089.pdf. Accessed 16 Apr 2020.
3. American Board of Medical Specialties. Standards for the ABMS Program forMaintenance of Certification (MOC): For implementation in January 2015.2014. http://www.abms.org/media/1109/standards-for-the-abms-program-for-moc-final.pdf. Accessed 16 Apr 2020.
5. McGivern G, Ferlie E. Playing tick-box games: Interrelating defences inprofessional appraisal. HumRelat. 2007;60(9):1361–85.
6. Chamberlain JM. Portfolio-based performance appraisal for doctors: a caseof paperwork compliance. Sociol Res Online. 2010;15(1):68–76.
7. Sehlbach C, Govaerts MJ, Mitchell S, Rohde GG, Smeenk FW, Driessen EW. Box/ticking and Olympic high jumping / physicians´ perceptions and acceptanceof national physician validation systems. Med Teach. 2018;40(9):886–91.
8. Holmboe ES, Ross K. Commentary: realizing the formative potential of multisourcefeedback in regulatory-based assessment programs. Acad Med. 2012;87(12):1657–9.
9. Lockyer J, Horsley T, Zeiter J, Campbell C. Role for assessment inmaintenance of certification: physician perceptions of assessment. J ContinEduc Health Prof. 2015;35(1):11–7.
10. Sargeant J, Bruce D, Campbell CM. Practicing physicians’ needs forassessment and feedback as part of professional development. J ContinEduc Health Prof. 2013;33(S1):54–62.
11. Sargeant J, Armson H, Chesluk B, Dornan T, Eva K, Holmboe E, van derVleuten C. The processes and dimensions of informed self-assessment: aconceptual model. Acad Med. 2010;85(7):1212–20.
12. Sargeant J, Lockyer J, Mann K, Holmboe E, Silver I, Armson H, Power M.Facilitated reflective performance feedback: developing an evidence-andtheory-based model that builds relationship, explores reactions and content,and coaches for performance change (R2C2). Acad Med. 2015;90(12):1698–706.
13. Sargeant J, Lockyer JM, Mann K, Armson H, Warren A, Zetkulic M, HolmboeE. The R2C2 model in residency education: how does it foster coaching andpromote feedback use? Acad Med. 2018;93(7):1055–63.
14. Sargeant J. Future research in feedback: how to use feedback and coachingconversations in a way that supports development of the individual as aself-directed learner and resilient professional. Acad Med. 2019;94(11S):9–10.
15. West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent andreduce physician burnout: A systematic review and meta-analysis. Lancet.2016;338:2272–81.
16. College Geneeskundige Specialismen. Besluit herregistratie specialisten.2015. https://www.demedischspecialist.nl/sites/default/files/Besluit%20herregistratie%2013 %20juli%202016.pdf. Accessed 16 April 2020.
17. Overeem K. Doctor performance assessment. Development and impact of anew system. PhD thesis, 2011.
18. Orde van Medisch Specialisten. Beleidsdocument “Individueel Functioneren vanMedisch Specialisten” Persoonlijk beter. https://www.demedischspecialist.nl/sites/default/files/Rapport%20IFMS%20Persoonlijk%20Beter.pdf. Accessed 20 Jan 2021.
19. Overeem K, Lombarts MJMH, Arah OA, Klazinga NS, Grol RP, WollersheimHC. Three methods of multisource feedback compared: a plea for narrativecomments and coworkers’ perspectives. MedTeach. 2010;32(2):141–7.
20. Violato C, Marini A, Toews J, Lockyer J, Fidler H. Feasibility and psychometricproperties of using peers, consulting physicians, co-workers, and patients toassess physicians. Acad Med. 1997;72:82–4.
22. Geeraerts GAG, Hoofwijk HA. Assessing Medical Professionals [in Dutch].Houten: BohnStafleu van Loghum; 2006.
23. Van der Meulen MW, Boerebach BCM, Smirnova A, Heeneman S, oude EgbrinkMGA, van der Vleuten CPM, Arah OA, Lombarts MJMH. Validation of theINCEPT: A Multisource Feedback Tool for Capturing Different Perspectives onPhysicians’ Professional Performance. J Contin Educ Health Prof. 2017;37:9–18.
25. Dolmans DH, Tigelaar D. Building bridges between theory and practice inmedical education using a design-based research approach: AMEE GuideNo. 60. Med Teach. 2012;34(1):1–10.
27. Centraal College Medisch Specialisten. Kaderbesluit CCMS. 2009. Availableon http://www.knmg.nl.
28. BoerebachBC, Lombarts KM, Arah OA. Confirmatory factor analysis of thesystem for evaluation of teaching qualities (SETQ) in graduate medicaltraining. Eval Health Prof. 2016;39(1):21–32.
29. Francois J, Sisler J, Mowat S. Peer-assisted debriefing of multisourcefeedback: an exploratory qualitative study. BMC Med Educ. 2018;18(1):1–7.
30. Bindels E, Van den Goor M, Scherpbier A, Lombarts K, Heeneman S. Sharingreflections on multisource feedback in a peer group setting: stimulatingphysicians’ professional performance and development. Acad Med. 2021;Accepted for publication.
31. Aalto AM, Heponiemi T, Josefsson K, Arffman M, Elovainio M. Socialrelationships in physicians’ work moderate relationship between workload andwellbeing—9-year follow-up study. Eur J Public Health. 2018;28(5):798–804.
32. Heponiemi T, Kouvonen A, Vänskä J, Halila H, Sinervo T, Kivimäki M,Elovainio M. Effects of active on-call hours on physicians’ turnoverintentions and well-being. Scand J Work Environ Health. 2008;34(5):356–63.
33. Dewa CS, Loong D, Bonato S, Thanh NX, Jacobs P. How does burnout affectphysician productivity? A systematic literature review. BMC Health Serv Res.2014;14(1):325.
34. Van Woerkom M, Bakker AB, Leiter MP. Positive psychology interventions inorganizations. J Occup Organ Psychol. 2021;94:221–9.
35. Tazzyman A, Bryce M, Ferguson J, Walshe K, Boyd A, Proce T, Tredinnick-Rowe J. Reforming regulatory relationships: the impact of medicalrevalidation on doctors, employers, and the General Medical Council in theUnited Kingdom. Regul Gov. 2019;13(4):593–608.
36. Waring J, Currie G. Managing expert knowledge: organizational challenges andmanagerial futures for the UK medical profession. Organ Stud. 2009;30(7):755–78.
37. Noordegraaf M. Hybrid professionalism and beyond:(new) forms of publicprofessionalism in changing organizational and societal contexts. J Prof Org.2015;2(2):187–206.
38. Noordegraaf M. Risky business: How professionals and professional fields(must) deal with organizational issues. Organ Stud. 2011;32(10):1349–71.
39. May C, Finch T. Implementing, embedding and integrating practices: anoutline of normalization process theory. Sociology. 2009;43(3):535–54.
40. Tazzyman A, Ferguson J, Hillier C, Boyd A, Tredinnick-Rowe J, Archer J, Regan deBere S, Walshe K. The implementation of medical revalidation: an assessmentusing normalization process theory. BMC Health Serv Res. 2017;17:749.
Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.
Bindels et al. BMC Health Services Research (2021) 21:800 Page 12 of 12
