1 ADVANCED ASSESSMENT IN ENHANCING PALLIATIVE CARE PRACTICE 7KNIM726 – 2019/20 Level: 7 Credits: 30 Module Leader: Jo De Souza Tel: 020 7848 3622 Email: [email protected]Module Deputy: Chris Mangar Tel: 020 7848 3583 Email: [email protected]_____________________________________________________________ This handbook must be read in conjunction with module information provided on KEATS, the King’s E-Learning And Teaching Service. You will be given access to KEATS on enrolment. Important information relating to assessment and related regulations can be found in the relevant programme handbook. This handbook can also be provided in alternative formats (such as large print) upon request to [email protected].
Session 6: General Physical Examination ............................................................................11
Session 7: Advanced Assessment & Clinical Examination of a Person with Palliative Care Needs .................................................................................................................................................11
Session 8: Using Secondary Data – Radiology.....................................................................11
Session 9: Using Secondary Data – Blood Tests ................................................................. 12
Session 10: Assessment of Common Respiratory ............................................................... 12
Session 11: Assessment of Common Gastrointestinal Symptoms (with demonstration). 12
Session 12: Assessment of Common Neurological Symptoms (with demonstration) ...... 13
Session 13: Assessment of Common Symptoms of Advanced Heart Failure .................... 13
Session 14: Assessment of Common Mental State Symptoms ........................................... 13
Day 6-10 ......................................................................................................................................... 14
Action from previous evaluations ........................................................................................ 17
3
Module Overview This standalone module has been developed in partnership with St Christopher’s Hospice. The first day will be delivered at KCL, Waterloo and the remainder at St Christopher’s Hospice, Sydenham.
This course aims to facilitate the personal and professional development of palliative care nurses and allied health professionals (AHPs) engaged in advancing their clinical practice and enhancing palliative care service delivery. The aim is to equip practitioners to work effectively within their organisational structure and to apply key principles to the development of their practice/service. The development of these skills will be underpinned by strategic concepts, appropriate theoretical frameworks and analytical tools, which will encourage practitioners to critically analyse the scope of their current role.
Module Outcomes
The course will enable practitioners to: 1. Critically examine the historical development of palliative care practice, in order to
understand the current context 2. Critically evaluate the organisation & delivery of relevant palliative care services,
demonstrating an in-depth understanding of social, political and professional influences which determine palliative care policy
3. Critically review the development of intra/inter professional working specifically in relevant palliative contexts
4. Critically examine legal and ethical issues influencing and inherent in advanced palliative care practice
5. Critically evaluate and apply models of problem solving and decision making and analyse influences on their own decision-making process
6. Demonstrate the necessary interpersonal skills, required to develop an effective therapeutic relationship with the patient and family/carers
7. Demonstrate the process of in-depth holistic assessment of patients with advanced progressive disease, including the analysis of primary and secondary data and physical examination when appropriate.
8. Demonstrate the coordination of complex decision-making processes involved in the management of care, particularly at the end of life.
9. Critically explore barriers and facilitators to role development, in relation to course learning outcomes
Academic Support Academic support is received through the course leaders (contact details on page 1). Opportunities for a tutorial to discuss the assessment can be arranged on an individual basis as required. The members of the teaching team can also be approached for advice regarding their particular area of expertise. Teaching arrangements The learning outcomes will be achieved through self-directed study, taught sessions, seminars, practice-based learning and the written assignment. The aim of the sessions is to provide a safe environment for stimulating interesting interaction. We hope to encourage your participation and look forward to discussion, questions and challenges. A variety of practitioner-centred, adult learning approaches will be adopted to facilitate shared learning, attitudinal change and professional reflection. These include lectures, discussion, skills training and supervised practice, small group work, guided professional practice, personal reflection and experiential learning.
4
Core reading
The following books are recommended as core reading prior to commencing the course. Please note; some do appear again for specific sessions. N.B. You will need to have refreshed your knowledge and understanding of relevant anatomy and physiology prior to commencing the module. Martini et al. (2018) is highly recommended. Try the university or local authority library. Barton, T., & Allan, D. (2015). Advanced Nursing Practice: Changing Healthcare in a Changing
World. London: Palgrave Macmillan.
Cox, C., Hill, M., & Lack, V. (Eds.), (2011). Advanced Practice in Healthcare: Skills for Nurses and Allied Health Professionals. London: Routledge.
Epstein, O., Perkin, D.G., de Bono, D.P., & Cookson, J. (2008). Clinical Examination. (4th ed.). Edinburgh: Mosby.
Ferrell, B., & Coyle, N. (2008). The Nature of Suffering and the Goals of Nursing. USA: Oxford University Press.
Gawande, A. (2015). Being Mortal: Illness, Medicine and What Matters in the End. London: Profile Books.
Hamric, A.B., Spross, J.A., & Hanson, C.M. (Eds.), (2013). Advanced Practice Nursing: An Integrative Approach. 5th ed. Elsevier Saunders, St Louis.
Innes, J.A., Dover, A.R., Fairhurst, K., Britton, R., & Danielson, E. (2018). Macleod’s Clinical Examination. Edinburgh: Elsevier.
Health Education England. (2017). Multi-professional framework for advanced clinical practice in England. Retrieved 21 October 2019 from https://www.hee.nhs.uk/sites/default/files/documents/Multi-professional%20framework%20for%20advanced%20clinical%20practice%20in%20England.pdf
Martini, F., Nath, J., & Bartholomew, E. (2018). Fundamentals of Anatomy and Physiology. (11th ed.). San Francisco: Pearson.
Rawles, Z., Griffiths, B., & Alexander, T. (2015). Physical Examination Procedures for Advanced Nurses and Independent Prescribers: Evidence and Rationale. (2nd ed.). London: Routledge.
Walshe, C., Preston, N., & Johnson, B. (2018). Palliative Care Nursing. (3rd ed.). UK: Open University Press.
Royal Pharmaceutical Society (2013). The RPS Advanced Pharmacy Framework. Retrieved 21 October 2019 from http://www.rpharms.com/faculty-resources/advanced-pharmacy-framework.asp
Assessment criteria All parts of the assessment must be completed and submitted by the dates below. Failure to submit assignments or attend examinations will result in a fail grade being awarded. Please read carefully the information in the Postgraduate Handbook regarding assignments/examinations.
Summative Assessments Academic Assignment (to be submitted electronically via KEATS): Overview of Summative Assessment: 1. A 3,000-word Academic Assignment:
Contribution to course assessment — 100% of the course mark. 2. Practice Assessment: Contribution to course assessment – compulsory (must be submitted having followed the guidelines given below). Academic Assignment (to be submitted electronically via KEATS): This assignment will demonstrate learning from the module, linking theoretical knowledge and practice experience. Title: Critical evaluation of an exemplar from practice demonstrating advancement of palliative care. Guidelines: The assignment should:
Respect confidentiality according to professional standards of practice, for example the NMC Code (2015)
Discuss the debate around the definition of advanced practice
Briefly describe and offer a rationale for a chosen exemplar of your role in advancing practice
Through analysis and synthesis of appropriate literature, offer a critical exploration of your role in relation to a defined model of advanced practice, using your exemplar as an illustrative example
Critical analysis of learning from the module in relation to developing your role as an advanced practitioner in your personal context
Include an introduction, body and conclusion
A complete reference list using the standard Harvard referencing scheme We would like you to write your assignment in lines with the submission guidelines from a nursing journal. We would suggest the use of the guidelines for the British Journal of Nursing. These will be distributed and discussed in class.
Practice Assessment Portfolio (to be submitted as hard copy)
This will comprise 4 workups of clinical assessments undertaken in practice verified by your mentor (no more than 1,000 words each). Please note that each of the four core systems taught on the course must have been clinically assessed within at least one of the workups. It must also include evidence that 2 of the above clinical assessments were observed and critically reviewed by the student’s mentor using the provided marking criteria. The above must be completed from your practice setting, and both the verification sheet and marking sheet completed and signed by your practice-based mentor prior to submission. Mentor in Practice: To undertake this course, you will need to be in clinical practice. Prior to commencing it is important to identify a practice-based mentor within your organisation. This should be a medical colleague who is a member of your multidisciplinary team and with whom you have regular contact and/or an advanced practitioner of any discipline who has undertaken an advanced assessment skills course and who is competent in physical examination of palliative care patients. In addition to the ten taught days it is essential that you allow four extra clinical study days. These four days should be used to observe and practice clinical assessment skills with your mentor(s) whenever possible and in particular those skills needed to undertake physical examination. Your mentor will be required to sign and verify your completion of the four clinical assessments required for your portfolio and will also be required to observe you undertaking two full clinical assessments and critically review your competence. Guidelines: Each of the 4 patient assessments must:
State how consent was obtained
Demonstrate a thorough and focused approach to history taking and systems review
Include rationale and outcomes of any physical examination
Include rationale for any secondary data used to assist in clinical decision making
Give some indication of differential diagnosis
Include an impression/diagnosis
Include a plan for management and justify how you made your decisions
Have been presented to or discussed with members of the multidisciplinary team
7
PLEASE NOTE THE FOLLOWING: 1. Informed consent Patients' informed consent MUST be obtained to undertake all assessments and use their data in either assignment. Consent may be obtained verbally but must be clearly documented in writing for each assignment. If the patient is not competent to give informed consent (e.g. confused, sedated, unconscious, cognitive difficulties) this must be obtained from the medical practitioner in-charge of the patient’s case. It is also recommended that you also gain assent (agreement) from the patient’s family or significant other. 2. Confidentiality Patients' anonymity MUST be respected at all times. You must state how you have maintained confidentiality in each assignment. Confidentiality of the patient/client must be maintained according to the NMC Code of Professional Conduct: Standards for Conduct, Performance and Ethics (NMC, 2015). Please note that confidentiality/ anonymity also applies to the ward/department, trust and health care professionals who must not be identified within the portfolio. 3. Unsafe practice Any issue regarded as unsafe practice (including a major breach of confidentiality) will constitute an automatic fail.
Submission date for summative assessment Academic Assignment: Tuesday 7 April 2020 by 11.59am
Practice Assessment Document: Tuesday 21 April 2020 by 11.59am
Coursework submission are provided on the modules KEATS site. It is essential that you use your candidate number on all assignments/examinations. Information on your candidate number will be available on the KEATS site. If you are unable to submit your work by the deadline please refer to the information on mitigating circumstances”. If you require further support in these circumstances, you are advised to contact KCLSU.
Late submissions will be accepted for 24 hours following the submission date. All work submitted late will be marked as normal but will be capped at the pass mark for the module. If your assignment is hard copy, please ensure you date stamp it and submit it to the submission room G15 JCMB. If your assignment is submitted electronically through TurnItIn, information about how to submit late will be provided on KEATS module sites under assessment information.
Results and resubmissions for coursework Students will receive a provisional (unratified) mark for their coursework 4 weeks following submission. According to the method of submission as detailed on your KEATS site, if your work was submitted online you will be able to download marked coursework from KEATS. Alternatively, if you submitted a hard copy you can collect your coursework and feedback from the Nightingale Student Hub. To collect a hard copy assignment from the Nightingale Student Hub, you must show your King’s student identification card. Alternatively, you may send a stamped addressed envelope to Nightingale Student Hub ensuring that this is large enough to accommodate your assignments and that you have applied sufficient postage. Hard copy assignments will be retained for four weeks; if you have not collected your assignment by then, it will be destroyed. The Hub is open Mondays, Tuesdays, Thursdays and Friday from 9am-5pm and from 9am-12pm on Wednesdays. Feedback for all assessments will include the award of a numerical grade which remains provisional until ratified by the Assessment Board. The dates for the Assessments Boards are available on KEATS. Ratified marks can be viewed via Student Records on the King’s Intranet, the Monday following the relevant Assessment Board. The marking criteria by which your work is judged are provided in full on KEATS. Please also refer to the section on plagiarism and how to avoid it. If you have a query about how to refer to a specific piece of work please ask your module leader, your group leader or a member of library staff for guidance or please use King’s Libguides site. There are KLaSS resources which all students are encouraged to work through. The feedback you receive on your assignment will guide you towards how to improve future assignments or how to maintain your existing high standard! If you are unsuccessful, it is recommended that you contact the module leader before submitting your second attempt. This will enable the module leader to provide you with an appropriate level of support as you prepare to re-submit your work. Module leaders may indicate another for support. For further information King’s College London’s Academic Regulations for the current academic year can be found here: https://www.kcl.ac.uk/campuslife/acservices/academic-regulations/index Up-to-date information will be available on the KEATS Assessments Hub area as well. Resubmission date Tuesday 14 July 2030 11.59 midday
Course Management Team Key teaching staff from both KCL & St Christopher’s Hospice, clinicians and user/client views are represented in the construction and ongoing evaluation of the course.
Aim: To increase understanding of relevant theoretical foundations to advancing practice in a palliative care context and the scope of the advanced practice role. Outcomes:
By the end of the day students should be able to:
Demonstrate awareness of appropriate theory regarding advanced practice
Critically explore the scope of ‘advancing’ practice
Critically discuss the current context of palliative care. Indicative reading:
Hill, B. (2017). Exploring the development and identity of advanced practice nursing in the UK. Nursing Management, 24, 36-40.
Oberle, K., & Allen, M. (2001). The Nature of Advanced Practice Nursing. Nursing Outlook. 49(3).
Por, J. (2008). A critical engagement with the concept of advancing nursing practice. Journal of Nursing Management, 16, 84-90.
Sheer, B., & Wong, F.K.Y. (2008). The development of advanced nursing practice globally. Journal of Nursing Scholarship, 40(3), 204-211.
10
Days 2-5 Aim: To equip students to critically evaluate their clinical knowledge and skills and the scope of the advanced practice role in their specific area of practice.
Session Outcomes:
Session 4: Assessment Tools and Documents
By the end of the session the student should be able to:
Critically reflect on the benefit or restriction of structured assessment tools
Critically evaluate the tools used within their own clinical setting.
Indicative reading:
NICE Evidence search. Retrieved 21 October 2019 from http://www.evidence.nhs.uk/search?q=nursing+assessment+tools
The Outcome Assessment and Complexity Collaborative (OACC). Retrieved 21 October 2019 from http://www.kcl.ac.uk/lsm/research/divisions/cicelysaunders/research/studies/oacc/index.aspx
Session 5: Advanced Assessment — A Structured & Systematic Approach
By the end of the session the student should be able to:
Understand and critically evaluate the principles of systematic, structured histories and systems reviews.
Critically evaluate knowledge and skills required for consideration of differential diagnosis.
Begin to understand the range of patient/client data required from both primary and secondary sources in order to make informed clinical decisions.
Understand and have practiced the skills required to undertake a structured history and systems review.
Indicative reading:
Coulehan, J.L., & Block, M.R. (2006). The Medical Interview: Mastering Skill for Clinical Practice. (5th ed.). Philadelphia: FA Davis.
Epstein, O., Perkin, D.G., de Bono, D.P., & Cookson, J. (2008). Clinical Examination. (4th ed.). Edinburgh: Mosby.
Fitzimmons, D., & Amedzai, S. (2008). Approaches to assessment in palliative care. In: S. Payne., J. Seymour & C. Ingleton (Eds.), Palliative Care Nursing. Principles and Evidence for Practice. (2nd ed., pp. 163-185). Maidenhead: Open University Press.
Innes, J.A., Dover, A.R., Fairhurst, K., Britton, R., & Danielson, E. (2018). Macleod’s Clinical Examination. Edinburgh: Elsevier.
Martini, F., Nath, J., & Bartholomew, E. (2018). Fundamentals of Anatomy and Physiology. (11th ed.). San Francisco: Pearson.
By the end of the session the student should be able to:
Understand the general principles of physical examination and be able to critically apply these to own practice.
Indicative reading:
Baid, H. (2006). The Process of Conducting a Physical Assessment: A Nursing Perspective. British Journal of Nursing. 15(3), 710-714.
Department of Health (2009). Reference Guide to Consent for Examination and Treatment [online]. Retrieved 21 October 2019 from https://www.gov.uk/government/publications/reference-guide-to-consent-for-examination-or-treatment-second-edition
Session 7: Advanced Assessment & Clinical Examination of a Person with Palliative Care Needs
By the end of the session the student should be able to:
Critically apply the principles of systems review and physical examination to the examination of a patient with palliative care needs.
Critically analyse the use of interpersonal skills for data collection. Indicative reading:
Doyle, D., Hanks, G.W., Cherny, N., & Calman, K. (Eds.), (2015). Oxford Textbook of Palliative Medicine. (5th ed.). Oxford: Oxford University Press.
Session 8: Using Secondary Data – Radiology
By the end of the session the student should be able to ‘rule in or rule out’ of a differential diagnosis by:
Recognise the full range of secondary data sources available to support clinical decision-making.
Critically evaluate the appropriate use of a secondary data when making informed clinical decisions in a palliative care context.
By the end of the session the student should be able to ‘rule in or rule out’ of a differential diagnosis by:
Critically discuss the rationale for using blood test results to inform clinical decisions in the management of people with palliative care needs.
Analyse common palliative care symptoms and signs including reversible emergencies and their application to interpretation of a range blood test results.
Session 10: Assessment of Common Respiratory
By the end of the session the student should be able to:
Understand the principles of history taking focused on the respiratory system of a person with palliative care needs.
Understand the principles of clinical examination of the respiratory system of a person with palliative care needs.
Analyse common respiratory and signs including potentially reversible emergencies and their application to history taking and clinical examination.
Understand the principles of documenting a respiratory examination Indicative reading:
Simpson, H. (2006). Respiratory Assessment. British Journal of Nursing. 15(9), 484-492.
Session 11: Assessment of Common Gastrointestinal Symptoms (with demonstration)
By the end of the session the student should be able to:
Understand the principles of history taking focused on the gastrointestinal system of a person with palliative care needs.
Understand the principles of an abdominal examination
Analyse common gastrointestinal symptoms and signs and their application to history taking and clinical examination.
Recognise bowel sounds and the implications of abnormalities.
Understand the principles of documenting an abdominal examination.
13
Session 12: Assessment of Common Neurological Symptoms (with demonstration)
By the end of the session the student should be able to:
Understand the principles of history taking focused on the neurological system of a person with palliative care needs.
Understand the principles of clinical examination of relevant cranial nerves, motor system and reflexes and sensation.
Analyse common neurological symptoms and signs including reversible emergencies and their application to history taking and clinical examination.
Understand the principles of documenting a neurological examination. Session 13: Assessment of Common Symptoms of Advanced Heart Failure (with demonstration)
By the end of the session the student should be able to:
Understand the principles of clinical examination and management of the cardiovascular system in heart failure
Analyse common heart failure signs and symptoms and apply this to the patient’s history and physical examination
Understand the principles of documenting a cardiovascular assessment
Session 14: Assessment of Common Mental State Symptoms
By the end of the session the student should be able to:
Understand the principles of history taking focused on mental health of people with palliative care needs.
Analyse different mental health assessment tools and their application to practice in a palliative care context.
Indicative reading:
Bryan, L., & Scott, S. (2008). The assessment of mental state in advanced stage of disease. End of Life Journal. 2(1).
14
Day 6-10
Aim: To equip students to critically evaluate their practice in light of the four pillars that underpin advanced clinical practice and the scope of the advanced practice role in their specific area of practice.
Outcomes:
By the end of the 5 days, participants should be able to:
Demonstrate an increased awareness of the possibilities of advancing practice in palliative care through the four pillars of advanced practice , advanced clinical skills, research, education and leadership.
Identify areas for development in personal skill bases, with regard to advancing practice Demonstrate through presentation and group discussion an understanding of the
implications of the chosen themes to their own service delivery and development Critically demonstrate advanced clinical decision-making skills related to patient
assessment.
Session 1
Aim: To broaden knowledge in relation to leadership and management of advancement in palliative care practice
Outcomes:
By the end of the day students should be able to:
Critically examine relevant theoretical perspectives on leadership and management of change
Critically analyse own role in relation to leadership and facilitation of change in practice
Critically evaluate the opposing arguments for and against the extension of the scope of the clinical role of the advanced palliative care practitioner
Critically analyse legal, ethical and professional issues related to undertaking advanced assessments.
Indicative reading:
Fischer, S.A. (2016). Transformational leadership in nursing: a concept analysis. Journal of Advanced Nursing, 72(11), 2644-2653.
Frankel, A. & PGCMS, R. (2019). What leadership styles should senior nurses develop? Hospital, 6, 8.
George, S. & Leasure, A.R. (2016). Application of transformational leadership principles in the development and integration of palliative care within an advanced heart failure program. Dimensions of Critical Care Nursing, 35(2), 59-65.
Murphy, L. (2005). Transformational leadership: a cascading chain reaction. Journal of Nursing Management 13, 128-136.
Sellgren, S., Goran, E., & Goran, T. (2006). Leadership styles in nursing management: preferred and perceived. Journal of Nursing Management 14, 348-355.
Hardy, S., Titchen, A., Manley, K., & McCormack, B. (2006). Re-Defining Nursing Expertise in the United Kingdom. Nursing Science Quarterly 19(3).
Hinchliff, S., & Rogers, R. (2008). Competencies for Advanced Nursing Practice. Chapter 2: The Legal and Regulatory Implications of Advanced Nursing Practice. London: Hodder Arnold.
15
Days 7-10 AIM: To consolidate learning from days 1-6. Outcomes:
By the end of the 4 days, participants should be able to
Demonstrate an increased awareness of the possibilities of advancing practice in palliative care through advanced clinical skills, research, education and leadership.
Identify areas for development in personal skill bases, with regard to advancing practice
Demonstrate through presentation and group discussion an understanding of the implications of the chosen themes to their own service delivery and development
Critically demonstrate advanced clinical decision-making skills related to patient assessment.
Session 2: Education
AIM: To broaden knowledge in relation to education in the advancement in palliative care practice
Outcomes: By the end of the day students should be able to:
Critically examine relevant theoretical perspectives on education in health care in relation to the delivery of palliative care
Critically evaluate current need for workforce development
Critically examine the role and responsibility of the advanced nurse as an educator
Critically analyse own role in relation to education to facilitate high quality practice
Session 3: Research
Outcomes: By the end of the day students should be able to:
Critically discuss the relationship between research and Advanced Nursing Practice
Identify a range of strategies for advancing palliative and end of life care through research
Critically examine how we can move from small ideas to changing practice
Indicative reading:
Cox, S., Murtagh, F.E., Tookman, A., Gage, A., Sykes, N., McGinn, M., Kathoria, M., Wilderspin, H. and Chart, L., 2017. A review of specialist palliative care provision and access across London–mapping the capital. London journal of primary care, 9(3), pp.33-37.
Michie, S. et al., (2011). The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci, 6(1), 42-52.
16
Session 4: Consolidation of Learning & Assignment Preparation
Outcomes:
By the end of the 5 days students should be able to:
Demonstrate an increased awareness of the possibilities of advancing practice in palliative care through the four pillars of advanced practice , advanced clinical skills, research, education and leadership.
Identify areas for development in personal skill bases, with regard to advancing practice
Critically demonstrate advanced clinical practice through completion of their written assignment and clinical portfolio.
Indicative reading
Walshe, C., Preston, N., & Johnson, B. (2018). Palliative Care Nursing. (3rd ed.). UK: Open
University Press.
Web Resources
AANPE UK. Retrieved 21 Oct0ber 2019 from http://aape.org.uk/about/
Health Service Journal. Retrieved 21 Oct0ber 2019 from https://www.hsj.co.uk/
NHS Scotland – Advanced Nursing Practice Toolkit. Retrieved 21 Oct0ber 2019 from http://www.advancedpractice.scot.nhs.uk/
NMC (2015). The NMC Code of Professional Conduct: Standards for Conduct, Performance and Ethics. Retrieved 21 Oct0ber 2019 from http://www.nmc-uk.org/Publications/Standards/The-code/Introduction/
NMC (2010) Advanced Nursing Practice Update. Retrieved 21 Oct0ber 2019 from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215935/dh_121738.pdf
NHS Institute for Innovation and Improvement. Retrieved 21 Oct0ber 2019 from http://www.institute.nhs.uk/
NHS Leadership Academy. Retrieved 21 Oct0ber 2019 from http://www.leadershipacademy.nhs.uk/
Module evaluation As part of the university’s Student Voice Strategy, King’s uses an electronic module evaluation system known as EvaSys. This provides an opportunity for you to feedback on different aspects of the module through a series of pre-set questions and qualitative comments. At the end of the module you will receive an automated invitation via your KCL email account to complete your evaluation online.
Please take the time to complete as your feedback is important. It informs ongoing developments to individual modules to ensure that the learning needs and expectations of the Faculty’s student community are met to a high standard.
To strengthen the feedback cycle, a report summarizing the quantitative results for the module as a whole and the module lead’s reflections on your feedback will be sent to you after the online evaluation survey has closed.
