KMC WorkshopGroup E
Monitoring and Evaluation
Clarification of Concepts
• Monitoring: vigilance of a process• Evaluation: assessment, value judgment about
a process and its results.• The group decided that it was going to focus
on monitoring, rather than on the specifics of evaluation
Clarification of Concepts
• Monitoring in KMC can have several meanings:– Vigilance of the implementation of KMC at different
levels• Global (WHO)• Regional• National• Local, etc.
– Vigilance of an ongoing program, again:• Global• Regional, etc.
Clarification of Concepts
• Prior to defining what to monitor during implementation, goals and plans for implementation should be stated.
• Main purposes of monitoring then, would be:– Surveillance of the compliance with
implementation tasks and steps, timetable, etc.– Quantification of performance (e.g. number of
trained health workers, etc.)– Quantification of achieved goals (e.g. mortality
reduction)
Clarification of Concepts• Monitoring of an ongoing program can also have
several purposes and objectives• The group identified monitoring a as a health
care quality assurance tool as a very important purpose.
• Most of the following discussion was centered then on “evidence-based quality assurance” for KMC programs
• The importance of monitoring for implementation was nevertheless acknowledged as well as the need for address it a next step
Objectives
• To identify elements and domains relevant for developing appropriate monitoring tools for quality assessment-improvement (quality assurance) of an ongoing KMC program.
• To generate a series of statements which can help to define good practices for monitoring KMC programs
• To develop the process, a hypothetical program for delivering KMC in a health facility was sometimes used.
• Extrapolation to other settings and to broader scopes can be made.
The EB-Quality improvement cycle• Evaluate current practice, identify problems
– Separation of mother and infant after birth– Low breast feeding rates– Undesired variability in practices and/or in outcomes
• Plan and implement interventions– Set quality standards for practice (e.g. clinical practice guidelines, evidence-
based)– Set quantitative goals– Implement practices
• Monitor – Compliance with requirements– Performance
• Compliance with guidelines• Frequency of selected outcomes
• Adjust performance according to monitoring• Evaluate results (close the cycle, go back to 1st step) and start all over
Delivering KMC implies • A KMC program:– Resources, administration, planning, management, put
together to deliver• The KMC intervention– Set of specific processes (interventions ) for caring for the
health care of newborn infants involving and empowering their mothers-families
– Using a specific method or technique• The KMC method, a complex non-pharmacological
intervention clearly standardized, defined and supported by scientific evidence.– The method is defined as EB recommendations, usually in
the form of structured detailed protocols.
Components of a KMC program as related to health care quality
• Structure• Processes• Outcomes
(Donabedian)
Structure
• Stable part of the Health care system, that provides the support and setting in which health KMC is delivered:– Physical structure– Administrative structure and processess– Norms– Resources• Technical• Human• Capital
Process• Actual specific health care interventions and
procedures employed for providing care– Delivered interventions should be backed by scientific
evidence supporting that they do more good than harm– They should be feasible:
• Available• Affordable• Proficiently performed
– Properly trained personal– Technically appropriate equipment
• Acceptable for– Target population and– Health care personnel
• Ethically appropriate
Outcomes
• Changes in different aspects of health associated with previously delivered interventions– Disease (condition) centered
• Mortality• Morbidity, complications and sequels• Time to event
– Patient centered• Growth and development• Satisfaction• Health related quality of life
Monitoring KMC programs for Quality assurance• Structure:
– Standards should be set• Rate of nurses to patients• Locative facilities for mothers to provide kangaroo position• Etc.
– Verification that minimum acceptable standards are met (Basic part of certification? Accreditation?)
• Process– Recommendations (evidence-based guidelines) have identified
processes known to do more good than harm– Compliance with recommendations is monitored
• Outcomes– Given that there should be evidence showing that recommended
processes do more good than harm not every specific health outcome needs to be monitored
– Nevertheless, given that despite having evidence-based recommendations, thing can go wrong (poor performance, other quality issues) some important and selected health outcomes should be monitored.
Proposed framework to identify what to monitor
Structure Process Condition centered outcome
Patient centered outcome
Method(Technique)Intervention
Program
An exampleElement Structure
MethodDoes the program has Guidelines or Protocols?Explicit, witten?
Clear identification of components?Standardized definitions of components?Identification of therapeutic goals for each component?Evidence-based recommended courses of action?Explicitly defined setting?
Explicitly defined target population?
Clear inclusion / exclusion criteriaClear role definition for health care personnel?
An example (cont.)Element Structure Process OutcomesIntervention Properly trained health care personnel
Compliance withInitiation Mother recruitment processes and inclusion criteria
Infant selection criteriaKP Kangaroo adaptation including monitoring
Kangaroo nutrition strategy initiationKN Apropiate support for breast feeding
Apropiate use of feeding methodsApproriate monitoring of feedingMaintenance of position
Follow up policies Discharge - Follow up proceduresmortalityKMC-related eventsHospital stay Clinic visits
Mother mood depresionBonding indicatorsAttachment indicatorsParents satisfaction
Examples of specific indicators• Performance indicators of a KMC program – LBW Infants in KP first visit to the clinic/total LBW
infants discharged from hospital• Minimum acceptable 90%• Optimal 95% (there are infants discharged after leaving KP)• Measures compliance with KP between discharge and first
visit to out KMC clinic– Hours a day a parent can stay at the NICU
• Minimum acceptable 12 /24• Optimal 24/24• Measures appropriateness of structure:
– Regulations and norms to remove access barriers– Amenities, comfort, space to allow parents to stay
Examples of specific indicators: the “SKIND” score
Item 0 1 2
SSC in 1st hour <50 >50 60 ‘
KP duration in the 1st 6 hours
< 4 h <5.5 >5.5 h
Initiate BF (Self attachment observed)
No 1-2 h 1st hour
Nutrition, Artificial feeding given?
AIF Clear fluids None
Delayed Procedures for 6 h
None bath all
Recommendations• Setting Standards for structure and making sound evidence-based
recommendations which guide processes is indispensable prior to proposing any reasonable plan for monitoring
• Identification of “key indicators” amenable for “universal” use can be attempted afterwards
• A group within INK should work on a guide for developing good monitoring practices for KMC quality assurance
• An inventory of available instruments for monitoring and assessing implementation and for quality assurance on ongoing KMC should be assembled– The group already identified several research and monitoring
instruments both general and specific– Instruments for monitoring the progress of Implementation – Instruments for monitoring quality of care