Page 1 of 20
05 June 2020 No.09
An overview of predictive scoring
systems used in ICU
UV Jaganath
Moderator: S Moodley
School of Clinical Medicine Discipline of Anaesthesiology and Critical Care
Page 2 of 20
CONTENTS Introduction ......................................................................................................................................... 3
Developing a predictive scoring system ..................................................................................... 4
Choosing a predictive scoring system ........................................................................................ 6
Acute Physiologic and Chronic Health Evaluation (APACHE) .......................................... 7
Sequential (Sepsis-related) Organ Failure Score (SOFA) ................................................. 10
Simplified Acute Physiologic Score (SAPS) ......................................................................... 11
Mortality Predictive Model (MPM) ............................................................................................ 12
Comparison of performance ......................................................................................................... 14
Use in the low to middle-income setting ................................................................................... 15
Other uses of predictive scoring systems ................................................................................ 16
Limitations ......................................................................................................................................... 17
Conclusion and recommendations ............................................................................................. 18
References ......................................................................................................................................... 19
Page 3 of 20
Introduction
With the rising healthcare costs and shortage of intensive care unit (ICU) beds,
clinicians need to appropriately triage patient admissions into ICU to avoid wasteful
expenditure and unnecessary bed utilisation. While the Critical Care Society of
Southern Africa does recognise the use of predictive scoring systems in ICU to aid in
triage, they have also acknowledged some if its pitfalls.1
Predictive scoring systems are tools that have been developed to describe the severity
of a disease process and subsequently predict outcomes in patients. These tools
typically utilise a combination of patient data, including clinical health information,
physiological and laboratory data to determine a numerical severity of disease score,
which in turn is used to determine outcomes, such as; length of hospital stay and
mortality rates.2-4
Predictive scoring systems can be divided into two broad categories5:
1. Single organ or disease specific scoring systems such as:
• European Systems for Cardiac Operative Risk Evaluation (EuroSCORE)
that is used to predict mortality after cardiac surgery.6
• The Model for End-Stage Liver Disease (MELD) score used in patients
with end-stage liver disease to predict mortality risk.7
• The Glasgow Coma Score (GCS) to assess and predict mortality after
head injury.8
2. Generic scoring systems for the use in all ICU patients, such as:
• Acute Physiologic and Chronic Health Evaluation (APACHE).
• Sequential (Sepsis-related) Organ Failure Score (SOFA).
• Simplified Acute Physiologic Score (SAPS).
• Mortality Predictive Model (MPM).
• Multiple Organ Dysfunction Score (MODS).
The ideal predictive scoring system should use objective and easily measured risk
factors or predictors upon which the outcome prediction is made.
The characteristics of the ideal predictive scoring system includes:5,9
1. Uses easily measurable variables (see below).
2. Has a high level of discrimination.
3. Well calibrated.
4. Validated for use in all patients in the ICU and in different countries or patient
cohorts.
5. Can predict length of hospital stay, quality of life after ICU and/or mortality.
Page 4 of 20
The idea variables for describing organ dysfunction should be:5,10
1. Simple, cheap and routinely available.
2. Reliable, reproducible and objective to measure.
3. Specific to the acute dysfunction of the organ being evaluated but not chronic
dysfunction.
4. It should not be affected by reversable, transient abnormalities that are
associated with practical or therapeutic interventions.
5. Abnormalities should only be in one direction.
6. Variable should be continuous rather than dichotomous.
It should be noted that the simultaneous use of more than one predictive scoring
system on the same patient should be seen as complementary, as opposed to
competitive or mutually exclusive, as their combined use may possibly offer a more
accurate indication of the true severity of the disease process and hence overall
prognosis.5
Developing a predictive scoring system
To appreciate the basis of predictive scoring systems, it is important to have an
understanding of the steps that are required to develop them. An overview of these
steps are as follows2,4,10,11:
1. Selecting the outcome variable(s)
• Mortality is most commonly used as it is an easy to define endpoint.
Being binary, it easily lends itself to statistical analysis.
• Long term prediction of quality of life after ICU and cost-effective
medical treatment may also be used, however, unlike mortality, they
cannot be represented as a binary endpoint, thus making them more
difficult to measure.
2. Selecting the patient population
• Population bias may be inherent. This is an important factor to
consider when one is developing or choosing to use a given scoring
system. This may be important in the South African setting whereby
our population demographics and disease profiles may not mirror
that of the population sample that the scoring system was originally
developed for and validated against.
• Ideally, the clinician should only use the predictive scoring system for
decision making if the said scoring system was developed for and
validated in a population group similar to that of the patient of interest.
• Should the accuracy of the scoring system be found to be inaccurate
in the population group being examined, it should then be updated or
modified for that population group. An example of such an
undertaking occurs with the APACHE scoring system.
Page 5 of 20
3. Selecting the variables or risk factors
• Selecting variables to be studied are usually done in one of two ways.
• In the first method, clinicians select variables that are believed to be
related to the chosen outcome (e.g. mortality). Such a technique was
employed to develop the APACHE II and SOFA scoring systems.
• In the second method, statistical methods (e.g. linear discriminant
function analysis) are used to narrow down the initial list of variables
to only those that are individually related to the chosen outcome.
Such a technique was employed to develop the APACHE III, SAPS
and MPM II scoring system.
• It is important that the variables identified are objective and easy to
obtain.
4. Data collection (predictors) and analysis (of the outcome)
• Prospective data collection is preferred as it allows for ongoing
analysis of the data being collected. This helps reduce the risk of bias
or errors which may reduce the accuracy of the predictive scoring
system.
• Another advantage of prospective data collection is that it allows for
evaluation of the timing of the measured variable to the outcome.
While the SAPS and MPM scoring systems use data collected within
1 hour of admission, the APACHE and SOFA scoring systems used
the worst physiological values within 24 hours of admission.
• Missing data from retrospective investigations may become
challenging and result in inappropriate associations of mortality with
a smaller number of organ system derangements, which may be
reduced with prospective data collection methods.
• Once the predictive factors are identified, they are analysed and
presented either as a cumulative score or as a single score which is
calculated from the sum of weights allocated to each individual
predictive factor (as used in APACHE and SAPS).
5. Developing the model
• This is done with the use of various statistical techniques including
multivariant linear regression models (used to assess relationships
between independent variables, i.e. the risk factors and the
dependent outcome i.e. mortality) and logistic transformation (used
when the predicting factors are not linearly related to the outcome).
• Logistic regression methods may be used to help identify additional
predictive factors that may impact the final outcome.
Page 6 of 20
6. Validation of the predictive scoring system
• This can be done by directly comparing the predictive scoring system
to a “gold standard” or to each other.
• Another technique involves developing the receiver operation
characteristic curve to aid with a more precise cut off point selection.
7. Evaluation of the predictive scoring system’s utility and impact
• Depending on the chosen scoring system, they may be used to triage
patients for ICU and assist in decision making. They may also be
used during clinical trials to prove that the two patient population
groups were similar.
8. Updating the system
• All predictive scoring systems should be updated as its calibration over
time may deteriorate because of case-mix and changes in both disease
profiles and therapeutic interventions. This tends to result in an
overestimation of mortality and subsequently poor discrimination
between outcomes (e.g. survivors vs non-survivors).
• This is important since past studies have shown that in certain
circumstances older scoring systems performed no better, or perhaps
worse than predictions made by clinicians.
Choosing a predictive scoring system
While there are many different predictive scoring systems to choose for, each with
their own advantages and limitations, some of the commonly used models include the
APACHE, SOFA, SAPS and MPM scoring systems. When choosing to use a
predictive model, is it important to select one that was most recently developed for and
validated in the population group being evaluated. Other factors that should be taken
into account include its user-friendliness, accessibility, feasibility, cost implications and
the outcome that is being evaluated (e.g. length of hospital stay vs predicted mortality
rate).2
An example of such limitations is that while the APACHE IV model is able to more
accurately predict mortality over the SAPS model, this benefit is off set by the fact that
it is more challenging and costly to use as it requires more variables and depends on
propriety software. Additional, APACHE IV is better calibrated for predicting length of
hospital stay when used in ICUs within the United States. In contrast, while the SAPS
model is cheaper and easier to use and better suited for use internationally, it is more
prone to case-mix effects and is unable to reliably predict length of stay when
compared to the APACHE IV model.2
A summary of each of these predictive scoring systems will be discussed separately.
Page 7 of 20
Acute Physiologic and Chronic Health Evaluation (APACHE)
The APACHE scoring system has been found to be efficient in predicting mortality and
estimating length of stay in ICU. There are four versions that is widely used, APACHE
I to IV.
The original APACHE model was developed in 1981 and consisted of two parts. The
former is the Acute Physiological Score (APS) which represents the degree of the
acute illness. A total of 34 physiological variables are measured and allocated a score
between 0 and 4, depending on the degree of severity. The worst value of each
variable measured within the first 32 hours of ICU admission are used. The latter part
of the score, the Chronic Health Evaluation (CHE) that is classified from A to D,
represents the physiological status of the patient before the illness. The classification
of A representing excellent health while a classification of D representing severely
failing health. The initial study demonstrated a direct relationship between the APS
score and the likelihood of mortality. With regards to the CHE, only class D was shown
to be an independent risk factor for mortality.12
Developed in 1985 as a modification of the original model, the APACHE II scoring
system uses a point score based on 12 physiological parameters that is routinely
investigated.13 These physiological measurements are done during the first 24 hours
after admission, in addition to this, the patients age, medical history and surgical
requirements are considered. If a variable has not been measured a point of zero is
allocated. A score between 0 and 71 is then calculated based on these measurements.
Generally, higher scores are more predictable of severe disease and subsequently
higher rates of mortality. From the APACHE II score, the estimated risk of in hospital
mortality is then calculated using a logistic regression equation, utilizing specific beta
co-efficient made for this purpose (Tables 1 and 2).4,14,15
The APACHE III was developed in 1991 using 26 variables. It comprises of two
components, namely the APACHE III score, ranging from 0 – 299, and the APACH III
predictive equation that uses the APACHE III score to predict in hospital mortality
rates.16 With regards to predicting ICU mortality in trauma patients, the APACHE III
has been validated against and found be as accurate as the Trauma Injury Severity
Score (TRISS).17
Developed in 2006, the APACHE IV is more complex and entails the input of 142
variables and 115 various disease groups, however web based calculations can be
done.18 Despite the APACHE IV being the most recent version, the APACHE II score
is still amongst the most commonly model in current clinical use.
When comparing the APACHE II to the APACHE IV model, both demonstrated good
discrimination capabilities, with an area under the receiver operating characteristics
curve of 0.805 and 0.832 respectively. With regards to predicting mortality, the cut-off
points with the best Youden index were 17 and 72 of the APACHE II and APACHE IV
respectively.13
Page 8 of 20
Table 1. Acute physiologic and chronic health evaluation (APACHE II)4,14
Page 9 of 20
Table 2. Acute physiologic and chronic health evaluation II-diagnostic category
weight4,14
Advantages: The measurements required to calculate the APACHE II score are
routine parameters that are monitored in the ICU thus no additional investigations are
needed. This predictive scoring system can predict sepsis with a single assessment
at 24 hours. Additionally, it has been validated in several countries and has been
proven to be highly reproducible.4,14,15
Disadvantages: The major setback of the APACHE scoring system is that patients
may have several co-morbid illnesses and therefore selecting only one main
diagnostic category may be challenging. The physiological parameters used in the
calculation of the APACHE score are dynamic and may be easily affected by factors
such as on-going treatment and resuscitation.4,14 Regarding the APACHE IV scoring
system, even though more updated, it is more complex and requires physiological
setting software resulting in added costs.18
Page 10 of 20
Sequential (Sepsis-related) Organ Failure Score (SOFA)
The SOFA score was developed in 1994 by the European Society of Intensive Care
Medicine, then revised in 1996. It was originally used to understand the natural
progression of individual organ failure and the interaction between failure of other
organs and to describe the sequence of complications (in terms of said organ
dysfunction or failure) in critically ill patients with sepsis.4,19 Intuitively, while any
calculation of morbidity or organ dysfunction should then relate to mortality to some
degree, the SOFA score was originally not intended to predict this outcome. However,
it has since been validated for use in critically ill patients with non-sepsis related organ
dysfunction and as a tool for predicting mortality rate.4,10,19 It measures 6 organ
systems with scores ranging between 0 – 4 for each (Table 5).
Interpretation: Irrespective of the initial SOFA score, an increase in the score within
48 hours of ICU admission is associated with a mortality rate of ≥ 50%, an unchanged
score was associated with a mortality rate of between 27 – 35% (if the initial score was
< 8) and 60% (if the initial score was ≥ 8), whereas a decreasing score is associated
with a mortality rates of < 6% and 27% if the initial score was < 8 or ≥ 8 respectively.20
In the setting of sepsis, a score of ≥ 2 is associated with a predicted mortality rate of ≥
10%, however, in the setting of septic shock with a score of ≥ 2 the predicted mortality
rate is around 40%.2,21 Further information regarding the interpretation of the of SOFA
score may be found in table 5.
Advantages: Unlike the APACHE and SAPS scores, which are models of
physiological assessment, the SOFA score may be considered to be a multi-organ
scoring system, and as such it can be used to track the response of organ dysfunction
to therapy over time.9 It is also easier and cheaper to use than models such as the
APACHE IV.
Disadvantages: In contrast to the above-mentioned predictive scoring systems, the
SOFA score does not predict individual outcomes, instead it helps identify high risk
patients at risk of sepsis related death as a group.2 It does not take chronic health
status into account.
Table 5. Sequential organ failure assessment score (SOFA)4
Page 11 of 20
Simplified Acute Physiologic Score (SAPS)
The SAPS II scoring system uses 17 variables i.e. 12 physiological variables which
are measured within the first 24 hours after admission, age of the patient, the type of
admission and three disease-related variables.22 Several of these variables are
assigned a score depending whether they are present or not, whilst the 12
physiological variables are scored according to a range of values. The SAPS II score
may vary between 0 –163 points. The probability of mortality is then calculated using
a logistic regression analysis.4,14
Advantages: The SAPS III which is the latest version has a greater potential for
universal use as it has been designed and validated in ICU settings amongst thirty-
five countries.
Disadvantages: The SAPS II was designed from a database of patients in North
America and Europe. Unfortunately this sample is not representative of the population
and ICUs in other countries where variability in structures and resources are related
to the outcome.23 Unlike APACHE IV, the SAP scoring system cannot be used to
predict length of stay, however it may be used to compare the use of resources
amongst ICUs.24
Table 3. Simplified acute physiology score (SAPS II)4,14
Page 12 of 20
Mortality Predictive Model (MPM)
The Mortality Probability Model was developed on an international sample of patients
between 1989 – 1990. It assesses patients probability of mortality at hospital
discharge, based on measurements attained within the first hour of ICU admission.25
There are three models. The first version of the model was developed to predict
mortality based on data from admission and after the first 24 hours in the ICU. Later,
models were developed to include data from 48 – 72 hours after ICU admission. This
model uses the patients’ chronic illnesses, acute diagnosis, some physiological
variables and other variables such as mechanical ventilation.4,14
Advantages: The MPM scoring system uses less physiological data in comparison to
other scoring systems thus this scoring system may be preferred in settings where
laboratory resources are constrained. The MPM III has good discrimination and
calibration.25
Disadvantages: The MPM excludes certain patient subsets such as cardiac surgery,
myocardial infarction, and ICU readmissions, which decreases its usefulness to some
ICUs.25
Page 13 of 20
Table 4. The Mortality Predictive Model (MPM)4,14
Page 14 of 20
Comparison of performance
Despite having a paucity of high-quality studies comparing the performance of the
various models to each, after evaluating the current evidence, the following
generalisations can be made:
1. When comparing the older models (APACHE II, APACHE III, SAPS II, and
SOFA) to each other, the APACHE II and III predictive scoring system were
shown to be superior in one study whereas the SAPS II was shown to be
superior in another.26 The accuracy of these model in predicting the actual
mortality rate was also limited. As such, it has been suggested that using a
combination of these scoring systems may result in an improvement in the
overall predictive performance.26
2. When comparing the newer models (APACHE IV, SAPS III and MPM III) to the
older models (APACHE II and III, SAPS II, MPM II and SOFA), the newer
models were found to perform better. This was evident by the fact that the older
models had a tendency to over-predict the mortality rate, whereas the newer
models had superior calibration, accuracy and discrimination characteristics.
However, despite these findings, these differences were not found to be
statistically significant. A possible reason for this finding is that the choice to
use one predictive scoring system over another may be due to ease of use and
local preferences.27
3. In general, all of these models have very good discriminatory values with an
area under the receiver operating characteristics curve between 0.80 – 0.90
while simultaneously demonstrating good calibration assessments.9 As such,
no single model has been shown to be significantly superior to another with
regards to predicting mortality.2,27
Page 15 of 20
Use in the low to middle-income setting
With regards to the South African context, a major cause of concern is that the above-
mentioned predictive scoring systems have been developed and validated
predominantly in high-income countries. As a result, the accuracy of these models
may be reduced given that our population and disease profile and therapeutic
intervention may not mirror that seen in high-income countries. While not specific to
South Africa, the performance these models to predict mortality in critically ill patients
in low or middle-income countries was demonstrated to be moderate at best.19,28
One reason for this relates to the lack of calibration of these models to the population
cohort. Ideally, if a predictive scoring system is to be used, it should exhibit good
calibration for that population cohort. However, it has been suggested that should a
model with a poor calibration demonstrate a good discriminatory ability, it may still be
of benefit if it is used it to identify high-risk patients for diagnostic and/or therapeutic
interventions.19,29
Another given reason is due to and the possibility of missing predictor variables due
to resource constraints. This problem may be overcome by developing scoring
systems with fewer and more commonly available variables (e.g. the Rwanda MPM
and TropICS models) or by allowing for the substation of certain inaccessible variables
with its more commonly available counterparts (e.g. substituting PaO2 for oxygen
saturation).19,28
A major advantage of developing setting-specific models is that they address the
above two problems. One such example is the Rwanda MPM (R-MPM), which was
developed using five easily attainable variables, namely: age, heart rate and GCS at
the time of ICU admission, hypotension or shock as a reason for ICU admission and
suspected or confirmed infection within 24 hours of ICU admission. Despite having
fewer variables, the R-MPM demonstrated better calibration, discrimination and
prediction compared to the MPM when used in the same population of Rwandan ICU
patients.28
Page 16 of 20
Other uses of predictive scoring systems
Apart from the clinical uses describe able, predictive scoring systems do play a role in
other aspects of medicine. Two key areas include its role during research and as a
quality care bench mark tool.
1. Research
• Scoring systems may be used in clinical trials to compare the baseline
risks between comparative groups to ensure that they are similar. This
is commonly used during clinical trials in patients with acute respiratory
distress syndrome or sepsis whereby possible therapeutic interventions
are being evaluated.2,11
2. Quality care bench mark
• Predictive scoring systems help evaluate the quality of care by
confirming that patients with the same or similar baseline mortality risks
are being compared. An example of such practices are studies that
compare ICU outcomes with other ICUs within the same hospital or in
other hospitals. The implications of such findings are that policies and
practices from ICUs with favourable mortality rates may then be adopted
and incorporated by other units to help improve their quality or care.30,31
Page 17 of 20
Limitations
The ICU is the perfect environment for using predictive scoring systems since both the
population group and patient care tends to be is well defined and the most significant
predictor of mortality is the severity of the illness. However, there are some limitations
with regards to their use as follows:32,33
1. The scoring system may not be validated in the population group that it is being
used to evaluate. Similarly, they may not be able to accurately predict outcomes
for specific patient population groups (e.g. the use of APACHE II was found to
be unreliable to predict outcomes in pregnant women receiving critical care) or
disease processes.34 The predictiveness of the system may be reduced due to
the unusual nature of a given population for which it was not designed or
validated in.33
2. The predictiveness of the scoring system deteriorates over time and as such,
failure to periodically update the system results in a gradual loss of
discrimination and/or calibration. The net effect it that an overestimation of the
predicted mortality rate may be seen.35
3. A phenomenon known as lead-time bias may occur. This was seen when
patient who were transferred in from other ICUs or hospitals had a higher
mortality rate than that predicted by the APACHE II scoring system. This
variable, the location of treatment, was subsequently added to the APACHE
scoring system but not the others mentioned above.36
4. The quality of care is better or worse than expected resulting in a lower or higher
patient mortality rate. In this scenario, the model will lose its predictive accuracy
as they will be a higher or lower actual survival rate due to the better or worse
quality of care respectively.33
5. Unlike SAPS, MPM and SOFA, models like the APACHE require proprietary
software and more data points to use, resulting in it being more burdensome,
however, the integration of electronic record keeping into health systems may
alleviate some of these challenges.2
6. When predicting mortality within 24 hours of admission into the ICU, the current
evidence suggests that scoring systems are not yet superior to clinical
judgment.37
Page 18 of 20
Conclusion and recommendations
Due to the limited availability of ICU resources in South Africa it is important that we
utilise multiple tools to aid in its rational use. After reviewing the literature, predictive
scoring systems do have a role to play in this. The accuracy of predictive scoring
systems will continue to improve with time. This is possible due to and improvements
in computing power and the integration of “big data” into our heath care systems.
When deciding to use predictive scoring systems, it is important for clinicians to be
aware that the predicted mortality rate relates to patients within a similar cohort and
not to an individual. As a result, while we may be able to predict the mortality of a
group of patients with a similar score, we are unable to predict which individual patient
may survive and who may die. Consequently, caution should be used when predicting
mortality for individual cases.
Conversely, as scoring systems allow for an objective assessment of the clinical status
of the patient, they may be used to assist the clinical decision-making since they mirror
the probability of mortality in a similar cohort of patients. Ultimately, predictive scoring
systems should be considered as a tool to assist, rather than replace the clinician.
Page 19 of 20
References
1. Joynt GM, Gopalan DP, Argent AA, et al. The Critical Care Society of Southern Africa Consensus Guideline on ICU Triage and Rationing (ConICTri); 2019.
2. Kelley MA, Manaker S, Finlay G. Predictive scoring systems in the intensive care unit. UpToDate 2020.
3. Saranya Avr SJ. Comparison of Different Scoring Systems Used in the Intensive Care Unit. Journal of Pulmonary & Respiratory Medicine 2015; 05(04).
4. Rapsang AG, Shyam DC. Scoring systems in the intensive care unit: A compendium. Indian J Crit Care Med 2014; 18(4): 220-8.
5. Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Critical care (London, England) 2010; 14(2): 207.
6. Nashef SA, Roques F, Hammill BG, et al. Validation of European System for Cardiac Operative Risk Evaluation (EuroSCORE) in North American cardiac surgery. Eur J Cardiothorac Surg 2002; 22(1): 101-5.
7. Kamath PS, Wiesner RH, Malinchoc M, et al. A model to predict survival in patients with end-stage liver disease. Hepatology 2001; 33(2): 464-70.
8. Zuercher M, Ummenhofer W, Baltussen A, Walder B. The use of Glasgow Coma Scale in injury assessment: a critical review. Brain Inj 2009; 23(5): 371-84.
9. Bouch DC, Thompson JP. Severity scoring systems in the critically ill. Continuing Education in Anaesthesia Critical Care & Pain 2008; 8(5): 181-5.
10. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive care medicine 1996; 22(7): 707-10.
11. Kollef MH, Schuster DP. Predicting intensive care unit outcome with scoring systems. Underlying concepts and principles. Crit Care Clin 1994; 10(1): 1-18.
12. Wong DT, Knaus WA. Predicting outcome in critical care: the current status of the APACHE prognostic scoring system. Can J Anaesth 1991; 38(3): 374-83.
13. Nagar VS, Sajjan B, Chatterjee R, Parab NM. The comparison of apache II and apache IV score to predict mortality in intensive care unit in a tertiary care hospital. International Journal of Research in Medical Sciences 2019; 7(5): 6.
14. Rapsang AG, Shyam DCJIjoccmp-r, official publication of Indian Society of Critical Care Medicine. Scoring systems in the intensive care unit: a compendium. 2014; 18(4): 220.
15. VijayGanapathy S, Karthikeyan VS, Sreenivas J, Mallya A, Keshavamurthy RJI, urology c. Validation of APACHE II scoring system at 24 hours after admission as a prognostic tool in urosepsis: A prospective observational study. 2017; 58(6): 453-9.
16. Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991; 100(6): 1619-36.
17. Darbandsar Mazandarani P, Heydari K, Hatamabadi H, Kashani P, Jamali Danesh Y. Acute Physiology and Chronic Health Evaluation (APACHE) III Score compared to Trauma-Injury Severity Score (TRISS) in Predicting Mortality of Trauma Patients. Emerg (Tehran) 2016; 4(2): 88-91.
18. Zimmerman JE, Kramer AA, McNair DS, Malila FMJCcm. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. 2006; 34(5): 1297-310.
19. Haniffa R, Isaam I, De Silva AP, Dondorp AM, De Keizer NF. Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review. Critical care (London, England) 2018; 22(1): 18.
20. Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA 2001; 286(14): 1754-8.
Page 20 of 20
21. Lambden S, Laterre PF, Levy MM, Francois B. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Critical care (London, England) 2019; 23(1): 374.
22. Le Gall J-R, Lemeshow S, Saulnier FJJ. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. 1993; 270(24): 2957-63.
23. Moreno RP, Metnitz PG, Almeida E, et al. SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. 2005; 31(10): 1345-55.
24. Rothen HU, Stricker K, Einfalt J, et al. Variability in outcome and resource use in intensive care units. 2007; 33(8): 1329-36.
25. Higgins TL, Teres D, Copes WS, Nathanson BH, Stark M, Kramer AAJCcm. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). 2007; 35(3): 827-35.
26. Saleh A, Ahmed M, Sultan I, Abdel-lateif A. Comparison of the mortality prediction of different ICU scoring systems (APACHE II and III, SAPS II, and SOFA) in a single-center ICU subpopulation with acute respiratory distress syndrome. Egyptian Journal of Chest Diseases and Tuberculosis 2015; 64(4): 843-8.
27. Juneja D, Singh O, Nasa P, Dang R. Comparison of newer scoring systems with the conventional scoring systems in general intensive care population. Minerva anestesiologica 2012; 78(2): 194-200.
28. Riviello ED, Kiviri W, Fowler RA, et al. Predicting Mortality in Low-Income Country ICUs: The Rwanda Mortality Probability Model (R-MPM). PLoS One 2016; 11(5): e0155858.
29. Nassar AP, Malbouisson LM, Moreno R. Evaluation of Simplified Acute Physiology Score 3 performance: a systematic review of external validation studies. Critical care (London, England) 2014; 18(3): R117.
30. Afessa B, Keegan MT, Hubmayr RD, et al. Evaluating the performance of an institution using an intensive care unit benchmark. Mayo Clin Proc 2005; 80(2): 174-80.
31. Zimmerman JE, Alzola C, Von Rueden KT. The use of benchmarking to identify top performing critical care units: a preliminary assessment of their policies and practices. Journal of critical care 2003; 18(2): 76-86.
32. Katsaragakis S, Papadimitropoulos K, Antonakis P, Strergiopoulos S, Konstadoulakis MM, Androulakis G. Comparison of Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score II (SAPS II) scoring systems in a single Greek intensive care unit. Critical care medicine 2000; 28(2): 426-32.
33. Patel PA, Grant BJ. Application of mortality prediction systems to individual intensive care units. Intensive care medicine 1999; 25(9): 977-82.
34. Ryan HM, Sharma S, Magee LA, et al. The Usefulness of the APACHE II Score in Obstetric Critical Care: A Structured Review. J Obstet Gynaecol Can 2016; 38(10): 909-18.
35. Nassar AP, Jr., Mocelin AO, Nunes AL, et al. Caution when using prognostic models: a prospective comparison of 3 recent prognostic models. Journal of critical care 2012; 27(4): 423 e1-7.
36. Escarce JJ, Kelley MA. Admission source to the medical intensive care unit predicts hospital death independent of APACHE II score. JAMA 1990; 264(18): 2389-94.
37. Sinuff T, Adhikari NK, Cook DJ, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Critical care medicine 2006; 34(3): 878-85.