CENTRE FOR SOCIAL SCIENCE RESEARCH
The International Labour Organization’s measure of legal health coverage: Is it
reliable?
CSSR Working Paper No. 408
Legislating and Implementing Welfare Policy Reforms
March 2018
Published by the Centre for Social Science Research
University of Cape Town 2018
http://www.cssr.uct.ac.za
This Working Paper can be downloaded from:
http://www.cssr.uct.ac.za/cssr/pub/wp/408
ISBN: 978-1-77011-395-4
© Centre for Social Science Research, UCT, 2018
About the author:
Danielle Pagano completed a Master’s degree in Development Studies at the University of Cape Town with a distinction in July 2017.
Email: [email protected]
Acknowledgements:
This paper is a product of the “Legislating and Implementing Welfare Policy Reforms” research project funded by the Economic and Social Research Council (ESRC) and Department for International Development (DfID) in the United Kingdom. The study
described in this paper benefited from the critical insight and support from many individuals, especially Nicoli Nattrass.
1
The International Labour Organization’s
measure of legal health coverage: Is it reliable?
Abstract
The International Labour Organization’s (ILO) 2014/2014 World Social Protection
Report included several indicators for quantifying the expansion of Universal Health
Coverage (UHC), including measures for both effective and legal health coverage
(LHC). Its measure for LHC, a critical component of UHC, purports to quantify
rights-based protection in the area of health for 47 African countries. This paper is
the second of a two-part series critiquing the ILO’s LHC measure. Paper 1, “The
ILO's measure of LHC: Is it conceptually strong?” analyses the strength of the ILO’s
concept of LHC. This paper explores the measure’s reliability through (i) an
investigation into the trustworthiness of the sources cited in the metadata for African
estimates; and (ii) a quantitative analysis. The quantitative analysis used simple
linear regressions to examine whether the measure could significantly predict de
facto access to primary health care in African countries, both for the national
average and the poorest fifth of country populations. The World Health
Organisation’s (WHO) measure of birth coverage (the percentage of births attended
by skilled health personnel) was used as a proxy indicator for access to primary
health care. This study’s findings indicate that only one-fourth of the ILO’s African
coverage estimates are reliable. Statistical results indicate that the ILO’s measure
cannot be used to predict access to primary health care in Africa) for total country
populations or for the poorest quintile.
Introduction
The objective of Sustainable Development Goal (SDG) 3 is to “ensure healthy lives
and promote well-being for all at all ages” worldwide (United Nations Development
Programme, 2016). A key mechanism, though certainly not the only one, for
achieving SDG 3 is implementing Universal Health Coverage (UHC) globally.
Often overlooked by rese-archers, legal health coverage (LHC), which is the
coverage that results from legal health frameworks, is a core component of UHC,
and a prerequisite for the fulfilment of SDG 3. As explained in Paper 1 of this two-
part series, “Legal health frameworks can be characterised as the legal policies and
mandates that protect access to health services and the right to health” (Clarke, Rajan
2
& Schmets, 2016:482). This study argues that achieving LHC requires reliable
measures to monitor its expansion across countries and over time. The International
Labour Organization’s (ILO) “World Social Protection Report” for 2014/2015,
which analyses countries’ advancement toward UHC, includes the only measure of
LHC that has estimates for more than a handful of African countries; thus, it has
potential significance for the expansion of UHC across Africa.
While Paper 1 deals with the ILO’s concept of LHC, this paper assesses the
reliability of the ILO’s LHC estimates. Accordingly, Paper 2 analyses the estimates
themselves, rather than their conceptual basis, and whether the ILO’s measure can
be used as a tool for monitoring LHC across Africa. I argue that, for the ILO’s
measure to be an accurate representation of LHC, it needs to possess the following
characteristics: it should be reliable; transparent as to how it quantified its estimates;
maintain conceptual validity; and possess at least some predictive power (i.e.,
statistical significance) when used as a predictor variable for health
access/outcomes. For the measure to be exceptional, its estimates must meet these
standards while remaining relevant and inclusive of marginalised subpopulations, in
particular “the poor”. These qualifications and related terminology are explained in
detail later in this paper.
Any measurement of these concepts at a national level should, ideally, be
accompanied by an estimate pertaining to the poorest residents of a country.
Estimates that treat a country’s population as one entity, rather than disaggregating
figures by characteristics that indicate societal marginalisation (e.g., sex or
socioeconomic class) can be misleading, especially in the presence of high
inequality. Disaggregated data allows estimates to more accurately reflect the reality
for marginalised populations, including those living in poverty, who often lack both
legal and effective coverage across much of the world (Anyanwu & Erhijakpor,
2009; Mills et al., 2012). Beyond the ethical imperative of this type of measurement
inclusivity, minimising inequalities is a key concern of stakeholders endeavouring
to achieve UHC and fulfil the post-2015 development agenda (Scheil-Adlung, 2015:
v).
1.1 Research aims
The expansion of UHC is unlikely to occur quickly without standardised, reliable
measures that allow for accurate monitoring to take place, especially across countries
and over time. Although UHC is comprised of many components, one of its most
important, and frequently neglected, is the “right to health”. In the World Social
3
Protection Report, the ILO claims that its LHC measure is an indicator of this. In
this study, I assess how reliable the measure’s estimates are for Africa. If the
estimates are not reliable, the measure should not be used to monitor the right to
health across the African continent. This study also asks whether the measure has
any predictive power for primary health care access across Africa. To investigate
these issues, I pose the following, more specific, questions:
1. How reliable and transparent are the ILO’s estimates of LHC for African
countries?
a) This is explored through an investigation into the ILO’s metadata,
with a focus on the consistency and quality of the sources that the
ILO cites for its coverage estimates.
2. Can the ILO’s LHC estimates significantly predict access to health care in
Africa? Birth coverage estimates are used as a proxy indicator for health
care access.
a) This question is answered using simple linear regression analysis.
The ILO’s LHC estimates (for African countries only) are input into
the regression as the predictor variable; countries’ respective birth
coverage estimates are input as the outcome variable.1 Only
countries that have estimates available for both LHC and birth
coverage are included in the regression.
3. Can the ILO’s LHC estimates significantly predict access to health care for
Africa’s poorest citizens?
a) This question is answered using a similar method to the one above;
however, instead of using aggregate national birth coverage (i.e.,
birth coverage for a country’s total population) as the outcome
variable, birth coverage for the poorest 20% of a country’s
population (the bottom wealth quintile) is used. For example, in
Question 5 above, Country A’s birth coverage for its entire
population is input as a data point for the outcome variable; in
Question 6, only the birth coverage average for the poorest quintile
of Country A is used as a data point for the outcome variable. The
predictor variable remains the same—the ILO’s measure is not
capable of being disaggregated.
1 National birth coverage estimates are obtained from the WHO’s Health Equity Monitor,
explained in detail later.
4
This paper uses a mixed-methods approach. The qualitative component investigates
the ILO’s methodology and metadata for its African estimates. The quantitative
component uses simple linear regressions to test the relationship between the ILO’s
LHC measure and access to basic health care in Africa; this part of the study
incorporates external data from the WHO’s Health Equity Monitor, explained in
further detail in Chapter 2.
1.2 Significance
As explained in Paper 1, this study is significant because it is the only study to date
that examines the reliability of the ILO’s World Social Protection Report LHC
estimates. Ensuring that the ILO’s legal health measure is transparent and reliable
matters because it is the only published measure with recent LHC estimates for most
of Africa. Yet, after examining the ILO’s LHC measure, specifically its estimates
for African States, I discovered many inconsistencies that led me to question the
reliability of both the measure and its utility for the African continent.
In addition to investigating its metadata, this study explores whether a statistical
relationship exists between the ILO’s measure of LHC and de facto access to basic
health care, proxied as birth coverage. Estimates for national birth coverage and for
the poorest quintile are incorporated into separate regressions. The reason for this is
to maintain inclusivity in the pursuit of UHC, which necessitates the development
of measures with real implications for the poor’s access to primary health care
services. If national averages were used for analysis, this study would only be able
to determine whether the ILO’s measure can successfully predict national access to
health services; it would be unable to ascertain whether the ILO’s measure can also
predict such access for the poor. This is because aggregate country statistics
habitually obscure the reality of the poor’s lived experiences (Gwatkin, 2000).
This study found that the ILO’s LHC measure is not a statistically significant
predictor (p≤.05) of access to primary health care services for African countries; nor
is it a statistically significant predictor of access for the poorest quintile. In addition
to the findings presented in Paper 1, these results provide further evidence that
indicates that there is a problem with the ILO’s measure – a good measure of LHC
should correlate with access to primary health care, even if it is not a perfect
association.
5
1.3 Paper overview
Chapter 2 explains this study’s research design. It operationalises key concepts,
especially reliability, in the context of this study. Furthermore, it explains both the
qualitative and quantitative research components, the WHO birth coverage dataset,
and the justifications for choosing this study’s approach. Lastly, the study’s
limitations and ethical considerations are addressed.
Chapter 3 presents the results from the investigation into the ILO’s metadata, which
point toward substantial shortcomings that the ILO either failed to make transparent
or was not aware of. The results suggest that the measure is not reliable, nor is it a
relevant measure for predicting Africans’ access to primary health care services,
especially for the most impoverished Africans.
Chapter 4 analyses this study’s results regarding the measure’s reliability and
predictive power for primary health care access; it also provides detailed case studies
illustrative of problems with the ILO’s measure. Additionally, it explains why
monitoring and evaluating true progress toward UHC requires measures that reflect
the reality of health care, not only for countries’ aggregate populations, but also for
their poorest citizens. If a well-known measure is reliable, but unable to predict
outcomes for the poor, this is a shortcoming that users of the World Social Protection
Report need to consider, or at least be made aware of.
Chapter 5 discusses the implications of this study’s findings for the LHC measure,
the World Social Protection Report, the ILO’s research, and the research of
intergovernmental organisations generally. Furthermore, it illustrates why the LHC
measure should not be used by development practitioners or researchers for gauging
health coverage. Additionally, it provides recommendations for the ILO regarding
the measurement of LHC.
2. Methodology
This section explains the methodology used to assess the reliability of the ILO’s
LHC estimates for African countries. Reliability is tested through an investigation
into the ILO’s metadata, in addition to testing the measure’s statistically significant
predictive power for health access. This study tested the predictive power of the
6
ILO’s measure for primary health access both for total (aggregate) African country
populations and the poorest fifth (quintile) of these populations.2
2.1 The metadata: Tables B.11 and B.11b
As explained in Part 1 of this series, to investigate the sources used by the ILO for
its estimates, I used two tables from the ILO’s World Social Protection Report. The
first is Table B.11: The multiple dimensions of health coverage (again, the
spreadsheet for this Table is called “effective coverage”). Table B.11’s first column
provides estimates for “health coverage as a percentage of the total population” for
most of the globe, including 54 African countries.3 Table B.11’s footnotes expand on
what it means by “total coverage”: “Coverage includes affiliated members of health
insurance or estimation of the population having free access to health care services
provided by the State”. Table B.11 then refers readers to an external document (not
actually contained in the World Social Protection Report itself): Table B.11b, “The
multiple dimensions of health coverage: percentage of the population covered
(members of health insurance or free access to health care services provided by the
State)”.
This Table can be downloaded from the ILO’s website.4 Table B.11b confirms that
the ILO relies primarily on external sources (i.e., non-ILO sources) for its LHC
estimates. The ILO uses varying types of sources for its estimates, including
academic journals, non-academic PowerPoint presentations, United States Agency
for International Development (USAID) reports, government publications, and
others. Table B.11b provides metadata detailing these sources for all countries that
are not members of the Organisation for Economic Cooperation and Development
(non-OECD).
2 It is important to note that conceptual problems with the measure are only discussed in Paper 1;
they are not considered here. For example, this paper does not discuss the problem that the ILO
quantifies LHC using insurance, or the fact that it does not consider judicialisation in its concept
of LHC. Instead, Paper 2 investigates whether the ILO quantifies its estimates reliably according
to its own definitions. 3 Botswana, Chad, Congo, Equatorial Guinea, Liberia, Malawi, and South Sudan are included in
Table B.11, but they are listed as having a coverage rate of (…), which means that data was not
available.
4 See http://www.social-protection.org/gimi/gess/RessFileDownload.do?ressourceId=37218
7
2.2 Qualitative component: Metadata and the
reliability of estimates
The qualitative component of this investigation tests reliability by conducting a
document review of the sources cited in the metadata of Table B.11b. Every
academic, Internet, and miscellaneous source referenced in Table B.11b for all 47
African countries included in the measure was examined. I also conducted extensive
Internet searches to locate source documents in cases where the URL provided by
the ILO was broken, incorrect, or otherwise missing. Each estimate’s source(s) are
detailed comprehensively by country in the appendix. At minimum, each country
summary includes the following information: Whether the ILO’s estimate in Table
B.11 was absent from the primary source it cited, whether the ILO’s estimate was
consistent with the source’s (e.g., 60% in Table B.11 and 60% in the primary source,
as opposed to 70% in Table B.11 and 60% in the primary source), and whether the
ILO’s estimate was misleading. An estimate was classified “reliable” once it met the
following conditions:
1. The ILO cited at least one source for the estimate, and I could locate that
source. The source was considered missing if I was unable to locate it after
a comprehensive Internet search using Google and Google Scholar.
2. The estimate appeared to include the entire population of a country, or at
least more than 90%.
3. The cited source seemed to be of good quality or, at minimum, acceptable.
a) Researcher discretion was used to determine whether a source’s
quality was “good” or “acceptable”, but I will list some examples
here for instructive purposes: the source was peer-reviewed; it
appeared to be professional or academic; and/or it was clear how the
source derived its estimates.
4. The estimate cited by the ILO could be verified/located in the source.
5. The estimate reflected the ILO’s definition of LHC (i.e., the percentage of
a country’s total population that has “health insurance or free access to
health care services provided by the State” (ILO, 2014).
If an estimate’s cited source(s) met the above requirements, it was coded as 1 to
indicate reliability; otherwise, 0 was used to indicate non-reliability. In some cases,
the ILO cited two sources for a country’s estimate. If the LHC estimate was based
on at least one reliable source, it was coded as 1, regardless of the second source’s
reliability. The four conditions listed above are not exhaustive in terms of why an
estimate was classified as a 0 or 1. As there were estimates available for 47 countries,
8
with many unique issues, it was impossible to completely standardise the analysis.
If an estimate’s reliability was unclear, I gave it the benefit of the doubt and coded
it as 1. In addition to the approaches already discussed, I emailed addresses listed as
official contacts on the ILO’s website to inquire about the measure’s methodology.
2.3 Quantitative component: Reliability and
predictive power
Apart from using sources of appropriate quality consistently, as well as referencing
sources correctly, a valid and reliable measure of LHC should correlate with access
to primary health care. Although LHC should not be expected to be a perfect
predictor of health access (i.e., a one-point increase in LHC predicts a one-point
increase in health access), given the significance of legal interventions for health
access, a reliable measure should still possess some statistical predictive ability
regarding access to primary health care in Africa. This study tests whether a
statistically significant relationship exists by conducting simple linear regressions.
The ILO’s LHC measure for Africa is included as the predictor variable and “birth
coverage” in Africa as the outcome variable. Two regressions are run, the first with
aggregate birth coverage as the outcome variable, and the second with birth coverage
of the poorest quintile (Q1) as the outcome variable. The null hypothesis for this
study is that there is no relationship between the ILO’s measure of LHC and birth
coverage in Africa. If LHC emerged as a significant predictor (p≤.05) for birth
coverage in Africa, this was taken as additional evidence of reliability.
Birth coverage was selected as the outcome variable because it is widely used in
academic literature as a proxy for access to primary health care (Stuckler et al., 2010;
ILO, 2008a; Lagomarsino et al., 2012). Birth coverage is defined as the “percentage
of births attended by skilled health personnel” (WHO, 2015). Data for this indicator
was obtained from the WHO’s (2016) Health Equity Monitor, an online database
with “reproductive, maternal, new born and child health indicators”. The Health
Equity Monitor primarily obtained its figures from the Demographic and Health
Surveys (DHS), a worldwide longitudinal survey initiative funded by the USAID
(Rustein & Johnson, 2004:2). The DHS initiative is notable due to its standardised
methodology, which provides an opportunity for comparative data analysis across
most of the African continent.
9
2.4 Predictive power for the poor’s primary health
access
Another reason birth coverage was selected as the proxy indicator for access to
primary health care is that birth coverage is one of the few health indicators with
publicly available data that can be disaggregated by wealth quintile for most African
countries. The Health Equity Monitor’s virtual toolkit allows for disaggregation “by
education, economic status, [and] place of residence (rural vs. urban)” (WHO, 2016).
So, in addition to measuring the predictive power of the ILO’s measure for African
country populations generally, this study also aims to assess its ability to predict
access to primary health care for the poorest 20% of Africans (the bottom quintile).
2.4.1 Operationalising “the poor”
To test the statistical power of the ILO’s measure in terms of predicting access to
primary health care for “the poor”, I first had to operationalise what it means to be
poor for analysis purposes. I classified individuals as poor using the WHO’s (2016)
economic status [wealth] index. Individuals were grouped into wealth quintiles
based on their country’s national distribution of wealth (as opposed to a global
wealth distribution, where the middle class of Country A may be poor when
measured against the rest of the world). Aggregate populations (i.e., a State’s total
population) were divided into these five wealth quintiles. Residents of a country
were classified as poor if they fell into the bottom 20% of their country’s wealth
distribution (the bottom quintile, or Q1), and “rich” if they were part of the top 20%
(the top quintile, or Q5). The Health Equity Monitor disaggregates country
populations into wealth quintiles this way. Thus, to utilise its data for rich-poor
comparisons, this study also employed their quintile data. Although classifying the
poor as those who make up the bottom 20% of a country’s wealth distribution may
appear arbitrary, it is a commonly used way of selecting the cut-off for the poverty
line (Rustein & Johnson, 2004). When a fixed poverty line is determined according
to a national wealth distribution, it can be used “for comparison across countries and
often shows similar results for health measures in different countries” (Rustein &
Johnson, 2004:6).
10
2.5 Limitations: Obstacles encountered during the
metadata review
For the qualitative component, many of this study’s limitations were ironically
caused by issues with the ILO’s metadata. For example, my findings demonstrate
that several of the ILO’s sources do not present any estimates, contrary to the
organisation’s claims. Hence, it is impossible to verify where the data came from in
those instances. Another difficulty was that the ILO did not always provide page
numbers for its estimates, even if the source was hundreds of pages long; this meant
that a figure could have been overlooked. I mitigated these issues by examining the
page number(s) the ILO cited for specific sources – if they included the page number
in the citation – as well as adjacent pages, sections of the source document(s) that
focused on coverage and/or health-related themes, and a source’s index.
Additionally, I used Window’s Control-F function to search for key terms, such as
“insurance” or “health” in whichever language the document was written in. The
statistic given for health coverage was also included in this search. For example, if
a country’s estimated coverage was 80.2%, then I searched for this number in the
document.
2.5.1 Testing the predictive strength of the ILO’s
LHC measure
Perhaps the biggest limitation of using regressions in this study was the impossibility
of matching the years for every country’s LHC estimate with its corresponding birth
coverage estimate. Indeed, even within each individual measure, the country
estimates are not all from the same year. In the WHO data, for example, the most
recent estimates on birth coverage by a skilled medical practitioner for the Central
African Republic and Burundi are from 2010, whereas the most recent estimate for
Chad is from 2004. The ILO’s estimates are similarly inconsistent. Furthermore, the
WHO’s data was collected at any point during the two to three years prior to the
listed survey year, so it does not specify which year the data was collected. For
instance, if it references 2010 for an estimate, the survey data could have been
collected in 2008 or 2007.
Ideally, a country’s birth coverage estimate and LHC estimate would be from the
same year, or even within five years, but this was not possible for several countries.
This issue was minimised by first running the regression with countries that had birth
11
coverage figures more than five years apart from their respective LHC estimates,5
and then rerun excluding them, to see if there was a significant difference. The effect
on the model was negligible, so they were retained to achieve greater statistical
power – in other words, a larger N-value. South Africa was the only country with
estimates available for both measures that was omitted from regressions; this is
because its birth coverage estimate was more than 10 years apart from its LHC
estimate. Fortunately, most of the birth coverage estimates came within five to six
years of their corresponding LHC estimates. Another limitation was the small N-
value, which resulted in diminished statistical power; however, data was not
available for every African country. Additionally, this study’s focus on the African
continent meant that it was not possible to add cases. A further limitation is that birth
coverage is not a perfect proxy for de facto access to health.
2.6 Ethical considerations
Regarding the subject of wealth classifications, it is important to note that those
deemed as either “poor” or “wealthy” do not necessarily identify with these terms or
agree that they apply to them. My intent was not to further marginalise “vulnerable
populations” with language; however, statistical analysis required that a quantitative
measure be designed to capture poverty in some form. Another important point to
take note of is that any criticism of the ILO’s measure is not meant to imply that the
organisation or its contributors purposefully misled readers or knowingly provided
unreliable figures.
2.7 Conclusion
To summarise, this paper classifies the ILO’s LHC estimates as reliable if they meet
the six conditions described above. If the ILO’s LHC measure has some value in
terms of predicting primary health access in Africa (i.e., if it is a statistically
significant predictor of total/aggregate birth coverage when used in simple linear
regressions), then this was taken as further evidence of reliability. The same method
was used to determine the measure’s value for predicting health access for the poor.
5 Morocco (6 to 7 years); Guinea-Bissau (7 to 8 years); Gambia (8 to 9 years).
12
3. Results
This study posed several research questions, mainly: how did the ILO develop its
LHC estimates for Africa, and are they reliable? Could its measure predict access to
primary health care for aggregate country populations in Africa or for the poor? The
findings are discussed below.
3.1 Reliability findings from the metadata
1. Nearly three out of four of the ILO’s LHC estimates for Africa are
unreliable.
2. The ILO’s measure is not transparent; in other words, it does not make its
limitations explicit to the reader.
3.2 Issues affecting the estimates
Out of the 54 African countries included in Table B.11, 47 have LHC estimates. For
the 47 African countries with data, the mean LHC estimate was 28.8%; the median
was 9.0%, and the standard deviation was very large, at 34.6 percentage points. After
reviewing the ILO’s metadata, this study found that only slightly more than one-
fourth (25.5%) of the organisation’s estimates were reliable. In other words, 12
country estimates were found to be reliable, and 35 (74.5%) unreliable. The list
below details some of the issues uncovered by this study. Not every issue applied to
every estimate, but these issues affected at least several estimates:
1. There is a numerical discrepancy between Tables B.11 and B.11b in terms
of a country’s health coverage estimate.
2. The ILO does not provide any reference information for a country’s
estimate.
3. The ILO cites the reference(s) for a country so poorly (e.g., without a page
number in a document several hundred pages long, or without the proper
title) that the figure or source itself is impossible or extremely time-
consuming to find.
4. The documents(s) cited for a country’s coverage estimate are of poor
quality or originate from dubious source(s).
5. The ILO’s estimate is either not contained in the source from which it
claims the estimate originates from, or the estimate is completely different
13
from the statistic actually found in the source where the figure is supposed
to be from.
6. The ILO’s coverage estimate is based on a statistic that excludes nearly
half or more than half of a country’s population; for example, it only
estimates coverage for those aged 15-49, as opposed to the entire
population.
7. The ILO reports qualitative data as a quantitative figure (i.e., it takes text
describing a country’s health coverage and reports it as a percentage).
3.2.1 Discrepancies between tables and a lack of
references
To begin with, Table B.11b includes only 43 African countries, omitting 11
additional countries that are included in Table B.11. Six of these 11 countries6 have
“total coverage” estimates available in Table B.11 (in Table B.11b it says “[no data
available]”). Even though Table B.11 sends the reader to B.11b for source
information, Table B.11b fails to reference any sources for six countries.
Additionally, Table B.11 provides a 5.0% coverage estimate for Comoros, and 65.0%
for Cape Verde, but Table B.11b states that no coverage estimates are available for
either country; it also fails to provide any references for either country. This means
that no references are provided for eight of Africa’s “total coverage” estimates.7 In
other words, the ILO fails to provide sources for nearly one-fifth of the African
countries with available LHC. This ratio does not even consider estimates where the
ILO cited a source, but I was unable to locate it. This ratio also does not consider
coverage estimates that fail to meet this study’s criteria for reliability, as listed in the
methodology section. There are also cases where Table B.11 lists a different year for
its coverage estimate than Table B.11b (for example, for the Central African
Republic, Gambia, and Guinea-Bissau).
6 Eritrea, Ethiopia, Libya, Mozambique, Seychelles, and Somalia. 7 Cape Verde, Comoros, Eritrea, Ethiopia, Libya, Mozambique, Seychelles, Somalia.
14
3.2.2 Percentage of country populations considered
in estimates
In addition to the estimates with no available source information, six estimates
exclude nearly half or more than half of that country’s population. The coverage
estimates for Madagascar, Namibia, Nigeria, Sao Tome and Principe, Swaziland, and
Zambia only include citizens aged 15-49.8 Although the ILO reports this in its
footnotes, readers must scour over the small print in Table B.11b, a document
external to the World Social Protection Report, to learn of this issue. The font size is
only Calibri point-8, and some estimates have an incorrect footnote (e.g., 22 instead
of 20). This supports my argument that the ILO fails to make major flaws in its
estimates transparent. See “footnote 15” in Figure 1 below for an example of this.
Figure 1: Large exceptions, small print
3.2.3 Reliability coding of metadata
Only 12 out of the 47 African country estimates meet the criteria of reliability
outlined in the methodology chapter. There are even issues with a few of the 12
“good” or “acceptable” estimates, although these problems were minor enough to
code them as 1 (reliable). This study’s results indicate that only slightly more than
25% of the ILO’s estimates have at least one coverage source that is reliable. For a
list of individual country coding, and the unique issues present for each country, 8 The ILO cites two sources for Egypt, one of which only includes ages 15-59.
10 Association africaine de l'économie et de la politique de la santé (AfHEA), Zohoré Olivier KOUDOU (2011). Accessibilité des services de santé en Afrique de
l’Ouest : le cas de la Côte d’Ivoire - 15 -17 mars 2011. Available at:
http://afhea.org/conference/conference2011/files/Presentations/Day%202/Parallel%20session%204/KOUDOU-Orale-Fr.pptx, p 5.
En Côte d’Ivoire, sur une population de près de 20 millions, seul 1,22 % bénéficie des services d’assurance maladie (Banque mondiale, 2008).
11 WHO, Health System profile Djibouti (2006). Available at: http://gis.emro.who.int/HealthSystemObservatory/PDF/Djibouti/Full%20Profile.pdf, p 52.
It is estimated that only 30 percent of the population has access to the public health system.12 ILO (2009). ILO considerations on the social health insurance reform project in Egypt, June 2009. Available at:
www.ilo.org/gimi/gess/RessFileDownload.do?ressourceId=14306, p 11 (Initial source : Hewitt Associates SA. 2008. Page 28.).
The current coverage is 38.8 million persons as of 2008 equaling about half of the population.
13 Fatma El-Zanaty et al (2008) Egypt Demographic Health Survey 2008. Concerns population aged 15-59. Available at:
http://www.measuredhs.com/pubs/pdf/FR220/FR220.pdf, p 278.14 African Development Bank Group. Annual report 2010. Available at: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/Chap%205%20-
%20Activities%20of%20the%20Boards.pdf15 Ghana National Social Protection Strategy. Republic of Ghana, Ministry of employment and social welfare (MESW) (January 2012). See also Demographic and
Health Surveys, Ghana Demographic Health Survey 2008, tables 3.8.1 and 3.8.2., Population aged 15-49. Available at:
http://www.measuredhs.com/pubs/pdf/FR221/FR221.pdf, p 48-49
16 Centre International de Développement et de Recherche (2011). Rapport d'activités 2010 Systèmes de santé et prévoyance sociale, Mai 2011. Available at:
http://cidr.org/IMG/pdf/Rapport_d_activites_SSPS_2010.pdf, p 9.
15
Table 1 below. Table 1 has six columns, each representing a distinct issue. The last
column, “Document not in English” had no bearing on whether an estimate was
coded as unreliable; however, it is meaningful to point this out, because it means that
those who do not speak French or Portuguese need to translate the source. Where the
ILO referenced more than one source, I affixed a- or b- to the country to show
findings for the individual source.
16
Table 1: Metadata coding
Country
estimates
Overall
categorisatio
n (reliable=1;
not
reliable=0)
Discrepanc
y between
Tables B.11
and B.11b
ILO lists
no
reference
s for
estimate,
or source
documen
t unable
to be
located
Cited
estimat
e is
differen
t from
source's
or not
present
in
source
Estimate
includes
less than
90% of
country's
populatio
n
Citation
missing
key
informatio
n or uses
incorrect
info (e.g.,
title of
source
document,
or page
number in
document
50 or more
pages)
Dubious
source
document
, or
source
document
does not
explain
how it
obtained
figure
Documen
t not in
English
Algeria 0 X X X X
Angola 0 X X X
Benin 1 X X
Burkina
Faso
0 X X X
Burundi 0 X X X
Cameroon 1 X X
Cape Verde-
a
0 X X X X
Cape Verde-
b
0 X X
17
Central
African
Republic
1 X
Comoros 0 X X
Congo,
Democratic
Republic of
0
X X
Côte
d’Ivoire
0 X X
Djibouti 0 X
Egypt- a 1 1 X
Egypt- b 0 X X
Eritrea 0 X X
Ethiopia 0 X X
Gabon 0 X X X
Gambia 1
Ghana- a 0 X
Ghana- b 0 X X
Guinea 0 X X
Guinea-
Bissau
1
Kenya 1 X
Lesotho 0 X
Libya 0 X X
Madagascar 0 X X
Mali 0 X X
Mauritania 0 X X
Mauritius 1
18
Morocco- a 0 X
Morocco- b 0 X X
Mozambiqu
e
0 X X
Namibia 0 X X
Niger 0 X X
Nigeria 0 X
Rwanda 0 X X
Sao Tome
and
Principe
0 X X
Senegal- a 0 X
Senegal- b 1 X
Seychelles 0 X X
Sierra
Leone
0 X
Somalia 0 X X
South
Africa
0 X X
Sudan 0 X X
Swaziland 0 X
Tanzania 1 X
Togo 0 X X
Tunisia 1 X X
Uganda 0 X
Zambia 0 X
Zimbabwe 1
19
3.3 Another ILO publication with even more
discrepant estimates?
In addition to the issues discussed above, there are a number of discrepancies
between the LHC figures found in the World Social Protection Report and those of
another ILO (2008b) publication, entitled Social health protection: an ILO strategy
towards universal access to health care. Social health protection contains Table
A2.2, titled “Formal health coverage”, which also lists LHC estimates for African
countries. Table B.11b cites Table A2.2 as its source for the following estimates: The
Central African Republic, Gambia, and Guinea-Bissau. The problem, however, is
that Table A2.2 was published in 2008, and Table B.11 claims that the estimates are
from 2011 (though Table B.11b lists 2008). 9 This means that the ILO lists the
incorrect years for the estimates in Table B.11, or that these estimates did not
originate in Table A2.2. Furthermore, Table A2.2 does not explain how these
estimates were initially obtained, though the foreword of the 2008 publication states
that “various international organisations and experts…contributed” to the overall
content (ILO, 2008b:xi).
Out of the 28 African countries that both Table A2.2 and Table B.11 have estimates
for, only six countries share the same estimate in both tables. For example, Table
A2.2 estimates Tunisia’s LHC figure at 99.9%, whereas Table B.11 claims it is
80.0%. This is in spite of the fact that Table B.11 claims that the 80.0% figure is
from 2005, before Table A2.2 was even created. If Tunisia’s estimate was calculated
before the creation of Table A2.2, both tables should have the same estimate. Table
B.11b lists seven of the 28 incongruous estimates as being from 2008 or earlier.10
Although Table A2.2 does not provide years for its estimates, several of them are
discussed in-depth in the text and are referred to using the present tense, indicating
that the figures are from 2008 (the document’s year of publication). For example,
the text says: “In Ghana with a per capita GDP of US$320, 18.7 per cent of to the
population is formally covered by a health protection scheme” (ILO, 2008b:19).
3.4 Inquiry to the ILO
In order to learn why there are so many discrepancies in the ILO’s estimates; to ask
the ILO how it arrived at the estimates without references; and to find out why it
9 Table B.11 does not provide a year for its estimate of the Central African Republic. 10 Only estimates that differ by more than 0.5% are included in this figure.
20
often presents different figures than the sources that it cites, I sent an inquiry in
December 2015 to three email addresses listed as contacts on the ILO’s website. The
email explained that I was a researcher from the University of Cape Town writing
about LHC in Africa, and that I was trying to discern how the ILO arrived at its
estimates in its World Social Protection Report for 2014/15. I provided several
examples of the discrepancies that I uncovered in the metadata and asked the
organisation to explain how it arrived at these figures, or to provide me with contact
information for someone who would be able to do so. To date, I have not received a
reply from the ILO. See Appendix A for the text of this letter.
3.5 Findings on the predictive power of the ILO’s
estimates for access to primary health care
1. The ILO’s LHC measure was not a significant predictor of aggregate birth
coverage in Africa.
2. The measure held no significant predictive power for the birth coverage of
Africa’s poorest residents.
To test whether the ILO’s measure could predict access to health care in Africa, a
simple linear regression was conducted using the ILO’s measure as the predictor
variable and aggregate birth coverage as the outcome variable. The ILO’s LHC
measure did not emerge as a significant predictor of aggregate birth coverage in
Africa [p≥.05]. These findings indicate a further lack of reliability for the ILO’s LHC
measure; a valid measure of LHC is likely to have a significant association with
access to primary health care in Africa. See Figure 2 below for the scatterplot of this
relationship, and Table 2 for the results from the simple linear regression.
21
Figure 2: ILO LHC and aggregate birth coverage
22
Table 2: Regression results: ILO LHC and aggregate birth coverage
Model Summaryb
Model R R Square
Adjusted R
Square
Std. Error of
the Estimate
1 .316a .100 .074 17.17411
a. Predictors: (Constant), ILO LHC estimate (%)
b. Dependent Variable: Average birth coverage of all
quintiles (%)
ANOVAa
Model
Sum of
Squares df Mean Square F Sig.
1 Regression 1143.881 1 1143.881 3.878 .057b
Residual 10323.250 35 294.950
Total 11467.131 36
a. Dependent Variable: Average birth coverage of all quintiles (%)
b. Predictors: (Constant), ILO LHC estimate (%)
Coefficientsa
Model
Unstandardized
Coefficients
Standardized
Coefficients
t Sig. B Std. Error Beta
1 (Constant) 59.087 3.485 16.953 .000
ILO LHC estimate
(%)
.194 .098 .316 1.969 .057
a. Dependent Variable: Average birth coverage of all quintiles (%)
The ILO’s LHC measure also failed to significantly predict birth coverage for the
poor [p≥.05], indicating that the measure is not inclusive of the poor. See Figure 3:
ILO LHC and birth coverage for the poor (Q1) below for the scatterplot of this
relationship, and Table 3 for the results from the simple linear regression.
23
Figure 3: ILO LHC and birth coverage for the poor (Q1)
24
Table 3: Regression results: ILO LHC and Q1 birth coverage
Model Summaryb
Model R R Square
Adjusted R
Square
Std. Error of the
Estimate
1 .317a .100 .075 20.0537
a. Predictors: (Constant), ILO LHC estimate (%)
b. Dependent Variable: Birth coverage of the poorest quintile
(%)
ANOVAa
Model
Sum of
Squares df Mean Square F Sig.
1 Regression 1568.822 1 1568.822 3.901 .056b
Residual 14075.335 35 402.152
Total 15644.157 36
Coefficientsa
Model
Unstandardized
Coefficients
Standardized
Coefficients
t Sig. B Std. Error Beta
1 (Constant) 34.806 4.070 8.552 .000
ILO LHC estimate
(%)
.227 .115 .317 1.975 .056
a. Dependent Variable: Birth coverage of the poorest quintile (%)
4. Discussion
An extensive investigation into the ILO’s metadata in Table B.11b provided
evidence that the ILO’s LHC estimates, which it conflates with “total effective
coverage” for Africa are not reliable. This remains true even though reliability was
assessed based on the ILO’s own indicator of LHC, provided in the World Social
Protection Report. In other words, this study’s reliability analysis did not consider
whether the ILO’s operationalisation of LHC captured the concept of rights-based
protection; the reliability assessment only determined whether the ILO’s estimates
25
were quantified according to its own indicator. This study confined the critique of
the ILO’s indicator to Chapter 5.
This study’s finding that the ILO’s LHC measure in the World Social Protection
Report was not a significant predictor of access to primary health care in Africa,
either for total country populations or for the poorest quintile, suggests that there are
problems with the ILO’s health measure. One would expect higher levels of LHC to
be strongly associated with higher birth coverage – yet this is not the case. This
chapter explains these findings in more detail by providing illustrative examples and
analysis.
4.1 Unreliable estimates and sources: Instructive
examples
Although detailed information on each country’s estimate can be found in Appendix
B, it is still prudent to discuss some instructive examples. Sudan’s metadata is a good
case to start with. The ILO’s measure estimates Sudan’s coverage rate at 29.7%. The
metadata claims that this figure originates from Table 22 on page 266 in the Sudan
statistical yearbook 2009 (Central Bureau of Statistics & Ministry of the Cabinet,
2009). Although there are health coverage statistics in this table, the figure 29.7% is
not listed at all, nor is there any estimate for the total Sudanese population. The
statistics that are included in the document are divided into seemingly random
categories, such as the health coverage rate for shepherds and “Martyrs Families”.
The document provides no explanation for these classifications.
Additionally, both a manual search and a search of the figure “29.7” using Window’s
Control-F search function confirmed that nothing in this document states that 29.7%
of Sudanese have healthcare coverage. In any case, even if the ILO’s estimate were
present in the source, the source itself seems to be of objectionable quality. There
are many significant errors and omissions immediately obvious to anyone casually
perusing the document. For instance, at the bottom of Table 22, the document says,
“The data don’t inclued Khartoum State [sic]” (Central Bureau of Statistics &
Ministry of the Cabinet, 2009). Parts of the text are in English and other parts in
Arabic, even on the same table, for no discernible reason, and often without a
translation. Moreover, it is unclear how the Sudanese government arrived at these
figures. The document does not explain its methodology, nor does it explain what it
means by many of its categories, such as “poor families”. Thus, this study coded
Sudan’s estimate as 0: unreliable.
26
South Africa’s LHC estimate is another good example. The ILO states that South
Africa has 100% LHC and bases this claim on a single qualitative source: a quote
from a South African government official, Andrew Donaldson, from a 2010 health
conference in Portland, Oregon. Not only is it problematic to transform qualitative
information into quantitative estimates, but the official never actually stated that
South Africa has 100% LHC. This is, in fact, what he said: “Most South Africans
have universal access to primary health care and hospital services that is free at the
point of service...for most users of the public system, services are free [emphasis
mine]” (2010:2). Besides not actually saying that all South Africans have health
coverage, the Minister appears to be referring to health care access while
overlooking legal protections.
Ghana is the last example I will discuss. Table B.11 presents Ghana’s coverage figure
as 73.9%. Table B.11b lists Ghana’s Demographic Health Survey as the source of
this estimate, specifically Tables 3.8.1 (“Health insurance coverage: women”) and
3.8.2 (“Health insurance coverage: men”). These tables provide the “Percent
distribution of women [and men] aged 15-49 by type of health insurance coverage,
according to background characteristics” (Ghana Statistical Service, Ghana Health
Service & ICF Macro, 2009). Estimates are provided for the percentage of
individuals in four insurance categories. A fifth column labelled “No health
insurance” quantifies the percentage of Ghanaians who have no health insurance at
all. These tables state that 60.1% of women aged 15-49 have no form of health
insurance, while 70.3% of men in this age demographic have none. In other words,
only 39.9% of women and 29.7% of Ghanaian men in that age group have health
insurance. It is therefore unclear as to how the ILO arrived at its 73.9% coverage
estimate for Ghana based on this information. Furthermore, even if the ILO had used
the actual estimates from the source, the tables only include individuals aged 15-49.
Based on the CIA’s (2016) Ghanaian population estimates, this would require
leaving out more than 48% of Ghana’s population in the coverage figure. The ILO
does mention that its estimate only “includes ages 15-49” in its footnotes for Ghana,
as shown in Figure 3; however, this omission of nearly half of Ghana’s population
is not overt. Finally, Table B.11b states that Ghana’s 73.9% coverage estimate is
from 2010, but in 2008 the ILO published a document estimating Ghana’s LHC
figure at 18.7%. It is questionable that Ghana’s LHC increased 55.2% in just two
years.
As discussed in the results section, Ghana’s LHC estimate is only one of at least six
African estimates that omitted nearly half or more than half of a country’s
population. Thus, the measure is extremely misleading. There is little use in having
27
a LHC measure that discounts more than half of a country’s population, unless the
measure is meant to look at specific age groups only. Looking at only half of a
country’s population for certain estimates means that coverage rates cannot be
compared across countries, as the estimates are not measuring the same concept. As
noted in the results section, even though the ILO briefly mentions that some of its
estimates are only for citizens aged 15-49, this information is buried deep in the
footnotes of its metadata. This is even more problematic when one considers that the
metadata is not even in the World Social Protection Report itself, but in an external
table (Table B.11b) referenced by the report. Similarly, if one wishes to learn what
percentage of a country’s population is included in the estimate, they must then look
up that country’s population and disaggregate this information by age group in order
to figure out what percentage of the population is absent from the estimate (e.g., how
many people fall outside the ages of 15-49). The examples given above are just a
few of the many problems with the ILO’s estimates.
4.2 Conclusion
As explained in Chapter 3, it is not possible to know how the ILO obtained many of
its health coverage estimates. Perhaps the ILO’s estimates were derived from other,
more appropriate sources than the ones cited by the metadata, but for unknown
reasons the organisation neglected to cite them. It is possible that some of the
estimates were acquired from insider knowledge, which might explain why many of
the estimates differ from the sources cited for them in the metadata. If this is the
situation, it is unclear why the ILO neglected to provide this information in its
metadata, or why it chose to cite sources it did not use. In any case, the ILO’s LHC
estimates are not trustworthy. Thus, the measure is not useful for monitoring or
evaluating legal health protection, and researchers and policymakers would be wise
to avoid incorporating it into studies or base policies on it.
5. Implications and recommendations
The original intent of this study was not to critique the ILO, but to investigate
whether its total coverage measure in Table B.11 was consistent with another
measure of health coverage: birth coverage. However, after discovering that “total
coverage” estimates were in fact quantified using the ILO’s indicator for LHC and
presented in a spreadsheet called “effective coverage”, it became necessary to
28
investigate the ILO’s measure in more depth. The result was that the primary focus
of this research became a critique of the ILO’s measure.
5.1 The importance of a reliable and transparent
measure
This raises the question of why it is so important to assess the measure’s
transparency and reliability. To restate an earlier point, the ILO’s LHC measure is
one of the only, if not singular, prominent measure that offers estimates on LHC for
most of Africa. Thus, it is critical that its methodology be clear and its estimates
cogent. Secondly, effective monitoring, evaluation, and implementation of UHC
globally require “common, comparable standards to measure [countries’] progress
over time and in comparison with other countries” (Lagomarsino et al., 2012: 941).
Because many scholars argue that a “legal [coverage] mandate…provides a
framework for social [health] mobilisation” (Stuckler et al., 2010: 33), LHC could
certainly become a factor during deliberations over where and how to invest
government and donor resources. If an NGO unknowingly uses unreliable estimates,
this could negatively impact the expansion of UHC and the right to health. To
reiterate an important point mentioned earlier, the ILO’s Director-General, Guy
Ryder, publicly recommended that policymakers and practitioners use the
information provided by the World Social Protection Report; doing so, however,
may pose a risk if the information is applied to real-world development projects.
5.2 Implications of findings
Although this study only examined the LHC measure’s African estimates (as
opposed to all its country estimates), I suggest that the measure’s estimates for other
continents not be used for any study of LHC globally, either. Because this study
found that nearly 75% of its African estimates were unreliable, researchers should
be wary of estimates for other regions, as well. Furthermore, the ILO’s indicator for
LHC excludes the most critical elements that make up rights-based protection. To
make matters worse, if one important measure included in the World Social
Protection Report is unreliable, confidence in the report’s other measures is
dampened. While there are not necessarily problems with the other measures,
additional research should be conducted to assess their methodology.
29
Overall, the implications of this paper’s findings are worrying. Because studies of
this nature currently receive little scholarly attention, this problem is probably not
unique to the measure itself, or even to the ILO. If policymakers are making
decisions based on unreliable health estimates, it could have negative consequences,
including for the poor. This problem has a relatively simple solution, however.
Greater oversight from academia would certainly encourage organisations to
develop better measures. Accordingly, I recommend that a greater number of
scholars examine measures that are not peer-reviewed, even when estimates
originate from well-respected organisations. I further argue that intergovernmental
organisations have an ethical responsibility to make their methodologies transparent.
5.3 Recommendations for the ILO
Based on this study’s findings, I recommend that the ILO put out a memo retracting
its LHC measure, explaining that it should not be used by researchers and
policymakers because its estimates are not reliable. In its statement, the ILO should
also acknowledge that neither the definition(s) of LHC it provides in the World
Social Protection Report, nor its operationalisation of the concept, adequately
capture rights-based protection. If the ILO decides to revisit and redesign its
measure, it needs to follow several guidelines. Firstly, it needs to revise its indicator
of LHC. Based on my review of what it means to have rights-based protection in
health, it is clear that one quantitative indicator cannot incorporate the most
important components of this protection. At a minimum, any measure must assess
coverage range, qualifying conditions, and basic entitlements; judicialisation of the
right to health and related enforcement mechanisms; and inclusivity of marginalised
populations, particularly the poor.
Regardless of the type of measure that the ILO chooses to use, the organisation needs
to use reliable sources for its estimates, and it needs to make the limitations of its
data explicit. In cases where none are available, the ILO needs to state this, as
opposed to using sources of poor quality. No estimate is better than an unreliable
one, as this may give readers a false perception of the state of a country’s health
protections. This is not to say that any lack of reliability or transparency of the
current measure was intentional on the part of the ILO, but more care should be taken
in the future. For recommendations for future studies on LHC, please refer to Paper
1 of this series.
30
References
Anyanwu, J. C. & Erhijakpor, A. E. O. 2009. Health expenditures and health
outcomes in Africa. Oxford, UK: Blackwell Publishing Ltd.
Central Bureau of Statistics & Ministry of the Cabinet 2009. Statistical year book
for the year 2009. The Republic of Sudan.
CIA. 2016. About: Ghana [Online]. CIA. Available:
https://www.cia.gov/library/publications/resources/the-world-
factbook/geos/gh.html.
Clarke, D., Rajan, D. & Schmets, G. 2016. Health systems strengthening, universal
health coverage, health security and resilience. Bulletin of the World Health
Organization, 94, 2.
Ghana Statistical Service, Ghana Health Service & ICF Macro 2009. Ghana
Demographic and Health Survey 2008. Demographic and Health Survey
Accra, Ghana: Ghana Statistical Service, Ghana Health Service, ICF Macro.
Gwatkin, D. R. 2000. Health inequalities and the health of the poor: what do we
know? What can we do? Bulletin of the World Health Organization, 78, 3-18.
ILO. 2008a. Global campaign on social security and coverage for all. Social health
protection: an ILO strategy towards universal access to health care. Geneva,
Switzerland: ILO.
ILO. 2008b. Paper 1: Social health protection. An ILO strategy towards universal
access to health care. Geneva, Switzerland: ILO.
ILO. 2014. World Social Protection Report: building economic recovery, inclusive
development and social justice. Geneva, Switzerland: International Labour
Office.
Lagomarsino, G., Garabrant, A., Adyas, A., Muga, R. & Otoo, N. 2012. Moving
towards universal health coverage: health insurance reforms in nine
developing countries in Africa and Asia. The Lancet, 380, 933-943.
31
Mills, A., Ataguba, J. E., Akazili, J., Borghi, J., Garshong, B., Makawia, S., Mtei,
G., Harris, B., Macha, J., Meheus, F. & McIntyre, D. 2012. Equity in financing
and use of health care in Ghana, South Africa, and Tanzania: implications for
paths to universal coverage. The Lancet.
Rustein, S. O. & Johnson, K. 2004. The DHS wealth index: DHS comparative
reports Calverton, Maryland: ORC Macro.
Scheil-Adlung, X. 2015. Global evidence on inequities in rural health protection.
New data on rural deficits in health coverage for 174 countries. In: Scheil-
Adlung, X. (ed.) Extension of Social Security. Geneva, Switzerland: ILO.
Stuckler, D., Feigl, A. B., Basu, S. & McKee, M. 2010. The political economy of
universal health coverage First Global Symposium on Health Systems
Research. Montreux, Sweden: WHO.
The National Bureau of Asian Research 2010. Universal health coverage:
Discussion highlights from a 2010 Pacific Health Summit workshop. In:
Karuna Luthra & Laura Cava Northrop (eds.) Pacific Health Summit 2010.
Portland, Oregon, USA: The National Bureau of Asian Research: Centre for
Health and Aging.
United Nations Development Programme. 2016. Goal 3: Good health and well-
being [Online]. UNDP. Available:
http://www.ua.undp.org/content/undp/en/home/sdgoverview/post-2015-
development-agenda/goal-3.html [Accessed April 3 2016 ].
WHO. 2015. Health Equity Monitor: Compendium of indicator definitions [Online].
WHO. Available:
http://www.who.int/gho/health_equity/outcomes/health_equity_compendiu
m.pdf?ua=1 [Accessed April 3 2016].
WHO. 2016. The Health Equity Monitor [Online]. WHO. Available:
http://www.who.int/gho/health_equity/en [Accessed April 16 2016].
32
Appendix: Detailed country metadata
The World Social Protection Report contains Table B.11, The multiple dimensions
of health coverage. Table B.11’s first column provides estimates for “health
coverage as a percentage of the total population” for most of the globe, including 54
African countries.11 Table B.11’s notes expand on what it means by coverage:
“Coverage includes affiliated members of health insurance or estimation of the
population having free access to health care services provided by the State”. Table
B.11 refers readers to Table B.11b: The multiple dimensions of health coverage:
Percentage of the population covered (members of health insurance or free access
to health care services provided by the State) for detailed source information on how
the ILO obtained coverage estimates for non-OECD countries.12 By exploring the
metadata provided for African coverage estimates in Table B.11b, I determined
which coverage estimates were reliable. This appendix includes detailed reliability
analysis on every African country with coverage estimates. As stated in the main
body of this paper, estimates were coded as reliable (1) if they met the following
conditions:
1. The ILO cited at least one source for the estimate, and I was able to locate
that source. The source was only considered to be missing if I was unable
to locate it after a comprehensive Internet search using Google and Google
Scholar.
2. The estimate appeared to include the entire population of a country, or at
least more than 90%.
3. The cited source seemed to be of good quality or, at minimum, acceptable.
4. Researcher discretion was used to determine whether a source’s quality
was ‘good’ or ‘acceptable’, but I will list some examples here for
instructive purposes: the source was peer-reviewed; it appeared to be
professional or academic; it was clear how the source derived its estimates.
5. The estimate cited by the ILO was able to be verified/located in the source.
6. The estimate reflected the ILO’s definition of LHC (i.e., the percentage of
a country’s total population that has “health insurance or free access to
health care services provided by the State” (ILO, 2014b).
11 Botswana, Chad, Congo, Equatorial Guinea, Liberia, Malawi, and South Sudan are included in
Table B.11, but they are listed as having a coverage rate of (…), which means that data was not
available. 12http://www.socialprotection.org/gimi/gess/RessourceDownload.action?ressource.ressourceId=3
7218
33
Unreliable estimates (i.e., those that do not meet the above conditions) were coded
as 0. Each country in this appendix has a 0 (unreliable) or 1 (reliable) next to its
name, and Table B.11b’s health coverage estimate in parentheses. I explain instances
where a discrepancy exists between Table B.11 and Table B.11b.
1- Algeria 0 (85.2%)
The source document is in French. The URL the ILO provides for the source is
broken, and the ILO’s citation does not include a title for the source; however, I was
able to find the source after searching the International Social Security Association
(ISSA) website. The International Social Security Association (2016) is an
international organisation that “provides access to information, expert advice,
business standards, practical guidelines and platforms for members to build and
promote dynamic social security systems worldwide”. It is affiliated with the ILO
and the UN (See https://www.issa.int/the-issa). The source document appears to be
a paper presented at the Regional Conference of the ISSA for Africa on the theme
"Social Security in the African context". It is not an academic work and contains no
sources.
The document’s author is Patricia Bougrine, who was apparently the Director of
Services for the National Social Insurance Fund for Salaried Workers in Algeria
(Directrice des prestations caisse nationale des assurances sociales des travailleurs
salaries Algérie) (Bougrine, 2005). It is unclear whether she works for the ISSA or
the Algerian government. The source discusses the social security system of Algeria
but does not contain the 85.2% coverage figure cited by the metadata on the page
provided (page 5), nor does the document offer the 85.2% figure anywhere else.
cited by the ILO on page 5 or anywhere else, nor does it appear to present any
overall health coverage figure (Bougrine, 2005).
2- Angola 0 (0% in Table B.11; ‘…’ in Table B.11b)
Table B.11b provides qualitative information on Angola’s coverage but states that
no data is available for a coverage estimate, despite the fact that Table B.11 estimates
Angola’s coverage at 0%. The source link for Angola’s metadata leads to a now-
retired website (Health Systems 20/20, which was a USAID project that closed in
2012).13 The ILO metadata does not state which page it obtained its estimate from 13 https://www.hfgproject.org/health-systems-2020-retires/
34
in the 132-page source document. Furthermore, the ILO sheet’s metadata says the
following: “Angola does not have any health insurance, such as social health
insurance, private health insurance, or community-based health insurance”.
However, the document cited by the ILO clearly states that, while Angola previously
did not have private health insurance, “Private health insurance options have
emerged since 2005 targeting companies and upper income households” (Connor,
Averbug & Miralles, 2010:xxi). Pages 23, 26, and 77 also explain that Angola has
private health insurance options. The report appears to be of a good academic
quality, with three qualified researchers collecting background data and conducting
qualitative interviews with stakeholders in Angola; however, the information the
ILO provides about the report is incorrect.
3- Benin 1 (9.0%)
The source document is in French. The URL works. The translated title of the report
is Study on the situation and prospects for social protection in Benin (Etude sur l'état
des lieux et les perspectives de protection sociale au Bénin) (Hodges et al., 2010).
The ILO states that 9% of people in Benin are covered by health insurance. While
this figure is found in the cited document, the ILO lists the incorrect page number in
a 94-page document (the figure is actually on page iv). The 9% coverage estimate
includes coverage from the Benin National Retirement Fund (Fonds National de
Retraite du Bénin- FNRB) (around 6%), mutual health insurance (less than 2%), and
private insurance (less than 1%) (Hodges et al., 2010: iv). The source seems to be
reliable; it has academic references.
4- Botswana* (…)14
The ILO did not provide a coverage estimate for Botswana in either table.
5- Burkina Faso 0 (1.0%)
The source document is in French. The translated title is Burkina Faso. Spending
review and performance of social protection (ILO, 2012). When I accessed the URL
provided by the ILO in April 2015, the document was not listed for download; 14 Asterisks (*) refer to countries that are listed in Table B.11, but not Table B.11b, despite B.11
stating that all non-OECD country sources can be found in B.11b.
35
nevertheless, I was able to locate it after an Internet search. The ILO metadata claims
that the document states that 1% of people in Burkina Faso have healthcare coverage;
however, the ILO metadata does not include a page number for this information in
a document that contains more than 100 pages, and I was unable to confirm the
verity of the statistic after searching the document.
6- Burundi 0 (28.4%)
The source document is in French. The translated title is Draft document as a basis
for discussions on national policy on social protection in Burundi. The ILO metadata
does not include a link for the source, and I was unable to find the document
referenced for Burundi’s coverage estimate after conducting both a Google search
and a search of the social-protection.org website.
7- Cameroon 1 (2.0%)
The source document is in French. The translated title is Platform promoters of
mutual health in Cameroon (Promuscam): Strategic plan for the promotion and
development of mutual health organisations in Cameroon (Republic of Cameroon,
2006). When I followed the link provided by the ILO, I received an error message,
but I was able to find another URL that leads to the PDF document. Page 28, the
page number provided by the ILO, states that less than 1% of Cameroonians are
covered by private health insurance. It also says that health insurance is compulsory
for staff of the private and public sectors through the National Social Insurance Fund,
but that this covers only 1% of the population. Although the ILO table states that 2%
of Cameroonians have health coverage, I was unable to find anything on the page
cited by the ILO that says 2%; the only statistic I was able to find says that 1% of
Cameroonians have mutual health insurance. I have still coded it as reliable because
it is only one percentage point off.
8- Cape Verde 0 (65.0% in Table B.11; ‘…’ in Table B.11b)
In Table B.11, Cape Verde is listed as having 65% coverage, but Table B.11b, which
is supposed to be the source of B.11, lists Cape Verde as having no data (…). The
first source is in Portuguese. The URL provided by the ILO is broken, and I was
unable to find the source document through a Google search and a search of the
36
website provided. The metadata did not include a year for the publication, and there
are a number of publications with the term “Protecção social” in the document title.
The second source is a two-page country brief on Cape Verde, entitled: Social
protection floor: Country brief for Cape Verde (ILO, 2010b). The URL provided by
the ILO works. This source does not provide any coverage estimates for Cape Verde.
9- Central African Republic 1 (6.0%)
The link does not take you to PDF of the cited report but instead to ILO’s Social
Protection Department website. I s able to locate another URL that leads to the actual
PDF document. The metadata states that the source’s title is Formal coverage in
social health protection, but this refers to the title of a table in the document (Table
A.2.2) instead of the report’s title, Paper 1: Social health protection. An ILO strategy
towards universal access to health care (ILO, 2008b). The citation in Table B.11b
does not provide information on which page the estimate was obtained from, despite
the fact that the source is over 100 pages long.
Below, I present the figures that Table A.2.2 provides for formal coverage
percentage (this includes State, SHI, PHI, Company/Trade-Based Union, and MHI
Total). Strangely, the ILO chooses to cite Table A2.2 as a source for only three
African countries (the Central African Republic, Gambia, and Guinea-Bissau), even
though Table A2.2 has coverage estimates for 27 African countries. Moreover, there
are discrepancies between the World Social Protection Report and Paper 1: Social
Health Protection for 19 African estimates. Many of these differences are extreme.
Some of the estimates provided by the source are much lower than the ILO’s World
Social Protection Report’s estimates, and some much higher. For example, Table
A2.2 states that Ghana has 18.7% health coverage, whereas the ILO metadata claims
that Ghana has 73.9% coverage. In the case of Tunisia, this table estimates a 99%
formal coverage rate, whereas the ILO metadata provides a figure of only 80%. This
would not be as confounding if the ILO did not obtain coverage estimates for CAR,
Gambia, and Guinea-Bissau from this source, while simultaneously choosing not to
use its additional 27 African coverage estimates.
There is a disparity between the coverage rates given for 21 of the countries included
in both documents. For countries where the coverage estimate differs between the
World Social Protection Report and Paper 1: Social Health Protection, countries
are bolded.
37
Table 4 below shows discrepancies in coverage estimates between Table A2.2 and
Table B.11b. I discovered after further investigation that the estimates in Table A2.2
can actually be found in a hidden Excel column of Table B.11b that says “SECSOC
estimate/formal coverage (%) OLD”. I was unable to verify whether the estimates
in Table A.2.2 are, in fact, older than the estimates in Table B.11b because Table
A2.2 does not provide years for its estimates. I gave the ILO the benefit of the doubt
and coded CAR, Gambia, and Guinea-Bissau’s coverage estimates as reliable.
Table 4: Discrepancies in coverage estimates between Table A2.2 and Table B.11b
Country Coverage estimate in Table
A2.2
Coverage estimate in Table
B.11b
Benin 0.5% 9.0%
Burkina
Faso
0.2% 1.0%
Burundi 13.0% 28.4%
Cameroon 0.1% 2.0%
Cape Verde 65.0% ...in Table B.11b; 65% in Table
B.11
Congo … 10.0%
Côte
d’Ivoire
5.0% 1.2%
Egypt 47.6% 51.1%
Gabon 55.0% 57.6%
Ghana 18.7% 73.9%
Guinea 1.1% 0.2%
Kenya 25.0% 39.4%
Mali 2.0% 1.9%
Mauritania 0.3% 6.0%
Morocco 41.2% 42.3%
Namibia 22.5% 28.0%
Niger 0.7% 3.1%
Rwanda 36.6% 91.0%
Senegal 11.7% 20.1%
Tanzania 14.5% 13.0%
Togo 0.4% 4.0%
Tunisia 99.0% 80.0%
Uganda 0.1% 2.0%
38
10- Chad* (…)
The ILO did not provide a coverage estimate for Chad in either Table B.11 or Table
B.11b.
11- Comoros 0 (5% in Table B.11; ‘…’ in Table B.11b)
Comoros is a hidden row in the Excel sheet for Table B.11b. Once it is unhidden,
the data states that its coverage estimate is (…), but Table B.11 provides a 5%
coverage estimate for Comoros. It is unclear why there is a discrepancy between the
two figures provided.
Congo (…)
Congo is a hidden row in the Excel sheet for Table B.11b. Once it is unhidden, the
data states that its coverage estimate is (…).
12- Congo, Democratic Republic of 0 (10%)
The source document is in French. The URL provided by the ILO works, and links
to a PowerPoint called Universal coverage: what model for the DRC? [translated]
(COOPAMI, 2010). The document was published by COOPAMI, whose stated
mission is to provide assistance and expertise to developing countries that want “to
develop or modernize a universal and sustainable social health protection based on
solidarity” (COOPAMI, 2009). COOPAMI is a staff service provided by Belgium’s
National Institute for Health and Disability Insurance (NIHDI). The organisation
intends to make its expertise in management of health insurance in Belgium
available to other countries in order to contribute to combat poverty and other social
issues. The source is not of an academic nature and contains no references for its
Congolese coverage estimates. Furthermore, it is unclear whether the 10% coverage
figure it provides refers to LHC or de facto access. This does not appear to be a
reliable source.
13- Côte d’Ivoire 0 (1.2%)
The source document is in French, and the URL provided by the ILO works. The
estimate in the source matches the
39
ILO's estimate, and says that it obtained 1.2% estimate for Côte d’Ivoire from the
World Bank, but the PowerPoint does not contain a bibliography. I was unable to
verify the statistic from the original source because the PowerPoint did not contain
enough information for me to locate it. Additionally, it is also unclear whether the
PowerPoint’s coverage estimate refers to effective access or LHC. This does not
appear to be a good source.
14- Djibouti 0 (30.0%)
The URL provided by the ILO for the source is broken, but I was able to locate the
source after an Internet search. The source is a WHO (2006:52) publication, Health
system profile: Djibouti, which states the following: “It is estimated that only 30
percent of the [Djibouti] population has access to the public health system” (WHO,
2006:52). The document does not explain who estimates a 30% coverage rate or how
the estimate was quantified, nor does it define what it means by “access to the public
health system”. This estimate also seems to be referring to effective access, as
opposed to LHC. Additionally, the document has a table on page 51, entitled
Population coverage by source, which seems to indicate that the WHO does not
know the health coverage rate for Djibouti’s population; coverage rates are listed as
dashes, and there is no key that explains what the dashes refer to.
15- Egypt 0 (51.1%)
The metadata references the publication, ILO considerations on the social health
insurance reform project in Egypt, as the first source for its 51.1% coverage estimate
(ILO, 2009). The URL provided for this reference works, but the metadata does not
40
include a page number for the estimate in the reference, despite the fact that the
document is 66-pages long. ILO considerations on the social health insurance
reform project in Egypt states the following:
The Egyptian health insurance covers various groups through different schemes,
most of them compulsory. The current coverage is 38.8 million persons as of 2008
equalling about half of the population. Most of the people not covered by HIO belong
to low-income groups. Coverage rates across regions vary considerably because of
different population structures in the regions (ILO, 2009:11).
ILO considerations on the social health insurance reform project in Egypt
references another source as the origin of this estimate: Hewitt Associates SA. This
source is also cited in the metadata: “(Initial source: Hewitt Associates SA. 2008.
Page 28.) [sic]”. The metadata provides no link or document title, only the name of
the organisation that produced the initial document. As there are a number of
organisations named Hewitt Associates, I was unable to locate the original source
or verify the statistic by conducting a Google search, so I have coded the estimate
as unreliable
.
16- Equatorial Guinea* (…)
17- Eritrea* 0 (5% in Table B.11; not listed in Table B.11b)
18- Ethiopia* 0 (5% in Table B.11; not listed in Table B.11b)
19- Gabon 0 (57.6%)
The URL provided in the metadata leads to a PDF document for “Chapter 5” of the
African Development Bank and African Development Fund (2010) publication,
Annual Report 2010, but it does not provide a page number where the statistic can
be found, even though Chapter 5 is over 100 pages and the Annual Report 2010 is
over 300 pages. The ILO measure claims that Gabon has a coverage rate of 57.6%;
however, I was unable to find any such statistic in their cited source document, nor
was I able to find this statistic once I looked through all of the Annual Report 2010’s
chapters (as opposed to just searching Chapter 5). A Control-F search of “57.6”
yielded no statistic of health care coverage. Furthermore, the only section of the
41
Chapter 5 that even mentions the term insurance or Gabon is on page 79, which does
not have any coverage statistics (see snapshot below).
20- Gambia 1 (99.9%)
See the explanation given earlier for the Central African Republic (the source used
for Gambia is the same source cited for CAR). The coverage figure for Gambia
(99.9%) can be found in Table A2.2, Formal coverage in social health protection
(ILO, 2008b: 85).
42
1- Ghana 0 (73.9%)
The metadata lists two sources, yet it only includes the URL for the second source,
which is broken. I managed to locate the second source anyway, the Ghana
Demographic Health Survey (Ghana Statistical Service, Ghana Health Service &
ICF Macro, 2009). The ILO measure states that Ghana has a 73.9% coverage rate
and that it obtained this figure from Tables 3.8.1 (Health insurance coverage:
Women) and 3.8.2 (Health insurance coverage: Men). The tables provide the
“Percent distribution of women [and men] age 15-49 by type of health insurance
coverage, according to background characteristics”. They also state that 60.1% of
women in this age demographic do not possess any form of health insurance, while
70.3% of men in that demographic have none. In other words, only 39.9% of women
and 29.7% of Ghanaian men in that demographic possess health insurance. It is
unclear how the ILO arrived at the figure of 73.9% coverage from this information.
Even if the ILO had used the correct figures from this page, the tables only include
ages 15-49. This would require leaving out more than 38% of Ghana’s population
in the coverage figure (CIA, 2016).
43
44
22- Guinea 0 (0.2%)
The source document is in French. The URL provided by the ILO works. The
translated title is Activity report 2010: health and welfare systems (Centre
International de Développement et de Recherche, 2011). The metadata states that
the ILO learned of the 0.2% coverage rate for Guineans on page 9, but there is
nothing on page 9 that states 0.2% of Guineans have health coverage, and I was
unable to find this figure anywhere else in the document.
45
23- Guinea-Bissau 1 (1.6%)
See my earlier explanation for the Central African Republic (the source used for
Guinea-Bissau is the same as the one cited for the Central African Republic). The
coverage figure for Guinea-Bissau (1.6%) can be found on page 85 in Table A2.2,
Formal coverage in social health protection (ILO, 2008b).
24- Kenya 1 (39.4%)
The URL provided by the ILO works. Table B.11b claims that the source it cites for
its estimate (which is another ILO document), Kenya developing an integrated
national social protection policy, states that the 39.4% of Kenyans have health
coverage. However, page 56 states that 39% of Kenyans have health insurance, not
39.4%, though this is likely due to rounding (ILO, 2010a). A Control-F search
confirmed that the percentage 39.4 does not appear anywhere in the document. The
39% coverage figure refers to insurance only; it includes “NHIF [National Hospital
Insurance Fund] members including voluntary, NHIF families, and micro insurance”
(ILO, 2010a: 56). The figures presented in the document appear to be based on
reliable information.
46
25- Lesotho 0 (17.6%)
The URL works. The citation does not include the page number for an 18-page
document, called Reaching universal coverage by means of social health insurance
in Lesotho? Results and implications from a financial feasibility assessment
(Mathauer et al., 2011). The figure cited by the metadata (See below) seems to
contain estimates of coverage if a hypothetical insurance scheme were to be
introduced, not actual coverage figures.
47
26- Liberia* (…)
27- Libya* (100% in Table B.11; not listed in Table B.11b)
28- Madagascar 0 (3.7%)
The URL works. The document is in French; the translated title is Madagascar
Demographic Health Survey 2008-2009 (Institut National de la Statistique & ICF
Macro, 2010). Once again, the statistic provided by the ILO excludes the population
below the age of 15 and above the age of 49. More than 48% of Madagascar’s
population did not fall into this age demographic as of 2014 estimates (CIA, 2014a).
Although the ILO metadata specifies that the approximate coverage rate only
includes ages 15-49, the statistic is extremely misleading. Unless an individual
interested in the statistics were to look at the metadata for every source, they would
not realise that the 3.7% coverage rate for Madagascar and a number of other
countries only include a very limited age group. It is not useful to have a measure
calculating coverage rates for a number of countries if they cannot be compared and
at times exclude more than 50% of the population.
48
31- Malawi* (…)
29- Mali 0 (1.9%)
The source document is in French, and the URL provided by the ILO works. The
1.9% coverage estimate provided in Table B.11b originates from Figure 4 in the
source document, for which the English translation is The evolution of mutual health
coverage, 2004-2008 (Doumbia, 2010:36). This figure seems only to be referring to
mutual health coverage while excluding other forms of health insurance, such as
49
government-sponsored health insurance. Once again, the coverage figure provided
is extremely misleading. It is unclear what the actual coverage rate for Mali is based
on this information.
33- Mauritania 0 (6.0%)
The source is in French (the translated title is Introducing health insurance in
Mauritania and the role of the CNAM) (Ould Mohamed El Mocta, n.d.). The URL
provided by the ILO works. It is unclear where the ILO obtained its 6% coverage
estimate, because Ould Mohamed El Mocta (n.d.) states that approximately 8% of
the Mauritanian population was covered by mandatory health insurance as of 2010.
This source is not academic and does not contain information on how the 8%
estimate was obtained.
34- Mauritius 0 (100%)
The URL provided in the metadata works. The source of the ILO’s estimate
(Mauritius case study: Local ownership of the MDGs) does not provide any
quantitative coverage estimates; the ILO says that Mauritius has 100% healthcare
coverage based on the following quotation: “Mauritius…provides free and
compulsory primary education and universal health care” (United Nations
Development Programme, Bureau for Resources and Strategic Partnerships &
MDGs Unit, n.d.:1). This document contains no bibliography and is not academic—
it is a summary of a UNDP case study.
30- Morocco 0 (42.3%)
The ILO measure claims that Morocco has 42.3% health coverage based on two
sources. The URLs work for both sources. For the first source, the metadata states
that the percentage can be found on page 35; however, there is no such statistic
provided on page 35, which instead seems to indicate that approximately half of
Morocco’s population has medical coverage: “Around 50 per cent of the population
will have some sort of medical cover from now on” (ILO, 2008a:35). A Control-F
search of the document also yielded 0 results for the number 42.3.
The second source is in French (the translated title is Symposium: The future of social
protection in Morocco) (Schmitt-Diabaté, 2008). The paper summarises a
50
symposium on social protection in Morocco. Nothing in the second source gives the
42.3% coverage rate, either. The quote the ILO has taken from this paper states that
AMO, which is a contributory health scheme for private sector employees, covers
just under 30% of the Moroccan population. For those living in poverty, which the
paper estimates as 6 million Moroccans, there is another health scheme called
RAMED, or the Medical Assistance Plan for the Economically Destitute. The issue
with using this as an estimate for coverage is that the programme had not been
implemented as of 2008, and the ILO states that its estimate is for 2007. According
to the figures presented in this source, RAMED would cover approximately 19% of
Morocco’s citizens, for a combined total of around 49% population coverage
between AMO and RAMED. This figure still differs from the ILO’s 42.3% estimate.
32- Mozambique* 0 (4.0% in Table B.11; not listed in Table B.11b)
35- Namibia 0 (28.0%)
The URL works. The figure for Namibia is taken from the Demographic and Health
Survey 2006-07 (Ministry of Health and Social Services & Macro International Inc.,
2008). Again, the 28.0% coverage statistic leaves out the population under the age
of 15 and above the age of 49. For Namibia, including only the ages of 15-49
excludes more than 41% of the population (CIA, 2014b).
51
52
36- Niger 0 (3.1%)
In French. The translated title is Feasibility study of mutual health insurance in
Niger. The metadata did not provide a link, and I was unable to find the original
source after conducting a search of both Google and the social-protection.org
website.
53
37- Nigeria 0 (2.2%)
The URL works. Although the coverage statistic the ILO provides is found in the
source document, it only includes the age demographic of 15-49, which neglects to
take into account more than half of Nigeria’s young population (CIA, 2014c).
54
38- Reunion* 0 (listed in B.11b but not B.11) (…)
39- Rwanda 0 (91.0%)
The URL provided in the metadata leads to Rwanda’s Ministry of Health website,
not the actual source document. Additionally, the citation does not include the title
of the source, a publication year, or a page number for where the statistic can be
found. I deducted through a search of the Ministry of Health website that the
document the metadata is referring to is The third integrated household living
conditions survey (EICV3): Main indicators report (National Institute of Statistics
55
of Rwanda & Oxford Policy Management, 2012), as the metadata mentions this
survey in a quote it uses in its citation for Rwanda. It is unclear why the ILO chose
to list 91% as its coverage estimate, as this contradicts the metadata. The metadata
states the following:
Participation in Mutuelles de Santé [mutual health insurance] attained 91%.
However according to the latest household survey Enquête Intégrale sur les
conditions de vie des ménages 2010-2011, 69 per cent of the population is covered
by health insurance in 2011”
If the 91% figure is outdated, then the ILO should have used the more recent and
accurate statistic from 2011 in the source document, which can be found in Table
4.2.5: Percentage of the population with health insurance. This table estimates that
approximately 68.8% of the Rwandan population is covered by health insurance,
65.3% of which is composed of mutual health insurance (National Institute of
Statistics of Rwanda & Oxford Policy Management, 2012:87-88).
56
40- Sao Tome and Principe 0 (2.1%)
The URL provided by the ILO works. The document is in Portuguese (the translated
title is Sao Tome and Principe Demographic and Health Survey 2008-2009)
(Instituto Nacional de Estatística, Ministério da Saúde & ICF Macro Inc., 2010).
Although the estimate provided by the ILO is present in the source, it again only
includes ages 15-49, which leaves out at least half of the population (CIA, 2014d).
57
58
41- Senegal 1 (20.1%)
The source document is in French. Table B.11b references two sources for Senegal’s
estimate, but both of the URLs provided in the metadata do not work. I found an
alternative link for the second source but was unable to locate the first. The translated
title of the second source is National strategy for the expansion of risk and disease
coverage in Senegal (Ministère de la Santé et de la Prévention, n.d.). The coverage
rate includes non-contributory schemes (state officials, Senegalese over 60-years-
old, students), voluntary contributory schemes (mutual health members, private
insurance subscribers), and mandatory contributory schemes (private sector
employees). The statistic cited by the ILO can be found in the document.
59
42- Seychelles* 0 (90% in Table B.11; not listed in Table B.11b)
43- Sierra Leone 0 (0.0%)
The URL works. This source states that “Health insurance is largely unknown in
Sierra Leone; almost no one is covered by a health insurance scheme” (Statistics
Sierra Leone & ICF Macro, 2009:45). However, the ILO came up with the figure of
0% coverage based on this quote. “Almost no one” having health insurance does not
literally mean that no one has health insurance. One could say that “almost no one”
has health insurance if 1% or even 5% of individuals have health insurance. The
ILO’s interpretation is problematic.
44- Somalia* 0 (20% in Table B.11; not listed in Table B11.b)
45- South Africa 0 (100%)
The URL provided by the ILO does not work. The 100% coverage estimate for South
Africa comes from a qualitative source: a quote from a South African government
official, Andrew Donaldson, at a health conference in Portland, Oregon. Not only is
it problematic to turn qualitative data into quantitative data, Donaldson never
60
actually stated that South Africa has 100% healthcare coverage. He said that: “Most
South Africans have universal access to primary healthcare and hospital services that
is free at the point of service...for most users of the public system, services are free
[emphasis mine]” (The National Bureau of Asian Research, 2010) (The National
Bureau of Asian Research, 2010).
46- South Sudan* (…)
47- Sudan 0 (29.7%)
The URL provided by the ILO does not work, but I found another that links to a PDF
copy of the document. The source is a report published by the Sudanese government,
called Statistical year book for the year 2009 (Central Bureau of Statistics &
Ministry of the Cabinet, 2009). The ILO measure lists Sudan as having a coverage
rate of 29.7%, which the metadata claims comes from Table 22 on page 266. There
is nothing on page 266 that lists this statistic for anything. Furthermore, there seems
to be no aggregate coverage rate listed at all. The coverage rates provided are
divided into seemingly random categories, such as the coverage rate for shepherds
and “Martyrs Families”. A Control-F search confirmed that nothing in this
document states that 29.7% of Sudanese have healthcare coverage. Moreover, even
if the listed statistic were there, the source seems to be of terrible quality. There are
a number of errors immediately obvious to anyone casually reading the document.
As an example, at the bottom of Table 22, it says, “The data don’t inclued Khartoum
State [sic]”. Parts of the document are in English and other parts in Arabic, even on
the same table, for no discernible reason, and sometimes without a translation. It is
also unclear how these figures were determined. This source does not appear to be
reliable.
48- Swaziland 0 (6.2%)
The URL works. As with the other countries for which figures are derived from a
Demographic and Health Survey, the statistic provided by the ILO excludes
Swaziland’s population below the age of 15 and above the age of 49 (Central
Statistical Office & Macro International Inc., 2008). More than 45% of Swaziland’s
population is excluded from this age range (CIA, 2014e). The ILO appears to have
added the percentage of women with health coverage and the percentage of men with
coverage and divided by two to obtain the 6.2% figure.
61
62
49- Tanzania 0 (13.0%)
The URL works. The ILO cites a PowerPoint presentation from the Second
Conference of the African Health Economics and Policy Association that seems to
have been showcased by the Ifakara Health Institute. The 13% figure that the
PowerPoint presents seems to refer to LHC. The statistic is actually from a secondary
source, as Slide 4 states that the figure was obtained from an author with the surname
“Humba” (Kuwawenaruwa & Ifakara Institute, 2011). Through a Google search, I
discovered that the author of the original source is most likely Emmanuel Humba,
who was at one point in time the Director-General of the National Health Insurance
Fund for Tanzania. I was unable to verify the statistic in the primary source because
63
the PowerPoint only contains the year and surname of the author; it does not have
a bibliography with additional information that would allow me to locate the
original source.
50- Togo 0 (4.0%)
The URL provided does not work. The metadata says that it obtained the 4%
coverage figure from the ILO’s website, which states the following: “Mutual health
organizations cover less than 4 per cent of the [Togolese] population” (ILO, 2014a).
The webpage says nothing about other types of insurance that could contribute to a
higher rate of coverage, such as private or public insurance. Therefore, the 4% figure
provided by the ILO seems only to include coverage by mutual health organisations.
51- Tunisia 1 (80.0%)
The source is in French. The translated title is Tunisia: Health sector study. The link
provided in the metadata leads to a page that requires payment to access. I found an
alternative link that provides the source for free. The document states that “Over
80% of the Tunisian population has access to health care as part of a health insurance
plan or that of a medical assistance programme” [translated] (Sameh et al., 2006); it
is unclear whether the figure of 80% reflects legal access or effective access.
52- Uganda 0 (2%)
The URL provided does not work, and I was unable to locate the source after
conducting an extensive Internet search.
53- Zambia 0 (8.4%)
The URL works. The estimate provided by the ILO only includes ages 15-49
(Central Statistical Office et al., 2009). More than 52% of Zambia’s population does
not fall into this age demographic (CIA, 2014f).
54- Zimbabwe 1 (1.0%)
The URL provided by the ILO does not work. I found another URL for the source.
The source document is called Zimbabwe health system assessment 2010 (Osika et
64
al., 2010:16). The source is of acceptable quality and states that “health insurance
covers less than 1 percent of the population”.
65
Detailed country metadata reference list
African Development Bank & African Development Fund 2010. Annual Report
2010. Abidjan, Côte d'Ivoire: African Development Bank.
Bougrine, P. 2005. L'assurance maladie de la sécurité sociale Mise en place des
régimes de soins de santé: L'expérience de la caisse nationale des assurances sociales
des travailleurs salaries. Regional Conference of the ISSA for Africa: Social security
in the African context. Lusaka, Zambia: International Social Security Association.
Central Bureau of Statistics & Ministry of the Cabinet 2009. Statistical year book
for the year 2009. The Republic of Sudan.
Central Statistical Office & Macro International Inc. 2008. Swaziland Demographic
and Health Survey 2006-07. Demographic and Health Survey Mbabane, Swaziland:
Central Statistical Office, Macro International Inc.
Central Statistical Office, Ministry of Health, Tropical Diseases Research Centre,
University of Zambia & Macro International Inc. 2009. Zambia Demographic and
Health Survey 2007. Calverton, Maryland, USA: Central Statistical Office, Macro
International Inc.
Centre International de Développement et de Recherche 2011. Rapport d'activités
2010 systèmes de santé et prévoyance sociale. Centre International de
Développement et de Recherche.
CIA. 2014a. Madagascar [Online]. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/ma.html
[Accessed July 8 2015].
CIA. 2014b. Namibia [Online]. CIA. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/print_wa.html
[Accessed March 17 2015].
CIA. 2014c. Nigeria [Online]. CIA. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/ni.html
[Accessed March 15 2015].
66
CIA. 2014d. Sao Tome and Principe [Online]. CIA. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/tp.html
[Accessed March 15 2015].
CIA. 2014e. Swaziland [Online]. CIA. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/wz.html
[Accessed April 23 2015].
CIA. 2014f. Zambia [Online]. CIA. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/za.html
[Accessed March 8 2015].
CIA. 2016. Ghana [Online]. CIA. Available:
https://www.cia.gov/library/publications/the-world-factbook/geos/gh.html
[Accessed December 20 2016].
Connor, C., Averbug, D. & Miralles, M. 2010. Angola health system assessment
2010. Bethesda, MD: United States Agency for International Development.
COOPAMI. 2009. About COOPAMI [Online]. COOPAMI. Available:
http://coopami.org/en/home.htm [Accessed September 9 2016].
COOPAMI 2010. Couverture universelle: quel modèle pour la République
démocratique du Congo? Brussels, Belgium: Commission Présidentielle,
COOPAMI
Doumbia, B. 2010. Contribution des mutuelles de santé à l'accès des populations aux
soins de santé de base au Mali: cas de la MIPROMA dans les quartiers de
Magnambougou, Sogoniko et Sokorodji en Commune VI du district de Bamako.
Ghana Statistical Service, Ghana Health Service & ICF Macro 2009. Ghana
Demographic and Health Survey 2008. Demographic and Health Survey Accra,
Ghana: Ghana Statistical Service, Ghana Health Service, ICF Macro.
Hodges, A., Mèdédji, D., Mongbo, J.-J. & O'Brien, C. 2010. United Nations
Children's Fund: Etude sur l'etat des lieux et les perspectives de protection sociale
au Benin UNICEF.
ILO 2008a. Decent work and social protection in six countries: umbrella paper.
Geneva, Switzerland: ILO.
67
ILO 2008b. Paper 1: Social health protection. An ILO strategy towards universal
access to health care. Geneva, Switzerland: ILO.
ILO 2009. ILO considerations on the social health insurance reform project in Egypt.
Geneva, Switzerland: ILO.
ILO 2010a. Developing an integrated national social protection policy. Geneva,
Switzerland: ILO.
ILO 2010b. SPF country brief Cape Verde. Social Protection Floor Initiative. ILO.
ILO 2012. Burkina Faso: revue des dépenses et de la performance de la protection
sociale. Geneva, Switzerland: Organisation internationale du Travail.
ILO. 2014a. Togo [Online]. ILO. Available: http://www.social-
protection.org/gimi/gess/ShowCountryProfile.action?id=447&lang=EN [Accessed
October 10 2016].
ILO 2014b. World Social Protection Report: building economic recovery, inclusive
development and social justice. Geneva, Switzerland: International Labour Office.
Institut National de la Statistique & ICF Macro 2010. Enquête démographique et de
santé Madagascar 2008-2009. Demographic and Health Survey. Antananarivo,
Madagascar: Ministère de l’Économie et de l’Industrie.
Instituto Nacional de Estatística, Ministério da Saúde & ICF Macro Inc. 2010. São
Tomé e Príncipe inquérito demográfico e sanitário 2008-09.
International Social Security Association. 2016. Mandate: excellence in social
security administration [Online]. Available: https://www.issa.int/en_GB/the-
issa/mandate [Accessed October 15 2016].
Kuwawenaruwa, A. J. & Ifakara Institute 2011. Willingness to pay for voluntary
health insurance in Tanzania. Second Conference of the African Health Economics
and Policy Association. Saly, Senegal: Ifakara Health Institute.
Mathauer, I., Doetinchem, O., Kirigia, J. & Carrin, G. 2011. Reaching universal
coverage by means of social health insurance in Lesotho? Results and implications
68
from a financial feasibility assessment. International Social Security Review, 64, 45-
63.
Ministère de la Santé et de la Prévention n.d. Stratégie nationale d’extension de la
couverture du risque maladie des Sénégalais. République du Sénégal, USAID.
Ministry of Health and Social Services & Macro International Inc. 2008.
Namibia: Demographic and Health Survey 2006-07. Demographic and Health
Survey Windhoek, Namibia; Calverton, Maryland: Ministry of Health and Social
Services, Macro International Inc.
National Institute of Statistics of Rwanda & Oxford Policy Management 2012. The
third integrated household living conditions survey (EICV3): main indicators report.
National Institute of Statistics of Rwanda, Oxford Policy Management.
Osika, J., Altman, D., Ekbladh, L., Katz, I., Nguyen, H., Rosenfeld, J., Williamson,
T. & Tapera, S. 2010. Zimbabwe health system assessment 2010. Bethesda,
Maryland: Health Systems 20/20 Project, Abt Associates Inc., USAID.
Ould Mohamed El Mocta, S. n.d. Présentation de l'assurance maladie en Mauritanie
et du rôle de la CNAM.
Republic of Cameroon 2006. Plan strategique pour la promotion et le developpement
des mutuelles de sante au Cameroun 2005-2015. In: Ministry of Public Health (ed.).
Yaoundé, Cameroon.
Sameh, E.-S., Johansen, A., Waters, H., Rahola, A., Gikic, D., Tanaka, M. &
Rahman, M. A. A. 2006. République Tunisienne: Etude du secteur de la santé
Washington, D.C., USA: The World Bank.
Schmitt-Diabaté, V. 2008. Colloque: L'avenir de la protection sociale au Maroc.
L'avenir de la protection sociale au Maroc. Casablanca, Morocco.
Statistics Sierra Leone & ICF Macro 2009. Sierra Leone Demographic and Health
Survey 2008. Demographic and Health Survey Calverton, Maryland: Statistics
Sierra Leone, ICF Macro.
The National Bureau of Asian Research 2010. Universal health coverage:
Discussion highlights from a 2010 Pacific Health Summit workshop. In: Karuna
Luthra & Laura Cava Northrop (eds.) Pacific Health Summit 2010. Portland,
69
Oregon, USA: The National Bureau of Asian Research: Centre for Health and
Aging.
United Nations Development Programme, Bureau for Resources and Strategic
Partnerships & MDGs Unit n.d. Mauritius case study: local ownership of the MDGs.
WHO 2006. Health system profile: Djibouti. Health system profile. Cairo, Egypt:
WHO.
![Universal Health Coverage UHC in Global Health · Universal Health Coverage [ UHC ] in Global Health Ariel Pablos-Méndez, MD, MPH ... Voice and Accountability matters more ! UHC](https://static.documents.pub/doc/80x56/5f0f7c537e708231d4446621/universal-health-coverage-uhc-in-global-health-universal-health-coverage-uhc-.jpg)
