The Health Information Literacy Knowledge Test (HILK):
Construction and results of a pilot study
Leibniz Institute for Psychology Information (ZPID)
Trier, Germany [email protected] [email protected]
Anne-Kathrin Mayer & Julia Holzhäuser
Health information literacy (HIL) comprises a set of abilities needed to recognize a health information need, search and evaluate relevant health information, and to use this information to make appropriate health decisions
relevant for people´s autonomy regarding health decisions
Everyday HIL is usually assessed by self-report measures or measures of basic reading skills and numeracy (e.g., TOFHLA, Parker et al., 1995; REALM, Davis et al., 1993)
So far no validated achievement test exists in the international research literature which is not focused on the academic context
Need for a knowledge test which goes beyond basic literacy and which is appropriate for adults of different age groups with middle to higher levels of education (adequate level of reading skills is assumed)
Importance of a knowledge test for research purposes: measurement of HIL in the general population to identify people with low HIL, determine a need for training programs of HIL and evaluate such interventions
Background
Development of an economic test of knowledge about seeking and evaluating everyday health information of adults based on a skill decomposition
Empirical construction in two steps:
1) Expert study: examination of the correctness of the answers and refinement of item formulations
2) Pilot study: considerable shortening of the test based on psychometric properties of the items and first examination of its validity
Aims
Construction of items was based on a skill decomposition derived from models of information problem solv-
ing as well as general models of information literacy (e.g. IPS, Brand-Gruwel et al., 2009; Big6, Eisenberg
& Berkowitz, 1990)
Skill decomposition (Four skills with two subskills each):
1. Definition of information need (1.1 Define the information problem / 1.2 Identify information needed),
2. Planning the search (2.1 Knowledge of information resources / 2.2 Determine search strategy),
3. Accessing information sources (3.1 Identify type of source / 3.2 Gain access (full text)),
4. Scanning information (4.1 Assessment of relevance and quality / 4.2 Orientation within source)
Questions require participants, e.g., to recognize adequate sources to satisfy a specific information need, or
to evaluate health materials with regard to their relevance or quality
The test focuses on different types of information sources and providers, e.g. books, libraries, and the inter-
net
Multiple choice questions with three answer options and an “I don´t know”-category (number of correct
answers ranging from 0 to 3)
Example items:
Test Construction
Subjects: N = 11 psychologists from the field of information literacy research (6 male, 5 female)
Materials: The experts worked on a preliminary paper-and-pencil test version of the HILK with 57 items
Results:
Four items were dropped because experts did not reach an acceptable consensus about the correct
answers
Experts’ feedback was used to refine several item formulations
Resulted in a revised test version with 53 items
Expert Study
Subjects and Procedure:
N = 138 psychology students, 70.3% Bachelor, 28.3% Master
Age: between 18 and 33 years (M = 22.61, SD = 2.71)
87% female and 13% male
Data was collected in computer laboratories or classrooms at the University of Trier in groups of up to 20 sub-
jects; the sessions took between 90 and 120 minutes
Participants were paid for their participation
Materials:
HILK: revised version with 53 items
Test of domain-specific scholarly information literacy (Leichner et al., 2013; k = 35)
Battery of self-report questionnaires assessing :
1) self-efficacy beliefs related to health-related information: SWE-IB-16 (Behm, 2015; k = 16), eHEALS (Soellner et al., 2014; k = 8)
2) current health status SF-12 (Bullinger & Kirchberger, 1998; k = 12) 3) health related control beliefs: FEGK (Ferring & Filipp, 1989; k = 29) 4) health anxiety: MK-HAI (Bailer & Witthöft, 2006; k = 14)
The HILK could be successfully shortened to a 24-item version based on psychometric properties of the items
(Item-total correlations range from rit = .11 to .38)
Reliability:
The shortened version demonstrates acceptable internal consistency (Cronbach’s Alpha = .71)
Guttman´s Lambda 6 of λ6 = .78 more suitable measure of reliability for heterogeneous constructs
Validity and associations with other measures:
The HILK is moderately correlated with the scholarly information literacy test for psychology (see Table 1 for
correlations)
sign for convergent validity
Significant correlation with a measure of self-efficacy beliefs related to health-related information (SWE-IB-
16), but not with self-efficacy beliefs focused on the internet and electronic resources (eHEALS)
No significant correlations found between achievement in the HILK and health related control beliefs, health
anxiety and the current health status
Significant group difference in HILK test performance between Bachelor (M = .79; SD = .09) and Master stu-
dents (M = .82; SD = .11; t(134) = 1.69; p < .05, one-tailed)
Results
Pilot Study
3rd European Health Literacy Conference, November 17-18, Brussels
References Bailer, J. , Witthöft, M. (2006). Modifizierte Kurzform des Health Anxiety Inventory (MK-HAI) [The health anxiety inventory - German modified version]. In A. Glöckner-Rist (Ed.), ZUMA-Informationssystem. Elektronisches Handbuch sozialwissenschaftlicher Erhebungsinstrumente. Version 10.0. Mannheim: Zentrum
für Umfragen, Methoden und Analysen. Behm, T. (2015). Informationskompetenz und Selbstregulation: Zur Relevanz bereichsspezifischer Selbstwirksamkeitsüberzeugungen [Information competence and self-regulation: The relevance of domain-specific self-efficacy] . In A.-K. Mayer (Ed.) Informationskompetenz im Hochschulkontext – Interdisziplinäre
Forschungsperspektiven (S. 151-162). Lengerich: Pabst Science Publishers. Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers & Education, 53(4), 1207-1217. Bullinger, M., & Kirchberger, I. (1998). Fragebogen zum Gesundheitszustand - Manual [Health survey - Manual]. Göttingen: Hogrefe-Verlag. Davis, T. C., Long, S. W., Jackson, R. H., Mayeaux, E. J., George, R. B., Murphy, P. W., & Crouch, M. A. (1993). Rapid estimate of adult literacy in medicine: a shortened screening instrument. Family medicine, 25(6), 391-395. Eisenberg, M. B., & Berkowitz, R. E. (1990). Information Problem Solving: The Big Six Skills Approach to Library & Information Skills Instruction. Norwood, NJ: Ablex Publishing Corporation. Ferring, D., & Filipp, S. H. (1989). Der Fragebogen zur Erfassung gesundheitsbezogener Kontrollüberzeugungen (FEGK): Kurzbericht [The Health-Related Locus of Control Questionnaire (FEGK)]. Zeitschrift für Klinische Psychologie, Psychopathologie und Psychotherapie, 18, 285-289. Leichner, N., Peter, J., Mayer, A.-K., & Krampen, G. (2013). Assessing information literacy among German psychology students. Reference Services Review, 41(4), 660-674. doi:10.1108/RSR-11-2012-0076 Parker, R. M., Baker, D. W., Williams, M. V., & Nurss, J. R. (1995). The test of functional health literacy in adults. Journal of General Internal Medicine, 10(10), 537-541. Soellner, R., Huber, S., & Reder, M. (2014). The concept of eHealth Literacy and its measurement. Journal of Media Psychology, 26, 29-38.
SWE-IB-16
α = .85
eHEALS
α = .77
ILT
α = .67
FEGK internal
α = .83
FEGK external
α = .81
SF-12 physical
SF-12 mental
MK-HAI
α = .89 HILK .15* -.02 .49** .07 -.08 .06 .17 -.08
Table 1. Correlations between HILK test performance and self-efficacy beliefs related to health-
related information (SWE-IB-16 and eHEALS), information literacy test (ILT), health related control be-
liefs (FEGK), current health status (SF-12) and health anxiety (MK-HAI).
* p < .05; ** p < .01 (one-tailed)
Book 1 Book 2 Book 3
Additional materials provided in a supplement Questions
The 24-item version of the HILK is an economic test of knowledge about seeking and evaluating everyday
health information which may be used for research purposes
More evidence is needed before applying it in research:
Studies needed with people from different age groups, fields of studies, and without academic background
Stronger indicators of validity needed, e.g. search tasks or measures of the general cognitive level
Conclusion
8. Look at the book covers on page 3 in the supplement. Which book contains
or which books contain probably the most balanced information (naming pros
and cons) about the topic “health outcomes of sport in older age”?
True Not true
Book 1
Book 2
Book 3
I don´t know
22. Please look at the list of contents on page 5 in the supplement. You would
like to find other publications about the treatment options for lung cancer as
easily as possible. How do you proceed?
True Not true
I read the chapter „Therapeutic approaches for lung cancer” and look up the cited works in the section “References”.
I go systematically through the section “References“ and look for all the sources which relate to treatment options accord-ing to the title.
I look for “treatment options“ in the index.
I don´t know
50. You want to find out if a website contains reliable information. How do you
proceed?
True Not true
I look for the legal notice for finding out which person or or-ganization is responsible for the content of the webpage.
I pay attention if the website is clearly designed and uses an appealing design.
I take care if the website contains advertisements – if yes, the information is useless.
I don´t know
11.2.2 The DNA structure .................................................................................................................. 13
11.2.3 The role of telomeres ............................................................................................................. 16
11.2.4 The human immune system ................................................................................................... 19
11.3 Cardiovascular diseases ......................................................................................................... 24
11.3.1 The structure of the cardiovascular system ........................................................................... 24
11.3.2 Hypertension .......................................................................................................................... 26
11.3.3 Embolism and stroke .............................................................................................................. 29
11.3.4 Coronary ................................................................................................................................. 33
11.4 Lung cancer ............................................................................................................................. 37
11.4.1 Lung anatomy ......................................................................................................................... 37
11.4.2 Risk factors for lung cancer ..................................................................................................... 40
11.4.3 Diagnostic procedures for lung cancer ................................................................................... 42
11.4.4 Therapeutic approaches for lung cancer ................................................................................ 45
11.5 Summary ................................................................................................................................... 47
References ................................................................................................................................ 50
Outlook .................................................................................................................................... 64
Glossary ................................................................................................................................... 17
Index ........................................................................................................................................ 20