Walden UniversityScholarWorks

School of Nursing Publications College of Health Sciences

Summer 6-2016

Development and Testing of an Instrument toMeasure Informatics Knowledge, Skills, andAttitudes Among Entry-Level Nursing StudentsDiane K. Whitehead

Lynne E. BryantNova Southeastern University

Jo Ann KleierNova Southeastern University

Follow this and additional works at: http://scholarworks.waldenu.edu/sn_pubs

Part of the Nursing Commons

This Article is brought to you for free and open access by the College of Health Sciences at ScholarWorks. It has been accepted for inclusion in Schoolof Nursing Publications by an authorized administrator of ScholarWorks. For more information, please contact ScholarWorks@waldenu.edu.

Development and Testing of an Instrument to

Measure Informatics Knowledge, Skills, and

Attitudes among Undergraduate Nursing

Students

Source:

OJNI Volume 20, Summer 2016

by Lynne E. Bryant, EdD, MSN, RN, CNE; Diane K. Whitehead, DNP, EdD, RN, ANEF; Jo

Ann Kleier, PhD, EdD, ARNP, ACNP-BC

Citation

Bryant, L., Whitehead, D., & Kleier, J. (July, 2016). Development and testing of an instrument to

measure informatics knowledge, skills, and attitudes among entry-level nursing students. Online

Journal of Nursing Informatics (OJNI), 20(2), Available at http://www.himss.org/ojni

Abstract

Informatics competencies in nursing education have long been and continue to be a concern.

This article reports on the development and psychometric testing of the Knowledge, Skills, and

Attitudes towards Nursing Informatics (KSANI) Scale to measure these constructs among entry-

level nursing students. A measurement instrument was developed based on the Quality and

Safety Education for Nurses (QSEN) Institute informatics competencies for pre-licensure

students (Cronenwett et al., 2007). Survey data were collected from a convenience sample of 300

undergraduate nursing students attending the 2014 Florida Student Nurses Association’s annual

convention. The data were subjected to Cronbach’s test to estimate the level of reliability as

internal consistency. At 0.90, the alpha for the overall scale exceeded the 0.70 benchmark for

acceptability. The scale items were clustered into the intended three factors – knowledge, skills

and attitudes – as well as into the added factor of opportunities. The instrument was found to be

sound and appropriate for the target population.

Nursing informatics combine the disciplines of nursing science, information science, and

computer science (McGonigle & Mastrian, 2015). Ever since the time of Florence Nightingale,

one of the critical roles of the registered nurse (RN) has been to collect and interpret data to

provide safe and effective patient care. Since the early 1980s, informatics competencies in

nursing education have been discussed in nursing literature (Staggers, Gassert, & Curran, 2001).

In 1992, the American Nurses Association (ANA, 2015) recognized the importance of

technology to nursing practice, identifying nursing informatics as a specialty practice. The 1999

Institute of Medicine (IOM) report calling for a safer health care system identified the use

information technology (IT) as a key factor toward meeting this goal. In 2010, the IOM

published The Future of Nursing, which recommended making technology an essential

component of nursing education. Both the American Association of Colleges of Nursing

(AACN, 2008) and the National League for Nursing (NLN, 2008) emphasized

that knowledge and skills in information management and patient care technology are critical

components in nursing education and accreditation.

Skiba, Connors, and Jeffries (2008) identified a lack of informatics competencies in nursing

education prior to 2008. Since that time, the American Association of Colleges of Nursing

(AACN) and the Robert Wood Johnson Foundation (RWJF) have partnered to support the

Quality and Safety Education for Nurses (QSEN) Initiative (AACN, 2016). One of the

components of the QSEN Initiative was the development of competencies in various areas

including informatics.

This research contributes to the development of a reliable and valid instrument based on the

QSEN competencies to test the informatics knowledge, skills, and attitudes of current RN

students in Florida.

Problem Statement

No succinct research instrument that addresses the informatics knowledge, skills and attitudes of

undergraduate nursing students was found. Several studies have sought to validate instruments

addressing informatics skills level in health professions students, but studies have not addressed

students’ perceptions of integration of informatics into their program of study and opportunities

to use a variety of technologies as opportunities for learning (Choi & Bakken, 2013; Sun &

Falan, 2013; Yoon, Yen, & Bakken, 2009).

Purpose

The purpose of this study was to develop a new instrument, the Knowledge, Skills, and Attitudes

towards Nursing Informatics (KSANI) Scale, and test it to assure it met the psychometric

standards for use among this population. The following research questions were addressed:

Do the items on the KSANI measure have content validity in that they are an adequate

measure of the common underlying dimensions of knowledge, skills and attitude toward

informatics among undergraduate nursing students?

Are the items on the KSANI internally consistent and reliable in that they correlate with

other items on the scale that purport to measure the same construct but do not correlate

with items on the scale that do not measure the same construct?

Significance

Although healthcare professionals seem to express positive attitudes toward computer

information systems (CIS) and their use in evidence-based practice, recent research continues to

identify a poor correlation between these positive attitudes and the use of CIS in practice (Melas,

Zampetakis, Dimopoulou, & Moustakis, 2014). Lavin, Harper, and Barr (2015) cited a current

lack of basic informatics education, coupled with the lack of expectation for graduates to exhibit

informatics competencies, as a continuing and critical issue impacting patient outcomes. Hern,

Key, Goss, and Owens (2015) identified that readiness to become proficient in informatics

competencies varies with the age of the provider. Although nursing programs have worked to

incorporate informatics into their curricula, there is no current research on the knowledge, skills,

and attitudes of undergraduate nursing students.

Literature Review

The 1999 IOM recommendation to establish a safer health care system identified IT as one of the

ways to meet this goal (National Research Council, 2012). The Health Information Technology

for Economic and Clinical Health (HITECH) Act, enacted as part of the American Recovery and

Reinvestment Act of 2009, was signed into law on February 17, 2009. This law promotes the

adoption and meaningful use of health information technology and provides the Department of

Health and Human Services (HealthIT.gov, 2015) the authority to establish programs to improve

health care quality, safety, and efficiency through the promotion of health information

technology.

In 2007, the Technology Informatics Guiding Education Reform (TIGER, 2014) initiative

developed informatics competency recommendations for nursing education, research and

practice. The AACN (2008) and the Accreditation Commission for Education in Nursing

(ACEN, 2013) have added informatics competencies to their standards. Since that time,

informatics outcomes have been added to the curricula of many entry-level nursing programs.

Over the past decade, research has been conducted on incorporating a variety of informatics

competencies into nursing curricula (Gallaghar-Lepak, Scheibel, & Gibson, 2009; Gardner &

Jones, 2012; Hern et al., 2015; Schwirian, 2013; Tellez, 2012). Choi (2012) revealed that

traditional pre-licensure students differed significantly from accelerated baccalaureate in science

of nursing (BSN) and RN-to-BSN students in their overall informatics competency. Choi and

De Martinis (2013) indicated that undergraduate students continued to perceive themselves as

lacking competence in applied computer skills and the clinical informatics role. To date, no

assessment of undergraduate students’ perception of their knowledge, skills, and attitudes has

been published.

Instrument Development

The QSEN Institute developed recommendations for informatics to be included in the education

of students. An instrument was developed based on the QSEN competencies (Cronenwett et al.,

2007). The scale was intended to measure three factors: perception of informatics knowledge,

perception of skills in using informatics, and attitudes toward the use of informatics. In addition,

questions were included to elicit if the participants were offered opportunities in their nursing

programs to apply informatics knowledge.

The 24-item instrument contained three sections, each with varying numbers of questions. The

estimated time to complete the survey was 10 minutes. Section 1, which contained 12 items

dealing with participant abilities related to specific competencies and the stem, I feel confident in

my ability to…, was included with the competency statements in this section. Section 2, which

contained four questions, addressed participant thoughts about various competencies, and the

stem, It is important to me that…, was included. Section 3, which contained eight questions, was

included to determine opportunities to apply informatics during participant educational

programs. The stem, In my nursing program I had opportunities to…, was included with the

competency statements.

Some QSEN competencies were divided into two different statements when there were two

action verbs included. For example, the competency, document and plan patient care in an

electronic health record, was included in the instrument as, I feel confident in my ability to

document patient care in an electronic health record, and a second item, I feel confident in my

ability to plan patient care in an electronic health record. Other items were adapted to in the

attitude about informatics section. The competency, appreciate the necessity for all health

professionals to seek lifelong, continuous learning of information technology skills, was also

adapted into two statements to elicit the participants’ personal attitudes about lifelong learning

for self and also for others. The resulting items were, It is important to me that I seek lifelong,

continuous learning of information technology skills, and, It is important to me that all health

professionals seek lifelong, continuous learning of information technology skills.

A four-point Likert scale was applied in each section. In Section 1, in which the statements

related to the participants’ confidence in their ability to, the scale ranged from 1 (not confident)

to 4 (extremely confident). Section 2 related to the participants’ attitudes, and the scale ranged

from 1 (not important) to 4 (extremely important). In Section 3, participants rated their

opportunities to apply informatics in their nursing programs. The scale in this section ranged

from 1 (no opportunity) to 4 (frequent opportunity).

A demographic survey was included to obtain information about the participants’ program types,

their progress in the programs, use of electronic heath records (EHRs), and education about

informatics. The survey asked participants to indicate the type of program and the semester in

which they were currently enrolled. Questions about the use of EHRs asked about use in a

current or previous place of employment. Information about the inclusion of an informatics

course in a program was requested, as well as whether the course was required and/or if the

participants had taken the course. Participants were also asked to indicate if they had taken an

informatics course outside of their nursing program.

Instrument Testing

The Florida Nursing Students Association (FNSA, 2015), under the auspices of the Florida

Nurses Association, has chapters in 50 undergraduate nursing programs throughout the state.

Each year, the FNSA presents a convention that is popular among its constituents and draws

attendees from across the state. In 2014, the convention was held in Daytona, FL; about 1,200

associate degree and undergraduate BSN nursing students attended. With permission from the

convention organizers and the Institutional Review Board (IRB), anonymous data were collected

from a convenience sample of the attendees. A priori sample size was determined based on the

24 items on the instrument. Using the rule-of-thumb of at least 10 participants per item

(Nunnally & Bernstein, 1994), the sample size was set at a minimum of 240 participants

providing complete data forms.

Two researchers attended the FNSA convention. One of the researchers circulated in the exhibit

area and distributed flyers announcing the opportunity to participate to the attendees. Those who

were interested were directed to the researchers’ exhibit table and were given an information

letter and the survey. To protect their confidentiality, the participants were encouraged to move

away from the table to complete the survey. Once the survey was completed, the participant

placed it in a sealed box and received a numbered ticket for a drawing for a $10 gift card from a

beverage company. At the conclusion of the convention, 20 tickets were drawn by a third party

and gift cards were distributed. The sealed box opened when the researchers returned to the

university campus and the data were entered in SPSS for statistical analysis.

Results

A total of 308 individuals submitted completed surveys. Of these, eight were eliminated due to

incomplete information relating to the type of nursing program in which they were

enrolled. Data analysis was carried out using the remaining 300 participants. Information

regarding the type of nursing educational program attended and the percentage of the program

completed is presented in Table 1.

The majority (n = 186, 62.0%) reported using an EHR in the current place of employment and an

even larger number (n = 200, 66.7%) had used an EHR at a previous place of employment at

some time. However, most participants (n = 237, 79.0%) reported that there was no informatics

course offered in their nursing programs, and of these, 196 (65.3%) had not completed an

informatics course outside of their nursing programs. Of the 61 (20.3%) participants that

reported that there was an informatics course offered, 41 (67.2%) indicated that the course was

required and 34 (55.7%) said they had completed the course. However, these 34 represented only

11.3% of the total sample.

To determine if the scale items were clustering into the intended three factors, a principal

component analysis (PCA) was conducted on the 24 items with orthogonal rotation

(varimax). The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis,

KMO = 0.89 and all KMO values for individual items were > 0.75, which is well above the limit

of 0.5. Bartlett’s test of sphericity, c2 (276) = 3332.35, p < 0.000, indicated that correlations

between items were sufficiently large for PCA. An initial analysis was also run to obtain

eigenvalues for each component in the data.

Four components had eigenvalues over Kaiser’s criterion of 1 and, in combination, explained

57.96% of the variance. The scree plot reached a stable plateau at four factors. Given the large

sample size (N = 300) and the convergence of the scree plot and Kaiser’s criterion on four

components, this is the number of components that were retained in the final analysis. Table 2

shows the factor loadings after rotation. The items that cluster on the same components suggest

that component 1 represents educational opportunity to apply informatics, component 2

represents knowledge of informatics, component 3 represents informatics skills confidence, and

component 4 represents attitude toward informatics.

In this component matrix, each of the variables has a substantial loading on only one factor

except for I feel confident in my ability to use information management tools to monitor

outcomes of care, which loaded nearly equally on the factor labeled knowledge of informatics

(0.474) and on the factor labeled informatics skills confidence (0.468). As this variable has a

substantial loading on two factors, it is considered as complex. After careful consideration, the

team chose to retain this item and attach it to the factor informatics skills confidence, for

conceptual reasons.

The data were subjected to Cronbach’s test to estimate the level of reliability in internal

consistency. At 0.90, the alpha for the overall scale exceeded the 0.70 benchmark for

acceptability. Corrected item-total correlation found that no item correlated below 0.30. These

values suggest that the items on the scale are measuring the same thing. Alpha values for the

individual factors are provided in Table 2.

Discussion

The instrument contains four factors, educational opportunity to apply informatics, knowledge of

informatics, informatics skills confidence, and attitude toward informatics, all of which have a

high internal consistency and reliability. One item loaded equally into both the knowledge of

informatics factor and the informatics skills confidence factor. In the knowledge, skills and

attitudes of informatics competencies, this statement, use information management tools to

monitor outcomes of care, is considered a skill competency (Cronenwett et al., 2007). Therefore,

it was decided to consider the item an informatics skills confidence statement. The instrument

appeared to be appropriate for students from both associate degree and baccalaureate nursing

programs. As a result of the psychometric testing, no changes were made in the individual items.

This instrument could be used to ascertain the knowledge, skills and attitudes about informatics.

Strengths and Limitations

The items included on the instrument were directly derived from QSEN recommendations

(Cronenwett et al., 2007). The data were collected at a convention attended by nursing students

from different programs within Florida; thus the sample consisted of students enrolled in many

programs within the state, increasing the generalizability of the findings. However, there is no

assurance that attendees represented all nursing programs in the state. Students attending a

conference may be different from those students that did not attend the conference. Participants

were asked to self-report and may have over-estimated their knowledge, skills or attitudes toward

informatics.

Conclusions and Recommendations

Several studies have sought to validate instruments addressing informatics skills level in health

professions students, but studies have not addressed students’ perceptions of integration of

informatics into their programs of study and opportunities to use a variety of technologies as

opportunities for learning (Choi & Bakken, 2013; Sun & Falan, 2013; Yoon, Yen, & Bakken,

2009). A research instrument was developed using the QSEN attitudes, knowledge, and skills

recommendations for inclusion in the education of pre-licensure students (Cronenwett et al.,

2007). The data were subjected to tests for psychometric estimates. Validity was supported by

feedback from an expert panel and factor analysis. Reliability as internal consistency was

supported by Cronbach’s alpha meeting benchmark. The instrument was found to be sound and

appropriate for the target population.

The researchers recommend that the KSANI instrument be used to explore the current

informatics knowledge, skills, and attitudes of undergraduate students in other nursing programs

throughout the United States to gain a more thorough picture of the current state of informatics

education in nursing programs.

References

Accreditation Commission for Education in Nursing. (2013). Accreditation manual. Retrieved

from: http://www.acenursing.org/accreditation-manual/

American Association of Colleges of Nursing. (2008). The essentials of baccalaureate education

for professional nursing practice. Washington, DC: Author. Retrieved from

http://www.aacn.nche.edu/publications/order-form/baccalaureate-essentials

American Association of Colleges of Nursing (2016). Quality & Safety Education for Nurses

(QSEN). Retrieved from http://www.aacn.nche.edu/qsen/home

American Nurses Association. (2015). Nursing informatics: Scope and standards of practice

(2nd ed.). Silver Springs: MD: Nursingbooks.org.

Choi, J. (2012). Comparative assessment of informatics competencies in three undergraduate

programs. Online Journal of Nursing Informatics, 16(2). Retrieved from

http://ojni.org/issues/?p=1700

Choi, J., & Bakken, S. (2013). Validation of the self-assessment of nursing informatics

competencies scale among undergraduate and graduate nursing students. Journal of Nursing

Education, 52(5), 275-282. doi: 10.3928/01484834-20130412-0

Choi, J., & De Martinis, J. (2013). Nursing informatics competencies: Assessment of

undergraduate and graduate nursing students. Journal of Clinical Nursing, 22(13-14), 1970-

1976. doi: 10.1111/jocn.12188

Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., Mitchell, P.,…Warren, J. (2007).

Quality and safety education for nurses. Nursing Outlook, 55(3), 122-131. doi:

http://dx.doi.org/10.1016/j.outlook.2007.02.006

Florida Nursing Students Association. (2015). FNSA regions. Retrieved from

http://www.fnsa.net/fnsa-regions.html

Gallagher-Lepack, S., Scheibel, P., & Gibson, C. (2009). Integrating telehealth in nursing

education: Can you hear me now? Online Journal of Nursing Informatics, 13(2), 1-16. Retrieved

from http://ojni.org/13_2/GallagherLepak.pdf

Gardner, C. L., & Jones, S. J. (2012). Utilization of academic electronic medical records in

undergraduate nursing education. Online Journal of Nursing Informatics, 16(2). Retrieved

from http://ojni.org/issues/?p=1702

HealthIT.gov. (2015). Health IT legislation and regulations. Retrieved from

http://healthit.gov/policy-researchers-implementers/health-it-legislation

Hern, M. J., Key, M., Goss, L. K., & Owens, H. (2015). Facilitating adoption of informatics and

meaningful use of electronic health records with nursing faculty. Journal of Nursing Education

and Practice, 5(3), 118-126. doi: 10.5430/jnep.v5n3p118

Institute of Medicine. (1999). To err is human: Building a safer health system. Washington, DC:

National Academy Press. Retrieved from

https://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/1999/To-Err-is-

Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf

Institute of Medicine. (2010). The future of nursing: Leading change, advancing health.

Washington, DC: National Academies Press.

Lavin, M. A., Harper, E., & Barr, N. (2015). Health information technology, patient safety, and

professional nursing care documentation in acute care settings. OJIN: The Online Journal of

Issues in Nursing, 20(2). doi: 10.3912/OJIN.Vol20No02PPT04

Melas, C. D., Zampetakis, L. A., Dinopoulou, A., & Moustakis, V. S. (2014). The significance of

attitudes towards evidence-based practice in information technology use in the health sector: an

empirical investigation. Behaviour & Information Technology, 33(12), 1248-1260. doi:

10.1080/0144929X.2013.774048

McGonigle, D., & Mastrian, K. (2015). Nursing informatics and the foundation of knowledge

(3rd ed.). Burlington, MA: Jones and Bartlett Learning.

National League for Nursing. (2008). Preparing the next generation of nurses to practice in a

technology-rich environment: An informatics agenda. Retrieved from:

http://www.nln.org/docs/default-source/professional-development-programs/preparing-the-next-

generation-of-nurses.pdf?sfvrsn=6

National Research Council. (2012). Health IT and patient safety: Building safer systems for

better care. Washington, DC: National Academies Press. Retrieved from

http://www.modernhealthcare.com/Assets/pdf/CH76254118.PDF

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY:

McGraw-Hill.

Schwirian, P. M. (2013). Informatics and the future of nursing: Harnessing the power of

standardized nursing terminology. Bulletin of the Association for Information Science &

Technology, 39(5), 20-24.

Skiba, D. J., Connors, H. R., & Jeffries, P. R. (2008). Information technologies and the

transformation of nursing education. Nursing Outlook, 56, 235-230. doi:

10.1016/j.outlook.2008.06.012

Staggers, N., Gassert, C. A., & Curran, C. (2001). Informatics competencies for nurses at four

levels of practice. Journal of Nursing Education, 40(7), 303-316.

Sun, X., & Falan, S. (2013). What is your informatics skills level?—the reliability of an

informatics competency measurement tool. Transactions of the International Conference on

Health Information Technology Advancement. Paper 31. Retrieved from

http://scholarworks.wmich.edu/cgi/viewcontent.cgi?article=1040&context=ichita_transactions

Technology Informatics Guiding Education Reform. (2014). Informatics competencies for every

practicing nurse: Recommendations for the TIGER collaborative. Retrieved from

http://www.aacn.nche.edu/education-resources/tiger.pdf

Tellez, M. (2012). Nursing informatics education past, present and future.CIN:

Computers, Informatics, Nursing,30(5), 229-233. doi: 10.1097/NXN.0b013e3182569f42

Yoon, S., Yen, P. Y., & Bakken, S. (2009). Psychometric properties of the self-assessment of

nursing informatics competencies scale. Studies in Health Technology Informatics, 146, 546-550.

Author Bios

Lynne E. Bryant, EdD, MSN, RN, CNE Lynne Bryant is an Associate Professor at Nova Southeastern University in Ft. Lauderdale,

FL. She earned a diploma in nursing from Johns Hopkins Hospital School of Nursing in

Baltimore, MD, a BSN from the University of Maryland in Baltimore, MD, a MSN from the

University of Pennsylvania in Philadelphia, PA, and an EdD from Florida International

University in Miami, FL. Her scholarly interests focus on the issues involving nursing

education.

Jo Ann Kleier, PhD, EdD, ARNP, ACNP-BC Jo Ann Kleier is a Professor and Director of Research at Nova Southeastern University in Ft.

Lauderdale, FL. She earned a BSN from the University of South Florida in Ft. Myers, FL, a

MSN from the University of Miami in Coral Gables, FL, an EdD from Florida International

University in Miami, FL, and a PhD from Barry University in Miami Shores, FL. Her scholarly

interests focus on the health protective behaviors of minorities.

Diane K. Whitehead, DNP, EdD, RN, ANEF Diane Whitehead is a faculty member at Walden University Doctor of Nursing Practice Program.

She was previously a Professor of Nursing at Nova Southeastern University in Ft. Lauderdale,

FL. She received a BSN from Florida State University in Tallahassee, FL, an MSN from the

University of Miami in Coral Gables, FL, an EdD from Florida International University in

Miami, FL, and a DNP from Walden University. Her areas of interest include nursing leadership

and management and community health nursing.