www.homeless.org/culture: Analyzing the Relationship between
Organizational Culture and HMIS Use among Homeless Service Providers
December 2009 By Courtney Cronley, PhD
UNIVERSITY OF TENNESSEE COLLEGE OF SOCIAL WORK OFFICE OF RESEARCH AND PUBLIC SERVICE
2
Acknowledgements
This study was funded by a Doctoral Dissertation Research Grant from the
United States Department of Housing and Urban Development’s Office of University
Partnerships. The author would like to thank the following groups for their participation
in and cooperation with this research: the East Tennessee Coalition to End Chronic
Homelessness; the Michigan Coalition Against Homelessness; Holland/Ottawa County
Continuum of Care (CoC); Out-Wayne Counties CoC; and Pontiac/Oakland County
CoC.
Contact
The author can be reached at [email protected] to answer any questions
regarding the content of this report. A full description of the methodology and results is
available through the dissertation abstracts in the Hodges Library at the University of
Tennessee, www.lib.utk.edu.
3
Table of Contents
Executive Summary…………………………………………………………….……………….4
Research Report
Introduction…………………………………………………………………………........5
Background…………………………………………………………………..…………..5
Research Questions…………………………………………...………………………..7
Design……………………………………………………………………………………. 7
Results…………………………………………………………………………………… 9
HMIS Use………………………………………………………………………... 9
Organizational Culture…………………………………………………………12
HMIS Use and Organizational Culture……………………………………… 15
Limitations………………………...…………………………………………………………….17
Summary and Recommendations.…………………………………………………………. 17
References……………………………………………………………………………………...20
4
Executive Summary
The United States Department of Housing and Urban Development (HUD) has
required federally-funded homeless service providers to participate in their homeless
management information systems (HMIS) since 1999. As of now, though, no one has
examined how and to what extent these technologies are being used. Theory and
research suggest that the technology dissemination is contingent upon the
organizational culture in which new resources are being used. This study represents the
first empirical analysis of HMIS use and explores the cross-level relationship between
staff members’ HMIS use and organizational culture.
Staff members at 26 homeless service providers completed the Organizational
Social Context (OSC) survey. Individual scores were aggregated to determine the
organizational culture of each organization. Data on HMIS use, measured as the
number of times that an individual attempted to log on to the system, were collected
from 142 individuals.
Results suggest that organizational proficiency is related to HMIS use and is
moderated by gender. The rate of log on attempts for male staff members increases in
organizations with higher levels of proficiency. Moreover, organizational culture results
revealed that the sample reported significantly higher levels of organizational
proficiency, rigidity, and resistance, compared to a national sample of children’s mental
health providers. The study concludes with the recommendation that policy makers view
HMIS implementation as an ongoing, cyclical process of interactions among the
organizational social context, the software, and the researchers developing the
technology.
5
Introduction
This study represents the first attempt to measure empirically the use of
homeless management information systems (HMIS). It stems from the Department of
Housing and Urban Development’s (HUD) 1999 mandate requiring all federally funded
homeless service providers to participate in an HMIS (HUD, 2008). To date only one
study has assessed HMIS implementation (Gutierrez & Friedman, 2005). This study
was not empirical, however. It was based on the authors’ observations and self-reports
from staff members at the organizations. Considering the funding that HUD has
allocated for this project and the expected implications of its use, it is critical that we
conduct systematic and objective assessments of the extent to which HMIS are being
used by organizations.
Background
Research demonstrates that use of new technologies, such as information
management systems and electronic referral systems, in human services can
significantly improve service provision and client outcomes (McCoy & Vila, 2002).
Studies show, though, that new technologies frequently are under or mis-used in
organizations (Carrillo, 2005, 2007; O’Looney, 2004; McCoy & Vila; Herie & Martin,
2002). Reasons for this include poorly designed technology, limited technical
competence, and leadership that do not support change (Carrilio, Packard, & Clapp,
2003).
This study considered how frequently homeless service providers were using
their HMIS and what factors were related to use. It relied on three theories to explain
6
technology use in organizations: diffusion of innovations; sociotechnical theory; and
organizational effectiveness. Table 1 describes each of the theories.
Table 1. Theory base
Theory Question
Diffusion of Innovations (Rogers, 2003) How do new ideas spread among people?
Sociotechnical theory of organizational effectiveness (Trist & Bamforth, 1954)
What is the relationship between technology and the social context in which it is used?
Organizational culture theory (Schein, 1992)
Do shared values, beliefs, and expectations develop in the organization’s social context and guide employee behavior?
Both the theories of diffusion of innovation and sociotechnical organizational
effectiveness consider how new ideas are adopted and what makes their adoption
successful. Diffusion of innovations argues that new ideas, such as the use of cell
phones, spread among people through social networks. One person tells another, who
tells another, etc. Sociotechnical theory argues that the use of new technology,
specifically in organizations, relies on a goodness of fit between the social context and
the technology. Leadership and work practices must accommodate a new technology
for it to be used, regardless of its effectiveness.
Organizational culture theory provides a conceptual bridge between diffusion of
innovations and sociotechnical theory. People in organizations behave according to
established values and expectations as well as shared history and leadership. New
ideas move into organizations to the degree that organizational leaders are aware of
and introduce the innovations. Staff members in organizations whose cultures support
these innovations, those that value proficiency and are willing to take risks with new
7
ideas, are most likely to use new technologies. Table 2 identifies the three main
constructs of organizational culture that were measured in this study: rigidity,
proficiency, and resistance.
Table 2. Organizational culture constructs
Construct Definition
Rigidity The degree to which organizations observe set policies and procedures for work processes.
Proficiency The degree to which organizations value staff competency and strive to provide the best possible services to clients.
Resistance The degree to which organizations invite change.
Research Questions
The study sought to answer the following two questions:
1) Does organizational culture influences staff members’ HMIS use?
2) Do individual characteristics (e.g. gender) interact with organizational culture
to influence staff members’ HMIS use?
Design
Four Continua of Care (CoC) in two states, Michigan and Tennessee,
participated in the study. Figure 1 shows the geographic distribution of the data
collection. A total of 26 homeless service providers and 142 staff members were
sampled. Data were collected in two-waves, 1) January – May, 2008, and 2) January –
May, 2009. Two variables were measured: organizational culture and HMIS use.
Organizational culture was assessed as an organizational-level variable, meaning that
each organization had a single score. HMIS use was assessed as an individual-level
variable, meaning that each individual had a different score. Figure 2 shows the
8
Figure 1. Distribution of data collection sites. The blue dots represent participating organizations.
structure of the sample. The analysis involved examining the relationship between
organizational culture scores and staff members’ HMIS scores. Data were collected in
two waves, 1) January – May, 2008, and 2) January – May, 2009. Two variables were
measured: organizational culture and HMIS use. Organizational culture was assessed
as an organizational-level variable, meaning that each organization had a single score.
HMIS use was assessed as an individual-level variable, meaning that each individual
had a different score. Figure 2 shows the structure of the sample. The analysis involved
examining the relationship between organizational culture scores and staff members’
HMIS scores.
The purpose was to determine if an organization’s culture score influenced how
frequently a staff member used an HMIS. Would staff members in organizations that
scored high on organizational proficiency be more likely to use the HMIS than staff
members in organizations scoring low or average? The study also assessed the
9
Nested Sample
CoC 1 CoC 2
CoC 3 CoC 4
8 Organizations4 Organizations
5 Organizations 7 Organizations
34 HMIS Users
44 HMIS Users 51 HMIS Users
13 HMIS Users
Figure 2. The sample consisted of 142 HMIS users, nested within 26 organizations, nested within four CoC. Each organization had an organizational culture score, and each staff member within each organization had an HMIS use score.
variables separately. It examined organizational culture across the sample to see if it
varied substantially among providers. In addition, the study considered how frequently
staff members were attempting to use the HMIS and if this frequency was similar across
organizations.
Results
HMIS – Variability in Use
The median number of times that staff members attempted to log on to the HMIS
was 47 times per year. Results suggest, however, that there is a large amount of variety
in how the organizations surveyed are using their HMIS. The number of times that staff
members attempted to log on to an HMIS ranged from 2 to 719 in one year. Looking at
use by organizations, one can group them into three main categories: 1) regular,
10
proportionate use; 2) irregular use; and 3) irregular, disproportionate use. Select
organizations reflecting these profiles are shown in Figures 3, 4, and 5. The user ID, on
the right, vertical axis, denotes individual staff members, in the organization. The
months are identified on the horizontal axis, beginning with 2 for February. The count,
on the left, vertical axis, denotes the number of times each month the staff member
attempted to log on to an HMIS.
Figure 3. Regular, proportionate use: A large number of staff members licensed to use the HMIS and many of them attempting to log on to the HMIS at least once every month.
11
Figure 4. Irregular use: A small number of staff members licensed to use the HMIS and attempting to log on infrequently and inconsistently.
Figure 5. Irregular, disproportionate use: A single staff member licensed to use the HMIS and attempting to log on inconsistently.
12
There was also variability among the CoC in organizational use of HMIS.
Aggregate organizational use of an HMIS ranged from a median of 33 in CoC 2 to a
median of 337 in CoC 3. Figure 6 shows the total number of times that staff members
attempted to log on to an HMIS in each CoC.
Figure 6. Pie graph showing the total number of times that staff members attempted to log on to the HMIS, by CoC. Results show great disparity in attempts from a low of 817 times in CoC 1 to a high of 6,106 times in CoC 4.
Organizational Culture – Variability across Organizations
Organizational culture results are reported as T-scores derived from a national
sample of children’s mental health providers (n = 100). A T-score of 50 means that an
organization’s culture shows average levels of proficiency, rigidity, and resistance
compared to the national sample. One standard deviation is 10 points higher or lower
than 50. Figure 7 shows that the homeless service providers in this study are markedly
different from the national sample. These organizations report average levels of rigidity
and resistance that are more than a full standard deviation above the mean (M = 60.39
13
60.39
58.11
64.11
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
Figure 7. A graph comparing the sample of homeless service providers (in blue) to a national sample of children’s mental health providers. It shows that the sample is markedly different from the national sample.
and 64.11, respectively) and a proficiency level that is nearly one standard deviation
above the mean (M = 58.11).
Among the homeless service providers, however, there is a great deal of
variability, particularly in proficiency scores, which ranged from a T-score of 36.30 to
71.07. Figure 8 show the two extreme organizational profiles found in the study. The
model organization shows a high proficiency score and low resistance score. The least
constructive organization shows the inverse relationship with a low proficiency score
and high resistance score.
14
50.6061.10
55.10
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
73.11
48.13
64.27
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
Figure 8. Examples of extreme organizational culture profiles. The organization on the left shows a model organization for disseminating innovations. The level of resistance is low, and the proficiency is high. The organization on the right shows a challenging site for innovation. Levels or rigidity and resistance are high while expectations of worker competency, proficiency, are low. Results also showed differences in organizational culture profiles among CoC.
Figure 9 graphs the average T-scores of all organizations in each of the four CoC.
There was an 8.33 point difference between CoC 3, in which organizations reported the
highest average rigidity score (M = 63.86) and CoC 2, where organizations showed the
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
Figure 9. The figure above compares average organizational culture profiles for all four CoC graphed against the normative sample of children’s mental health providers.
15
lowest average rigidity score (M = 55.53). Similarly, there was an 8.54 point difference
in average resistance scores between the highest in CoC 3 (M = 67.88) and the lowest
in CoC 2 (M = 59.34).
HMIS Use and Organizational Culture
Figures 10, 11, and 12 compare select organizations’ HMIS use with their
organizational culture profiles. The blue line indicates the select organization. The dark
green line represents the sample average.
60.56
59.2353.89
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
Figure 10. This organization show regular, proportionate use of the HMIS. Compared to the sample average (green line), the organization reports lower levels of rigidity and resistance (blue line) and a higher level of proficiency.
65.72
57.6953.38
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
Figure 11. The organization above shows irregular use of the HMIS. One staff member never uses the HMIS and other staff members do not access the system for entire months at a time. The organizational profile shows resistance that is almost two standard deviations above the mean.
16
64.11
55.54
69.90
20
30
40
50
60
70
80
Rigidity Proficiency Resistance
T S
co
re
Figure 12. A single staff member uses the HMIS, suggesting disproportionate use, and the use is sporadic and irregular, swinging between a low of less than 10 times a month to a high of nearly 40 times. The corresponding organizational culture profile shows a level of resistance that is nearly two standard deviations above the mean. Results also showed that one component of organizational culture, proficiency,
affected staff members’ use of an HMIS, although this effect was moderated by gender.
Specifically, in organizations with high levels of proficiency, men were more likely to use
an HMIS. Figure 13 shows this interaction between gender and proficiency.
48.13 52.91 57.69 62.47 67.253.25
6.53
9.82
13.10
16.38
Proficiency
Lo
g o
n A
ttem
pts
Female (0)
Male (1)
Figure 13. Men’s use of an HMIS increases as organizational proficiency increases, while women’s use is unaffected. Interestingly, the increase for men doesn’t occur until organizations show markedly higher levels of proficiency compared to the average organization.
17
Limitations
This study shared many of the limitations common to studies of organizations.
Every effort was made, however, to minimize the effects of limitations to the accuracy of
the results and interpretation. Primary limitations include:
Sample – the small, purposive sample limits the results to the organizations and
communities included in the study.
Measurement – the study relied on log on attempts to measure HMIS use. This
measurement may not measure the variety or the substantive ways in which staff
members interact with an HMIS.
Data collection – the study did not collect individual-level data about staff
members’ familiarity with technology. Factors such as education, job title, and
technical training may affect an individual’s use of an HMIS, above and beyond
the organizational culture.
Summary and Recommendations
Results from this study can be summarized in Figure 14. Dissemination of the
HMIS into homeless service providers has been an ongoing, cyclical process moving
from initial efforts by HUD to disseminate the technology to service providers adopting
it, to front-line practitioners who are expected to use the system in daily activities. HUD
designed HMIS to act as real-time data bases that case managers use as they serve
clients. As organizations adopt the technology and staff members use it, this research
shows that not all organizations are using the data base in the real-time. Possible
explanations for this finding may be that organizations are maintaining dual record
keeping systems with paper assessments that they transfer to an HMIS. Alternatively,
18
Figure 14. The diagram shows the dynamic interplay among different factors that contribute to dissemination of innovations into organizations.
they may be allocating HMIS work to a single staff member who receives data from
other staff members and enters everything en masse.
Policy makers and researchers have a responsibility to assess the
implementation and consider ways in which the technology or the methods of
dissemination should be altered to maximize successful use of HMIS as tools of
practice for service providers.
The results of the study point to two major policy area recommendations:
1) Organizational culture influences how organizations are able to adopt and
implement an HMIS.
a. Implement organizational culture change interventions aimed at reducing
resistance and rigidity while emphasizing proficiency.
b. Provide long-term funding that supports technical assistance for the
organizational change.
19
c. Act patiently. Organizational change and full implementation of technology
are long-term, non-linear processes that face numerous setbacks and
challenges.
2) Organizations are using the HMIS differently and often in ways contrary to the
original design.
a. Provide organizational specific training that supports the unique service
environments of each provider.
b. Monitor implementation through more implementation evaluations to
ensure fidelity to the original design.
20
References
Carrilio, T. (2005). Management information systems: Why are they underutilized in the social services? Administration in Social Work, 29(2), 452-462.
Carrilio, T. (2007). Using client information systems in practice settings: Factors affecting social workers’ use of information systems. Journal of Technology in Human Services, 25(4), 41-62.
Carrilio, T. E., Packard, T., & Clapp, J. D. (2003). Nothing in – nothing out: Barriers to the use of performance data in social service programs. Administration in Social Work, 27(4), 61 – 75. Gutierrez, O. & Friedman, D. H. (2005). Managing project expectations in human services information systems implementations: The case of homeless management information systems. International Journal of Project Management, 23. Herie, M. & Martin, G. W. (2002). Knowledge diffusion in social work: A new approach
to bridging the gap. Social Work, 47(1), 85-95. McCoy, H. V. & Vila, C. K. (2002). Tech knowledge: Introducing computers for coordinated care. Health and Social Work, 27(1), 71-74. O’Looney, J. (2005). Social work and the new semantic information revolution.
Administration in Social Work, 29(4), 5-34. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press. Schein, E. (1992). Organizational culture and leadership. San Francisco: Jossey-Bass. Trist, E. L. & Bamforth, K. W. (1951). Some social and psychological consequences of the longwall method of coal-getting. Human Relations, 4, 3-38. The United States Department of Housing and Urban Development. (2007). The Annual
Homeless Assessment Report to Congress. Washington D.C.: Author.
The University of Tennessee College of Social Work 224 Henson Hall Knoxville, TN 37966-3333 865-974-3176 www.csw.utk.edu