Delft University of Technology The role of organizational...

Delft University of Technology

The role of organizational strategy in the user-centered design of mobile applications

Eshet, Eyal; de Reuver, Mark; Bouwman, Harry

Publication date2017Document VersionAccepted author manuscriptPublished inCommunications of the Association for Information Systems

Citation (APA)Eshet, E., De Reuver, M., & Bouwman, H. (2017). The role of organizational strategy in the user-centereddesign of mobile applications. Communications of the Association for Information Systems, 40(1), 315-331.[14].

Important noteTo cite this publication, please use the final published version (if applicable).Please check the document version above.

CopyrightOther than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consentof the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Takedown policyPlease contact us and provide details if you believe this document breaches copyrights.We will remove access to the work immediately and investigate your claim.

This work is downloaded from Delft University of Technology.For technical reasons the number of authors shown on this cover page is limited to a maximum of 10.

C ommunications of the

A

I

S

ssociation for nformation ystems

Accepted Manuscript


Eyal Eshet

Turku Centre for Computer Science (TUCS)

Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Finland

[email protected]

Mark de Reuver

Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands

Harry Bouwman


Received date: 02/02/2016 Accepted date: 4/12/2016

Please cite this article as: Eshet, Eyal; de Reuver, Mark; Bouwman, Harry: The role of organizational strategy in the user-centered design of mobile applications, Communications of the Association for Information Systems (forthcoming), In Press.

This is a PDF file of an unedited manuscript that has been accepted for publication in the Communications of the Association for Information Systems. We are providing this early version of the manuscript to allow for expedited dissemination to interested readers. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered, which could affect the content. All legal disclaimers that apply to the Communications of the Association for Information Systems pertain. For a definitive version of this work, please check for its appearance online at http://aisel.aisnet.org/cais/.

http://aisel.aisnet.org/cais/


C ommunications of the

A

I

S

ssociation for nformation ystems

Research Article ISSN: 1529-3181

Volume 34 Paper XXX


Eyal Eshet

Turku Centre for Computer Science (TUCS)

Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Finland

[email protected]

Mark de Reuver


Harry Bouwman


Abstract:

Gathering insights on users and their usage context is at the core of the User-Centered Design (UCD) approach. The efficient and effective usage of these insights is strategically important to organizations. With the proliferation of mobile applications, gaining timely and relevant insights is increasingly challenging due to the heterogeneous and dynamic context of use, the abundant availability of information on usage behaviour as well as the intense time constraints imposed by the highly competitive mobile market. This paper develops a research model that considers strategy foci as motivators affecting the efficient and effective usage of insights on users and context. Mediating effects of UCD resources, like time and financial constraints, organizational practices, as well as UCD competences are examined. To test the model a survey was conducted with 100 mobile practitioners and PLS was used to estimate the model. The model shows that focus on an innovation strategy greatly affects data usage on user and their context, directly and indirectly, i.e. mediated by organizational practices and UCD competences. Strategies with a focus on cost have no direct effect on the usage of user insights, but lead to negative impacts on UCD competences and organizational practices.

Keywords: IS development; Organizational strategy; Mobile applications; Interaction design; Survey


Communications of the Association for Information Systems 557

Volume 34 Paper XXX

1 Introduction

User-centered design (UCD) is a well-established approach to the design and development of usable and

useful interactive systems. According to the approach, the design of interactive systems is largely

informed by insights on users and their context of use. Gaining these insights is strategically important to

organizations, as emphasized by research in various disciplines, e.g. Management of Information

Systems (MIS) (Robey and Markus, 1984), Strategic Management (Boland, 1978), Software Engineering

(Schmidt et al. 2001), Human-Computer Interaction (Gould and Lewis, 1985), the emerging fields of User

Experience (UX) design (Hassenzahl and Tractinsky, 2006) and Interaction Design (Sharp et al. 2007) as

well as international standards (ISO 9241-210, 2010). Commonly, organizations use an idiosyncratic

approach to collect and analyze user data. Regardless of the approach, an efficient and effective use of

this data, the dependent variable in our research, is important to the success of the project, and therefore

relevant to the implementation of the organizational strategy. Understanding the influence of different

strategies and work procedures on the UCD practice is also relevant to IS design management.

The new mobile era, labeled the ‘mobile apps era’ (Eshet and Bouwman, 2014), introduces challenges to

the UCD practice of mobile interactive systems (i.e. mobile applications), specifically regarding designers’

effort to gain relevant insights on users and context of use. Firstly, in contrast to the typically single

context of use in stationary computing (e.g. desktop and web-applications), the use of mobile computing

devices (e.g. mobile phone) is characterized by multi-context of use due to the mobility of people and the

devices they carry with them (Henfridsson and Lindgren, 2005). In addition, the affordability of mobile

devices leads to an increased diversification of users, and new classes for situated activities that are

made possible with mobile computing (Johnson, 1998). Consequently, the diversity of users and variety of

use contexts put a strain on the effective collection and analysis of relevant user data. Secondly, rapid

technological developments in recent years, for instance in embedded sensors and wireless technology,

allow the real-time collection and analysis of actual usage behaviour data. The so-called ‘Big Data’ leads

to a continuous flow of data about user behaviour, which is further fuelled by insights from a growing

amount of market research companies. With the excess availability of usage data, the efficient use of such

insights, i.e. understanding the relevant from the ordinary, becomes more difficult. Lastly, application

stores like those provided by Apple and Google have democratized the development and distribution of

mobile applications, which resulted in an increasingly competitive and dynamic mobile market (Bergvall-

558 The role of organizational strategy in the user-centered design of mobile applications

Volume 34 Paper XXX

Kåreborn and Howcroft, 2011). Moreover, Agile approaches are increasingly adopted for the design and

development of mobile applications (Eshet and Bouwman, 2014), though the Agile principle on short

system delivery cycles limits the time to understand users (Seffah et al. 2005). Given the competitive

pressures and resource constraints for UCD practice, the effective and efficient usage of user and context

data becomes more strategically important.

Organizational strategy and its relation to a specific aspect in the UCD practice of mobile applications, i.e.

the efficient and effective usage of data on users and context of use, has not been a topic of research, to

our knowledge. The new mobile era introduces challenges beyond BYOD (bringing your own devices) or

channel strategies, affecting a company into its capillaries. Understanding how an outside-in perspective

on organization strategies (De Wit and Meyer, 2010) is combined with an inside-out strategies focused on

providing resources and building competences in order to support the UCD practice is utmost important.

From an outside-in perspective views on cost focused strategies vis-à-vis strategies focused on innovation

are relevant to consider (Christensen, 1997; Porter, 1985; Tidd et al. 2005). From an inside-out

perspective the availability of UCD resources and capabilities, as extensively discussed in strategic

management and IS literature needs attention (Barney, 1991; Mata et al. 1995; Wernerfelt, 1984).

Research on the relation between IS and resources and capabilities are rather high level and focus in a

generic way on IT assets, processes or IS Capabilities (Wade and Hulland, 2004) rather than on specific

practices. IS research has paid extensive attention to ‘Design Science” (Cross, 2001; Hevner et al. 2004;

Peffers et al. 2007; Sein et al. 2011), though research on the effect of organizational strategy on the UCD

practice has yet to be developed within IS research. Research in Human-Computer Interaction (HCI)

emphasizes the relation between the UCD practice and resources and competences as well as

management level support (e.g. Rosenbaum et al. 2000; Venturi et al. 2006), though a relation between

the organizational strategy and the UCD practice has not been established. Particularly with the

increasing dominance of mobile-based applications, attention to the UCD practice is utmost important, let

alone with the emergence of Big Data.

Seen the identified gap in literature, the objective of this study is to obtain insights into the impact of

organizations’ strategies on the UCD practice. Given the abundance availability of data on users and

context, in this paper we focus on how innovation and cost focused strategies affect a specific aspect of

the UCD practice, that is, the efficiency and effectiveness usage of data on users and context during the


Volume 34 Paper XXX

design and development of mobile applications. Moreover, following HCI-based views on the UCD

practice, the relation between organizations’ strategies and the UCD practice is examined using UCD

resources and competences, as well as organizational practices, as mediating effects. By that, this paper

is unique because we contribute to how time and financial constraints, UCD capabilities as well as

organizational practices mediate between strategy and the usage of data on users and context. We also

contribute to strategic management literature and to “Design Science” approaches in IS by connecting a

strategy focus with the UCD practice of mobile applications. As far as we are aware, a quantitative

research focusing on this relation has not been introduced yet.

We conducted a survey among user experience (UX) designers and interaction designers, software

developers, project managers and owners who are active in the design and development of mobile

applications to collect their perceptions and views. The study was carried out when the use of mobile

applications became a common practice in work and non-work activities of people in many western

countries. Relating organizational strategic aspects to the UCD practice is important for managers. By

doing so we also relate IS research with a focus on strategy to research on the UCD practice in HCI.

The remainder of this paper is organized as follows. In the next section the core concepts of the paper are

defined. In addition, hypotheses are developed based on relevant literature, and a research model is

provided. Section three explains the overall research methodology. Section four presents the study

results, followed by a discussion of the results in section five. The final section concludes the paper, points

out limitations and suggests follow up research.

2 Core Concepts and Hypotheses Development

2.1 Core Concepts

2.1.1 UCD Practice

UCD is a formalized and standardized approach (ISO, 2010) to the design and development of usable and

useful interactive systems. According to the approach, the development of usable and useful interactive

systems requires a focus on users and context of use throughout the design and development process.

Practically, this focus is operationalized through UCD activities, such as interview, observation, usability

testing, and market research (Sharp et al. 2007). Hence, UCD practice refers to the set of activities that

aim at gaining insights on users, context of use, and usage behaviour during the design and development


Volume 34 Paper XXX

of interactive systems. Those who are involved in the UCD practice use different job titles, commonly

interaction designer, user experience (UX) designer, usability engineer, and user researcher (Sharp et al.

2007).

2.1.2 Mobile Application

A mobile application is considered here from a sociotechnical perspective. Socially, mobility is an attribute

of the human rather than the application, and is roughly defined as people’s ability to move between

locations that vary in their spatial, temporal, and social settings (Henfridsson and Lindgren, 2005). People

carry their computing devices with them while moving, and use the devices in varied multi-contextual

settings. Accordingly, mobile devices are considered here as devices that are carried-on by people (Dix et

al. 2000), in particular smartphones and tablet computers. The software program that runs on these

mobile devices is defined here as ‘mobile application’. Technically, a mobile application can take the form

of a native platform application (i.e. mobile app), web-based solution (i.e. HTML5) or any hybrid solution.

2.1.3 Organizational Strategy

Porter (1985) describes strategy in terms of the activities an organization chooses to perform, either

different activities from rivals or similar activities in different ways, in order to withstand competition in the

long run. These activities are not merely support the organization in dealing with the external competition,

but essentially guide the internal operations, action, and behaviour of the organization and its employees

(Mintzberg, 1987). In this study we limit the scope of strategies to two distinct approaches - cost-

leadership strategy and innovation-focused strategy.

2.1.4 Organizational Work Practices

Apart from a strategy, organizations have internal cultures with distinct characteristics that have an impact

on the daily work practices, including on the UCD practice (Iivari, 2006). In this study we consider

organizational culture in terms of established work practices that contribute to creativity, open

collaboration and sharing of ideas within the work environment.

2.1.5 UCD Competences

A competence can be described as a qualification that makes someone fit to perform a particular activity

(Ritter and Gemünden, 2004), in this case, activities that relate to the UCD practice. Hence, UCD


Volume 34 Paper XXX

competences refer to the qualified capabilities and experience of practitioners to deal with the UCD

practice.

2.1.6 UCD Resources

Resources denote everything that is controlled by the organization, such as assets, capabilities,

organizational processes, knowledge and information, which enable the organization to cost-effectively

implement its strategy (Barney, 1991). In this study, we limit UCD resources to the financial and time

resources that are allocated to the UCD practice.

2.1.7 Efficient and effective usage of data on users and context of use

Data on users and context of use refers to the insights gained through the UCD practice. In this study, we

use the dependent variable to evaluate the efficient and effective usage of this data, the extent to which

the data provide understanding on the context of use, as well as the role of the data in the success of the

mobile application.

2.2 Hypotheses Development

2.2.1 Cost-Leadership Strategy

Organizational strategy can be conceptualized from two perspectives, i.e. outside-in and inside-out (De

Wit and Meyer, 2010). One approach that is typical for outside-in views in Strategic Management literature

is Porter’s (1985) approach on strategic positioning. Porter emphasizes two core types of competitive

advantage i.e. cost and differentiation. In combination with the scope of activities, this leads to three

strategies: cost leadership, differentiation or focus. Treacy and Wiersema (1993) refined Porter’s ideas by

focusing on operational excellence, product leadership, and customer intimacy. In practice, a focus on

cost and operational excellence leads to cost awareness and optimization to reduce costs. Hereafter, this

strategy is labeled ‘cost-leadership’.

The cost-leadership strategy implies that also with regard to the UCD practice, cost awareness is key, and

hence, has an impact on the collection and usage of data on user behaviour and user context. Usage of

existing secondary data and low cost alternatives, like using freely available Internet reports on mobile

usage, for gaining insight on user behaviour and usage context would be favoured in a cost leadership

strategy. Since there are no estblished theoretical insights on the relation between strategy and the UCD


Volume 34 Paper XXX

practice, we anticipate a direct and positive impact of cost-leadership strategy on the efficient and

effective usage of data on users and context of use.

Hypothesis 1: Cost-leadership strategy is positively related to the efficient and effective usage of

data on users and context of use.

In response to outside-in models, Barney (1991) developed the resource-based view on strategy, focusing

on how rare resources and capabilities give firms a competitive advantage. If resources and capabilities

are rare, hard to imitate, and there are limited alternatives this will contribute to reinforcing a company’s

strategy. Hence, resources and capabilities play an important role in dealing with the critical contingencies

that firms are facing. The competences and experience of UCD practitioners may be part of these critical

resources and capabilities.

In IS literature, research into resources and capabilities is quite extensive, though on a high-level. In this

study, we focus on resource and capabilities that are specifically relevant for the UCD practice. Insights on

the work of practitioners, who are involved with the UCD practice, show that their work is affected by

resource constraints, in terms of time and budget (Monahan et al. 2008; Rosenbaum et al. 2000;

Vredenburg et al. 2002), the integration of design practice techniques within an organization (Bygstad et

al. 2008; Gulliksen et al. 2006), organizational culture (Iivari, 2006), organizational work practices in terms

of internal communication (Rosenbaum et al. 2000; Venturi et al. 2006), and management support

(Gulliksen et al. 2006; Rosenbaum et al. 2000; Venturi et al. 2006). In addition, the competences of

designers, i.e. skill acquisition and the degree of expertise, are an important element to consider (Hertzum

and Jacobsen, 2001; Gulliksen et al. 2006; Suwa and Tversky, 2001).

In this study we consider the concepts of organizational work practices, UCD competences, and UCD

resources as defined in the section 2.1. These concepts are mediating the relation between organizational

strategies and the efficient and effective usage of data on users and context of use. Stimulating creative-

and collaboration- focused work practices require stimulation of competences as well as significant time

and budget resources. Existing HCI studies and literature mainly highlight the challenges in promoting and

implementing UCD activities within an organization (Gulliksen et al. 2004; Rosenbaum et al. 2000; Venturi

et al. 2006) and provide guidelines for dealing with the institutionalization of the UCD practice (Mayhew,


Volume 34 Paper XXX

1999; Schaffer, 2004). Since there is no existing literature in this area, hypothesis formulation is based on

generic insights from innovation literature relating to business strategy.

With regard to organizational work practices, we anticipate the practices to be more regulated with a cost-

leadership strategy, leaving less room for creativity, open collaboration, and sharing of ideas.

Hypothesis 1a: Cost-leadership strategy is negatively related to organizational work practices.

Similarly, we anticipate a negative impact of cost-leadership strategy on UCD resources and UCD

competences.

Hypothesis 1b: Cost-leadership strategy is negatively related to UCD resources.

Hypothesis 1c: Cost-leadership strategy is negatively related to UCD competences.

2.2.2 Innovation-Focused Strategy

Similarly to Porter’s differentiation strategy and Treacy and Wiersema’s product leadership view, emerging

and transformational strategies (Mintzberg et al. 2009) with a focus on the impact of disruptive or

incremental innovation (Christensen 1997; Tidd et al. 2005), adopt an outside-in approach. What these

strategic approaches have in common is how companies and organizations respond to changes in their

external environment, like technology innovation, changes in consumer demand or competitor behaviour.

These strategic approaches are labeled in this study ‘innovation-focused’ strategy. The changes in

consumer demand and behaviour, as is in the case with mobile applications, may affect an organization

and require a response from top-management, such as by offering new products and services.

In innovation-focused strategy, making use of UCD activities requires in depth engagement with users,

such as by conducting contextual interviews, participatory observation, focus groups and analyses of data

collected via device-embedded sensors. In principal these are time and resource consuming activities and

therefore less effective and efficient. Therefore, we anticipate a direct and negative impact of innovation-

focused strategy on the efficient and effective usage of data on users and context of use:

Hypothesis 2: Innovation-focused strategy is negatively related to the efficient and effective

usage of data on users and context of use.


Volume 34 Paper XXX

With regard to the mediating concepts of organizational work practices, UCD competences, and UCD

resources, we anticipate an innovation-focused strategy to leave more room for creativity, open

collaboration and sharing of ideas as well as a positive impact on UCD resources and UCD competences.

Hypothesis 2a: Innovation-focused strategy is positively related to organizational work practices.

Hypothesis 2b: Innovation-focused strategy is positively related to UCD resources.

Hypothesis 2c: Innovation-focused strategy is positively related to UCD competences.

2.2.3 UCD Resources

Applying UCD activities to gain understanding on users and the context of use is much dependent on the

availability of financial and time resources. Several studies observed that budget and time constraints

have a significant influence on the UCD practice (e.g. Monahan et al. 2008; Rosenbaum et al. 2000;

Vredenburg et al. 2002). These resources may have a determinant role on the approach that practitioners

take during the UCD practice, for instance how much effort, if any, is put into user and context studies and

how evaluation is carried out. Thus, we anticipate the availability of UCD resources to have a direct and

positive impact on the efficient and effective usage of data on users and context of use.

Hypothesis 3: UCD resources are positively related to the efficient and effective usage of data

on users and context of use.

On a more general level it can be assumed that UCD resources also impact UCD competences and

organizational work practices. More budget and time also implies that there will be more opportunities to

develop UCD competences and more attention to new ideas and to feedback, as well as joint teamwork.

Therefore, we anticipate a direct and positive impact of UCD resources on the mediating roles of

organizational work practices and UCD competences.

Hypothesis 3a: UCD resources are positively related to organizational work practices.

Hypothesis 3b: UCD resources are positively related to UCD competences.

2.2.4 Organizational work practices

Prior studies observed that internal communications within the organization (Rosenbaum et al. 2000;

Venturi et al. 2006), management support (Gulliksen et al. 2004; Rosenbaum et al. 2000; Venturi et al.

2006), and involvement of cross-functional teams (Rosenbaum et al. 2000) are key success factors for he


Volume 34 Paper XXX

UCD practice in organizations. Iivari (2006), examining the role of organizations in facilitating user

involvement, observed that organizational culture have influence on UCD practice. Hence, we anticipate a

direct and positive impact of organizational work practices on the efficient and effective usage of data on

users and context of use.

Hypothesis 4: Organizational work practices are positively related to the efficient and effective

usage of data on users and context of use.

2.2.5 UCD Competences

Last, the competences of practitioners who are involved in the UCD practice are relevant. Prior studies

have observed that the knowledge and experience of UCD practitioners largely affect the outcome of

usability evaluation (Hertzum and Jacobsen, 2001), and the approach and implementation of the UCD

practice (Gulliksen et al. 2006). Suwa and Tversky (2001) observed the superiority of experienced

designers over novices in generating new ideas from external representations, such as sketches.

However, merely having a competence is not enough; the competence should be put into an efficient and

effective use that generates value to the organization (Ritter and Gemünden, 2004). Highly competent and

experienced practitioners, besides being more aware to the importance of UCD practice in bringing value

to the organization, are better equipped to create such value. Experienced practitioners are more informed

about the overall importance of and the different possibilities to understand users and the context of use.

Moreover, experienced practitioners are more aware of the need for financial and time resources to the

UCD practice. Hence, we anticipate a direct and positive impact of UCD competences on the efficient and

effective usage of data on users and context of use.

Hypothesis 5: UCD competences are positively related to the efficient and effective usage of

data on users and context of use.

2.3 Research Model

Based on the hypotheses formulated above, Figure 1 presents the research model in this study. Next, we

explain how the data was collected and how the model was tested.


Volume 34 Paper XXX

Figure 1. The research model

3 Methodology

To test our research model, a survey method with the PLS-SEM technique was selected. Surveying by

means of a questionnaire is commonly used for testing hypotheses (Johnson and Turner, 2003; Lazar et

al. 2010). SEM is especially useful to test models that include mediation, and makes it possible to test the

structural and measurement parts of a model at the same time.

3.1 Sample

Data was collected by means of an online questionnaire among practitioners who are involved in the

design and development of mobile applications. This was emphasized both in the questionnaire invitation

and in the introduction to the questionnaire. Acknowledging the fact that the UCD practice takes place in a

complex organizational environment that involves multiple stakeholders with distinct backgrounds and

worldviews (Suchman, 2002), different expectations (Krippendorf, 2006), and different criteria for project

success evaluation (Baxter and Sommerville, 2011), we were interested in responses from practitioners in

the following three project roles: (a) UCD specialists (e.g. UX designer, interaction designer, usability

engineer), who implement the UCD practice and make use of its gathered data; (b) project managers, who

are familiar with the organizational strategy and have decision-making power on the allocation of UCD

competences and UCD resources; and (c) software developers, who also make use of the data that is

gathered from the UCD practice. Practitioners in these project roles are both on the management level

(i.e. project managers) as well as on operational level (i.e. UCD specialists and software developers).

Especially in small firms, operational-level practitioners are commonly also involved in strategy making

and decisions. Therefore, the broad range of project roles justifies that our respondents are

knowledgeable both on the UCD practice and on organizational strategy. However, the parametric size of

these project roles’ populations is difficult to estimate, as there are no relevant databases. In particular

those involved in the UCD practice have numerous job titles and educational backgrounds. Thus, data


Volume 34 Paper XXX

was collected based on non-probabilistic sampling, which is considered a valid and common practice

(Lazar et al. 2010) in cases where it is not possible to apply a strict random sampling.

Potential respondents were contacted through multiple channels, particularly individual e-mail invitations

and the mailing lists of professional communities (e.g. HCI, UXPA, IXDA). In addition, a link to the

questionnaire was posted on relevant and active discussion forums (e.g. LinkedIn groups, Google+

communities), and on Twitter, using relevant hash (#) tags. The survey was available online for 4 weeks

during February – March 2013.

We received a total of 100 responses from 20 countries, mainly from practitioners based in Finland (46%),

US (12%), Sweden (9%), Netherlands (5%) and Israel (5%) with other countries representing a smaller

percentage. Most of the respondents work as UX designers (33%), project managers (19%), software

developers (8%) and project owners (7%).

Respondents work primarily in companies (as opposed to freelancers) of different sizes. About 28% of the

companies have 10 or fewer employees, 24% have 11-50 employees, 7% have 51-100 employees, 18%

have 101-1000 employees, and the remaining 23% have more than 1000 employees. The median size of

companies is 46 and the mode is 5. In terms of business sectors, software (36%) and usability/UX

consulting (18%) were the two largest categories, with other sectors, including education,

telecommunications, design, technology research and gaming representing a single-digit percentage. At

face value, the respondents work in environments in which we would expect them to be active – small,

medium-sized and large enterprises mainly in the software business and usability/UX consulting. Although

we cannot claim representativeness, we assume that the respondents represent a common sample for

our population.

While other studies (e.g. Clemmensen et al. 2013; Ji and Yun, 2006) found differences between usability

specialists and software developers, we extensively tested difference between the three groups (UCD

specialists, project managers, and software developers) for the core constructs based on ANOVA, but

were unable to find any significant differences. Therefore, we conclude that the sample is homogeneous

enough to conduct SEM.


Volume 34 Paper XXX

3.2 Measures

The measures for the core concepts in this study are based on established items from existing studies, as

presented in Table 1. Furthermore, the questionnaire was pilot-tested following Dillman’s (2000) three-

stage recommendations. Questions were grouped into four sections: (1) Addressing the context of use in

mobile application design – the perceived importance of contextual aspects and the use, purpose, and

perceived effectiveness of methods to gather data on users and the context of use; (2) Utilizing the

collected data – usage and evaluation of methods for data analysis and informing the design; (3)

Organizational settings – business sector, size, organizational practices, strategy and competitive

environment, and (4) Demographic data – geographical location, experience with UCD practice, and main

role in mobile application projects.

Confirmatory factor analysis, using Warp PLS, showed acceptable levels of convergent and discriminant

validity. Convergent validity was acceptable for all constructs. Factor loadings exceeded -70 and all

average variance extracted (AVE) were above .60 (Fornell and Lacker, 1981). Construct reliability was

acceptable as Composite Reliability is above .80, exceeding the .60 benchmark. Multi-collinearity was not

significant since the average of full collinearity VIF equals 1.099, and full collinearity VIF equals 1.445,

were way below the 3.3 benchmark.

Table 1: Confirmatory Factor Analysis

Discriminant validity is acceptable, as the average squared correlation of any pair of constructs does not

exceed the average of the respective average variance extracted (see Table 2).

Table 2: Interconstruct correlations and Square Root of AVE

4 Results

We trimmed the original research model in Figure 1 by omitting insignificant paths. The final structural

regression model as shown in Error! Reference source not found.Figure 2, presented a good fit

(Tenenhaus GOF equals .432). Overall, the explained variance of evaluation of data usage is moderate

(R2= .42).


Volume 34 Paper XXX

Figure 2. Structural Regression Model

With regard to our hypotheses, we address the implications in the discussion section. Table 3 gives an

overview of the hypotheses: seven are accepted, four rejected, and two hypotheses show an opposite

direction. We will discuss the results in more detail in the next section.

Table 3: Overview of Accepted and Rejected Hypotheses

5 Discussion

In this section, we discuss the results and the hypotheses in greater detail. First of all, it is interesting to

see that the model itself, which relates the organizational context and innovation strategies to the UCD

practice, particularly with regard to mobile applications and how UCD specialists, project managers and

software engineers use data on users and the dynamic context of mobile use, has a high predictive value

and explains the efficiency and effectiveness of the UCD practice to a significant extent. To our

knowledge, this is one of the first attempts to connect the broader stream of research on strategy and

innovation to UCD practice, and also one of the first approaches that go beyond qualitative or descriptive

models with regard to the UCD practices, to develop an explanatory model. However, the model remains

limited in the sense that it is focused on the UCD practice, while the proof of the pudding will be in the

connection of the model with evaluation of the actual usage of mobile applications. In other words, our

study only tested the efficiency and effectiveness of the UCD practice in informing the design of mobile

applications, not their usage. This means that we can only claim to contribute to how the UCD practice

leads to better user informed designs, while taking strategy orientation and how design teams operate

within constraints of an organizational setting into account. Next, we discuss the specific insights.

There is no main effect between a cost-leadership strategy and the efficiency and effectiveness in which

data on users and use context is used in the UCD practice of mobile applications (hypothesis 1). The

effect is mediated by the availability of UCD resources, organizational work practices and UCD

competences (hypotheses 1 a-c). It is striking that only the relation with UCD resources is positive,

contrary to our expectations (hypothesis 1b), while the other two relations are negative (hypotheses 1a

and 1c). This implies that cost focused companies are prepared to invest but don’t positively contribute to

favorable organizational work practice and UCD competences. This finding confirms traditional insights

from strategic management literature regarding the cost focused strategies in which optimization is core.


Volume 34 Paper XXX

In contrast to cost-leadership strategy, there is a relation between an innovation-focus strategy and

efficiency and effectiveness in which data on users and use context are used in the UCD practice of

mobile applications (hypothesis 2). Strikingly, there is no relation between an innovation-focused strategy

and the availability of UCD resources (hypotheses 2b). Apparently it is more important to develop positive

organizational work practices and UCD competences than providing budgets and time for the UCD

practice (hypotheses 2a and 2c). This finding confirms insights from innovation management, in which

specifically the discussion on open innovation and knowledge management illustrates the importance of

having an open exchange of information, insights and knowledge. Facilitating internal communication

within an organization was previously emphasized as a key to the UCD practice (Rosenbaum et al. 2000;

Venturi et al, 2006). The connection between an outside-in strategy focus with an inside-out perspective

with a focus on UCD resources and competences and organizational work practices proves to be fruitful.

We will discuss the relation between the latter in more detail.

Although earlier studies emphasize the significant influence of budget and time-related constraints on the

UCD practice (e.g. Monahan et al. 2008; Rosenbaum et al. 2000; Vredenburg et al. 2002), this does not

affect the UCD practice with regard to the efficient and effective usage of data on users and context

(hypothesis 3). Organizations rely mainly on the practitioners’ UCD competences and organizational work

practices. However it is striking that there is a negative relation between UCD resources (time and budget)

and organizational work practices. This implies that due to temporal and financial constraints UCD

practitioners rely on open communication and collaboration (hypothesis 3a, but then with a reversed

direction). The impact of UCD resources on UCD competences shows a moderate positive contribution

(hypothesis 3b).

Creation and sharing of ideas and knowledge within the organization, in order to streamline the UCD

practice proves to be highly relevant (hypothesis 4). This relation is the strongest. Design teams that are

open and sharing are more likely to make use of all kind of data sources that are relevant to them in a

more effective way.

Our results show that the UCD competences of practitioners are essentially the key determinant of an

effective data usage (hypothesis 5), at least in the context of mobile application design. The hypothesis

supports the idea that competent practitioners are more likely to make efficient and effective use of the

resources, by finding alternative solutions to a design problem and capitalizing on earlier experiences.


Volume 34 Paper XXX

However, the actual qualifications and experience of practitioners with regard to the design and

development of mobile applications were not studied. In future research, a more extensive

operationalization of these two concepts, in relation to a practitioner's specific competences may better

explain their role in the UCD practice.

Our study contributes to strategic management literature by showing how different high-level strategic

orientations affect the specific UCD practice on the operational level. Our tested hypotheses provide a

basis for theorizing on how strategic decisions in organizations affect the day-to-day work practices,

particularly with regard to the UCD practice. Moreover, we contribute to IS literature by showing how the

UCD practice in IS design and development is affected by both strategic orientation as well as the

availability of resources and competences in organizations. Given the complex, multifaceted nature of

organizations and their operation, our study also contributes an interdisciplinary approach by combining

theories from strategic management, IS management and the HCI disciplines.

On a practical level, our findings imply that practitioners, especially those in managerial positions, in

organizations that design and develop mobile applications, should take steps to stimulate organizations

and ensure they have practitioners with relevant skills and experience to produce better user-informed

designs in a timely manner. As we explained before, understanding users and their context of use during

the UCD practice in the mobile apps era requires new competences. In the highly competitive mobile

business market, obtaining such competences may be part of the rare capabilities that give an

organization a competitive advantage.

6 Conclusion, Limitations and Future Research

In this paper, we have shown that there are links between the organizational strategy and the way

practitioners, particularly UCD specialists such as UX and interaction designers, deal with data on user

and context of use when designing mobile applications. The innovation-focused strategy has a direct

impact on the way designers work, i.e. their work practices, and individual UCD competences, but also

with regard to how efficient and effective they use data sources on users and the context of use. However,

when dealing with the dynamic user context and needs, practitioners rely on their competences and

experience from earlier projects, mainly due to project resource limitations.


Volume 34 Paper XXX

This is the first study that relates organizational context and innovation strategies with UCD practices,

particularly with regard to the way in which practitioners make use of data on users and the context of use.

To our knowledge, this is also the first study that tries to develop more explanatory models with regard to

the UCD practice in the HCI discipline. The connection between an outside-in strategy focus with an

inside-out perspective that focuses on UCD resources, UCD competences, and organizational work

practice, proves to be fruitful. It is clear that the latter play an important mediating role.

Due to this rather innovative character, we have tried to establish a theoretical basis in strategic and

innovation management literature, as well as a focus on resource-based views. In doing so, we think that

an important under-researched interdisciplinary domain is addressed. Developing explanatory models,

instead of descriptive and more qualitative models, that take a strategy perspective as a starting point,

and pays attention to moderating resources and capabilities in explaining a specific UCD practice, may

extend our knowledge on the UCD practice and more generally on the design practice. Research focusing

on outcome expectations from the UCD practice, both with regard to constraints and competences, needs

to be extended. The same goes for the way information on users and their context exactly plays a role in

the UCD practice. An efficient usage of the data may be hindered by the quality of the data, a lack of

precision or even by information overload. With regard to further research we see two distinct avenues.

On the one hand, we want to advocate the development of more sound research models and a more

detailed and precise operationalization of core concepts to be tested in large-scale surveys. These

research models should take the actual usage of mobile, or any other type of, applications, into account.

On the other hand, we want to propose more detailed qualitative research into everyday design practice.

More extensive research will provide deeper insights, which again are open to testing by more quantitative

research.

On a practical level, this research highlights the importance of having an open innovative strategy, i.e.

encouraging the contribution of ideas and facilitating the communication of information within an

organization, to achieve a better informed design. Moreover, practitioners with relevant skills and

experience in terms of understanding user needs and context of use can be critical assets, giving

organizations a competitive advantage.

The limitations of this study are related to the data we collected, as well as the model we tested. One of

the main issues with regard to research into those who are involved in UCD practice is that data is


Volume 34 Paper XXX

collected based on a convenience sample and that is highly dependent on the willingness on the part of

practitioners to participate. We put a lot of effort in collecting data by addressing respondents in several

ways. To further research in this domain, it is necessary to involve more UX and interaction designers,

project managers and software developers, not only in the interest of science, but also because research

that is based on more representative samples can help improve their working practices.

With regard to the limitations of the model, we did not consider the socio-spatial context in which the

designed application is being used. For instance, whether the application is used for specific work-related

activities in relatively stable and predictable context of use or for non-work activities in more diverse and

dynamic context of use. Including a factor on the context of use would allow a more thorough analysis on

the tendency of UCD practitioners to collect user and context data, as well as analyzing the interactions

between organizational factors and context of use factors and their influence on UCD practitioners.

Moreover, developing and testing alternative models is necessary and expedient. With this paper, we

wanted to connect strategic and innovation management research to research into UCD practice, and in

this way open new interdisciplinary research venues.


Volume 34 Paper XXX

Acknowledgments

The authors greatly thank the reviewers and the editor for their extremely useful and constructive

comments on earlier drafts of this paper. The authors also wish to thank Balsamiq Studios and Rally

Software for giving away a few licenses of their software to motivate participation in the study. The

research work for this study was funded by TUCS - Turku Centre for Computer Science and by a grant

from the Foundation for Economic Education in Finland (Liikesivistysrahasto).


Volume 34 Paper XXX

References

Barney, J. (1991). Firm Resources and Sustained Competitive Advantage. Journal of Management, 17(1), 99–120.

Baxter, G., & Sommerville, I. (2011). Socio-technical systems: From design methods to systems engineering. Interacting with Computers, 23(1), 4–17. http://doi.org/10.1016/j.intcom.2010.07.003

Bergvall-Kåreborn, B., & Howcroft, D. (2011). Mobile Applications Development on Apple and Google Platforms. Communications of the Association for Information Systems, 29, 565–580.

Boland, R. J., Jr. (1978). The Process and Product of System Design. Management Science, 24(9), 887–898.

Bygstad, B., Ghinea, G., & Brevik, E. (2008). Software development methods and usability: Perspectives from a survey in the software industry in Norway. Interacting with Computers, 20(3), 375–385.

Christensen, C. M. (1997). The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail. Harvard Business Press.

Clemmensen, T., Hertzum, M., Yang, J., & Chen, Y. (2013). Do Usability Professionals Think about User Experience in the Same Way as Users and Developers Do? In P. Kotzé, G. Marsden, G. Lindgaard, J. Wesson, & M. Winckler (Eds.), Human-Computer Interaction – INTERACT 2013 (pp. 461–478). Springer Berlin Heidelberg.

Cross, N. (2001). Designerly Ways of Knowing: Design Discipline Versus Design Science. Design Issues, 17(3), 49–55.

De Wit, B., & Meyer, R. (2010). Strategy Synthesis: Resolving Strategy Paradoxes to Create Competitive Advantage (3 edition). Australia: Cengage Learning.

Dillman, D. A. (2006). Mail and Internet Surveys: The Tailored Design Method -- 2007 Update with New Internet, Visual, and Mixed-Mode Guide (2nd ed.). Wiley.

Dix, A., Rodden, T., Davies, N., Trevor, J., Friday, A., & Palfreyman, K. (2000). Exploiting space and location as a design framework for interactive mobile systems. ACM Trans. Comput.-Hum. Interact., 7(3), 285–321.

Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18(1), 39–50.

Gould, J. D., & Lewis, C. (1985). Designing for usability: key principles and what designers think. Commun. ACM, 28(3), 300–311.

Gulliksen, J., Boivie, I., & Göransson, B. (2006). Usability professionals—current practices and future development. Interacting with Computers, 18(4), 568–600.

Gulliksen, J., Boivie, I., Persson, J., Hektor, A., & Herulf, L. (2004). Making a difference - a survey of the usability profession in Sweden. In Proceedings of the third Nordic conference on Human-computer interaction (NordiCHI’04) (pp. 207–215). ACM Press.

Hassenzahl, M., & Tractinsky, N. (2006). User experience - a research agenda. Behaviour & Information Technology, 25(2), 91–97.

Henfridsson, O., & Lindgren, R. (2005). Multi-contextuality in ubiquitous computing: Investigating the car case through action research. Information and Organization, 15(2), 95–124.

Hertzum, M., & Jacobsen, N. E. (2001). The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods. International Journal of Human-Computer Interaction, 13(4), 421–443.

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design Science in Information Systems Research. MIS Quarterly, 28(1), 75–105.

Iivari, N. (2006). “Representing the User” in software development—a cultural analysis of usability work in the product development context. Interacting with Computers, 18(4), 635–664.


Volume 34 Paper XXX

ISO. (2010). ISO 9241-210:2010 Ergonomics of human-system interaction-Part 210: Human-centred design for interactive systems. Geneva: International Organization for Standardization.

Ji Yong Gu, & Yun Myung Hwan. (2006). Enhancing the Minority Discipline in the IT Industry: A Survey of Usability and User-Centered Design Practice. International Journal of Human-Computer Interaction, 20(2), 117–134.

Johnson, B., & Turner, L. A. (2003). Data collection strategies in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of Mixed Methods in Social & Behavioral Research (pp. 297–319).

Johnson, P. (1998). Usability and Mobility; Interactions on the move. In Proceedings of the First Workshop on Human-Computer Interaction with Mobile Devices.

Krippendorff, K. (2006). The Semantic Turn: A New Foundation for Design. CRC Press.

Lazar, J., Feng, J. H., & Hochheiser, H. (2010). Research Methods in Human-Computer Interaction. John Wiley & Sons.

Mata, F. J., Fuerst, W. L., & Barney, J. B. (1995). Information Technology and Sustained Competitive Advantage: A Resource-Based Analysis. MIS Quarterly, 19(4), 487–505.

Mayhew, D. J. (1999). The Usability Engineering Lifecycle: A Practitioner’s Handbook for User Interface Design. Morgan Kaufmann.

Mintzberg, H. (1987). The strategy concept I: Five Ps for strategy. California Management Review, 30(1), 11–24.

Mintzberg, H., Ahlstrand, B., & Lampel, J. (2009). Strategy Safari: The Complete Guide Through the Wilds of Strategic Management. Pearson Prentice Hall.

Monahan, K., Lahteenmaki, M., McDonald, S., & Cockton, G. (2008). An investigation into the use of field methods in the design and evaluation of interactive systems. In Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction - Volume 1 (pp. 99–108). Swinton, UK: British Computer Society.

Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, 24(3), 45–77.

Porter, M. E. (1985). Competitive Advantage: Creating and Sustaining Superior Performance. Simon and Schuster.

Ritter, T., & Gemünden, H. G. (2004). The impact of a company’s business strategy on its technological competence, network competence and innovation success. Journal of Business Research, 57(5), 548–556.

Robey, D., & Markus, M. L. (1984). Rituals in Information System Design. MIS Quarterly, 8(1), 5–15.

Rönkkö, M., & Peltonen, J. (2012). Software Industry Survey. Aalto University, Software Business Lab. Retrieved from http://www.softwareindustrysurvey.org/

Rosenbaum, S., Rohn, J. A., & Humburg, J. (2000). A toolkit for strategic usability. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI’00) (pp. 337–344). ACM Press.

Schaffer, E. (2004). Institutionalization of usability: a step-by-step guide. Addison-Wesley Professional.

Schmidt, R., Lyytinen, K., & Mark Keil, P. C. (2001). Identifying software project risks: An international Delphi study. Journal of Management Information Systems, 17(4), 5–36.

Seffah, A., Desmarais, M. C., & Metzker, E. (2005). HCI, Usability and Software Engineering Integration: Present and Future. In A. Seffah, J. Gulliksen, & M. C. Desmarais (Eds.), Human-Centered Software Engineering — Integrating Usability in the Software Development Lifecycle (pp. 37–57). Springer Netherlands.

Sein, M., Henfridsson, O., Purao, S., Rossi, M., & Lindgren, R. (2011). Action design research. MIS Quarterly, 35(1), 37–56.


Volume 34 Paper XXX

Seyal, A. H., Awais, M. M., Shamail, S., & Abbas, A. (2004). Determinants of Electronic Commerce in Pakistan: Preliminary Evidence from Small and Medium Enterprises. Electronic Markets, 14(4), 372–387.

Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design: beyond human-computer interaction (2nd edition). Wiley.

Suchman, L. (2002). Located accountabilities in technology production. Scandinavian Journal of Information Systems, 14(2).

Suwa, M., & Tversky, B. (2001). Constructive perception in design. In J. Gero & M. L. Maher (Eds.), Computational and cognitive models of creative design V (pp. 227–239). University of Sydney, Australia: Key Centre of Design Computing and Cognition.

Tidd, J., Bessant, J., & Pavitt, K. (2005). Managing Innovation: Integrating Technological, Market and Organizational Change. John Wiley & Sons.

Treacy, M., & Wiersema, F. (1993). Customer intimacy and other value disciplines. Harvard Business Review, 71(1), 84–93.

Venturi, G., Troost, J., & Jokela, T. (2006). People, Organizations, and Processes: An Inquiry into the Adoption of User-Centered Design in Industry. International Journal of Human-Computer Interaction, 21(2), 219–238.

Vredenburg, K., Mao, J.-Y., Smith, P. W., & Carey, T. (2002). A survey of user-centered design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 471–478). New York, NY, USA: ACM.

Wade, M., & Hulland, J. (2004). Review: The Resource-Based View and Information Systems Research: Review, Extension, and Suggestions for Future Research. MIS Quarterly, 28(1), 107–142.

Wernerfelt, B. (1984). A resource-based view of the firm. Strategic Management Journal, 5(2), 171–180.


Volume 34 Paper XXX

About the Authors

Eyal Eshet is a post-doc researcher at IAMSR, Åbo Akademi University, Turku, Finland. He has several years of work experience in software programming and design in industrial context, which shaped his scope of research interest on the design practice. He is currently finalizing his dissertation on the User-Centered Design of mobile systems.

Mark de Reuver is Assistant Professor at ICT Section, Faculty of Technology Policy and Management Delft University of Technology, The Netherlands. He received his PhD degree in 2009 from Delft University of Technology. Mark has published more than 65 journal and conference articles in the area of mobile service innovation, platform governance, mobile service business model and smart living.

Harry Bouwman is a Finnish Distinguished Professor at IAMSR, Åbo Akademi University, Turku, Finland, and an associate professor at ICT Section, Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands. His research is focused on ICT and organizations; ICT Management; strategy, business models, and enterprise architecture; and on mobile cloud computing and mobile services.

Copyright © 2015 by the Association for Information Systems. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than the Association for Information Systems must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specific permission and/or fee. Request permission to publish from: AIS Administrative Office, P.O. Box 2712 Atlanta, GA, 30301-2712 Attn: Reprints or via e-mail from [email protected].

Date post:	08-Jun-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Delft University of Technology The role of organizational...

Documents