Implementing COBIT 5 in Small and Medium Enterprises David Miguel Mendonc ¸a da Silva Thesis to obtain the Master of Science Degree in Information Systems and Computer Engineering Supervisors: Prof. Miguel Leit ˜ ao Bignolas Mira Silva Prof. R´ uben Filipe de Sousa Pereira Examination Committee Chairperson: Prof. Ana Paiva Supervisor: Prof. Miguel Leit ˜ ao Bignolas Mira Silva Member of the Committee: Prof. Henrique O’Neill November 2018
Transcript
Implementing COBIT 5 in Small and Medium Enterprises
David Miguel Mendonca da Silva
Thesis to obtain the Master of Science Degree in
Information Systems and Computer Engineering
Supervisors: Prof. Miguel Leitao Bignolas Mira SilvaProf. Ruben Filipe de Sousa Pereira
Examination Committee
Chairperson: Prof. Ana PaivaSupervisor: Prof. Miguel Leitao Bignolas Mira SilvaMember of the Committee: Prof. Henrique O’Neill
November 2018
Acknowledgments
First and foremost, I would like to begin by thanking my whole family for their encouragement, caring,
and patience over all these years. My family gave me the financial and emotional support that was
essential to overcome this journey, without their effort nothing would be possible.
Furthermore, I would like to express my gratitude to my dissertation supervisors Prof. Miguel Mira da
Silva and Prof. Ruben Pereira for their availability, useful insights, guidance and sharing of knowledge
that allowed me to learn as a researcher and conclude this master thesis.
Last but not least, to all my colleagues and friends who have shared this experience with me or who
have unconditionally supported me over the years. With them I lived the best and worst moments of my
academic life, it was thanks to them that I also grew as a person and became what I am today.
Finally, I would like to thank all the participants that collaborated in this research, making it possible.
Their valuable feedback was essential to the completion and success of this research.
To each and every one of you – Thank you!
Abstract
Information Technology (IT) has become fundamental for most organizations since it is vital to their sus-
tainability, development, and success. This pervasive use led organizations to a critical dependency on
IT. Despite the benefits, it exposes organizations to several risks. Hence, a significant focus on Enter-
prise Governance of IT (EGIT) is required. EGIT involve the implementation of processes, structures
and relational mechanisms to support the business/IT alignment and the creation of business value from
IT investments. In order to support an EGIT implementation, there are broad and complete best prac-
tices frameworks from which the COBIT 5 is a reference. This kind of frameworks are considered highly
complex and require considerable investments and resources which, in general, are extremely scarce in
Small and Medium Enterprises (SMEs). However, no specific guidance is provided to help these organi-
zations. Therefore, the problem addressed in this research is the lack of support for the implementation
of COBIT 5 in SMEs. To solve this problem, this research proposes a solution that identifies the fun-
damental mechanisms to implement effective EGIT in SMEs and then, establishes the correspondence
between the EGIT mechanisms and the COBIT 5 components that support its implementation. The
proposed solution was evaluated according to several methods, including qualitative semi-structured in-
terviews with experts and specific methods to evaluate IT artifacts. Finally, this research work followed
the Design Science Research principles and guidelines.
Keywords
Information Technology; Enterprise Governance of IT; Enterprise Governance of IT Mechanisms; COBIT
5; Small and Medium Enterprises.
iii
Resumo
Tecnologia de Informacao (TI) tornou-se fundamental para a maioria das organizacoes, uma vez que e
vital para a sua sustentabilidade, desenvolvimento e sucesso. Este uso difundido conduziu as organizacoes
a dependencia critica de TI. Apesar dos benefıcios, isto tambem expoe as organizacoes a diversos
riscos. Por isso, e necessario um foco significativo na Enterprise Governance of IT (EGIT). EGIT
envolve a implementacao the processos, estruturas e mecanismos relacionais para suportar o alin-
hamento negocio/TI e a criacao de valor comercial a partir de investimentos em TI. De forma a apoiar
uma implementacao de EGIT, existem frameworks de boas praticas completas e abrangentes das quais
o COBIT 5 e uma referencia. Este tipo de frameworks e considerado altamente complexo e requer
investimentos e recursos consideraveis que, em geral, sao extremamente escassos na Pequenas e
Medias Empresas (PMEs). Portanto, O problema abordado nesta investigacao e a falta de apoio para
a implementacao do COBIT 5 em Pequenas e Medias Empresas. Para resolver este problema, esta
investigacao propoe uma solucao que identifica os mecanismos fundamentais para implementar EGIT
eficaz em Pequenas e Medias Empresas e, em seguida, estabelece a correspondencia entre os mecan-
ismos EGIT e os componentes COBIT 5 que suportam a sua implementacao. A solucao proposta foi
avaliada de acordo com diversos metodos, incluindo entrevistas qualitativas semi-estruturadas com es-
pecialistas e metodos especificos para avaliar artefactos de TI. Por fim, the trabalho de investigacao
seguiu os principios e diretrizes da Design Science Research.
Palavras Chave
Tecnologias de Informacao; Enterprise Governance of IT ; Mecanismos Enterprise Governance of IT ;
[19]. SMEs’ employees tend to be hired for their business skills to ensure core business survival and,
therefore, may be unaware of the potential benefits and costs of IT [67].
In turn, adopting a broad and complete EGIT framework requires capable IT staff with EGIT exper-
tises which normally do not exist in SMEs. Most organizations maintain small internal IT departments,
responsible for IT management, that are focused on short-term solution and operational efficiency [66].
Generally, SMEs lack long-term vision of their business and tend to adopt a more operational than a
strategic view [69], [70]. Thus, these departments tend to search outsourced IT-enabled business solu-
tions [68]. For this reason, SMEs must often be dependent of outsourcing and external consultants for
service and support, including to implement EGIT [19]. Outsourcing can present problems such as the
instability of service providers and a lack of service level agreements, making SME extremely vulnerable.
Additionally, time can also be a problem for these enterprises since the owner and managers are
regularly overloaded with other business priorities [71].
To summarize, the resources limitations and the lack of IT knowledge within the organization will have
a negative impact in the perception and adoption of IT. Undoubtedly, these SMEs’ characteristics make
the implementation of broad and large EGIT frameworks extremely difficult. They are complex and costly
to implement, hence SMEs perceive it as a frightening and unpractical implementation process [28].
Thus, the best approach is to scale-down and adapt the existing frameworks to fit within those particular
SMEs [72].
3.5 Minimum Baseline of EGIT Mechanisms
The complexity of broad best-practice frameworks for EGIT implementation, and especially COBIT 5,
is a problematic issue. It requires the coordination of a large number of components, namely struc-
tures, processes and relational mechanisms. According to De Haes et al. [24], problems with COBIT 5
implementation starts at an early phase when practitioners have to decide what are the processes to
implement and its implementation order.
A possible and suitable solution is to identify a capable minimum baseline of EGIT mechanisms that
could serve as starting point or basis to effectively implement EGIT in organizations [8, 23]. However,
different organizational contexts may imply different EGIT mechanisms [73]. Therefore, it is clear that the
minimum baseline will not be suitable and sufficient to all organizations. The minimum baseline should
be used as a roadmap to implement the most significant EGIT mechanisms in specific organizational
contexts [31]. However, it should be adapted and supplemented with other mechanisms as required by
the organization environment [8].
In the literature, were found several researches aimed to identify minimum baselines to support and
facilitate EGIT implementations in distinct environments. De Haes and Van Grembergen [8] provided a
20
minimum baseline of EGIT mechanisms for Belgian financial services organizations that is regarded as
the necessary set of mechanisms to implement EGIT in this sector. Pereira et al. have also contributed
by identifying minimum baseline of EGIT mechanisms for two different Portuguese sectors, namely
the financial industry [32] and the healthcare industry [31]. Analyzing the differences in the obtained
baselines, it becomes clear that the contingency factors can definitely have a huge impact in EGIT.
Afterwards, Bianchi et al. identified a baseline for implementing EGIT in universities based on in-
depth interviews involving universities of Brazil, Portugal and Netherlands [33]. The resultant baseline
was compared with all the previously identified minimum baselines, namely the baselines for the Por-
tuguese financial and healthcare industries and for the Belgian financial industry, allowing to detect the
similarities and dissimilarities between them.
Bartens et al. applied a slightly different approach, his research was mainly focused on the pro-
cesses, specifically COBIT 5 processes that could be a basis to effectively implement COBIT 5 reducing
its inherent complexity. However, the other enablers defined by COBIT 5 are also extremely relevant and
the authors suggest that further research should consider also the other enablers [23]. This approach
can be useful but we consider that a mixture of mechanisms is essential to the effective implementation
of EGIT, so it should not comprise only the processes mechanisms but also other types of mechanisms.
All these researches followed the same procedure, several experts were interviewed in order to eval-
uate an overarching list of EGIT mechanisms in terms of its effectiveness and ease of implementation.
Subsequently, each of them was asked to elect the 10 most important mechanisms based on the previ-
ous evaluation and their personal experience. Their answers were instrumental in supporting the choice
of the minimum baselines. The only exception was the research developed by Bartens et al. in which
the experts analyzed and evaluated the list of COBIT 5 processes instead of a list of EGIT mechanisms,
as in the other studies.
All the aforementioned authors have stated that further investigations are needed in order to identify
and evaluate other contingencies influencing the EGIT mechanisms. As an example, further investiga-
tions could “address the impact of specific contingencies such industry, geography and size” [8].
Therefore, we consider that a minimum baseline of EGIT mechanisms to effectively govern IT in
SMEs would definitely contribute to facilitate EGIT implementation in this specific context.
3.6 ITIL implementation on SMEs
Initially, we started by searching in the literature for concrete cases of COBIT5 implementation in SMEs
but there was no relevant material. Consequently, we tried to search for implementation of similar
frameworks, such as Information Technology Infrastructure Library (ITIL).
The pervasive use of IT leads to organizations increasingly dependents of IT services to satisfy busi-
21
ness needs and objectives. It is mandatory that IT services achieve their expected function. Therefore,
Information Technology Service Management (ITSM) emerged to define quality rules to guarantee per-
formance and to satisfy customer needs as a result of efficient service management practices during
services life cycle [74].
The most common framework for ITSM is ITIL [75]. ITIL is globally recognized as a reference frame-
work presenting guidance to IT service providers on the provision of IT services in accordance to the
customers’ demands regarding to functionality, quality of service and transparency. Generally, it provides
processes and procedures considered efficient, reliable and adaptable to organizations of all sizes [75].
This standard defines a service life cycle divided in five stages: Service Strategy, Service Design,
Service Transition, Service Operation and Continual Service Improvement. Each stage is influenced
and dependent on the others to receive inputs and provide feedback [75]. However, identification of the
first process to implement is a complex question especially for SMEs. The implementation order is also
one of the problems recognized in ITIL implementations [76], as it is in COBIT 5 implementations.
To address this problem J.A. Calvo-Manzano et al. performed two different surveys [76]. The first sur-
vey, directed to SMEs, had the objective of determining which ITIL processes are used in organizations
and which processes will be implemented in the future.
The top-three processes already used include two processes from Service Operation, the Request
Fulfillment process and the Problem Management process which include incident, request and problem
management, and one process from Continual Service Improvement, the Seven-step improvement pro-
cess. Therefore, analyzing the results, we can perceive that SMEs are mostly focused in maintaining
and improving their critical business operations to guarantee that IT services are delivered effectively
and efficiently.
The top-three process that are planned to implement in the future are Knowledge Management, IT
Service Continuity Management or Supplier Management or Change Evaluation and, finally, Design
Coordination or Information Security Management or Event Management. Thus, the processes to be
implemented are related with Service Design and Service Transition contributing to correctly design new
services and to properly build and deploy services, respectively.
The second survey, directed to experts, intends to know which is the order for implementing the
ITIL processes in SMEs according to their experience. The results showed that the first process to be
implemented is the Incident Management process, the second process is the Service Level Management
or the Service Catalogue Management. Lastly, the third process is Service Asset or Configuration
Management Process. Additionally, the experts interviewed were asked to identify the criteria used to
perform the prioritization of ITIL processes. The most referred criteria were Quick Wins, Strengthen
Service Support, Customer Services and Demands prioritization.
From this results, we can notice that the first ITIL process to implement belongs to the Service
22
Operation stage. Furthermore, one of the most used criteria in the prioritization was quick wins which
allow the business operations to easily perceive the positive impact that ITSM processes would have if
effectively implemented. Other common criteria were also focused in the business operations such as
Strengthen Service Support or Customer Services.
Finally, after analyzing the results of both surveys it is clear that an ITIL implementation in a SME
will typically start with processes from to the Service Operation stage, contributing to the focus of these
enterprises in business operations.
Philipp Schmidtbauer et al. investigated whether ITIL is suitable for SMEs as it is or some adaptations
are necessary [77]. His research is focused on a case study at Nordex, a wind turbine manufacturer
from Germany, to explore the processes and changes when introducing ITIL Service Operation in real-
life SME. In this case, the ITIL implementation followed a project-based approach. It started by the
Service Operations processes due to the fact that they have strong operation focus and to build the
basis for further implementation activities.
At beginning, specific staff were selected to ITIL training on Foundation Level. The processes flow
diagrams were then customized to fulfill the requirements of Nordex, resulting in a 3-layered process.
It involved the modeling and documentation of the ITIL processes in a predefined format. Thus, all
employees understand the processes diagrams and can suggest improvements.
According to the literature and the Nordex case study, the author concluded that there is no standard
procedure to implement ITIL but the project-based ITIL implementation used showed that general project
activities can be useful. This case study demonstrated that the implementation of Service Operations
processes based on ITIL can be a viable and beneficial solution but the processes and roles have
to adapted to the IT department and resources available in a SME. As ITIL implementation depends
fundamentally on the needs and context of the enterprise, the experiences collected in the case study
should be considered only as inspiration.
3.7 Related Work
As part of the literature review, the authors searched for empirical studies that specifically addressed
EGIT mechanisms in order to understand the existing body of knowledge and how the findings of this
study will contribute to knowledge advancement [78]. As aforementioned, the review of the related
work allows to close highly researched areas and reveal others where further research is required [48].
Thus, Table 3.1 presents a set of relevant empirical studies regarding the use of EGIT mechanisms in
organizations. The column “SMEs” represents if the study was focused on SMEs.
There are some relevant empirical studies about EGIT mechanisms, but few are focused on SMEs.
One of them, developed by Huang, Zmud and Price [17], addresses specifically SMEs but it exam-
23
Table 3.1: Empirical Research on EGIT Mechanisms
Source Description SMEs[79] Examine empirically four EGIT mechanisms that influence the overall effective-
ness of EGIT in Australian Public Sector organizations.No
[8] Studies the effectiveness and ease of implementation of EGIT mechanisms anda provides minimum baseline of EGIT mechanisms, focusing only on Belgian fi-nancial services organizations ranging from 100 to over 1000 employees.
No
[17] Research that examine qualitative data of three SME case sites with focus onthe influence of EGIT mechanisms related with two specific aspects: IT steeringcommittees and IT-related communication policies.
Yes
[80] Investigate EGIT mechanisms in a multi-sourced IT environment. Presents a real-life example of EGIT mechanisms at a leading multinational financial servicesprovider and proposes a framework of mechanisms suitable for this context.
No
[81] Examine empirically the EGIT mechanisms that influence the overall effectivenessof EGIT. Investigate the relationship of effective EGIT, the extent of IT outsourcingdecisions, and the level of IT Intensity.
No
[50] Research based on surveys of SMEs in the Australian tourist accommodationindustry regarding their use of EGIT mechanisms to define a framework of thecore elements to implement in this context.
Yes
[16] Investigate if Brazilian companies that have adopted EGIT mechanisms have im-proved their financial performance, by measuring pre and post adoption perfor-mance indicators.
No
[32] Exploratory study that intends to elicit and validate possible EGIT mechanismspatterns and identify the most relevant EGIT mechanisms for financial servicesorganizations based six interviews in Portuguese organizations.
No
[31] Exploratory research aiming to elicit ITG mechanisms patterns based on casestudies analysis and to draw conclusions about ITG mechanisms for Portuguesehealthcare industry based on six semi-structured interviews in large healthcareservices organizations.
No
[33] Exploratory research to identify an EGIT mechanisms’ baseline for universitiesbased on six case studies comprising of in-depth interviews three large and publicuniversities in Brazil, Portugal and the Netherlands.
No
ines the influence of EGIT Mechanisms related with only two specific aspects of EGIT: the IT steering
committees and communication policies. The results presented by this study are definitely interesting.
However, the authors believe that evaluating all aspects of EGIT in a holistic way will also be a great
contribution. Therefore, further research in this topic is required.
The second research that focused on SMEs, developed by Wilkin [50], intends to investigate and
identify the core EGIT mechanisms to implement in the context of SMEs that operate in the Australian
tourist accommodation industry. This study motivated us to investigate the fundamental EGIT mecha-
nisms for SMEs adopting a more comprehensive approach. The authors consider that it will be interest-
ing to evaluate the EGIT mechanisms not being restricted to only one specific industry.
Clear and unambiguous definitions of the roles and the responsibilities of the involved parties are a crucial prerequisite for an effective ITG. It includes governance/alignment tasks for business and IT people and it is the responsibility of the board and executive management to communicate and to make sure that they are clearly understood throughout the whole organization. The best idea is to document all roles and responsibilities.
Establish, agree on and communicate roles and responsibilities of IT personnel, as well as other stakeholders with responsibilities for enterprise IT, that clearly reflect overall business needs and IT objectives and relevant personnel’s authority, responsibilities and accountability.
Process APO01 Manage the IT Management Framework
Practice
APO01.02 Establish roles
and responsibilities
2 IT Organization
Structure [Structure]
The possibility of effective governance over IT is of course also determined by the way the IT function is organized and where the IT decision-making authority is located in the organization. The adoption of a particular mode is influenced by different determinants, such as history, economies of scale, size, industry, etc. Decision-making structures are the natural approach to generate commitment within the organization.
Position the IT capability in the overall organizational structure to reflect an enterprise model relevant to the importance of IT within the enterprise, specifically its criticality to enterprise strategy and the level of operational dependence on IT. The reporting line of the CIO should be commensurate with the importance of IT within the enterprise.
Process APO01 Manage the IT Management Framework
Practice
APO01.05 Optimize the
placement of the IT function.
3
IT Project Steering
Committee [Structure]
Steering committee composed of business and IT people focusing on prioritizing and managing IT projects.
A group of stakeholders and experts who are accountable for guidance of programmes and projects, including management and monitoring of plans, allocation of resources, delivery of benefits and value, and management of programme and project risk.
Organizational Structure
Project and Programme
Steering Committee
4
IT Budget Control and Reporting [Process]
Processes to control and report upon budgets of IT investments and projects.
Implement a cost management process comparing actual costs to budgets. Costs should be monitored and reported and, in the case of deviations, identified in a timely manner and their impact on enterprise processes and services assessed.
Process APO6 Manage Budget
and Costs
Practice APO06.05
Manage costs
5
Strategic Information
System Planning [Process]
Formal processes to define and update the IT strategy of the organization, including aligning IT with business goals, exploiting IT for competitive advantage, directing efficient and effective management of IT resources, and developing technology policies and architectures. These processes should assure the IT priorities and investments are strictly aligned with the mission, objectives and goals of organization
Create a strategic plan that defines, in co-operation with relevant stakeholders, how IT-related goals will contribute to the enterprise’s strategic goals. Include how IT will support IT-enabled investment programmes, business processes, IT services and IT assets. Direct IT to define the initiatives that will be required to close the gaps, the sourcing strategy and the measurements to be used to monitor achievement of goals, then prioritize the initiatives and combine them in a high-level road map.
Process APO02 Manage Strategy
Practice
APO02.05 Define the Strategic
Plan
49
6 Service Level
Agreement (SLA) [Process]
A Service Level Agreements (SLA) is defined as “a written contract between a service provider of a service and the customer of the service”. The functions of SLAs are: Define what levels of service are acceptable by users and are attainable by the service provider; define the mutually acceptable and agreed upon set of indicators of the quality of service. Three basic types of SLAs can be defined: in-house, external and internal SLAs.
Define and prepare service agreements based on the options in the service catalogues. Include internal operational agreements.
Process APO09 Manage Service
Agreements
Practice APO09.03 Define and
prepare service agreements
7
Shared Understanding of Business/IT
Objectives [Relational]
Mechanism that promote the mutual understanding of business and IT objectives and plans by business and IT people and the respect of each other’s contribution. Therefore, business and IT people can accurately interpret and anticipate actions and, if necessary, coordinate adaptively. This mechanism is considered a paramount for attaining and sustaining business/IT alignment.
Understand current business issues and objectives and business expectations for IT. Ensure that requirements are understood, managed and communicated, and their status agreed on and approved.
Process APO08 Manage
Relationships
Practice APO08.01 Understand
business expectations
8
Informal Meetings
(Business and IT Seniors)
[Relational]
Informal meetings, with no agenda, where business and IT senior management talk about general activities, directions, etc. (e.g. during informal lunches)
9 IT Leadership
[Relational]
Ability of CIO or similar role to articulate a vision for IT’s role in the company and ensure that this vision is clearly understood by managers throughout the organization. The goal is the coordination across the organization.
Communicate awareness and understanding of IT objectives and direction to appropriate stakeholders and users throughout the enterprise.
Process APO01 Manage the IT Management Framework
Practice
APO01.04 Communicate management objectives and
direction
50
Next, the IT Project Steering Committee definition clearly indicates that it consists in a group of peo-
ple, from business and IT, with the purpose of manage and prioritize the IT projects. This suggests
that it corresponds to a structure in the organization. Therefore, the authors focused on the Organiza-
tional Structures Enabler and based on the descriptions provided, identified the Project and Programme
Steering Committee as a correspondence for this mechanism. Only one mechanism was clearly an Or-
ganizational Structure, the authors believe that it can be related with the accumulation of responsibilities
that generally exists in SMEs.
There is only one mechanism that is not mapped in a COBIT 5 component, it is the Informal Meetings
between Business/IT Seniors. Given the name and description of this mechanism, the authors decided
to look at the APO Domain, more specifically to the Process APO08 - Manage Relationships. Obviously,
this process can help and support the establishment and management of the relationship between
business and IT. However, the process description includes: “Manage the relationship between the
business and IT in a formalized and transparent way that ensures a focus on achieving a common and
shared goal of successful enterprise outcomes...”. The mechanism intends to establish an informal
relationship, through meetings without a predefined agenda, while this practice recommends to manage
this relationship between business and IT in a formalized way. As can be easily perceived, this is the
opposite of the mechanism purpose. Since there is no correspondence with this EGIT mechanism, the
authors consider that the COBIT 5 framework do not support its implementation.
Analyzing the resulting mapping, the authors identified several interesting findings. First, the pro-
posed mapping does not contain any Practice from the EDM domain. This domain encompasses the
governance processes, including the responsibilities of the board for evaluating, directing and monitor-
ing the use of IT assets to create business value. It was not expected since this is a minimum baseline
of mechanisms that should be considered a good starting point to implement effective EGIT. However,
the lack of EDM Practices can be associated with the fact that the authors tried to identify only the Prac-
tice or Organizational Structure that was more related with each mechanism to maintain an acceptable
level of complexity in this initial phase of the EGIT implementation. Anyway, the EDM practices that
are relevant to the selected practices should appear associated with an input of those practices. Thus,
the practitioners know that there is a practice from EDM domain that should produce an input that will
contribute to the correct implementation of that practice.
Furthermore, all the identified Practices are from APO Processes. This domain addresses the plan-
ning and organization of the enterprise IT in order to effectively contribute to the achievement of the
business objectives, thereby including the strategic alignment between business and IT. This fact is ex-
tremely interesting since the authors identified that SMEs lack long-term vision of their business and
tend to adopt a more operational view than a strategic one. Their small IT departments are focused on
operational efficiency and short-term solutions. Therefore, the authors consider that the mechanisms
51
elicited based on the practitioners’ feedback and now mapped with APO Practices can contribute to
mitigate this problem that still exists in this type of organizations. Considering that the baseline should
be seen as a good starting point, the authors believe that it is extremely important to begin with the
implementation of practices that promote the planning and the strategic alignment from an early phase.
Only one baseline mechanism did not have a correspondence to a COBIT 5 component. Therefore,
the authors identified a correspondence for eight of the nine mechanisms included in the minimum
baseline, thereby suggesting there are guidelines and best practices provided by the COBIT 5 framework
that can also be relevant and appropriate for SMEs that intend to implement an effective EGIT. As always
some adaptation may be required. The identified correspondences contradict the idea that the COBIT 5
framework is only suitable for large organizations and that the provided best practices are not applicable
to SMEs.
With this mapping, the authors intend to facilitate the implementation of COBIT 5 in SMEs by showing
and indicating which are the main components that could support the implementation of almost all the
EGIT mechanisms present in the minimum baseline for SMEs. This solution can help to overcome
the problem related to the lack of orientation in the initial phases, namely to choose the processes or
practices to be implemented.
6.2 Evaluation
As in the previous chapter, this section will address the Evaluation Phase of the DSR but now for the
proposed mapping between the baseline mechanisms and the COBIT 5. As stated by Hevner et al. DSR
evaluation is a crucial phase of the research process [6]. However, the design researcher should balance
the interests of practitioners and researchers. The practitioners are concerned with the applicability
and usefulness of an artifact whereas the researchers are focused on the validity of the artifact and in
ensuring the rigor in the process [87]. Therefore, this artifact will be evaluated according to the Wand
and Weber method, Osterle et al. principles and expert interviews.
6.2.1 Wand and Weber Method
The developed mapping between the EGIT mechanisms and COBIT 5 components will be evaluated
using the Wand and Weber Method [91], which enables the analysis of the ontological effectiveness
of this mapping. This evaluation method is based on the ontological deficiencies that can be found in
the mapping, namely Incompleteness, Redundancy, Overload and Excess. These shortcomings can be
briefly described as:
• Incompleteness: Is each element of the first set mapped to an element of the second set? If so,
52
the mapping is considered complete. Otherwise, it is incomplete.
• Overload: Is each element of the second set mapped only by an element of the first set? If not,
the mapping is overloaded.
• Redundancy: Is each element of the first set mapped to more than one element of the second
set? If so, the mapping is redundant.
• Excess: Is each element of the second set mapped by an element of the first set? If not, the
mapping is excessive.
As referred, the proposed mapping is present in Table 6.1. In the next paragraphs, this mapping will
be evaluated according to this four ontological deficiencies.
Starting by incompleteness, it is easy to verify that the mapping is considered incomplete since there
is no correspondence between the mechanism Informal Meetings between Business/IT Seniors and a
COBIT 5 component. This fact means that COBIT 5 does not specify a concrete practice that support
and guide the implementation of the mechanism which is recognized as fundamental to SMEs. How-
ever, this incompleteness is not totally surprising because frameworks like COBIT 5 are often criticized
for being more appropriate for large organizations. Moreover, there is only one mechanism without cor-
respondence and COBIT 5 also support this relationship between Business and IT, through Process
APO08 - Manage Relationships, but not in an informal way as stated by this mechanism.
The mapping is not overloaded. This deficiency is not verified in the proposed mapping since there
are not two different mechanisms that were mapped to the same COBIT 5 component. It happens be-
cause the authors tried to map each mechanism to the most related Organizational Structure or Practice
and not only at the Process level. In conclusion, the baseline mechanisms extracted address relevant
and distinct aspects of the implementation of effective EGIT in SMEs. Therefore, the identified baseline
suggests the adoption of a holistic approach, which is highly recommended for EGIT implementation.
Furthermore, the proposed mapping is not redundant. This deficiency was not identified because, as
aforementioned, the authors tried to map only the most related practice and not all the related practices
in order to avoid a huge number of related COBIT 5 components. The increase of COBIT 5 practices
related will definitely increase the complexity associated with its implementation, which can act as barrier
to the implementation of EGIT in these organizations. Therefore, the authors selected the practice
that seems more related and appropriate to support the implementation of that mechanism. If a more
comprehensive approach is adopted, redundancy will certainly exist and is not surprising given the
complex nature of the COBIT 5 framework.
Finally, this mapping is clearly excessive. However, this was also expected since the authors decided
to establish the mapping with COBIT 5 Practices and Organizational Structures. As already referred,
53
COBIT 5 defines more than 200 Practices, that compose the 37 processes, and 26 different organiza-
tional structures or roles whereas the elicited baseline is composed only by 9 EGIT Mechanisms. This
is a minimum baseline that should be seen as an initial roadmap to the implementation of EGIT in SMEs
in general. The authors consider that, in this case, this shortcoming is normal and do not affect the
purpose and value of the proposed mapping.
6.2.2 Osterle et al. principles
As before mentioned, scientific research should be characterized by abstraction, originality, justification
and publication. Therefore, there are four basic principles to which all constructed artifact must comply
[34]:
• Abstraction: Each artifact must be applicable to a class of problems - The authors consider the
proposed mapping is useful and relevant to managers and practitioners from SMEs that desire to
implement the fundamental EGIT mechanisms for these organizations by adopting the guidelines
and best practices provided by the COBIT 5 framework.
• Originality: Each artifact must substantially contribute to the advancement of the body of knowl-
edge - After the literature review performed, the authors verified there are no studies addressing
the adoption of COBIT 5 in SMEs. More specifically there are no studies addressing the funda-
mental EGIT mechanisms for SMEs and their correspondence to COBIT 5 components, including
the recommend Practices or Organizational Structures. Therefore, this artifact contributes to the
advancement of the current body of knowledge and the originality principle was satisfied.
• Justification: Each artifact must be justified in a comprehensible manner and must allow for its
validation - As the first artifact, the proposed mapping is based on previously published articles
and in COBIT 5 official documentation. The authors consider that the motivation and research
problem presented justify the need for the proposed mapping. All the steps applied in construction
and validation of the artifact are carefully described in this report.
• Benefit: Each artifact must yield benefit – either immediately or in the future – for the respective
stakeholder groups - The stakeholders that will benefit from this artifact are the managers and
practitioners from SMEs. The proposed mapping enables these practitioners to know the CO-
BIT 5 components that support the implementation of the baseline mechanisms. Therefore, the
practitioners can access and follow the best practices and guidelines specified by the COBIT 5
framework, such as the recommended activities and respective inputs/outputs, which will be an
important help in implementing these EGIT mechanisms.
54
6.2.3 Expert interviews
As previously referred, qualitative interview has been broadly used in IS research. This is considered as
a suitable and valuable method to collect important feedback according to interviewee’s perception [46].
Furthermore, several authors declared that expert interviews are an appropriate method to apply in DSR
in order to evaluate the created artifact [34], [87], especially when other evaluation methods may not be
feasible [6].
Therefore, the authors decided to perform expert interviews, through a semi-structured approach,
with professionals who have a great deal of knowledge about EGIT and, more specifically, about the
COBIT 5 framework. The flexibility of the semi-structured interviews allowed the authors to use Likert
scales to assess the experts’ perceptions on various aspects of the mapping and to ask open ques-
tions to probe and extend their answers. Thus, the authors can understand the rationale behind their
evaluations.
The authors contacted several experts by e-mail to present the research and invite them to participate
in this evaluation. When accepted, the interview was scheduled and two documents were shared with
the interviewee (see Table 6.2). One document was the proposed mapping whereas the second was
the questionnaire containing the questions that will guide the interview. Thereby, all the interviewee had
enough time to analyze the proposed mapping.
Table 6.2: COBIT 5 experts’ details.
The interviews, through Skype, started with an introduction to the thesis and the purpose of our
work, followed by the questionnaire which is composed by three distinct parts (Appendix C). The first
part involves a few questions about the academic qualifications and personal experience of the intervie-
wees. The second part is related with the individual evaluation of each correspondence presented in the
mapping, while the third part comprise a set of questions that address several evaluation criteria for IS
55
artifacts. All these questions were answered based on a 5-point Likert scale [86].
As it can be seen in Table 6.2, a broad spectrum of professional profiles was considered, including
researchers, managers, directors and professors. These interviewees have a vast knowledge of EGIT
in general, obtaining an average of 4.5, and more specifically of COBIT 5, with an average of 4.3.
Despite not being a requirement, almost every interviewee had previous experience in SMEs which can
be extremely relevant and advantageous for gathering significant feedback through this evaluation. After
providing ratings for the 8 correspondences, the interviewees then provided ratings for a subset of the
criteria proposed by Prat et al. [7] to evaluate IS artifacts.
6.2.3.A Evaluation criteria by Prat et al.
Based on general systems theory, Prat et al. [7] proposed a hierarchy of criteria to evaluate IS artifacts.
This hierarchy of evaluation criteria was derived from all the criteria proposed in design-science research
literature and was structured according to the dimensions of a system, such as goal, environment,
structure, activity, and evolution. For each dimension, a set of criteria and sub-criteria were specified
and described. Thus, the proposed hierarchy provides a holistic of view of the evaluation criteria.
From this overarching hierarchy [7], the authors selected the criteria that were considered the most
appropriate and relevant to evaluate our artifact. Next, the selected criteria are presented in Table 6.3
with a brief description addressing how it will be applied in the context of our artifact and research.
These criteria and the achievement of the predefined objectives will be evaluated based on semi-
structured interviews which are an adequate method to collect feedback, as aforementioned. This feed-
Table 6.3: Evaluation criteria selected from the hierarchy proposed by Prat et al. [7].
56
back will be essential to realize the opinions and experience of people that can use and be directly
affected by the proposed solution. In this way, the authors will be able to conclude if the predefined
objective are actually achieved or not.
6.2.3.B Results Analysis
In this section, the authors will present and discuss briefly the results obtained regarding the evaluation
of the proposed mapping. All the data collected is presented in Appendix D. Table D.1 contains the
ratings for the individual evaluation of the nine correspondences, whereas Table D.2 comprises the
ratings given by the experts regarding the evaluation criteria selected. Starting with the evaluation of the
correspondences established in this mapping, the average results can be seen in Figure 6.1. Herein,
the letter C represents the correspondences.
Figure 6.1: Average ratings of correspondences
Thus, it is possible to verify that five out of the eight correspondences obtained an average rating
equal to or higher than 4.4. The authors consider that this is a fairly good and satisfactory average
value, thereby suggesting that these correspondences are appropriate and little improvement can be
made. Furthermore, the vast majority of the experts interviewed provided a positive feedback (4 - Agree
or 5 - Strongly Agree) regarding these correspondences:
• C1 and C5 obtained an average value of 4.40 - For both, 90% of the respondents answered
positively and 50% gave the maximum classification;
• C2 got an average value of 4.50 - 90% of the experts responded positively and 60% evaluated with
maximum rating;
57
• C6 obtained an average of 4.60 - 100% of the experts interviewed gave a positive rating and 60%
gave the maximum classification;
• C4 achieved an average of 4.70 - 100% of the experts provided a positive answer and 70% rated
the correspondence with maximum value.
The referred correspondences got quite acceptable ratings. However, as can be seen in Figure 6.1,
there are three correspondences that got lower average values when compared with the ones already
mentioned. Thus, the authors will now focus on these correspondences in order to investigate and
understand the reasons for these lower ratings, according to the qualitative feedback provided by the
experts. These correspondences obtained the following results:
• C3 got an average value of 3.50 - Just 40% of the experts answered positively and only 10% of
them gave the maximum rating;
• C7 obtained an average of 3.90 - 70% of the experts responded positively and only 20% rated the
correspondence with the maximum value;
• C9 got an average value of 4.00 - Just 70% of the experts provided a positive answer and 40%
gave the maximum classification.
This three correspondences got much lower average values and the authors consider that can be
much higher. Therefore, the authors will present some of the most consensual reasons, provided by the
experts, for these ratings. This qualitative feedback will be fundamental to identify several possibilities
of improvement of the proposed mapping, namely regarding the C3, C7 and C9.
The C3 is the correspondence between the structural mechanism IT Project Steering Committee
and the COBIT 5 organizational structure Project and Programme Steering Committees. This was the
correspondence with the worst ratings and the reasons behind it are consensual. The majority of the
experts stated that, despite agreeing that the COBIT 5 organizational structure is a proper matching,
they also consider that an effective implementation of the EGIT mechanism will imply more than solely
create an organizational structure. The proposed mapping is incomplete. Based on their feedback, this
organizational structure should base their work on the COBIT 5 best practices. Therefore, this majority
recommended implementing part of the Process BAI01 - Manage Programme and Projects in order to
achieve a suitable and sufficient correspondence.
Next, the C7 is the correspondence between the relational mechanism Shared Understanding of
Business/IT Objectives and the Practice APO08.01 - Understand business expectations. This was the
second worst correspondence and the rationale behind this ratings is somehow consensual. Three
of the interviewees have clearly stated that, besides the proposed practice is definitely related with the
EGIT mechanisms, the implementation of this mechanism implies the existence of mutual understanding
58
of objectives and plans and, furthermore, the achievement of a strategic alignment. According to the
experts’ feedback, this mutual understanding and strategic alignment should start from the board of
directors and not only at the management level. Thus, the experts suggested searching for a related
Process or Practice from the EDM domain, which comprises the Governance processes.
The C9 correspondence identifies the Practice APO01.04 - Communicate management objectives
and direction as a possible solution for the implementation of the relational mechanism IT Leadership.
The average value is not so bad as the previous ones but given the feedback gathered from the experts,
the authors considered that this correspondence can be improved. Three experts have declared that this
type of leadership is much more than simply communicate the IT objectives and direction to appropriate
stakeholders, the proposed practice is not sufficient. Based on their opinion, the communication is im-
portant and the Practice APO01.04 is relevant and applicable, however this mechanism depends heavily
on the personal capabilities of the CIO or similar role that generally is responsible for this leadership.
One of the interviewees suggested to check the Process APO02 - Manage Strategy because there can
be something relevant for this mechanism implementation. Given this feedback, the authors suggest to
investigate the Enabler People, Skills and Competences which was not consider in this research.
Finally, regarding the lack of a correspondence to the EGIT mechanism Informal Meetings between
Business and IT Seniors. Seven of the nine interviewees understood and agreed with the vision followed
by the authors and explained in section 6.1. However, several experts recommended to search in other
COBIT 5 Enabler, the Culture, Ethics and Behavior, which was also not considered in the scope of this
research. Therefore, investigation addressing the other COBIT 5 Enablers can be an excellent future
work.
Now, Figure 6.2 presents the rating given by the experts according to the Prat et al. criteria. From
the five selected criteria, there are two criteria that obtained less than 4.00. The authors consider
that these average values are relatively low and then, there are some improvements that can be done.
First, the Efficacy criteria addresses the degree to which the artifact produces its desired effect and
got an average of 3.90. The authors believe that the obtained value can be extremely higher if the
aforementioned problems are appropriately solved. Despite some of the problems identified, 80% of the
interviewees gave a positive answer about this criteria. Thus, the authors consider that a large part of
the mapping goal was achieved.
The other criteria that got a lower average value was the Level of Detail with an average of 3,50. It is
a relatively low value for a criteria that can affect the understandability and perception of the correspon-
dences established. Moreover, it can affect the perceived usefulness of the proposed artifact. According
to experts’ opinion, the problem of this criteria is related to the definitions provided for some of the EGIT
mechanisms, which were extracted from the literature. Thus, further analysis of the literature may be
required to extract more detailed definitions. The authors believe that if this aspect is improved, the
59
Figure 6.2: Evaluation of Prat et al. criteria
correspondences established could achieve higher classifications.
The criteria of Consistency and Ease of Use obtained quite good ratings, with an average of 4.40.
Furthermore, as it can be seen in Figure 6.2, 90% of the experts responded positively and 50% of them
gave the maximum rating. These ratings substantiate the quality of the proposed artifact regarding these
two criteria.
Regarding the Utility for people criteria, the artifact obtained an average of 4.00. Despite not being
an excellent result, the authors consider it is a reasonable and interesting value. Again, as can be seen
in Figure 6.2, 80% of the experts interviewed provided a positive feedback about this specific criteria.
Thus, these results suggest that the proposed mapping would be useful in practice and could be an
added value for practitioners and managers from SMEs that intend to implement effective using the
COBIT 5 framework.
It is important to reinforce that the large majority of the COBIT 5 experts interviewed had previously
experience in SMEs, therefore, the authors consider that the feedback provided about the evaluated
criteria is extremely significant and close to reality.
In spite of not being an objective, the Ease of use of this mapping by people without a great knowl-
edge of COBIT 5 framework was also evaluated. As expected, the obtained result was very low, with
an average value of 2.50. Moreover, only 20% of the experts gave a positive answer. This fact is not
surprising given the complexity that is associated with the COBIT 5 framework, which take some time to
This chapter consists in a brief summary of the work performed during this thesis. Next, all the con-
tributions resulting from this research, as well as all the limitations identified, will be presented in the
following sections. Finally, the communication activities and the future work related to this research will
also be detailed. In this chapter, the authors will also analyze and verify if the predefined objectives for
the proposal were achieved.
As aforementioned, De Maere and De Haes [38] recommend that researchers adopting DSR in the
ITG area adhere to the guidelines defined by Hevner et al. [6]. The purpose of the guidelines is to help
researchers and reviewers to understand the requirements for an effective DSR. These guidelines will
be verified and discussed throughout the following sections.
During this research work, the authors proposed and developed two different artifacts. However, it
is important to refer that the first artifact was fundamental for the construction of the second one. First,
the minimum baseline of EGIT mechanisms for SMEs was identified. This baseline resulted from the
elicitation of the fundamental EGIT mechanisms for these organizations, according to the feedback of
experts and practitioners. Then, the second artifact was constructed. This artifact consists of a mapping
between the identified EGIT mechanisms and the COBIT 5 components that can support and help in its
implementation.
The proposed artifacts constitute a solution intended to solve a significant and relevant organiza-
tional problem: the lack of support for the implementation of COBIT5 in SMEs. As referred, SMEs
represent 99% of all businesses in the EU and are also dependent on IT. Nowadays, EGIT is cru-
cial to manage and control their IT-related assets but little empirical research addressed the EGIT in
SMEs. COBIT 5 is recognized as the best and most complete EGIT framework. However, the COBIT 5
framework involves an exorbitant number of interrelated components and this type of organizations are
normally more constrained in terms of IT resources, making COBIT 5 implementation a complex and
frightening task. Furthermore, the interviews performed with experienced members of ISACA, including
Mike Hughes (ISACA International Board Member Director) and Marc Vael (ISACA Belgium Chapter
President), contributed to substantiate and validate the relevance of the identified problem.
Regarding the guidelines provided by Hevner et al. [6] and considering the information presented in
previous paragraphs, the authors conclude that Guideline 1 - Design as an artifact and Guideline 2 -
Problem relevance were both fulfilled.
The DSR evaluation is a crucial phase of the research process [6]. The proposed artifacts were
rigorously evaluated based on well-executed evaluation methods. First, the minimum baseline of EGIT
mechanisms was evaluated through qualitative semi-structured interviews with experts and based on the
Osterle et al. principles. Next, the proposed mapping was evaluated using different methods, including
the Wand and Weber method and the Osterle et al. principles. Furthermore, semi-structured interviews
with COBIT 5 experts were performed to evaluate the correspondences established and the criteria
62
selected from the hierarchy of evaluation criteria for IS artifacts proposed by Prat et al. Thus, the authors
consider that the Guideline 3 - Design evaluation proposed by Hevner et al. was also satisfied in this
research.
7.1 Objectives evaluation
As referred in Chapter 4, the main purpose of the proposed solution is to facilitate COBIT 5 imple-
mentation in SMEs. Therefore, the authors defined that the proposed solution should comply with the
following objectives:
• Objective 1: Identify the fundamental mechanisms to implement effective EGIT in SMEs;
• Objective 2: Establish the correspondence between the fundamental EGIT mechanisms for SMEs
and the Processes and Organizational Structures defined in COBIT5;
Based on the results obtained through the various evaluation methods applied, the authors will draw
conclusions regarding achievement of this objectives.
Since the minimum baseline of EGIT mechanisms for SMEs was entirely constructed based on the
feedback collected through semi-structured interviews with IT experts knowledgeable in the context of
SMEs, the authors consider that the objective 1 was accomplished.
As can be perceived, the objective 2 is related with the proposed mapping. Given the results obtained
in the evaluation, including the evaluation of the correspondences and the selected criteria, the authors
conclude that objective 2 was partially achieved. Despite having 80% or more of positive answers about
several criteria, such as Goal efficacy, Consistency, Utility for people and Ease of use, there are some
improvements that can be performed, especially regarding the incomplete correspondences and the
level of detail presented.
In spite of one of the objectives not being totally achieved, the authors consider the obtained results
demonstrated that the proposed solution could be useful and advantageous for practitioners from SMEs
that intend to implement effective EGIT in the organization by adopting the best practices of the COBIT
5 framework. Therefore, as intended, this solution can facilitate the COBIT 5 implementation in SMEs.
7.2 Contributions
With the work performed during this research, the authors expect to contribute not only to the specific
research problem, but also to the advancement of the existing body of knowledge. Based on both scien-
tific and practitioner perspectives, this research provided several interesting and relevant findings about
63
the EGIT in SMEs. The two major contributions of this research are the artifacts produced. However,
there are other minor contributions associated.
Regarding the minimum baseline of EGIT mechanisms for SMEs, the authors started by performing
qualitative interviews with several IT experts with knowledge and experience in these organizations.
In this interviews, the experts had to evaluate an overarching list of 46 EGIT mechanisms existent in
the literature. This evaluation comprised two important parameters: the difficulty of implementation
and potential effectiveness in SMEs’ context. Afterwards, each interviewee had to select the ten most
important mechanisms based on their professional experience in SMEs. This is also a contribution since
each one of the 46 EGIT mechanisms were individually evaluated and then, the ten fundamental ones
were selected by several experts. This information can be extremely useful when the practitioners and
managers of SMEs are studying the implementation of certain EGIT mechanisms.
Based on this feedback, the authors constructed the minimum baseline of EGIT mechanisms for
SMEs, which is one of the major contributions of this thesis. Additionally, the authors analyzed the qual-
itative feedback gathered during the interviews to understand the reasons behind certain classifications
and to reveal other interesting findings regarding the perception of some EGIT mechanisms in SMEs.
Finally, a cross-study comparison with similar studies was performed. This allowed the authors to com-
pare the results obtained in previous studies focused on other contingencies and to draw conclusions
about how the EGIT mechanisms are perceived differently depending on the contingency. This qualita-
tive analysis and cross-study comparison also contributes to the advancement of the existing body of
knowledge.
Finally, the other major contribution of this thesis is the proposed mapping between the baseline
mechanisms and the COBIT 5 components. This artifact intends to establish the correspondence be-
tween each EGIT mechanism and a COBIT 5 component, a Process or an Organizational Structure, that
support its implementation. The mapping enables the practitioners to know and recognize the COBIT
5 components that support the implementation of these mechanisms. Therefore, the practitioners can
access and adhere to the best practices and guidelines specified by the COBIT 5 framework, such as
the responsibilities description or the recommended activities and respective inputs/outputs, which will
be an important help in implementing these EGIT mechanisms.
A correspondence was established for eight of the nine mechanisms included in the baseline, thereby
suggesting that the COBIT 5 framework provides guidelines and best practices that can also be relevant
and appropriate for SMEs. Therefore, this mapping intend to facilitate the implementation of COBIT 5 in
SMEs by identifying the main components that could support the implementation of almost all the EGIT
mechanisms present in the minimum baseline for SMEs.
Based on all the contributions mentioned, the authors conclude that the Guideline 4 - Research
Contributions was clearly fulfilled. Additionally, the Guideline 5 - Research rigor is related with the
64
effective use of the knowledge base, including the theoretical foundations and research methodologies.
DSR relies upon the adherence to appropriate data collection and analysis techniques to construct and
evaluate the artifact. Therefore, the authors believe that Guideline 5 was also satisfied by applying the
following techniques throughout the construction and evaluation of the artifacts: literature review, Wand
and Weber method, Osterle et al. principles and semi-structured interviews with experts. All these
methods were described in previous chapters.
7.3 Limitations
As the contributions, also the limitations are related with one of the constructed artifacts. Regarding the
minimum baseline, the authors identified some limitations. First, all the collected data was limited to the
eleven semi-structured interviews performed with IT experts and only one person of each organization
was interviewed. More interviews with IT experts can be performed, acquiring an even bigger sample to
reinforce and strengthen the results of the mechanisms evaluation and the identified baseline. However,
the authors believe that the eleven semi-structured interviews performed were a good starting point for
drawing interesting conclusions in a research area that is currently scarce and limited.
Another limitation is the fact that the transcript of the qualitative data was not presented due to space
limitations. However, the authors consider that the most significant findings extracted from the qualitative
feedback, collected through the interviews with the experts, were presented. Furthermore, there are
additional contingencies factors that could affect the reality of the SMEs, such as the geography, strategy
and culture of the organization, and they were not addressed in this research.
There are also other limitations related to proposed mapping. First, three correspondences, the C3,
C7, and C9, were assessed as incomplete by the experts. However, several possible improvements to
these correspondences were presented based on the feedback collected during the interviews. Another
limitation related to this artifact is the low Level of Detail presented. According to the experts, this is
related with the detail of the EGIT mechanisms definitions that were taken from the literature.
Finally, another limitation exists in this research, related to the DSR process. As referred, this re-
search process consists inherently in an iterative and incremental activity, where the evaluation phase
should provide and contribute to the design phase with valuable feedback. Therefore, the constructed
artifacts can be improved and completed until the requirements and constraints of the problem are sat-
isfied. However, in this research, a second iteration of the DSR process was not applied given the time
restrictions associated with the development of this thesis. Therefore, the authors consider that this is a
limitation of our research.
Thus, the Guideline 6 - Design as a search process is extremely difficult to fulfill. First, performing
several iterations is a central part of the DSR process and, as referred, in this research only one iteration
65
was accomplished due to time limitations. Moreover, there are no other competing solutions to address
the same problem situation. The comparison of the proposed solution with other developed by experts
is impossible. Therefore, the authors conclude that the Guideline 6 was not fulfilled.
7.4 Communication
The results of DSR must be presented both to technology-oriented as well as management-oriented
audiences. This enables the practitioners to benefit by applying the constructed artifact and allows the
researchers to build a cumulative knowledge base [6]. As aforementioned, the review process prior to
scientific publications is part of the evaluation. This section addresses the Diffusion phase of the DSR
process where the results obtained are shared with the interested communities.
Throughout the development of this thesis, two scientific papers were submitted. The first paper,
addressing the minimum baseline of EGIT mechanisms for SMEs, was already accepted, presented
and published in the 20th IEEE International Conference on Business Informatics 1:
• Silva, D., da Silva, M. M., & Pereira, R. (2018). Baseline Mechanisms for Enterprise Governance
of IT in SMEs. In 2018 IEEE 20th Conference on Business Informatics (CBI). Vienna, Austria.
IEEE. [92]
This paper was submitted in an intermediate stage of the research, when only seven interviews
were performed to evaluate the EGIT mechanisms and elicit a minimum baseline. Thus, the paper was
accepted as research-in-progress.
Afterwards, the authors completed the research with more interviews and submitted an article pre-
senting the results obtained to the Information Systems Management Journal 2, which is awaiting ac-
ceptance.
All the papers developed by the authors were also sent to all practitioners and experts that partici-
pated in the research. Finally, all the research performed in the scope of this thesis was presented and
described in this thesis report. Therefore, the authors conclude that the Guideline 7 - Communication of
research was also satisfied.
7.5 Future Work
Further research on this topic may focus on interesting aspects such as the identification of new EGIT
mechanisms specifically used in SMEs, the impact of the other EGIT contingency factors in SMEs’
context or the differences between public and private SMEs regarding the EGIT mechanisms, as those1https://cbi2018.big.tuwien.ac.at/2https://www.tandfonline.com/toc/uism20/current
1. IT Strategy Committee The IT Strategy Committee operates at the board level. The IT Strategy Committee – composed of board and non-board members – should assist the board in governing and overseeing the enterprise’s IT-related matters. This committee should ensure that IT is a regular item on the Board’s agenda and the Board has the information required to achieve the ultimate objectives of IT Governance. This committee has to work in close relationship with the other board committees and with management in order to provide input, and to review and amend the aligned enterprise and IT strategies [2],[13],[57].
2. IT Audit Committee (at level of board of directors)
Independent committee at the level of the board of directors overviewing (IT) assurance activities [2]. This committee should: identify the key business processes that depend on IT and identify key risks areas and constantly measure the risk level and systematically and carefully examine their controls efficiency [53].
3. CIO on Board The presence of the CIO on Board will ensure that IT will be a regular item on the board’s agenda and that it will be addressed in a structured manner. That presence will also enhance the ability of the board to understand the role of IT in business strategy and to map the ITG role of the executive team. The CIO should report on a regular basis to the board [10],[52],[57].
4. CIO on Executive Committee
CIO is a full member of the executive committee [2]. This ensures that IT is part of the executive team agenda’s where most strategy discussions begin and end. With that interaction IT can be an enabler of the organization [2],[55].
5. CIO reporting to CEO and/or COO
CIO has a direct reporting line to the CEO and/or COO [2].
6. IT Steering Committee The IT steering committee is situated at executive level. It is responsible for determining business priorities in IT investment [2]. It assists the Executive in the delivery of the IT strategy, overseeing the day-to-day management of IT service delivery and IT projects. IT steering committee focuses particularly on implementation [13], tracking IT investments, setting priorities and allocating scarce resources [1].
7. IT Governance Function/Officer
Structure in the organization responsible for promoting, driving, managing IT governance processes and reporting to CIO [2],[55]. The implementation of this structure sends a strong message that IT governance is important and provides a continual focus on the issue by dedicating a resource and holding a senior manager accountable for IT governance initiatives [55].
8. Security / Compliance / Risk Officer
Function responsible for security, compliance and/or risk, which possibly impacts the IT [2].
9. IT Project Steering Committee
Steering committee composed of business and IT people focusing on prioritizing and managing IT projects [2].
10. IT Security Steering Committee
Steering committee composed of business and IT people focusing on IT related risks and security issues [2].
78
11. Architecture Steering Committee
Committee composed of business and IT people providing architecture guidelines, advises on their applications and directing IT architecture design [1],[2]. The main goal of this committee is identify, communicate and enforce architecture and IT standards, strategic technologies and ensure that the architecture is compliant with legislative and regulatory
requirements [1],[10],[54],[55].
12. Integration of Governance / Alignment Tasks in Roles and Responsibilities
Clear and unambiguous definition of the roles and the responsibilities of the involved parties are fundamental for an effective IT Governance [13],[57]. It includes governance/alignment tasks for business and IT people [2] and it is the role of Board and Executive management to communicate these roles and responsibilities and to make sure that they are clearly understood throughout the whole organization [13],[57].
13. IT Councils IT Councils often report to the executive committee and contain overlapping memberships. This councils can provide a focused environment to consider several levels of policies and investments [54] and to discuss new technologies and new ways technology can be leveraged across the organization [27]. The huge items can then go to the executive committee with informed recommendations [54].
14. IT Leadership Councils IT Leadership Councils – composed by business unit IT representatives - are particularly important for large multi-business enterprises where there is a mix of responsibilities for infrastructure services, some enterprise-wide and others at business-unit level that need to be governed and managed [22],[54].
15. Business/IT Relationship Managers
Business/IT relationship managers Business/IT relationship managers act as the intermediary between the business and IS, playing a critical daily two-way role by helping IS understand how business operates and giving the business units an entry point to IS. They play an important role in communicating mandates and their implications and supporting the needs of business units managers while help them see benefits rather than inconveniences [10],[15].
16. IT Investment Committee
Committee responsible for evaluating and approving major capital expenditures, ensuring that all the IT investments approved are aligned with organization’s strategies and deliver value within acceptable risk boundaries [22],[55].
17. IT Expertise at Level of Board
Members of the board of directors have expertise and experience regarding the value and risk of IT. A lack of board oversight for IT activities is dangerous; it will put the firm at risk in the same way that failing to audit its books would [2],[10].
18. IT Organization Structure
The possibility of effective governance over IT is of course also determined by the way the IT function is organized and where the IT decision-making authority is located in the organization. The adoption of a particular mode is influenced by
Centralized In a centralized IT organization, all IT decision-making and the IT budget are in one place, they are much easier to manage and require much less effort to organize [55]. Promotes efficiency and standardization of IT infrastructure [57].
Decentralized In a decentralized organization, each decentralized IT function has developed its own IT governance processes, infrastructure and applications [55]. Promotes effectiveness and flexibility for the development of applications [57].
79
different determinants, such as history, economies of scale, size, industry, etc. Decision-making structures are the natural approach to generate commitment within the organization [10],[57].
Federal Hybrid organizations that have both centralized and decentralized components. Most infrastructure and enterprise wide applications are centralized in a corporate IT organization and operated as a shared service with chargebacks, while business units retain control over specific applications and development resources [55]. This model tries to achieve both efficiency and standardization for the infrastructure, and effectiveness and flexibility for the development of applications [13],[57].
19. Strategic Information System Planning
Formal processes to define and update the IT strategy of the organization [2], including aligning IT with business goals, exploiting IT for competitive advantage, directing efficient and effective management of IT resources, and developing technology policies and architectures. These processes should to assure the IT priorities and investments are aligned with the mission, objectives and goals of organization [2],[13].
20. IT Performance Measurement - IT Balanced Scorecard (BSC)
An important part in the implementation process of strategic alignment is the performance measurement of IT and of IT related to the business. BSC has been applied in the IT function and its processes. Recognizing that IT is an internal service provider, the proposed perspectives of BSC should be changed accordingly, with corporate contribution, user orientation, operational excellence, and future orientation. Linking the business BSC and
the IT BSC is a supportive mechanism for ITG [2],[13],[57].
21. Portfolio Management Process to prioritize and manage IT-related investments, projects and assets by means of investment programs in which business and IT people are involved (includes business cases, information economics, ROI, payback) [2],[55]. A strong IT portfolio management process is in place to ensure that all IT investments are optimized and deliver the optimal value to the organization [55].
22. Chargeback Chargeback is an accounting mechanism for allocating central IT costs to business units. The purpose of chargeback is to allocate costs so that business units IT costs reflect the use of shared services while the shared services unit matches its costs with the business it supports. When IT understands its costs and charges out accordingly, chargeback processes demonstrate the cost saving resulting from shared services. Enterprises with effective costing mechanism find that chargeback can foster useful discussions between IT and business units about IT charges, leading to better- informed ITG decisions [2],[10],[15].
23. Service Level Agreement A Service Level Agreements (SLA) is defined as “a written contract between a service provider of a service and the customer of the service”. The functions of SLAs are: Define what levels of service are acceptable by users and are attainable by the service provider; define the mutually acceptable and agreed upon set of indicators of the quality of service. Three basic types of SLAs can be defined: in-house, external and internal SLAs. The differences between those types refer to the parties involved in the definition of the SLA [2],[13],[57].
80
24. IT Governance Frameworks/ Standards
Generically, a framework is a set of guiding principles and good practices that are explicitly designed to be adapted by adopting organizations. Frameworks are distinguished from standards that are designed for monolithic adoption. An IT Governance Framework/Standard is the set of guidelines and good practices to govern and manage IT-related issues and activities [24].
25. IT Governance Assurance and Self-assessment
Process to perform regular self-assessments or independent assurance activities on the governance and control over IT [2].
26. Project Governance / Management Methodologies
Processes and methodologies to govern and manage IT projects [2].
27. IT Budget Control and Reporting
Processes to control and report upon the usage of established budgets for IT investments and projects [2].
28. Benefits Management and Reporting
Processes to monitor the planned business benefits and report the actual situation in terms of benefits realization during and after the implementation of IT investments / projects [2].
29. Business/IT Alignment Model
Model that conceptualize and direct the process and goal of achieving competitive advantage through developing and sustaining a symbiotic relationship between business and IT. One of the most used models is the well-known Strategic Alignment Model (SAM) [13].
30. IT Governance Maturity Models
To implement and improve an IT Governance framework, organizations need to have a self-diagnosing tool [13]. To be able to self-assess, measure and benchmark the IT Governance performance, organizations can use a maturity model. This is a method of scoring based on a variety of attributes that enables the organization to grade itself from non-existent (0) to optimized (5), contributing to determine the “as-is” and the “to-be” position. When these positions are known, gaps can be determined, projects defined, and specific actions can be defined to move towards the desired level of governance maturity [13],[57].
31. Demand Management Demands for IT resources come from all directions and in all forms. Some
demand is routine, other demand is strategic and complex.Demand
management forces all IT demand through a single point, where the demands can be consolidated, prioritized and fulfilled [55].
32. Architectural exception process
Technology standards are critical to IT and business efficiency. But occasionally exceptions are not only appropriate, they are necessary. Enterprises use the exception process to meet unique business needs and to gauge when existing standards are becoming obsolete. Without a viable exception process, business units ignore the enterprise
wide standards and implement exceptions with no approval.The
effectiveness of the architecture exception process depends on the ability of the IT unit to research and define standards and on the enterprise’s commitment to technology standards [10],[27].
33. Job-rotation IT staff working in the business units and business people working in IT [2]. Employees have the opportunity to rotate between different IT and business functions contributing to widen their knowledge and increase mutual insight in the business and IT [2].
81
34. Business/IT Co-location Physically locating business and IT people close to each other [2], enforcing daily contacts between them by the physical landscape of the working environment [2].
35. Cross-Training Training business people about IT and/or training IT people about business [2].
36. Knowledge Management (On IT governance)
Mechanism to communicate, share and distribute knowledge about IT governance framework, responsibilities, tasks, etc. Portals have become the premier method to implement this mechanism [2],[55].
37. Business/IT Account Management
Bridging the gap between business and IT by means of account managers who act as in-between [2].
38. Senior management giving the good example
Senior business and IT management acting as “partners” [2],[8].
39. Informal Meetings between Business and IT Senior Management
Informal meetings, with no agenda, where business and IT senior management talk about general activities, directions, etc. (e.g. during informal lunches) [2],[54].
40. IT Leadership The ability of the CIO or similar role to articulate a vision for IT’s role in the company and ensure that this vision is clearly understood by managers throughout the organization. Hence, we can say that the goal of IT leadership is to have coordination across the organization [2],[15],[8].
41. Organizational Internal Communication
Internal communication regularly addresses general IT issues [2],[8].
42. IT Governance Awareness Campaigns
Campaign to explain to business and IT people the need for ITG. Working with managers who stray from desirable behaviors is a necessary part of generating the potential value of governance processes. Therefore, it is necessary to communicate with those managers in order to educate them
for IT issues [2],[10].
43. Partnership Rewards and Incentives
Mechanism that consists in giving rewards and incentives, such as financial rewards, to employees that follow organization’s strategy and contribute to the achievement of performance objectives [13],[52].
44. Shared Understanding of Business/IT Objectives
Mechanism that promote mutual understanding of business and IT objectives and plans by business and IT people and respect of each other’s contribution. Therefore, business and IT people can accurately interpret and anticipate actions and, if necessary, coordinate adaptively. This mechanism is considered a paramount for attaining and sustaining business/IT alignment [2],[52].
45. Senior Management Announcements
Senior management announcements clarifying priorities and demonstrating commitment usually get a great deal of attention throughout an organization [10].
46. Office of CIO or ITG IT Governance needs an owner to ensure that individual mechanisms reinforce rather than contradict one another and to communicate governance processes and purposes. This mechanism also needs to ensure alignment between IT governance and the governance of organization’s other key assets (financial, human, physical, IP and relationship) [10].
This interview is part of a Master Thesis about implementing COBIT 5 in Small and Medium
Enterprises and will be conducted by the student David Miguel Mendonça da Silva, under the
supervision of the professors Miguel Mira da Silva (IST) e Rúben Pereira (ISCTE).
The objective of this interview is to identify the fundamental mechanisms for the
implementation of effective Enterprise Governance of IT (EGIT) in Small and Medium
Enterprises (<250 employees) based on the experience and knowledge of professionals in
the area.
The questionnaire is divided in two sections:
1. Questions regarding your education level and personal experience with IT in Small and
Medium Enterprises.
2. Questions regarding the evaluation of EGIT Mechanisms.
General information:
• This study will be conducted with IT professionals who are knowledgeable about the
COBIT 5 framework.
• The interview time is approximately 30 min. Feel free to interrupt at any time. I would
like to record the interview with your consent and authorization.
• The purpose of this interview is solely academic, your personal information and your
organisation will be protected by confidentiality.
• The results of this research may be submitted to conferences and academic journals.
All information from the interviewee and the organization will be confidential.
• The results obtained will be shared with all those involved in the interviews.
84
• Personal Information
In the following questions, mark the correct option with X:
1. Age range:
A) [20-30] □; B) [30-40] □; C) [40-50] □; D) [50-60] □; E) [60+] □;
2. Education Level:
A) Specialist □; B) Bachelor □; C) Master □; D) Doctor □;
3. Position:
A) CIO □; B) IT Director □; C) IT Manager □;
D) IT Operational □; E) Other:
4. IT Experience (years):
5. Experience in Small and Medium Organization (<250 employees)(years):
6. Number of SMEs you worked for:
7. From your experience in SMEs, indicate your experience in each sector:
A) Public sector (years): B) Private sector (years):
8. If you are currently working in a small organization, please indicate:
A) Total number of employees: B) Total number of IT employees (including outsourcing):
85
• EGIT Mechanisms evaluation:
The following table contains an overarching list of EGIT mechanisms extracted from the literature. Evaluate each of these mechanisms (line) according to the following criteria (column): Effectiveness - defined as the extent to which the mechanism contributes to the attainment of IT-related goals and objectives. Ease of implementation - defined as the amount of time and effort required for implementation.
This evaluation will be based on the following numerical scales:
• What is the ease of implementation of a particular EGIT Mechanism? Rate between 0 and 5.
Evaluate with 0 (zero) if it is not easy to implement. Evaluate with 5 (five) if it is considered to be extremely easy to implement.
• What is the effectiveness of a particular EGIT Mechanism? Rate between 0 and 5. Evaluate with 0 (zero) if it is considered to be ineffective. Evaluate with five (5) if you think it is extremely effective.
86
Mechanisms Ease of
implementation Effectiveness
1. IT Strategy Committee
2. IT Audit Committee (at level of board of directors)
3. CIO on Board
4. CIO on Executive Committee
5. CIO reporting to CEO and/or COO
6. IT Steering Committee
7. IT Governance Function/Officer
8. Security / Compliance / Risk Officer
9. IT Project Steering Committee
10. IT Security Steering Committee
11. Architecture Steering Committee
12. Integration of Governance / Alignment Tasks in Roles and Responsibilities
13. IT Councils
14. IT Leadership Councils
15. Business/IT Relationship Managers
16. IT Investment Committee
17. IT Expertise at Level of Board
18. IT Organization Structure
18.1. Centralized
18.2. Decentralized
18.3. Federal
19. Strategic Information System Planning
20. IT Performance Measurement - IT Balanced Scorecard (BSC)
21. Portfolio Management
22. Chargeback
23. Service Level Agreement
24. IT Governance Frameworks/ Standards
25. IT Governance Assurance and Self-assessment
26. Project Governance / Management Methodologies
87
27. IT Budget Control and Reporting
28. Benefits Management and Reporting
29. Business/IT Alignment Model
30. IT Governance Maturity Models
31. Demand Management
32. Architectural exception process
33. Job-rotation
34. Business/IT Co-location
35. Cross-Training
36. Knowledge Management (On IT governance)
37. Business/IT Account Management
38. Senior management giving the good example
39. Informal Meetings between Business and IT Senior Management
40. IT Leadership
41. Organizational Internal Communication
42. IT Governance Awareness Campaigns
43. Partnership Rewards and Incentives
44. Shared Understanding of Business/IT Objectives
45. Senior Management Announcements
46. Office of CIO or ITG
If there are other mechanisms that are not present in this list, please list them:
88
Given your evaluation of EGIT Mechanisms, select the 10 minimum (baseline) mechanisms that you consider fundamental to effectively implement IT Governance in Small and Medium Enterprises. Enter the name of the mechanisms or the number that identifies it in the previous table.
o Correspondence 8: No component related to this mechanism was detected! - Do you know any component (process or structure) that is related with this type of
mechanism? If yes, which component?______________________________
o Correspondence 9:
1 2 3 4 5
93
Part 2 - The following questions will address the evaluation of certain characteristics regarding the artefact and its use in practice. Classify fom 1 to 5 using the scale presented in each question:
1. Do you agree that this mapping achieves its desired goal ?
3. Do you consider the presented mapping is internally consistent ? (the procedure is the same for each correspondence: each correspondence presents the same information, the level of detail is equal for every correspondence, etc.)
4. Classify the utility of this mapping between general EGIT Mechanism and COBIT 5
components if you were a practitioner that aims to start implementing effective EGIT in your Small and Medium Organization (SMEs) adopting COBIT 5 framework. (Would it be advantageous to know the fundamental mechanisms for SMEs and the correspondent COBIT 5 component that could support the implementation of those mechanisms?)
Scale: {1 = Not Useful, 2 = Little Useful, 3 = Moderately Useful, 4 = Useful, 5 = Very Useful} 1 2 3 4 5
5. Classify how easily can you identify which are the COBIT 5 components that support the implementation of a certain EGIT Mechanism.
