A Case Study on Consistency Management of Business and IT … · A Case Study on Consistency...

SoSyM Special Issue on Enterprise Modeling manuscript No.(will be inserted by the editor)

A Case Study on Consistency Management of Business and IT ProcessModels in Banking

Moises Castelo Branco1(�), Yingfei Xiong2,3(�), Krzysztof Czarnecki1, Jochen Kuster4, HagenVolzer4

1 Generative Software Development Laboratory, University of Waterloo, Canada2 Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, China3 Institute of Software, School of Electronics Engineering and Computer Science, Peking University, China4 IBM Research-Zurich, Switzerland

The date of receipt and acceptance will be inserted by the editor

Abstract Organizations that adopt process modelingoften maintain several co-existing models of the same bu-siness process. These models target different abstractionlevels and stakeholder perspectives. Maintaining consis-tency among these models has become a major chal-lenge for such organizations. Although several academicworks have discussed this challenge, little empirical in-vestigation exists on how people perform process modelconsistency management in practice. This paper aimsto address this lack by presenting an in-depth empir-ical study of a business-driven engineering process de-ployed at a large company in the banking sector. Weanalyzed more than 70 business process models devel-oped by the company, including their change history,with over 1000 change requests. We also interviewed 9business and IT practitioners and surveyed 23 such prac-titioners to understand concrete difficulties in consis-tency management, the rationales for the specification-to-implementation refinements found in the models, stra-tegies that the practitioners use to detect and fix incon-sistencies, and how tools could help with these tasks. Ourcontribution is a set of eight empirical findings, some ofwhich confirm or contradict previous works on processmodel consistency management found in the literature.The findings provide empirical evidence of 1) how busi-ness process models are created and maintained, includ-ing a set of recurrent patterns used to refine business-level process specifications into IT-level models; 2) whattypes of inconsistencies occur; how they are introduced;and what problems they cause; and 3) what stakeholdersexpect from tools to support consistency management.

Key words Business processes – consistency manage-ment – process refinement patterns – empirical study

1 Introduction

Business Process Modeling (BPM) is increasingly usedby enterprises to improve their agility and operationalperformance by better aligning their IT infrastructurewith their business needs. Typically, a BPM-driven sys-tem development process involves the participation andcollaboration of many stakeholders (e.g. Business An-alysts, Systems Analysts, IT Architects and Develop-ers). These roles and responsibilities may be organisa-tionally defined, be the result of the adopted develop-ment process, or simply reflect the different competen-cies and capabilities of the people involved. The distri-bution of responsibilities and roles usually results in thecreation of different models of the same business process.These models vary from business-oriented ones, whichare technology-independent and easily understandableby business people, to IT-oriented ones, which are con-structed by taking into consideration technicalities ofexisting systems. Specialized modeling languages havebeen developed to represent such models, including Busi-ness Process Modeling Notation (BPMN) [1] for business-level models and Web Services Business Process Execu-tion Language (BPEL) [2] for IT-level executable mod-els. Since its 2.0 version, BPMN can also express exe-cutable models [3].

The multitude and heterogeneity of models createdto describe a business process at different levels of ab-straction and from different stakeholder perspectives leadoften to inconsistencies among the models. Inconsisten-cies arise because the models overlap—for example, theycontain elements that refer to common aspects of sys-tems and other enterprise resources, such as organiza-tional structure and flow of communication, and makeassertions about these aspects that may be contradictoryunder certain conditions. On the positive side, inconsis-tencies highlight different perceptions and goals of the

2 M.C. Branco et al.

stakeholders involved in the development process andthey can be intentionally introduced to indicate aspectsof a process which deserve additional information elic-itation and further development. On the negative side,inconsistencies can cause development delays, increasedcosts, and operational and audit failures.

To manage consistency of multiple business processmodels, researchers have proposed different approaches[4–9], each targeting a sub-problem of consistency man-agement. In practice, companies also employ their ownprocesses to manage the consistency among multiple mod-els. It is not clear to what extent the academic approachesare adopted by the companies and what remaining chal-lenges are still faced by practitioners. Conversely, manyacademic approaches are based on assumptions of howmodels are handled in practice, and some assumptionsare even contradictory. For example, Zerguini [10] andSoffer [11] assume that models at different levels of ab-straction are related in a strict top-down fashion via hie-rarchical refinements, whereas Weidlich et al. [12] pro-pose that non-hierarchical refinements should also beconsidered. It is not clear which of these assumptionsare true in practice. Thus, we need empirical evidenceto support such claims about consistency managementof business process models.

This paper presents an in-depth empirical study onconsistency management of business process models ina large company in the banking sector. The study isdesigned to answer an initially broadly-scoped researchquestion: how do people manage consistency of relatedbusiness- and IT-level process models in practice?

The IT department of the company has more than300 employees and is responsible for more than 200 infor-mation systems, including BPM-supported ones. Morespecifically, we apply three research methods. We firstanalyse more than 70 models in five BPM projects andtheir change history involving more than 1000 changerequests, in order to understand the relation betweenthe models and how they evolve over time. The analyzedmodels include business-level ones written in BPMN andIT-level ones written in BPEL. Second, we interview 9professionals ranging from Business Analysts to IT De-velopers in order to understand how they collaboratedto create and maintain the models. Finally, we conducta survey with 23 professionals to further understand therelevant issues revealed by the artifacts and interviews.As a result, we elicit several research questions that werenot asked up front, but emerged during the study as weanalysed the artifacts, interviews and the survey (seeSection 4).

Our main contribution is a set of eight empirical find-ings (F). These findings emerged from investigating anddistilling eight research questions (RQ) derived duringthe study from our initial goal. The following list sum-marizes the research questions and the corresponding

findings, divided in three categories.

A. Modeling Methods

RQ1: What development process is used to create busi-ness- and IT-level process models? We describe the soft-ware development process used at the studied organiza-tion, including the types of process models created andtheir purpose and stakeholders. We also characterize themodels in terms of their sizes, language constructs used,and the type of evolution they undergo.

F1: Process models are created and maintained at sev-eral levels of abstraction (Section 5.1).

RQ2: How are business- and IT-level process models re-lated and how do they differ? We identify a set recur-rent patterns used by the developers to refine abstract,business-level models into more concrete, IT-level mod-els. We found instances of these patterns in the studiedmodels and developers confirmed them. The patterns re-flect the relationships between the business- and IT-levelmodels and provide evidence of both hierarchical andnon-hierarchical refinements.

F2: Business and IT process models are related by bothhierarchical and non-hierarchical refinement patterns (Sec-tion 5.2).

B. Consistency Issues

RQ3: How do business- and IT-level process models evolveover time? We provide evidence on how the models co-evolve. We analyze cases where the business-level processmodels lag behind the IT-level models and vice versa,and provide the main reasons behind these inconsisten-cies.

F3: Process models undergo parallel maintenance (Sec-tion 6.1).

RQ4: How do differences between business- and IT-levelprocess models affect consistency? We investigate differ-ent types of differences between business- and IT-level,such as as coverage, behavior, and level of detail, anddetermine which of these are most likely to cause incon-sistencies.

F4: Coverage and behavioral differences affect consis-tency most (Section 6.2).

RQ5: How do BPM stakeholders define consistency be-tween business- and IT-level process models? We sum-marize how different BPM stakeholders define consis-tency between business- and IT-level models.

F5: Stakeholders have a subjective notion of consistency(Section 6.3).

RQ6: Can inconsistencies cause problems in practice?We give two concrete examples of serious incidents caused

A Case Study on Consistency Management of Business and IT Process Models in Banking 3

by inconsistencies between business- and IT-level processmodels.

F6: Inconsistencies can cause severe problems (Section 6.4).

C. Tool Support

RQ7: Are the BPM stakeholders satisfied with the toolsupport for the development process they currently em-ploy? We discuss how stakeholders evaluate the currenttool support for developing process models, and polltheir opinion on different approaches to align business-and IT-level models.

F7: The majority of the surveyed stakeholders wouldprefer a single model for Business and IT (Section 7.1).

RQ8: How are inconsistencies dealt with? We provideevidence on how the stakeholders deal with inconsisten-cies in the studied company and what tool support theyexpect in helping them to better perform their tasks.

F8: Inconsistencies should be detected and communi-cated at the time they occur, along with proposed fixes(Section 7.2).

The remainder of the paper is structured as follows.Section 2 provides background on BPM and describesthe running example, the models of an Automated TellerMachine (ATM), which we use throughout the paper.Section 3 discusses related work on model consistencymanagement. Section 4 describes the empirical studydesign, presenting details about the organization, theanalyzed projects and artifacts, and the conducted in-terviews and survey. Section 5 presents the findings re-garding the process and modeling methods used by thestudied company, including the refinement patterns cat-alog. Section 6 presents the findings related to the issuescaused by inconsistencies. Section 7 presents the stake-holder expectations regarding the tools for consistencymanagement. Section 8 analyzes the threats to the va-lidity of the work. Finally, Section 9 summarizes the keyresults, lessons learned, and concludes the paper.

2 Business Process Modeling

A business process is a collection of related, structured orad-hoc activities (tasks) that produce a specific output,such as service or product, for a particular customer ormarket [13]. Structured processes, which our study fo-cuses on, are usually modeled as workflows, i.e., flowsof activities. Typical examples of business processes arePurchasing, Manufacturing, Marketing, and Sales. A bu-siness process begins with a mission objective and nor-mally ends with achievement of the objective. The ac-tivities of a process interact with IT assets to capture,transform, or report business data. As with processes,

the data may be structured, such as a new order con-forming to some well-defined schema, or ad hoc (unstruc-tured) data, such as an e-mail [14].

In practice, a range of business to IT-oriented stake-holders create and use business process models for spe-cific purposes, including requirements elicitation, docu-mentation, simulation, and execution [15]. Each modelmust be appropriate for its target audience and purpose—having adequate level of detail, focusing on relevant as-pects, and neglecting irrelevant ones [8]. This goal can beachieved by creating either several separate models, eachfocused on particular set of stakeholders and purposes,or a single model with multiple views [16].

Figure 1 shows three models, each representing theprocess of using an Automated Teller Machine (ATM)system at different level of abstraction. We will use thesemodels, which are versions of real process models fromone of the studied projects (project P4, Section 4.3),as our running example. The two specifications are thesame as the original models, except that it has its la-bels translated from Portuguese to English. Also, the ITmodel is translated from BPEL to BPMN. For check-ing consistency, we focus on the control flow of the pro-cess models. BPMN and BPEL control flow constructsare similar in the sense that each can be mapped intothe other, according to the OMG specification of BPMN2.0 [3]. The control flow of the original models was en-tirely preserved in these examples. Note that the originalmodels, as represented in their respective modeling tools,also have detailed information as attributes of nodes andflows, such as the communication protocols and the ad-dresses of the services used.

The first model (Figure 1.a) represents a business-level process specification, which is created and main-tained by Business Analysts. The second one (Figure 1.b)is a refinement of the first one, created and maintainedby IT Systems Analysts. These stakeholders use suchmodels to align the modeled process with the existingservice infrastructure; specify how the process interactswith IT assets; and ensure that the process is sound andfree of design flaws, such as incomplete data objects anddeadlocks. The third model (Figure 1.c), created by ITArchitects and Developers, refines the second model andrepresents the executable process implementation thatgoes into production. The executable process is imple-mented on top of an ISO8583 service infrastructure [17]and the codes that appear in the names of some tasks,such as 0200 and 9010, are types of messages of this pro-tocol. Note that the final refinement (Figure 1.c) con-sists of multiple, modularized executable models. Thesemodels orchestrate the actual services provided by theIT service infrastructure.

The models in Figure 1 are expressed in BPMN.The notation represents activities by rounded rectan-gles, events by circles, gateways by diamonds (rhombi),


and sequence flows by arrows (see Appendix for legend ofBPMN symbols used in the example). Each model has astart, usually modeled by an start event (e.g., Customerinsert Card into ATM ), and a flow of activities that isgoverned by decisions (e.g., Card is Valid? ) and excep-tions (e.g., 8s Timeout). Each model also has an endpoint, which represents the achievement of the process:either a value delivered to a user or the termination ofthe process because of an error or a user decision (e.g.,Cancel Transaction).

3 Related Work

We discuss related work in four groups. First, we dis-cuss the general problem of Business-IT alignment inBPM. Second, we introduce the general area of consis-tency management and discuss related work addressingspecific consistency management tasks. Third, we turnto work on consistency management of business processmodels. Finally, we review empirical work related to ourstudy. Throughout the discussion, we point out caseswhere our findings confirm or contradict some of the as-sumptions found in the related works, as summarized inTable 1.

Business-IT Alignment in BPM

Bridging the gap between business and IT abstractionlevels is a standard topic in enterprise systems engineer-ing [27, 28]. Bieberstein et al. define business-IT align-ment as “a dynamic state in which a business organiza-tion is able to use information technology (IT) effectivelyto achieve business objectives—typically improved finan-cial performance or marketplace competitiveness.” [15]

A key aspect of the business-IT alignment is estab-lishing a correlation between business specifications andIT implementations. We discuss some of the existing ap-proaches below. All approaches are based on assump-tions on how specifications and implementations are cre-ated and how traceability between them is establishedand managed. Some approaches deal with issues of trans-forming business processes into executable models.

Buchwald et al. [20] present an approach which allowsfor a transfer of business requirements into executableprocesses. Their approach provides a three-level model-ing method that automatically maintains an intermedi-ate model called Business-IT-Mapping Model (BIMM)to describe how activities from the business process aretransferred into activities of the system process. A BIMMmanages correspondences between model activities bymeans of transformation operations such as rename, in-sert, remove, merge, and split. The technique tackles the

problem of synchronizing parallel maintenance of differ-ent perspectives on the same business process (confirm-ing F3). A limitation of the approach is that it onlyconsiders consistency in terms of coverage, i.e., whetheror not corresponding business-relevant activities are cor-rectly mapped between the models. In practice, consis-tency involves other aspects, such as control flow (seeSection 6.2).

Tran et at. [29] present a modeling framework real-ized as a view-based reverse engineering tool-chain. Theframework maps process descriptions onto appropriatehigh-level or low-level views. The framework can be ex-tended with support for different modeling languages, in-cluding BPMN and BPEL. Although the approach sup-ports representing process structures at different levelsof abstraction, it does not support consistency manage-ment among these views when they are independentlyedited (see Section 6.1).

Delgado et al. [18] provide a methodology for in-cremental development of business processes, based onthe joint application of Model Driven Development andService Oriented Computing paradigms. Their proposedmethodology recognizes the need of integrating businessand IT people into the development life-cycle and con-veying the right level of detail as output of each de-velopment stage. Our work confirms such a need (F1),by providing evidence on how business and IT peoplecollaborate to create process models throughout the de-velopment process (see Section 5.1).

Decker [25] proposes patterns for introducing a pro-cess support layer that solves incompatibilities betweenbusiness- and IT-level process models. The work assumesthat a single process model for business and IT is in-herently undesirable (contradicting F7) and that bothperspectives are hierarchically related to each other.

Consistency management

Consistency management is a set of methods and toolsfor establishing and maintaining consistency among soft-ware artifacts, such as models, code, documentation, andtest cases, which are usually created and used by mul-tiple stakeholders [21, 30]. Existing works divide consis-tency management into a set of tasks [24, 30, 31]. Theremainder of this subsection introduces these tasks andthe corresponding related work in general; the next sub-section discusses the related work specific to BPM.

– Defining consistency properties: Assuming a setof software models and a set of correspondence rela-tions among their elements, consistency is a propertyof these models and their correspondences [32, 33].Such a property is typically defined as a consistency


Cust

om

er

Inse

rts C

ard

in

to A

TM

Identif

y

Cust

om

er

Card

Card

is

Valid

?

Cust

om

er

Sele

cts

Tra

nsa

ctio

n

Yes

Cust

om

er

Pro

vides

Tra

nsa

ctio

n

Deta

ils

Cust

om

er

Pro

vides

PIN

Valid

ate

PIN

PIN

is

Valid

?

Pro

cess

With

dra

w

Consu

lt B

ala

nce

Consu

lt

Sta

tem

ent

Attem

pts

<= 3

?

Yes

Cance

l

Tra

nsa

ctio

nN

o

No

Concl

ude

Tra

nsa

ctio

n

Yes

No

(a) Business Specification

Identif

y

Cust

om

er

Card

Card

is

Valid

?

Yes

Cust

om

er

Pro

vides

Tra

nsa

ctio

n

Deta

ils

Cust

om

er

Pro

vides

PIN

Due to

PIN

?

No

Pro

cess

With

dra

w

Consu

lt B

ala

nce

Consu

lt

Sta

tem

ent

Attem

pts

<= 3

?

Yes

Cust

om

er

Inse

rts

Card

into

AT

M

Tra

nsa

ctio

n

Cance

led b

y

No

No

Tra

nsa

ctio

n

Cance

led b

y

Yes

Tra

nsa

ctio

n

Auth

orize

d?

No

Concl

ude

Tra

nsa

ctio

n

Debit

Acc

ount

Pro

cess

Pendin

g D

ebit

Pro

cess

Pendin

g

Tra

nsa

ctio

nC

ance

l

Tra

nsa

ctio

n

Auth

orize

T

ransa

ctio

n

Cu

sto

me

rC

usto

me

r

(b) Technical Specification

Identif

y

Cust

om

er

Card

9300

Card

is

Valid

?

Yes

Cust

om

er

Pro

vides

Tra

nsa

ctio

n

Deta

ils

Cust

om

er

Pro

vides

PIN

Due to

PIN

?

.

No

Dis

pense

Cash

Print

Bala

nce

Print

Sta

tem

ent

Attem

pts

<= 3

?

Yes

Get C

ard

Identif

icatio

n9310

Initi

aliz

e

Tra

nsa

ctio

n

Para

mete

rs

Cust

om

er

Inse

rts

Card

into

AT

M

Tra

nsa

ctio

n

Cance

led b

y

Cance

l Tra

nsa

ctio

n

No

Dis

pla

y

Cance

ling

Mess

age

Cance

l Tra

nsa

ctio

n

Check

Tra

nsa

ctio

n

Table

Susp

ect

of F

raud?

Send S

ecu

rity

Notif

icatio

n

Yes

Update

Tra

nsa

ctio

n

Table

No

Tra

nsa

ctio

n

Cance

led b

y

Auth

orize

With

dra

w0200

Request

Bala

nce

9000

Request

Sta

tem

ent

9000

Cance

l

Cance

l

Get

Auth

oriza

tion

0210

Get

Bala

nce

9010

Get

Sta

tem

ent

9010 C

ance

l

Yes

Tra

nsa

ctio

n

Auth

orize

d?

Pro

cess

Tra

nsa

ctio

n

8s

Tim

eout

8s

Tim

eout

No

Concl

ude

Tra

nsa

ctio

n

Send

Com

ple

tion

Confir

matio

n

0202

Yes

Debit

Acc

ount

Add P

endin

g

Debit

Rebate

Fee

Charg

eN

o

Cu

sto

me

rC

usto

me

r

(c) Executable Process

Fig. 1: ATM Process Models

rule, expressed in some logic and interpreted in aknowledge domain. Knowledge domains range fromwell-formedness of language constructs to industry-

and organization-specific policies, such as legal reg-ulations and organization-specific IT standards [32].For example, a reasonable policy is to require that


Table 1: Summary of Findings

Confirms Contradicts Section

Modeling Methods

F1—Processes are developed and main-tained in several levels of abstraction

Delgadoet al. [18],Koehler etal. [9]

— Section 5.1

F2—Business and IT process models arerelated by both hierarchical and non-hierarchical refinement patterns

Weidlich etal. [12]

Zerguini [10],Soffer [11],Dijkman etal. [19]

Section 5.2

Consistency Issues

F3—Process models undergo parallel main-tenance

Buchwaldet al. [20],Weidlich etal. [12]

— Section 6.1

F4—Coverage and behavioral differencesaffect consistency most

Weidlich etal. [12]

— Section 6.2

F5—Stakeholders have a subjective notionof consistency

Spanoudakiset al. [21]

Weidlich etal. [12,22,23]

Section 6.3

F6—Inconsistencies can cause severe prob-lems

Spanoudakiset al. [21],Nuseibeh etal. [24]

— Section 6.4

Tool Support

F7—The majority of the surveyed stake-holders would prefer a single model for Bu-siness and IT

— Decker [25] Section 7.1

F8—Inconsistencies and fixes should bepresented as they occur

Hegedus etal. [26]

Weidlich etal. [23]

Section 7.2

every business-relevant task in an executable model(e.g., Identify Customer Card 9300 in Figure 1.c) isreflected in its business-level specification (IdentifyCustomer Card in Figure 1.a); conversely, a purelytechnical task (Initialize Transaction Parameters inFigure 1.c) should not be reflected in the specifica-tion.

– Matching the models: This task deals with findingcorrespondence relations among elements of differ-ent models. For example, Identify Customer Card inFigure 1.a corresponds to Identify Customer Card inFigure 1.b, and to both Identify Customer Card 9300and Get Card Identification 9310 in Figure 1.c. As wediscuss in Section 6.1, process model matching is of-ten challenging because identifying correspondencesmay require uncovering tacit knowledge, which maybe only in the heads of the original creators of themodels or may be lost entirely. Unless the correspon-dences have been recorded (e.g., via unique IDs), mo-del alignment requires matching the models usingdomain- or organization-specific heuristics (e.g., byname and model structure). Examples of approachesthat match different types of artifacts include docu-ment to code traceability recovery [34] and generic,graph-based matching [35]. A related area is schemaintegration, and in particular, schema matching, whichdeals with establishing correspondences among data-base schemas (see surveys on this topic [36,37]). It is

not clear how the existing techniques can be tailoredto the problem of aligning process models. Our workpresents evidence on how business and IT processmodels are related and how the maintenance processis done. This evidence will help developing appropri-ate alignment techniques.

– Checking consistency: Once the models are aligned,consistency is checked by evaluating the consistencyrules. Spanoudakis and Zisman distinguish four typesof approaches to consistency checking: logic-basedapproaches, model checking, specialized model anal-yses, and human-centered collaborative exploration[21]. The adopted consistency management policy isthus subjective (confirming F5) and specifies the cir-cumstances that will trigger the checks.

– Diagnosing causes of inconsistencies: This taskidentifies the source, the cause, and the impact ofan inconsistency [21]. The source of an inconsistencyis the set of elements of software models that vio-late a consistency rule [24]. The cause of an inconsis-tency could be conflicting stakeholder goals or justa mistake in one or more of the conflicting models.The impact of an inconsistency are the consequencesthat the inconsistency has on the modeled system.Spanoudakis and Zisman include a survey of diagno-sis approaches in their paper [21] and discuss nega-tive consequences of undetected inconsistencies in amodel-based system (confirming F6).


– Fixing inconsistencies: The final task is to fix in-consistencies. Ideally one or more fixes should be au-tomatically proposed to the user. For example, Nen-twich et al. [38] give approach that generates ab-stract fixes from first-order logic rules. An abstractfix specifies only the locations to be changed and theuser needs to complete the edits. Egyed et al. [39]present an approach that generates concrete fixesfor UML models, based on predefined inconsistencyrules. Ameluxen et al. [40] propose an approach inwhich models are checked and corrected using graphtransformation rules. Pinna et al. propose using anautomated planning system, which does not requiredefining operations manually [41]. Similarly, Xionget al. use constraint solving techniques to generatea list of fixes, and ensure them to satisfy a set ofproperties [42].

Consistency management of process models

We summarize work on consistency management in thecontext of BPM.

– Defining consistency properties: Weidlich et al.categorize differences among related process mod-els that can cause inconsistencies into the followingtypes [12]:– Model coverage differences are differences of what

the related models describe in terms of function-ality. For example, a particular task can exist inone model, but may be missing in the other.

– Behavioral differences are differences in how aparticular functionality is implemented in each ofthe models. For instance, the execution sequenceof corresponding tasks might differ.

– Information density differences are differences inthe level of detail. For example, one model mighthave two or more tasks that decompose a singlecorresponding task from another model.

We used and confirmed (F4) the above categoriesto investigate how they affect consistency (see Sec-tion 6.1). Behavioral consistency typically involvessome notion of behavioral equivalence, such as traceequivalence or bisimulation. For example, Kuster [43]provides a behavioral consistency notion for object-oriented behavioral models. In contrast, Weidlich etal. view the consistency of two process models as adegree of consistency rather than a strict binary cri-terion [12,22]. An example of such notion are behav-ioral profiles [23]; they replace strict criteria such astrace equivalence with less strict degree of trace sim-ilarity. They build on properties of free-choice Petrinets and give a numeric degree of consistency rangingfrom 0 (inconsistent models) to 1 (consistent mod-els).

– Matching the models: Effective matching tech-niques applied to business process models requireheuristics that are notation and application specific[44–46]. Discovery of effective heuristics usually re-quires studying the differences among such models.In this context, for example, Dijkman [7] presents an-other classification of frequently occurring differencesbetween similar business processes in general, suchchanging names and types of activities and modify-ing the flow structure. Zerguini [10], Soffer [11] andDijkman [19] present solutions for matching hierar-chically related process models. Our study providesan in-depth analysis of differences between processmodels targeting different levels of abstraction andshows that non-hierarchical correspondences need tobe taken into account (F2) (see Section 5.2. Basedon our findings, we have recently presented an al-gorithm to automatically detect correspondences be-tween BMPN process models across levels of abstrac-tion [47]. The algorithm has two phases and combineslexical and structural correspondences over ProcessStructure Trees (PSTs) [48] of the input models. Thefirst phase matches the PST nodes using region andmodel element matching criteria adapted from previ-ous work on matching ASTs [49]. The second phaseestablishes additional correspondences based on theposition of the nodes in the PSTs.

– Checking consistency: Checking consistency of bu-siness process models may involve checking simplestructural rules, such as that each business relevanttask in the executable models is reflected in the busi-ness level specification, or analyzing behavioral prop-erties using model checking or specialized algorithms(e.g., [23]). Two special representations of processmodels are used in model comparison: process struc-ture trees [50] and process model terms [51]. Thefirst representation represents the essential structureof processes as trees, allowing their easy matchingand structural comparison. The second representa-tion gives a canonical representation of process mod-els and allows efficiently checking for a particular re-laxed form of behavioural equivalence. Weidlich etal. [12, 22, 23] propose generic frameworks for check-ing consistency of process models, based on task or-dering. Our findings reveal that consistency checkingshould actually take into account subjective project-and domain-specific differences among the models(F5) (see Section 6.3).

– Diagnosing causes of inconsistencies: The pro-cess model differences classified by Weidlich et al. [12]represent potential causes of inconsistencies. Estab-lishing the actual root causes of the inconsistencies,such as the conflicting goals of stakeholders, usuallyrequires additional knowledge that is not present inthe models. We are not aware of any work investigat-ing how diagnosis of inconsistencies among processmodels is done in practice.


– Fixing inconsistencies: Our findings show thatstakeholders prefer immediate notification and edit-ing quick-fixes (F8), integrated to the modeling tools,instead of an offline approach (see Section 7.2). Kusteret al. present an approach to synchronize changesand fix inconsistencies on multiple process views viaa shared process model [52]. The approach supportsviews on different abstraction levels, i.e., business-and IT-oriented ones. Hegedus et al. recently pro-posed an approach to fix model inconsistencies basedon state-space exploration and evaluated it on BPMNmodels [26]. Kuster et al. also discuss the changemanagement and inconsistency resolution in BPM[53,54].

Empirical Research

We are not aware of any empirical research on consis-tency management in BPM, yet empirical studies existin related areas.

Hutchinson et al. [55] address the relative absenceof empirical studies of industrial model driven engineer-ing (MDE) practices by describing lessons learned fromthree case studies. They applied a combination of re-search methods, such as interviews and questionnairesurveys for collecting data and deriving lessons learnedfrom MDE practices adopted by three companies. Com-pared to their work which focuses on MDE in general,our work focuses on BPM and consistency management.

Zapf and Heinzl [56] present an empirical study ofprocess refinement patterns in the call center domain.They compare different process partitioning strategies astypical design patterns in call centers. The analysis pro-vides insight to the question under which circumstancesa specific pattern is used. Our study provides empiricalevidence of how process refinement patterns are appliedin the domain of banking applications.

4 Study Design

4.1 Methodology

The study was designed to answer the following, broadly-scoped research question:

How do people manage consistency of related business-and IT-level process models in practice?

We initially left our problem statement open so thatwe could discover which facts about this subject reallymatter to the practice of BPM. We also decided to firstfocus on understanding the emergent consistency man-agement process used at BNB, both in terms of the pre-scribed procedures and how the participants actually

perform the tasks, in the context of the overall devel-opment process.

To answer this question, we adopted a structuredcombination of three research methods: 1) artifact study,2) semi-structured interviews and 3) electronic survey.The combination allowed us to gradually refine our un-derstanding of how consistency is managed and to tri-angulate multiple sources to improve confidence in ourfindings. We now briefly summarize each of the methods.

First, we analyzed business-level and IT-level modelsto understand the correspondences between them. Wewere interested in discovering the degree to which thesemodels differ, the refinement patterns applied, and thetype of information represented in each model.

Second, we interviewed relevant stakeholders at thestudied organization to understand details about thedevelopment process, collaboration patterns among theprofessionals involved, reasons for applying the refine-ments we found, when and how the consistency amongthe models is maintained, and the challenges faced dur-ing consistency maintenance.

Third, based on the artifact analysis and the inter-views, we created an electronic survey with questions todisambiguate unclear points and to solidify our initialfindings. We collected responses to this survey from alarger set of stakeholder than those interviewed.

The following sections give more details about thestudied organization and the applied methods.

4.2 The Organization

The Bank of Northeast of Brazil (BNB) is a major fi-nancial institution in Brazil. It is controlled by the fed-eral government and oriented towards regional develop-ment. The Information Technology Area of the Bankcontains over 300 professionals, responsible for maintain-ing more than 200 information systems in operation.Joining to these numbers are five outsourced softwaredevelopment companies, adding up to a virtual work-force of 1500 professionals responsible for developmentand maintenance of these systems. The systems are de-veloped using a broad range of technologies, includingconventional mainframe transactions and web-based ser-vices. Since 2007, BNB has used Business Process Man-agement based on the WebSphere family of productsfrom IBM, including Business Modeler, Integration De-veloper, Business Monitor, and Process Server. The de-velopment process is based on the Rational Unified Pro-cess (RUP), extended to include business process mod-eling. The first version of the development process wascustomized by BNB with consulting provided by IBM.


4.3 Artifact analysis

We analyzed five BPM projects, containing more than 70models in total (see Table 2). The development processat BNB entails iterative and multi-staged model refine-ment, resulting in three types of models: business speci-fications, technical specifications, and executable imple-mentations (cf. Figure 1). Table 2 lists the number ofmodels of each type. It is important to mention thatthe project P1 was the first one developed at BNB (pi-lot project), and its initial development was conductedwith IBM consultancy. BNB took advantage of the pilotproject to create 23 generic and reusable IT level pro-cesses (services), e.g., for logging and auditing. As theybelong to P1, they count as implementation models inthis project. That explains the large number of imple-mentation models as part of this project (29). Table 3gives the model sizes in number of elements of differenttypes.

Table 2: BPM Projects

Number of Models

Project Domain Business Technical Implementation

P1 Customer Registration 2 2 29P2 Credit Backoffice 6 6 6P3 Credit Risk Assessment 2 2 4P4 ATM 1 1 3P5 Procurement 3 3 4

We analyzed the models by manually inspecting andidentifying corresponding elements and model fragments(typically single-entry and single-exit regions [57]) basedon names and structural similarity. The analysis reliedon the domain knowledge of the first author; we clarifiedany unclear cases with the creators of the models. As alast step, we classified the correspondences into recurringrefinement patterns presented in Section 5.2.

BNB manages the change of software artifacts us-ing two IBM products—ClearQuest (workflow of changerequests) and ClearCase (artifact repository). Businessemployees open change requests to the IT departmentusing ClearQuest. Every request has a unique ID, a tex-tual description and several parameters, such as prior-ity and nature of the change (e.g., legal, evolution). Re-quests follow a sequence of steps, for example to groupthem into projects (when applicable) before they ar-rive to IT. IT Managers assign IT professionals (ProjectManagers, Architects, Developers) to every request. IT

Table 3: Model Sizes

Number of Model ElementsPools Tasks Gateways Events Flows

P1Business Spec. 11 59 38 25 149Technical Spec. 11 78 46 36 164Implementation 11 123 56 43 186





technicians only can change artifacts in ClearCase byhaving an assigned change request. When artifacts arechanged, ClearCase stores the change request ID in thechange log.

We recovered the change log of all projects and alsothe textual descriptions associated with every changerequest (from the ClearQuest database). Our objectivewas to find the reasons for changing the artifacts in eachproject we analyzed. Our first step was matching the tex-tual description of each change request with the actualartifacts changed. The aim of this process was to discoverhow inconsistencies were introduced by regular mainte-nance. For example, by finding a particular change inAugust 2009 that had affected only the business modelof the project P1, we realized from the description ofthe request that this change had re-established the con-sistency between the business specification and the pro-duction process (implementation). A new project wasbeing started on the business side requiring an updatedspecification to build on. Then, we recorded any suchcases to clarify with the people involved. In total, wemanually inspected more than 1000 change requests, asshown in Table 4.

Ultimately, the artifact analysis gave us a rich evi-dence on how business and IT process models are relatedin BNB and what kinds of differences they display. Thisknowledge was used to further inquire the practitionersand understand the underlying reasons behind the dif-ferences.

4.4 Interviews

We used semi-structured in-depth interviews. The dura-tions ranged from one to three hours, and the interviewswere informal: although organized around a number of


Table 4: Change Requests

Project Change Requests Analyzed

P1 388P2 234P3 176P4 78P5 207

Total 1083

themes, we allowed each respondent to follow her owninterest. The themes ranged from respondent’s back-ground, current role and experience, to practical work-ing scenarios with BPM and personal feelings on howthe tools should be improved.

The interviewees’ roles were selected from those hav-ing personal responsibility in editing BPM models. AnIT Manager was also interviewed because of his expe-rience in several projects. These roles served as a rep-resentative sample of a larger population of profession-als who later answered the survey. Statistics about theroles involved, their experiences with BPM, and the in-terview durations are shown in Table 5. Section 5.1 pro-vides more details about the responsibilities of each roleand the artifacts they produce.

We created transcripts of each interview and submit-ted them for approval of the respondents. Subsequently,we classified and categorized recurrent facts mentionedin the interviews, such as what consistency aspects arerelevant; when and how inconsistencies are detected andfixed; and which tool support would help to performthese tasks. Sample questions asked are the following:

What is your current role? What types of tasks doyou perform? How much experience do you have withBPM?

What are the roles involved in creating and maintain-ing business- and IT-level models?

What tools and architecture- and company-specificguidelines and methodologies impact the content and formof these models?

What collaborations exist between the different roles?

How do different roles coordinate and communicatewhen they make changes?

Are there examples where inconsistencies were de-tected?

Are there examples where inconsistencies had unde-sirable consequences?

Table 5: Interviews

Interview Role Num. Projects Duration (h)

1 IT Systems Analyst 2 1:452 IT Systems Analyst 2 1:323 IT Systems Analyst 3 1:404 IT Manager 6 1:105 IT Architect 4 3:016 IT Developer 2 2:347 Business Analyst 4 1:258 IT Architect 12 2:109 IT Architect 8 1:52

Total 17:09

4.5 Survey

We created a questionnaire to strengthen our initial ana-lysis and also to disambiguate conflicting and overlap-ping facts from the interviews. For example, during theinterviews some respondents mentioned that task order-ing affects consistency, whereas others mentioned thatit may not be important. Then we included the follow-ing question in the survey: Corresponding tasks mustobey exactly the same relative order, and the respon-dents could chose between four answers: Necessary allthe times; Important, but not always; May be importantsometimes; and Irrelevant. We also added open fields,so that the respondents could provide comments and ex-amples supporting their answers. The questionnaire wasdivided into six groups of questions: Alignment of Bu-siness and IT Models, Tool Customization, Refinement,Change Management, Consistency Checking, and FixingActions. In total, 23 professionals answered it as a websurvey. Figure 2 shows the distribution of answers perprofessional role. Please refer to Section 5.1 for detailsabout the roles and the work that they perform.

It is important to mention that BPM is a relativelyrecently adopted technology in BNB. At the time of thisstudy, there were few BPM projects in production—theyinclude the ones we used in this study—plus 5 new pro-jects in early phases of development (i.e., the businessmodels were under discussion). Around 30 profession-als — including business and IT oriented ones — hadbeen conducting these projects. Our survey collected 23answers from this population, yielding a participationof 76%. The other professionals in the bank work withother technologies and programming languages, rangingfrom traditional mainframe to web-based platforms.

The complete report of the survey and the commentsmade by the respondents are available online at our website.1

From the survey and the data we collected in theprevious two phases, we found that our main researchquestion can be divided in the following sub-questions:

1 http://gsd.uwaterloo.ca/empiricalstudybpm

http://gsd.uwaterloo.ca/empiricalstudybpm


Business Analyst

IT Manager

IT Systems Analyst

IT Architect

IT Developer 2

7

4

3

7

Fig. 2: Survey Answers per Professional Role

1. What development process is used for creating business-and IT-level process models?

2. How business- and IT-level process models are relatedand how do they differ?

3. How do business- and IT-level process models evolveover time?

4. How do differences between business- and IT-levelprocess models affect consistency?

5. Can inconsistencies cause problems in practice?6. How do BPM stakeholders define consistency between

business- and IT-level process models?7. Are the BPM stakeholders satisfied with the develop-

ment process they currently employ?8. How are inconsistencies dealt with?

For each of this sub-questions (see Table 1), we distillour findings in the next sections.

5 Empirical Findings on Modeling Methods

5.1 F1—Processes are developed and maintained inseveral levels of abstraction

Summary The state of the art recognizes the need forspecialized models (or specialized views) in business pro-cess modeling (e.g., [9, 18]), such that specific needs ofthe stakeholders are respected in terms of concepts, mod-eling notation, and level of detail. Our work confirmssuch a need in the analyzed case study, by providing ev-idence on how process models are created, ranging frombusiness- to IT-oriented ones.

The development process adopted by BNB starts byBusiness Analysts producing a Business Specification(Figure 1.a) which focuses on the concepts and rules rel-evant to the business level. The business specification

is refined by IT Systems Analysts to create a Techni-cal Specification (Figure 1.b). The technical specifica-tion has two objectives: a) to ensure that the process issound and free of design flaws, such as incomplete dataobjects, deadlocks, and lack of synchronization; and b)to adapt the specification to the existing service infras-tructure, making it clear and understandable to devel-opers and outsourcers. The business and technical spec-ifications are written in BPMN. The technical specifi-cation is subsequently refined by IT Architects and ITDevelopers to implement the executable process (Fig-ure 1.c). Executable processes are written in BPEL. Nat-urally, several other artifacts are part of the develop-ment process, for instance, glossaries, requirement docu-ments, use cases, architecture documents, business rulesdescriptions, and test cases. IT Managers are also in-volved in negotiating deadlines, assigning IT profession-als, and contracting external services and manhours. Be-low are descriptions of the main roles involved in devel-oping a BPM project in BNB:

– Business Analyst: Define and simulate the businessprocess in terms of organizational structure (lanes,pools), business items (information to flow), resources(e.g., people who interact with the process), tasks(human and automated), business rules and Key Per-formance Indicators (time, costs, etc.). The businessprocess is created in BPMN. Business rules and usecases to each business task are specified in naturallanguage in associated specification documents.

– IT Manager: Produce contracts for meeting the bu-siness requests. Assign IT personnel to projects andcontract outsourcers.

– IT Systems Analyst: Provide technical support forBusiness Analysts; correct and adjust the BPMN mo-del; clarify business items and rules; detail tasks andflows; specify Use Cases for each task, gateway, con-ditional flow, and event.

– IT Architect: Create a BPEL model out of the BPMN.Refine the BPEL. Describe service interfaces, inte-gration methods (queue manager, message broker,service bus), design human tasks, produce an Archi-tecture Document, Technical Use Cases, Design Mod-els, and Deployment Plan.

– IT Developer: Produce code (BPEL, Java, other lan-guages). Create testable builds.

The consequence of this development process is thatthree different process models for each project are cre-ated and maintained. This is considered suitable by BNBto effectively separate concerns and to convey the rightinformation to diverse stakeholders. The common usecases for consistency management throughout the devel-opment process are the following:

– Change propagation: By applying a development pro-cess based on RUP, requirements are created or up-


dated by carrying out the business modeling disci-pline. Business-level process specifications are upda-ted and the changes should be propagated across re-lated IT-level models. Similarly, due to incident reso-lution or time constraints it is possible that a processrunning into production is modified before updatingits specification. Later the specification needs to beupdated.

– Validation: Audits often require checking productionprocesses against high-level specifications and controlpoints of legal reference models, such as Basel II andSarbanes-Oxley.

It is important to note that the actual workflow isdefined by the business-level model together with busi-ness rules. In particular, detailed cases, such as whenwithdrawals are authorized, are specified in the businessrules document, and might not be visible in the work-flow of the process model. These points are more effec-tively captured as rules, and adding them to the diagramwould result in visual clutter. The business-level modelis intended to give an overall, high-level flow, and thestakeholders know that they also need to review the bu-siness rules document in addition to the process models,in order to cover all the business-relevant details.

The following quote provides a summary of the de-velopment process obtained in an interview from an ITSystems Analyst :

“The development is done in several iterations foraccomplishing the project milestones. This is managedby the project manager following the same methodologyused for any other software project. The objective of theinception phase is to clarify what should be done, then allthe requirements should be clear at the end of this phase.Most of the collaboration is performed by business ana-lysts and system analysts, although the architect is alsoinvolved in some meetings to anticipate possible integra-tion issues, such as data replications and unavailable ser-vices or application components. The artifacts discussedin the inception phase are mainly BPMN models, usecases for tasks, and business rules. In the elaborationphase the objective is to eliminate all the architecturalrisks and know how the project should be implemented.The main artifacts are the integration model (BPEL),the architecture document and the technical use cases.Most of the collaboration is done by the architect and thedevelopers. Systems analysts still collaborate with archi-tects and developers in the elaboration and constructionphases when a business rule or a use case is not wellunderstood.”

In the rest we will focus only on the business spec-ification and the IT implementation and refer them asbusiness (level) model and IT (level) model, respectively.

5.2 F2—Business and IT process models are related byboth hierarchical and non-hierarchical refinementpatterns

Summary Existing works argue the need for non-hie-rarchical refinements when deriving IT process modelsfrom their business specifications (e.g., [12]), while otherworks propose the transition from business to IT processmodels in a strictly hierarchical fashion (e.g., [10,11,19]).Our study provides evidence of the need for both typesof refinements (hierarchical and non-hierarchical).

Using the ATM case study (P4 )(see Section 2) asrunning-example, we now present the refinement pat-terns we identified. We chose P4 as illustration becauseit is the smallest one and it also contains concrete in-stances of all the patterns we found in the other pro-jects. Although the executable model is implemented inBPEL, for simplicity, we remodeled it in BPMN 2.0 [3],preserving the salient refinements patterns applied in thereal project. Naturally, mismatches that stem from usingdifferent languages pose further complications; however,the problem of managing consistency of related processmodels is generic and independent of any specific lan-guage [12].

Table 6 shows statistics of the pattern occurrencesacross the models.

Table 6: Refinement Occurrences

Occurrences

Refinement Pattern P1 P2 P3 P4 P5

Add properties 27 32 12 8 6Add script task 21 13 4 1 4Add protocol task 31 16 2 5 4Add boundary event 34 9 9 6 6Add technical exception flow 15 14 3 2 4Change activity name 14 5 2 3 2Change activity type 12 3 11 4 2Refactor gateway 6 8 - 1 3Split task into block 28 24 4 1 2Split workflow 25 3 4 5 4Suppress specification activity 11 7 5 1 6

Add properties

Description Parameters for grounding the executablemodel on top of the underlying IT infrastructure areadded during the implementation.

Motivation Several properties of tasks, gateways, flows,events, etc., are added to the implementation-level mo-del, such as application or service URLs, protocol types


(e.g., http or https), transactional behavior (e.g., com-mit before, commit after, participates, etc.). Such prop-erties do not change the workflow and may be tool orplatform-specific.

Example Each ISO8583 sending or receiving task shownin Figure 1 (e.g., Identify Customer Card 9300 and GetCard Identification 9310 ) has parameters that includethe message queue, authentication method, security pro-tocol, and message encoding.

Add script task

Description Script tasks are used to initialize variablesand implement business rules and non-functional require-ments that access or transform business objects data,e.g., logging steps of the workflow.

Motivation This type of task is frequently used becauseit has significantly better performance than calling ex-ternal services.

Example Figure 3 shows a task created in the ATM ap-plication for initializing several parameters of a transac-tion object, which controls user actions across the work-flow. Such kind of task in the IT model does not haveany correspondence in the business model.

Initialize

Transaction

Parameters

(a) Executable

Fig. 3: Add Script Task

Add protocol task

Description An asynchronous service can be imple-mented by a connection-less request or reply protocol.

Motivation It is common to implement a business taskby using an asynchronous connection-less service. In suchcases, the protocol needs to compose and send a messageand, after that, wait for a response.

Example Figure 4 shows an example where the busi-ness task Identify Customer Card is implemented on topof the ISO8583 protocol by sending a identification re-quest message (9300) and waiting for a validation mes-sage (9310).

Identify

Customer Card

(a) Business and Technical Specifications

Identify

Customer Card9300

Get Card

Identification9310

(b) Executable

Fig. 4: Add Protocol Task

Add boundary event

Description Boundary events are used to divert thenormal flow under special conditions, for example, be-cause of a particular action performed by the operatoron a human task.

Motivation The reason to divert the flow can be merelytechnical or too low-level to be represented in the busi-ness model. Such conditions can be implemented as re-sult of requirements and use cases that describe a humantask in detail.

Example Figure 5 depicts an example of boundary eventadded to human tasks to capture the customer’s decisionto cancel the transaction at any time. Another examplecan be seen in Figure 1, where boundary events wereadded to asynchronous receiving tasks (e.g. Get State-ment 9010 ) to cancel the transaction in the case of atimeout of 8s.

Customer

Provides

Transaction

Details


Customer

Provides

Transaction

Details

Transaction

Canceled by

Customer

(b) Technical and Executable

Fig. 5: Add Boundary Event

Add technical exception flow

Description Technical exception flows are included todivert the flow in case of technical exceptions, such asan unavailable service or a permission denied.


Motivation Technical exceptions are not expected tobe represented in the business model, because they im-plement non-functional requirements elicited during theelaboration phase of the development process.

Example Figure 6 shows examples of technical excep-tions flows added for dealing with service errors, in whichthe transaction parameters are saved and the system ad-ministrator is notified to complete the transaction later.

Consult

Balance


Consult

Balance

Print

Balance


Fig. 6: Add Technical Exception Flow

Change activity name

Description The name of a business activity can bechanged to facilitate the identification of an IT servicethat has a similar but different name.

Motivation IT specialists can decide to use technicalnames in model elements for facilitating maintenance.

Example Figure 7 shows an example.

Consult

Statement


Print

Statement

(b) Executable

Fig. 7: Change Activity Name

Change activity type

Description The type of a model element can be changedbecause of an implementation decision.

Motivation It is easier for business people to stick withbasic modeling constructs (such as plain tasks and gate-ways), while other types of model elements are more suit-able to implement the business intent.

Example Figure 8 shows an example were a human taskrepresented in the business model was implemented byan event.

Customer

Inserts Card

into ATM


Customer Inserts

Card into ATM


Fig. 8: Change Activity Type

Suppress specification activity

Description Business elements can be suppressed dur-ing the implementation.

Motivation Some elements of the business specifica-tion may be considered redundant, not subject to au-tomation, or subsumed by a particular task at the im-plementation level. Typical examples for applying thisrefinement pattern are:

– Combine several business tasks into a single servicecall (the service provided is coarser than the businesssteps described),

– Combine human tasks into a single human task, withthe separate steps of the human task being describedelsewhere as a screenflow, for example.

– Ignore manual business tasks, for example, “Sendcontract to the post office.”

Example Figure 9 shows a case where the two humantasks described in the business model were collapsed intoa single human task in the technical and implementationlevels.

Split task into block

Description A single business task can be implementedby a combination of services.


Customer

Selects

Transaction

Customer

Provides

Transaction

Details


Customer

Provides

Transaction

Details


Fig. 9: Suppress Specification Activity

Motivation To implement a specification task, it maybe necessary to combine several existing services, includ-ing additional control flow logic to organize the way theservices should be called to achieve the specified func-tionality.

Example Figure 10 illustrates such scenario, where atechnical specification task, Authorize Transaction, issplit into a block of ISO8583 service calls, organized asan exclusive gateway that controls the type of authoriza-tion required for each transaction type.

Authorize

Transaction

(a) Technical Specification

Authorize

Withdraw0200

Request

Balance

9000

Request

Statement9000

Get

Authorization0210

Get

Balance

9010

Get

Statement9010

(b) Executable

Fig. 10: Split Task into Block

Split workflow

Description The specification workflow can be splitinto smaller workflows that should be orchestrated bya main flow.

Motivation The typical reason for this pattern is thecreation of specialized and reusable workflows, such asfor logging and auditing purposes.

Example In Figure 11, the task Cancel Transaction wasimplemented by a specialized subflow that includes fraudcontrol and is reused by other projects. It is common touse web service interfaces or event triggering for callingthe subflows.

Cancel

Transaction


Cancel Transaction

Check

Transaction Table

Suspect of Fraud?

Send Security

Notification

Yes

Update

Transaction Table

No

(b) Executable

Fig. 11: Split Workflow

Refactor gateway

Description A business level gateway may need to berefined to take into account the technical behavior of theservices involved.

Motivation IT services may impose constraints on thecontrol flow. For example, the business model may spec-ify tasks executing in parallel; however, in the imple-mentation the corresponding IT services are called insequence to avoid deadlocks.

Example Figure 12 shows an example where the busi-ness specification has a rule for checking the maximumnumber of times that a customer can enter a wrong PIN.In the actual implementation, checking the validity ofthe PIN is a particular result of the transaction autho-rization. In this particular project, some of the othercases where the transaction is not authorized are alsorelevant to the business (e.g., insufficient funding). How-ever, since the business analysts did not know how thesystems were implemented, they specified such cases aspart of business rules of three business tasks: ProcessWithdraw, Consult Balance and Consult Statement. Bu-siness rules documents are produced together with busi-ness process models (see Section 5.1). The business an-alysts did not consider necessary to change the businessmodel to approximate it to the actual system, at whichpoint the workflows became different.


PIN is

Valid?

Yes

No


Due to

PIN?

Yes

Transaction

Authorized?

No

Yes

No


Fig. 12: Refactor Gateway

Hierarchical and non-hierarchical refinements

A refinement pattern is hierarchical if it is possible tofit the refined model elements into a collapsed subpro-cess that preserves the original number of incoming andoutgoing sequence flows, otherwise it is non-hierarchical.The pattern Split task into block (see Figure 10) is an ex-ample of hierarchical refinement whereas Refactor gate-way (see Figure 12) is an example of non-hierarchicalone. Other examples of non-hierarchical refinements canbe seen in Figure 1, where flows were added to divertthe main workflow in case of timeouts. It is important tonote that this specific example of refactoring a gatewayis non-hierarchical, but not all instances of the patternare such—for example, refining a parallel flow into a se-quence would be hierarchical.

Interestingly, several approaches for aligning businessand IT perspectives are based on the assumption thathierarchical refinements are sufficient in practice [10,11,19]. Our case study clearly shows that non-hierarchicalrefinements occur and are relevant in practice. One couldargue that non-hierarchical refinements are present inthe case study since the tools used there have not en-forced hierarchical refinements, as in, for example, theapproach by Dijkman et al. [19]. However, the major-ity of surveyed stakeholders express the need to sup-port both hierarchical and non-hierarchical refinementsin practice, as shown in Table 7.

We believe though that settling the question of whetherhierarchical transformation can be expressive enough tocreate models for different perspectives requires furtherresearch.

Table 7: Refinement Patterns Needed by Stakeholder

Strictlyhierar-chical

Any typeof refine-ment

Role does notneed to applyrefinements

Business Analyst 13% 87% 0%IT Systems Analyst 13% 87% 0%IT Architect 9% 91% 0%IT Developer 9% 78% 13%

6 Empirical Findings on Consistency Issues

6.1 F3—Process models undergo parallel maintenance

Summary Existing works recognize that process mod-eling needs support for parallel maintenance of modelsor views that target different levels of abstraction andstakeholder perspectives (e.g., [20]). Our study providesevidence on such a need, by analyzing the change his-tory of five real-world projects. The histories cover boththe development prior production and also changes af-ter the projects went into production. Their analysisshows that, while the majority of changes affect onlyartifacts other than the related business- and IT-levelmodels (i.e., use case descriptions, databases, services,and application components), about 10% of changes af-fect both models simultaneously, about 20% affect onlyIT-level models, and about 5% affect only business-levelmodels. The number of changes varies according to theprojects’ life-cycles, with the majority of business-model-only changes occurring at project start. The analysisalso revealed (i) cases where the business models werechanged in response to earlier IT model changes, in or-der to eliminate inconsistencies considered as undesiredby the stakeholders; and (ii) cases where changes firstspecified in the business models where later implementedin the IT models; these cases were viewed as acceptable“controlled inconsistencies” by the stakeholders.

As announced in Section 4, we report on a substan-tial dataset featuring in total 5 BPM projects, 74 mod-els (business and IT ones) and more than 1000 changesmade on these models throughout their life-cycles. Aproject at BNB contains the versions and baselines ofall the artifacts of a system, including models, textualdocuments—such as use cases and business rules—andsource code. We inspected the change history of eachproject to identify when inconsistencies were introducedby day-to-day maintenance and when they were foundand fixed.

We classified each change request with respect to themodel types being affected. For example, Only Businessmeans that a request has changed solely the businessmodel but not the IT model, whereas None means thatthe request has changed neither the business model northe IT model (but other resources, such as databases,services, and application components). The changes to


the artifacts ranged from adding or modifying a singlemodel element (e.g., a task or flow) to applying multiplepatterns in multiple places.

Figure 13 shows the distribution of the changes pertype in each project and Table 8 shows the correspondingpercentage numbers.

0

100

200

300

P1 P2 P3 P4 P5

Num

ber

of C

hang

es

Only Business Only IT Both None

Fig. 13: Distribution of Changes per Type

Table 8: Percentage of Changes per Project

Change Type

Project Only Business Only IT Both None

P1 2% 24% 10% 64%P2 3% 22% 11% 64%P3 4% 23% 9% 64%P4 9% 24% 13% 54%P5 3% 13% 3% 81%

Figure 14 shows the temporal distribution of 388changes made on project P1 throughout its three firstyears. Figure 15 shows the first-year stacked distributionof changes for the other projects.

The analysis of the change history revealed that in-consistencies were introduced mainly due to parallel main-tenance. The following two cases summarize situationswhere the inconsistencies were detected and fixed:

– Case 1 : Update only the business model because atleast one previous maintenance request that shouldhave affected both the business and the IT modelhas been made only in the IT model. This is consid-ered undesirable inconsistency by the stakeholders,

and the business model is being updated to addressit. Audits often motivate this type of maintenance.Another reason is when a new project requires anaccurate business-level model for AS-IS analysis.

– Case 2 : Update only the IT model to reflect, e.g.,a planned process optimization that has previouslybeen made only in the business model. This is con-sidered a controlled inconsistency by the stakehold-ers.

For project P1 we identified changes made only inthe business model because of Case 1 in January 2009and August 2009. Also, we identified requests becauseof Case 1 in projects P2 and P5 in March 2010 andJuly 2010, respectively. In project P3, we identified aprocess optimization made initially in the business modelbecause of Case 2 in May 2009.

Some stakeholders complain that the current toolsupport to plan and manage future changes (controlledinconsistencies) is deficient, although they mitigate theissue by using resources of the artifact repository. Themain complaint is the lack of tool features for easily com-paring, differencing, and merging process models, as theyare widely available for textual source code.

The stakeholders also complain about the tool sup-port for managing traceability and correspondence linksamong this multiplicity of models. This is particularlycritical when specifications are updated and given to out-sourcers. From time to time, the correspondences needto be reestablished and described using textual artifactsand model annotations, which is time-consuming whenmaintained manually. Another important aspect of cor-respondence links among process models is that theyare domain- and project-specific. For example, we foundcorrespondences that can be understood only by havingknowledge of existing systems used in BNB. Then, au-tomatic techniques for deriving correspondences shouldhave means to include specific domain knowledge as partof the matching methods.

Our analysis confirms the challenges of establishingcorrespondences among process models identified in [12].Since establishing correspondences may require uncover-ing tacit knowledge—present only in the original modelcreators’ heads or lost entirely—, a fully automatic ap-proach with high recall does not seem realistic. More re-search is necessary to understand the trade-offs betweenautomatic and manual efforts to establish and managecorrespondence links, integrated into the developmentprocess.

The following quote was made by an IT Systems An-alyst :

“The main problem with our BPM development ismaintaining traceability among such models and artifactsover time - this often requires considerable rework and


Jan/2

008

Feb/2008

Mar/

2008

Apr/2008

May/2

008

Jun/2

008

Jul/2

008

Aug/2008

Sep/2008

Oct

/2008

Nov/2008

Dec/

2008

Jan/2

009

Feb/2009

Mar/

2009

Apr/2009

May/2

009

Jun/2

009

Jul/2

009

Aug/2009

Sep/2009

Oct

/2009

Nov/2009

Dec/

2009

Jan/2

010

Feb/2010

Mar/

2010

Apr/2010

May/2

010

Jun/2

010

Jul/2

010

Aug/2010

Sep/2010

Oct

/2010

Nov/2010

Dec/

2010

0123456789

101112131415

Nu

mb

er

of

Ch

an

ge

s

Only BusinessOnly ITBothNone

Fig. 14: P1 Change History

Num

ber

of C

hang

es

0

5

10

15

20

May

/200

9

Jun/

2009

Jul/2

009

Aug/2

009

Sep/2

009

Oct/20

09

Nov/2

009

Dec/2

009

Jan/

2010

Feb/

2010

Mar

/201

0

Apr/2

010


(a) P2

Num

ber

of C

hang

es

0

5

10

15

Oct/20

08

Nov/2

008

Dec/2

008

Jan/

2009

Feb/

2009

Mar

/200

9

Apr/2

009

May

/200

9

Jun/

2009

Jul/2

009

Aug/2

009

Sep/2

009


(b) P3

Num

ber

of C

hang

es

0

5

10

15

Apr/2

009

May

/200

9

Jun/

2009

Jul/2

009

Aug/2

009

Sep/2

009

Oct/20

09

Nov/2

009

Dec/2

009

Jan/

2010

Feb/

2010

Mar

/201

0


(c) P4

Num

ber

of C

hang

es

0

5

10

15

Nov/2

009

Dec/2

009

Jan/

2010

Feb/

2010

Mar

/201

0

Apr/2

010

May

/201

0

Jun/

2010

Jul/2

010

Aug/2

010

Sep/2

010

Oct/20

10


(d) P5

Fig. 15: First Year Change History

is specially critical when outsourcers are involved. I saythat we could have much better tool support for managingthis.”


6.2 F4—Coverage and behavioral differences affectconsistency most

Summary The state-of-the art discusses types of differ-ences among process models that may affect the consis-tency among them [12]. Our work provides results of asurvey where real practitioners were asked to evaluate towhat extent such types of differences impact their notionof consistency.

From the artifact analysis, we prepared examples ofthe three types of model differences defined in [12] (brieflyexplained in Section 3) and asked the respondents to an-swer two questions:

Please indicate to what extent the following types ofdifferences affect the notion of consistency between Bu-siness and IT models; and

Please indicate to what extent the following types ofdifferences may be tolerated or ignored when checkingconsistency between Business and IT models.

The answers to these two questions are shown inTable 9 and Table 10, respectively. We also asked therespondents to rank a set of consistency aspects thatwere frequently mentioned in the interviews. The resultsare shown in Table 11. 86% of the respondents supportthat a difference in coverage always affects consistency,68% support that a difference in behaviour sometimesaffects consistency, and 68% support that a differencein density does not affect consistency. 74% support thatcorresponding tasks between the models must obey thesame relative order. We also collected open answers fromthe respondents explaining their understanding of thesetypes of differences.

Table 9: How Differences Affect Consistency

Type of Mismatch AlwaysAffects

SometimesAffects

Does not Af-fect

Coverage (There is a differencebetween WHAT is modeled.)

86% 14% 0%

Behavior (There is a differencein HOW a certain scenario is im-plemented.)

14% 68% 18%

Density (There is a differ-ence in the LEVEL OF DE-TAIL a certain scenario is im-plemented.)

0% 32% 68%

Table 10: How Differences are Tolerated

Type of Mismatch NeverTolerated

SometimesTolerated

AlwaysTolerated

Coverage (There is a differencebetween WHAT is modeled.)

50% 50% 0%

Behaviour (There is a differ-ence in HOW a certain scenariois implemented.)

14% 77% 9%

Density (There is a differ-ence in the LEVEL OF DE-TAIL a certain scenario is im-plemented.)

0% 59% 41%

Table 11: Consistency Aspects Mentioned in theInterviews

Consistency Aspect Necessaryall thetimes

Important butnot always

May beimportantsome-times

Irrelevant

Corresponding modelelements have thesame names

30% 70% 0% 0%

Corresponding tasksmust obey the samerelative order

74% 26% 0% 0%

Corresponding taskshave the same types(service, human etc.)

22% 61% 13% 4%

Corresponding gate-ways have the samenumber of incomingand outgoing flows

9% 52% 30% 9%

Corresponding busi-ness objects must haveexactly the same fields

13% 52% 30% 4%

Every task in the bu-siness model has atleast one correspond-ing task in the IT mo-del

9% 70% 22% 0%

Every gateway in thebusiness model has atleast one correspond-ing gateway in the ITmodel

13% 70% 17% 0%

Every event in the bu-siness model has atleast one correspond-ing event in the IT mo-del

9% 70% 22% 0%

Our analysis leads us to propose the notion of Bu-siness Relevance, which seems to be crucial wheneverstakeholders check consistency. If a mismatch is consid-ered relevant to the business it should be fixed, other-wise it is ignored. Although this definition is subjective,we noticed that typically differences that are consideredtechnical details of implementation are ignored. For ex-ample, Figure 3 shows a case of Coverage mismatch thatis not business relevant: the added script task is essen-tially a detail of implementation and does not have anycorrespondence in the business-level model. Similarly,Figure 6 and Figure 10 show respectively examples ofBehaviour and Density differences that are also not con-


sidered business relevant: both are details of implemen-tation.

The results confirm the postulates presented in [12]on how differences affect consistency. We extend the pos-tulates by adding the concept of business relevance, whichcan be summarized as follows:

– Difference in Coverage is what most affects consis-tency, as long as it is business relevant.

– Difference in Behavior is relevant when it affects taskordering.

– Difference in Density does not seem to affect consis-tency. It is generally considered an implementationdetail and thus not relevant to business.

We believe that more investigation is necessary tounderstand how one can allow the stakeholders to de-fine and manage these types of differences individually ineach project. The same type of mismatch can be consid-ered to affect or not to affect consistency, depending onits business relevance. This is why the answers to theseprevious two questions were subjective, consistently withresults from the interviews, as we discuss in Section 6.3.

6.3 F5—Stakeholders have a subjective notion ofconsistency

Summary Although the surveyed practitioners are ableto assess to what extent certain types of differences amongprocess models affect a notion of consistency, they donot have a definitive agreement on its meaning. Exist-ing works confirm the subjective nature of consistencyin software modeling (e.g., [21]), while other works pro-pose the use of consistency measures in terms of tracesimilarity (e.g., [23]). Our work concludes that more re-search is necessary to understand how the stakeholderscould manage their own concepts of consistency, whichmay vary depending on the role and project.

We asked specific questions in the interviews aimingto understand how the BPM practitioners would defineconsistency among process models; how they realign in-consistent process models; and how they decide when themodels are consistent enough. The following quotes arerepresentative of what we obtained regarding this point:

IT Systems Analyst:“This task is somehow subjectiveand the criteria of consistency may vary from personto person, but I would say that it is not hard or com-plex to be performed manually. At the end of the daywe always achieve a good understanding of possible ad-justments that should be made in the models in order toregain their consistency. Anyway, I would say that welack better tool support for doing this task: today we ba-sically print the models (or some parts), stick them on

the wall and visually inspect them together. It would bebetter having some online support for checking consis-tency, for example, whenever a potential inconsistencyis detected, the tool could highlight that and the modelercould take an immediate action.”

IT Architect:“It is not technically complex to reestab-lish consistency of business and IT models, although it isoften laborious and time-consuming: we need to do it byvisual inspection. It is not really complex because it is nota very strict thing; for example, we do not really need tocompute all possible traces - this level of consistency isoften too much. In general, some discrepancies in termsof traces may be considered important to be adjusted andothers can be just ignored.”

We interpret the practical evidence as confirmationthat differences between the models and potential in-consistencies are the inevitable result of the need to de-scribe complex systems from different perspectives, todistribute responsibilities to different stakeholders in thesoftware development life cycle, and to allow them towork independently without requiring a continuous rec-onciliation of their models and views for, at least, certainperiods of time. This is why stakeholders do not have adefinitive notion of consistency and think that at theend of the day they are always capable of regaining an‘agreement’ with respect to a consistency level, which isnot really ‘strict’. The benefit of working collaborativelywith process models indicates that inconsistencies needto be “managed”, that is detected, analysed, recordedand possibly resolved [21].

Interestingly, some works accept that it is hard forpeople to agree on the meaning of consistency betweenbusiness and IT models [12,22]. A more recent work fromWeidlich et al. [23] considers process model consistencyas a degree measure in terms of trace similarity, ratherthan a binary (true or false) criterion. Although this per-spective is more consistent with our finding, we did notsee a clear trend towards defining consistency in termsof trace similarity in the case study. In particular, asdiscussed in Section 7.2, the majority of business and ITstakeholders prefer to see concrete model differences af-fecting consistency rather than metric values describingthe degree of consistency.

As we started discussing in Section 6.2, we believethat more research is necessary to understand how thestakeholders could manage their own concepts of consis-tency. A potential way to investigate is to keep track ofthe correspondences among business-relevant model el-ements or fragments. Whenever a change affect such abusiness-relevant correspondence, the stakeholders shouldbe notified to decide whether to fix or not potential is-sues. Taking into account other artifacts, like separatelycaptured business rules, is another challenge.


6.4 F6—Inconsistencies can cause severe problems

Summary It is natural to expect that undesired incon-sistencies in software artifacts may cause problems, forexample, by delaying a new version of a system (e.g.,[21,24]). We provide examples of unnoticed inconsisten-cies that: (i) delayed business and impacted customers(production process instances needed to be canceled andrecreated) and (ii) affected compliance regulations, sub-jecting the company to fines. Such problems can causeserious financial and corporate image losses. Our moti-vation to present such cases is reinforcing the need forbetter support in process modeling for dealing with mul-tiple abstraction levels.

We identified two particular cases in which inconsis-tencies caused troubles. The first case was caused by anincomplete technical-level process specification (a prob-lem of business-relevant coverage mismatch, Section 6.2).An (inconsistent) technical-level process specification,the corresponding IT model and several other artifacts(use cases, architecture document etc.) were sent to anoutsourcer as part of a maintenance project. When up-dating the IT model, a developer inadvertently removedthe functionality shown in Figure 16. The developer wasnew to the team and thought that this functionalityshould be deleted from the IT model: the developer didnot see any reasonable correspondence in the specifica-tion, and also no reference to it in the architecture docu-ment. As a result, the problem passed unnoticed duringthe tests and the phase of user approval, and was discov-ered very late when the project went into production.

This was considered a severe problem, because somerunning instances of the process had to be canceled andrecreated, delaying business. In outsourcing, the com-munication throughout a project usually observes a rigidschedule and the external developers cannot directly talkto business or systems analysts: double-checking the un-derstanding of a specification is not as simple as in inter-nal development. Although the test cases were improvedafter the incident, similar incidents can still happen if thebusiness and IT-level models are incomplete, as there areno specified tests to capture every possible issue.

The second case was similar, but this time it wasdiscovered by a regular audit procedure, where projectsand their artifacts were inspected for consistency. Un-clear points were marked to be explained. It turned outthat the specification was outdated, and a notificationwas issued to correct the problem. This is also a severeproblem, because business specifications are used for sat-isfying regulation purposes. Whenever a compliance is-sue is reported by an audit procedure the company issubject to fines and the managers are subject to legalresponsibility.

Is proposal rural

credit?

Get PRONAF

loans

Yes

No

Fig. 16: Functionality Inadvertently Removed∗

∗ PRONAF is a business acronym in BNB that means acredit line for family agriculture.

One of the Business Analysts made the followingcomment about such incidents:

“It is somehow frustrating that BPM has not solvedour problem of reliably communicating with outsourcersby using process models as specifications. In practice thetechnology is preferable for internal development, wherethe communication between business and IT is straight-forward. There is always a risk of something is miss-ing in one or another model, or some correspondencenot being completely understood. We have to maintainheavy textual documents describing the correspondencesbetween specifications and implementations, which is cum-bersome and time-consuming. Today the quality team isspending a huge effort to guarantee that such problemsof misunderstanding the models do not require to cancelproduction process instances. This may affect customersand negatively impact the image of the company.”

7 Empirical Findings on Tool Support

7.1 F7—The majority of the surveyed stakeholderswould prefer a single model for Business and IT

Summary It is generally accepted that a single modelfor business and IT is undesirable [25]. Mixing businessand IT concepts may produce cluttered models that arehard to understand and maintain, specially on the bu-siness side. Curiously, most of the specialists from BNBwould prefer a single model for both business and IT.We actually do not interpret this finding as strictly con-tradicting the state-of-the-art. We rather interpret thisresult as a twofold message: (i) people want a single mo-del (i.e., a single source of truth) for the overlappingpart of related business and IT models, and (ii) they aredissatisfied with the current tool support for managingconsistency of multiple (independently edited) processmodels.

We asked the practitioners to answer which develop-ment method they consider more effective for keeping


Business and IT perspectives consistent. We ultimatelywanted to find out how happy they are with the currentdevelopment process and tool support for consistencymanagement. The results are shown in Figure 17.

Maintain only one model by mixing business and IT information to serve as the Business

Model as well as the IT Model

Create separate models for Business and IT and maintain their alignment when

necessary

Change the IT systems whenever needed in order to enact a pure Business Model

Create wrappers on top of existing IT systems in order to enact a pure Business Model

0 %

31 %

31 %

38 %

Fig. 17: Preferred Approach to Enforce Consistency

We examined the answers with respect to the roles ofthe respondents—the first question of the survey askedrespondents to provide their roles. Curiously, we noticedthat all the respondents who answered Create wrapperson top of existing IT systems in order to enact a pureBusiness Model were Business Analysts. They manifestthat pure business models are the ones really neededby the company and that the IT department should doanything necessary to enact them directly. Dealing withsome ‘pollution’ of information in a single model is evenconsidered preferable by business people over the burdenand the risk of losing consistency between different mod-els. None of the IT specialists chose that answer. Regard-ing the remaining answers, most of the IT stakeholders(38% of the answers) are skeptical whether it would beactually possible to enforce consistency by maintainingdifferent models for Business and IT.

Surprisingly, having a single model for Business andIT is generally considered undesirable by existing works[25]. When maintaining a single model, the companymight run into the problem that business analysts andmanagers could no longer understand the resulting mo-del. They might not recognize how their business is re-flected in the resulting model. Another problem of thisapproach occurs when the business model is used to sat-isfy compliance check points. Mixing Business and ITconcepts can force changing the terminology or the levelof granularity of business concepts, making the modelless clear and less useful for fulfilling the regulations.

Collating the answers and comments from the prac-titioners, given in both the interviews and the survey, weinterpret this result as a twofold message:

– People want a single model for their common (over-lapping) aspects, i.e., a single source of truth—thatis the same as consistency. Nevertheless, there maybe disjoint parts for business- and IT-specific con-cepts, which could live in separate models. For exam-ple, all the IT aspects that are not business relevant(see Section 6.2) could be maintained in a differentplace, other than the ’single’ model. The same appliesfor business concepts that are not system-supported,such as manual tasks. This does not completely con-tradicts the literature, which mainly refers to the spe-cific parts.

– There is a dissatisfaction of the users with the cur-rent tool support for managing consistency of mul-tiple process models. More research is necessary tounderstand whether a single model for the commonpart would be feasible as solution in practice. Anopen problem is that the existing process metamod-els do not reflect the situation of two overlappingmodels for business and IT. That is, alternatively tosynchronizing two separate models, a new metamodelcould be developed to reflect the multiple views usecase better. A possible direction is to allow customviews on a shared model [52].

In addition, the use of a single (i.e., unique) model forboth common and specific parts may not be technicallypossible, as some respondents pointed out:

IT Systems Analyst 1: “I sympathise with the idea ofhaving a single process model, as it would eliminate thisburden of synchronizing business and IT processes. How-ever, I still have some unclear points in my mind on howthis would work: 1 – If the language is the same, mostprobably the mechanism of having modeling perspectivesis critical, since the business roles should stick with theirbasic building blocks, while on the IT side we have fullmodeling capability. How would this work in practice?By hiding or showing things, like model elements? Is itreally possible to do this? What if by adding transactionscopes and controls we need to split the original processand thus drastically change the business view? It is notclear for me whether you can just hide or show things.2 – It seems that you expect improving the collaborationbetween business and IT, but what exactly do you expectthat tools would do for improving collaboration? For methe collaboration today is already good with the currenttools, although there is a lack of automated support forchange propagation and synchronization. However, I dothink that the tools also lack a better integration withthe development process, such as iteration planning andfine-grained change traceability.”


IT Systems Analyst 2: “I believe in a single modelonly if we can still have specialized views for businessand IT – I do not know how this would differ from hav-ing different models, since in practice we may implementthe business model only partially, or split it into severalpieces. On the other hand, if the tool enforces a uniquemodel for both business and IT and does not give anyfreedom of changing it in parallel for particular users, Iam afraid that people would create two different modelsanyway.”

IT Architect:“For me a single model is viable andideal when you have a highly mature IT service infras-tructure, with several business services already availableand aligned with the business objectives. In case you needto implement many things from scratch, it is almost in-evitable having the business model only as a referenceand the executable model more similar to the reality ofexisting systems.”

Business Analyst: “The ideal solution is having onlythe business model, because it is in the end the consum-able asset of the company. With a single model, the align-ment will be enforced by the technology, which is good. Inthe case of technical issues preventing the enactment ofa pure business model, it should be possible to solve thatby other means instead of changing the model itself.”

7.2 F8—Inconsistencies and fixes should be presentedas they occur

Summary Existing approaches propose quick-fixes, gen-erated during the editing of the models [26], other workspropose off-line reports where the practitioners can as-sess the degree of consistency of related process mod-els [23]. The surveyed practitioners prefer instantaneousfixing actions, integrated to the modeling tools.

We asked the respondents about their preferences onhow to check whether the models are consistent and howpotential inconsistencies should be automatically pre-sented by the tools. Figure 18 shows that the respondentsseem to prefer looking at concrete model differences,which may be grouped into high-level model changes,rather than metric measures associated with a degree ofconsistency as proposed in [23].

With respect to fixing actions, most of the respon-dents would prefer having quick fixes, automatically gen-erated by the tools during the modeling task, as shownin Table 12.

An IT Architect has made this comment in the sur-vey:

Look at groups of model differences

ranked by the degree they affect

the alignment

Look at metric values associated with

model elements and fragments

representing how much they are aligned

Other (please specify) 4 %

13 %

83 %

Fig. 18: Preferred Method for Aligning Models

Table 12: How Fixing Actions Should be Presented

Instantly, during themodeling task

As an offline report,when required

For Business Stakeholders 86% 14%For IT Stakeholders 95% 5%

“I think that one of the main reasons for the lackof alignment between business and IT is not related tohow we create business and IT models or related to whatcontents they should have or not. I believe that the devel-opment process plays an important role in this: today wetry to minimize the lack of alignment by enforcing a closerelationship between the technical modeller and the busi-ness analyst. This is good for new projects, but it oftenfails in day-to-day for several reasons: in practice manychanges are minor, which leads to accumulating someinconsistencies considered not critical until a big changeis necessary. Usually most of the change requests madeby a business role are described only textually and thebusiness model is not even touched – the problem here isthat the business analyst believes that only the productionprocess should be updated and its documentation does notneed updating. It is hard to enforce a policy requiringthe business analyst to always update the business mo-del, because the one who knows when the documentationshould be updated is the business analyst anyway. Thereare long periods of maintenance that affect primarily theexecutable model, so during the life cycles of small pro-jects you accumulate several small ‘waterfalls’ of textualrequirements in the sense that the business model (as itshould also be part of the requirements) is ‘forgotten’. Ithink that the best way to address this is by showing po-tential inconsistencies immediately, whenever the modelsare changed. This would make people aware to keep themodels always consistent to a sufficient level. We canalso manage the inconsistencies by planning when theyshould be resolved in future projects.”


8 Threats to Validity

Many empirical studies suffer from limitations such asthe number of subjects not being representative of theentire population, the differences between developmentmethods and tools employed across different organiza-tions, and so on. In particular, in the domain of pro-cess modeling there are still more challenges, such as(to the best of our knowledge) the absence of availableopen-source projects ranging from business-level speci-fications to IT-level implementations, and also the factthat companies which adopt such technologies usuallyconsider process artifacts extremely sensitive and confi-dential. As a result, there is little data from practice todrive BPM research.

Our study is subject to three main limitations: 1)the limited scope of participants and the fact that thestudy is based on a single company; 2) potential misun-derstandings, as it involved translating interviews andsurvey responses from Portuguese to English, and talk-ing to subjects with varied experience levels and skills;and 3) potential errors in the survey because the surveyresponses are based on subjective and relatively quickassessments of the respondents.

We think that these limitations do not invalidate ourresults. While we believe that the numbers of analyzedartifacts and interviewed and surveyed people are sub-stantial, we do not intend to draw any general conclu-sions about all development processes of process models.Clearly, different development processes and organiza-tional cultures will likely lead to different results. Fur-thermore, while the refinement patterns are grounded inthe studied artifacts, interviews and questionnaires re-veal subjective perceptions of the participating subjects.We invite the reader to focus on the overall findings andnot on specific numbers. We expect that most BPM de-velopment processes that are similar to the one we in-vestigated would face similar difficulties in maintainingconsistency of related business and IT process models.

9 Conclusions

We have performed a comprehensive study of an in-dustrial BPM-driven development process, including theanalysis of more than 70 models, 17 hours of interviewswith practitioners, and inspection of around 1000 changerequests in 5 BPM projects. Our study covers severalaspects of consistency management, including types ofinconsistencies, causes, impacts, and tool preferences.

The findings detailed in our study highlight some lim-itations in the way that state-of-the-art BPM solutionswork:

– Development process: Effective consistency manage-ment appears to require a progressive disclosure ap-proach, in which models are created by a smoothprogression from high-level specifications to IT-levelmodels, preserving a chain of manageable correspon-dences. Today, related models are initially createdusing common refinement patterns, but then main-tained separately for satisfying the needs of differentstakeholders, possibly in different languages.

– Hierarchical and non-hierarchical refinements: Re-finements should not be restricted to hierarchy. Ac-cordingly, a progressive disclosure modeling approachshould take that fact into consideration.

– Stakeholders need a way to define consistency prop-erties: Consistency is a subjective notion. The samepair of models may or may not be considered incon-sistent. The notion of business relevance influencesstrongly the consistency rules.

– There is a lack of support for parallel maintenance:Parallel maintenance requires differencing and merg-ing techniques, something lacking in the major tools.

– Detection of inconsistencies: Inconsistencies shouldbe detected and communicated as soon as they oc-cur and then managed according to a clearly definedprocess.

We hope that these findings will help researchers andtool builders improve tool support for business processmodeling. Such improvements would take into accountthe common refinement operations used by developersand include rapid detection and presentation of incon-sistencies to the user, including possible fixes.

Acknowledgment

We would like to thank the Bank of the Northeast ofBrazil (Banco do Nordeste – BNB) for granting us fullaccess to people and artifacts, fundamental for conduct-ing this study. This work was partially supported by anIBM PhD CAS Fellow Scholarship, the Ontario ResearchFund’s Research Excellence Project on Model-IntegratedSoftware Service Engineering, and the National NaturalScience Foundation of China under Grant No. 61202071and No. 61121063.

References

1. D. Miers and S. A. White, BPMN Modeling and Refer-ence Guide Understanding and Using BPMN. Light-house, Pt, FL, USA: Future Strategies Inc., 2008.

2. OASIS, “Web Services Business Process ExecutionLanguage (WSBPEL) TC,” http://www.oasis-open.org/committees/tc home.php?wg abbrev=wsbpel.

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsbpel

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsbpel


3. Object Management Group, “Business Process Modeland Notation (BPMN) Version 2.0,” http://www.omg.org/spec/BPMN/2.0/.

4. M. Weidlich, R. Dijkman, and M. Weske, “Deciding be-haviour compatibility of complex correspondences be-tween process models,” in Proceedings of the 8th interna-tional conference on Business process management, ser.BPM’10. Berlin, Heidelberg: Springer-Verlag, 2010, pp.78–94.

5. R. Dijkman, “Diagnosing differences between businessprocess models,” in Proceedings of the 6th InternationalConference on Business Process Management, ser. BPM’08. Berlin, Heidelberg: Springer-Verlag, 2008, pp. 261–277.

6. C. Li, M. Reichert, and A. Wombacher, “On measuringprocess model similarity based on high-level change op-erations,” in Proceedings of the 27th International Con-ference on Conceptual Modeling, ser. ER ’08. Berlin,Heidelberg: Springer-Verlag, 2008, pp. 248–264.

7. R. Dijkman, “A classification of differences between sim-ilar business processes,” in Proceedings of the 11th IEEEInternational Enterprise Distributed Object ComputingConference. Washington, DC, USA: IEEE ComputerSociety, 2007.

8. M. Henkel, J. Zdravkovic, and P. Johannesson, “Service-based processes: design for business and technology,” inProceedings of the 2nd international conference on Ser-vice oriented computing, ser. ICSOC ’04. New York,NY, USA: ACM, 2004, pp. 21–29.

9. J. Koehler, R. Hauser, J. Kuster, K. Ryndina, J. Van-hatalo, and M. Wahler, “The role of visual modeling andmodel transformations in business-driven development,”Electron. Notes Theor. Comput. Sci., vol. 211, pp. 5–15,April 2008.

10. L. Zerguini, “A novel hierarchical method for decompo-sition and design of workflow models,” J. Integr. Des.Process Sci., vol. 8, pp. 65–74, April 2004.

11. P. Soffer, “Refinement equivalence in model-based reuse:Overcoming differences in abstraction level,” J. DatabaseManag., vol. 16, no. 3, pp. 21–39, 2005.

12. M. Weidlich, A. P. Barros, J. Mendling, and M. Weske,“Vertical alignment of process models - how can we getthere?” in CAiSE 2009 Workshop Proceedings: BPMDS,2009, pp. 71–84.

13. T. H. Davenport, Process innovation: Reengineeringwork through information technology. Boston, MA,USA: Harvard Business School Press, 1993.

14. C. Rolland and N. Prakash, “Bridging the gap be-tween organisational needs and ERP functionality,”Requirements Engineering, vol. 5, pp. 180–193, 2000,10.1007/PL00010350.

15. N. Bieberstein, S. Bose, M. Fiammante, K. Jones, andR. Shah, Service-Oriented Architecture Compass: Busi-ness Value, Planning, and Enterprise Roadmap. UpperSaddle River, NJ, USA: Prentice Hall PTR, 2005.

16. R. Bobrik, M. Reichert, and T. Bauer, “View-based pro-cess visualization,” in BPM, 2007, pp. 88–95.

17. International Organization for Standardization, Fi-nancial transaction card originated messages – In-terchange message specifications – Part 1: Mes-sages, data elements and code values. [On-line]. Available: http://www.iso.org/iso/iso catalogue/catalogue tc/catalogue detail.htm?csnumber=31628

18. A. Delgado, F. Ruiz, I. G.-R. de Guzman, and M. Piat-tini, “A model-driven and service-oriented framework forthe business process improvement,” Journal of SystemsIntegration, vol. 1, no. 3, pp. 45–55, 2010.

19. R. M. Dijkman, D. A. C. Quartel, L. F. Pires, andM. J. v. Sinderen, “A rigorous approach to relate en-terprise and computational viewpoints,” in Proceedingsof the Enterprise Distributed Object Computing Confer-ence, Eighth IEEE International. Washington, DC,USA: IEEE Computer Society, 2004, pp. 187–200.

20. S. Buchwald, T. Bauer, and M. Reichert, “Bridging thegap between business process models and service com-position specifications,” in Service Life Cycle Tools andTechnologies: Methods, Trends and Advances. IdeaGroup Reference, November 2011, pp. 124–153.

21. G. Spanoudakis and A. Zisman, “Inconsistency man-agement in software engineering: Survey and open re-search issues,” in in Handbook of Software Engineeringand Knowledge Engineering. World Scientific, 2001, pp.329–380.

22. M. Weidlich, G. Decker, M. Weske, and A. Barros, “To-wards vertical alignment of process models - a collectionof mismatches,” Hasso Plattner Institute, Tech. Rep.,2008.

23. M. Weidlich, J. Mendling, and M. Weske, “Efficient con-sistency measurement based on behavioural profiles ofprocess models,” IEEE Transactions on Software Engi-neering, vol. 99, no. PrePrints, 2010.

24. B. Nuseibeh, S. Easterbrook, and A. Russo, “Leveraginginconsistency in software development,” in in SoftwareDevelopment”, Computer, 33(4):24-29, IEEE Computer.Society Press, 2000, pp. 1–33.

25. G. Decker, “Bridging the gap between business processesand existing IT functionality,” in Proceedings of the 1stInternational Workshop on Design of Service-OrientedApplications (WDSOA). Amsterdan, The Netherlands:ICSOC, 2005, pp. 17–24.

26. A. Hegedus, A. Horvath, I. Rath, M. C. Branco, andD. Varro, “Quick fix generation for DSMLs,” in Pro-ceedings of IEEE Symposium on Visual Languages andHuman-Centric Computing VLHCC 2011. IEEE, 2011.

27. H.-M. Chen, “Towards service engineering: Service ori-entation and Business-IT alignment,” in Proceedings ofthe Proceedings of the 41st Annual Hawaii InternationalConference on System Sciences, ser. HICSS ’08. Wash-ington, DC, USA: IEEE Computer Society, 2008, pp.114–.

28. D. Werth, K. Leyking, F. Dreifus, J. Ziemann, andA. Martin, “Managing SOA through business services: abusiness-oriented approach to service-oriented architec-tures,” in Proceedings of the 4th international conferenceon Service-oriented computing, ser. ICSOC’06. Berlin,Heidelberg: Springer-Verlag, 2007, pp. 3–13.

29. H. Tran, U. Zdun, and S. Dustdar, “View-based integra-tion of process-driven SOA models at various abstractionlevels,” in Proceedings of First International Workshopon Model-Based Software and Data Integration MBSDI2008. Springer, 2008, pp. 55–66.

30. J. Kuster, “Consistency management of object-orientedbehavioral models,” Ph.D. dissertation, UniversitatPaderborn, 2004.

http://www.omg.org/spec/BPMN/2.0/

http://www.omg.org/spec/BPMN/2.0/

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=31628

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=31628


31. A. Finkelstein and I. Sommerville, “The ViewpointsFAQ,” Software Engineering Journal, vol. 11, no. 1, pp.2–4, Jan. 1996.

32. W. Emmerich, A. Finkelstein, C. Montangero, S. An-tonelli, S. Armitage, and R. Stevens, “Managing stan-dards compliance,” Software Engineering, IEEE Trans-actions on, vol. 25, no. 6, pp. 836–851, 1999.

33. Z. Diskin, Y. Xiong, and K. Czarnecki, “Specifyingoverlaps of heterogeneous models for global consis-tency checking,” in Proceedings of the First InternationalWorkshop on Model-Driven Interoperability, ser. MDI’10. New York, NY, USA: ACM, 2010, pp. 42–51.

34. A. Marcus and J. I. Maletic, “Recovering documentation-to-source-code traceability links using latent semantic in-dexing,” in Proceedings of the 25th International Confer-ence on Software Engineering, ser. ICSE ’03. Washing-ton, DC, USA: IEEE Computer Society, 2003, pp. 125–135.

35. Z. Xing, “Model comparison with GenericDiff,” in Pro-ceedings of the IEEE/ACM international conference onAutomated software engineering, ser. ASE ’10. NewYork, NY, USA: ACM, 2010, pp. 135–138.

36. J. Euzenat and P. Shvaiko, Ontology matching. Heidel-berg (DE): Springer-Verlag, 2007.

37. E. Rahm and P. A. Bernstein, “A survey of approachesto automatic schema matching,” The VLDB Journal,vol. 10, pp. 334–350, December 2001.

38. C. Nentwich, W. Emmerich, and A. Finkelstein, “Con-sistency management with repair actions,” in SoftwareEngineering, 2003. Proceedings. 25th International Con-ference on, 2003, pp. 455 – 464.

39. A. Egyed, E. Letier, and A. Finkelstein, “Generating andevaluating choices for fixing inconsistencies in uml de-sign models,” in Automated Software Engineering, 2008.ASE 2008. 23rd IEEE/ACM International Conferenceon, 2008, pp. 99 –108.

40. C. Amelunxen, E. Legros, A. Schurr, and I. Sturmer,“Checking and enforcement of modeling guidelines withgraph transformations,” in Applications of Graph Trans-formations with Industrial Relevance, 2008, pp. 313–328,LNCS 5088, Springer.

41. J. Pinna Puissant, T. Mens, and R. Van Der Straeten,“Resolving Model Inconsistencies with Automated Plan-ning,” in Proceedings of the 3rd Workshop on Living withInconsistencies in Software Development. CEUR Work-shop Proceedings, 2010, pp. 8–14.

42. Y. Xiong, A. Hubaux, S. She, and K. Czarnecki, “Gener-ating range fixes for software configuration,” in ICSE’12:34th International Conference on Software Engineering,2012, pp. 58–68.

43. J. M. Kuster, “Towards inconsistency handling of object-oriented behavioral models,” Electronic Notes in Theo-retical Computer Science, vol. 109, pp. 57 – 69, 2004,proceedings of the Workshop on Graph Transformationand Visual Modelling Techniques (GT-VMT 2004).

44. R. Dijkman, M. Dumas, L. Garcia-Banuelos, andR. Kaarik, “Aligning Business Process Models,” in 2009IEEE International Enterprise Distributed Object Com-puting Conference. IEEE, Sep. 2009, pp. 45–53.

45. B. van Dongen, R. Dijkman, and J. Mendling, “Mea-suring Similarity between Business Process Models,” inProceedings of the 20th Int’l Conference on Advanced

Information Systems Engineering (CAiSE 2008), ser.Lecture Notes in Computer Science, Z. Bellahsene andM. Leonard, Eds., vol. 5074. Montpellier, France:Springer Verlag, 2008, pp. 450–464.

46. M. Weidlich, R. Dijkman, and J. Mendling, “The ICoPframework: identification of correspondences betweenprocess models,” in Proceedings of the 22nd internationalconference on Advanced information systems engineer-ing, ser. CAiSE’10. Berlin, Heidelberg: Springer-Verlag,2010, pp. 483–498.

47. M. C. Branco, J. Troya, K. Czarnecki, J. Kuster,and H. Volzer, “Matching Business Process Work-flows Across Abstraction Levels,” in Proceedings of15th ACM/IEEE International Conference on ModelDriven Engineering Languages and Systems, ser. MOD-ELS 2012. ACM/IEEE, 2012.

48. J. Vanhatalo, H. Volzer, and F. Leymann, “Faster andMore Focused Control-Flow Analysis for Business Pro-cess Models Through SESE Decomposition,” in ICSOC2007, ser. LNCS. Berlin, Heidelberg: Springer-Verlag,2007, pp. 43–55.

49. B. Fluri, M. Wursch, M. Pinzger, and H. Gall, “ChangeDistilling: Tree Differencing for Fine-Grained SourceCode Change Extraction,” Software Engineering, IEEETransactions on, vol. 33, no. 11, pp. 725 –743, nov. 2007.

50. J. Vanhatalo, H. Volzer, and J. Koehler, “The refinedprocess structure tree,” in Proceedings of the 6th Inter-national Conference on Business Process Management,ser. BPM ’08. Berlin, Heidelberg: Springer-Verlag, 2008,pp. 100–115.

51. C. Gerth, J. M. Kuster, M. Luckey, and G. Engels, “Pre-cise detection of conflicting change operations using pro-cess model terms,” in Proceedings of the 13th interna-tional conference on Model driven engineering languagesand systems: Part II, ser. MODELS’10. Berlin, Heidel-berg: Springer-Verlag, 2010, pp. 93–107.

52. J. Kuster, H. Volzer, C. Favre, M. C. Branco,and K. Czarnecki, “Supporting different processviews through a shared process model,” IBMResearch Zurich, Tech. Rep., 2012. [Online]. Available:http://domino.research.ibm.com/library/cyberdig.nsf/papers/FA822A5E450EB08685257A1600462337/$File/rz3823.pdf

53. J. M. Kuster, C. Gerth, A. Forster, and G. Engels, “De-tecting and resolving process model differences in theabsence of a change log,” in Proceedings of the 6th Inter-national Conference on Business Process Management,ser. BPM ’08. Berlin, Heidelberg: Springer-Verlag, 2008,pp. 244–260.

54. J. M. Kuster and K. Ryndina, “Improving inconsistencyresolution with side-effect evaluation and costs,” in MoD-ELS, 2007, pp. 136–150.

55. J. Hutchinson, M. Rouncefield, and J. Whittle, “Model-driven engineering practices in industry,” in Proceedingof the 33rd international conference on Software engi-neering, ser. ICSE ’11. New York, NY, USA: ACM,2011, pp. 633–642.

56. M. Zapf and A. Heinzl, “Evaluation of generic processdesign patterns: An experimental study,” in BusinessProcess Management, Models, Techniques, and Empir-ical Studies. London, UK, UK: Springer-Verlag, 2000,pp. 83–98.

http://domino.research.ibm.com/library/cyberdig.nsf/papers/FA822A5E450EB08685257A1600462337/$File/rz3823.pdf




57. J. Vanhatalo, H. Volzer, and F. Leymann, “Faster andmore focused control-flow analysis for business pro-cess models through sese decomposition,” in Proceed-ings of the 5th international conference on Service-Oriented Computing, ser. ICSOC ’07. Berlin, Heidel-berg: Springer-Verlag, 2007, pp. 43–55.


Appendix

Basic BPMN Notation

Sequence

(a) Flow

Service TaskService Task User TaskUser Task Send TaskSend Task Receive TaskReceive Task Sub-ProcessSub-Process

(b) Tasks

StartStart EndEnd ErrorError TimerTimer SignalSignal Terminate

(c) Events

ExclusiveExclusive ParallelParallel JoinJoin

(d) Gateways

Date post:	21-Apr-2020
Category:	Documents
Upload:	others
View:	6 times
Download:	0 times

A Case Study on Consistency Management of Business and IT … · A Case Study on Consistency...

Documents