The Automated Discovery of Declarative Process ModelsClaudio Di Ciccio
Humboldt-Universität zu Berlin, 7 December 2016
www.wu.ac.atwww.wu.ac.at/infobizwww.wu.ac.at/infobiz/team/diciccio
Control-flow discovery
?
Objective: understanding the temporal structure that best describes the process behind the event log
SEITE 3
Knowledge-intensive Processes
SEITE 4
Knowledge-intensiveProcesses
SEITE 5
Mining flexible processes
SEITE 6
Declarative modelling ofprocesses
Usage of constraints “Open model”
Declare state-of-the-art language
If A is performed,B must be performed,
no matter if before or afterwards(responded existence)
Whenever B is performed,C must be performed afterwards
and B can not be repeateduntil C is done
(alternate response)SEITE 7
Workflow Netsas process models
SEITE 8
Imperative v declarative
SEITE 9SEITE 9
Declarative processesDECLARE
Declarative process modelling
“Open model” Specify constraints for
permitted behaviour Every execution that
complies with them is acceptable
Works best with flexible processes
The set of DECLARE templates is extendible
SEITE 11
A fragment of declarative process model If an abstract is submitted, a new paper
had been written or will be written
After the paper submission, a confirmation email is sent
After the paper submission, the paper will be reviewed;there can be no review without a preceding submission
A paper can be accepted only after it has been reviewed
After the rejection, no further submission follows
Paper cannot be both accepted and rejected
SEITE 12
Submit abstract Write new paper
Submit paper Send confirmation email
Submit paper Review paper
Review paper Accept paper
Reject paper Submit paper
Accept paper Reject paper
= activation task
Responded existence(Submit abstract, Write new paper)
Response(Submit paper, Send confirmation email)
Succession(Submit paper, Review paper)
Precedence(Review paper, Accept paper)
Not succession(Reject paper, Submit paper)
Not co-existence(Accept paper, Reject paper)
Template Tasks
A fragment of declarative process model
SEITE 13
Submit abstract
Write new paper Submit paper
Send confirmation email
Review paper Accept paper
Reject paper
Declare:Existence Constraint Templates
Existence(n, A)Activity A occurs at least n times in the process instanceBCAAC ✓ BCAAAC ✓ BCAC ✗ (for n = 2)
Absence(A)Activity A does not occur in the process instanceBCC ✓ BCAC ✗
Absence(n+1, A)Activity A occurs at most n-1 times in the process instanceBCAAC ✗ BCAC ✓ BCC ✓ (for n = 2)
Exactly(n, A)Activity A occurs exactly n times in the process instanceBCAAC ✓ BCAAAC ✗ BCAC ✗ (for n = 2)
Init(A)Activity A is the first to occur in each process instanceBCAAC ✗ ACAAAC ✓ BCC ✗
Absence(2, A) ≡ AtMostOne(A)
Existence(1, A) ≡ Participation(A)
SEITE 14
Declare:Relation Constraint Templates
RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓
SEITE 15
Declare:Relation Constraint Templates
RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓Response(A, B)If A occurs in the process instance, then B occurs after ABCAAC ✗ CAACB ✓CAC ✗ BCC ✓
SEITE 16
Declare:Relation Constraint Templates
RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓Response(A, B)If A occurs in the process instance, then B occurs after ABCAAC ✗ CAACB ✓CAC ✗ BCC ✓AlternateResponse(A, B)Each time A occurs in the process instance, then B occurs afterwards, before A recursBCAAC ✗ CAACB ✗ CACB ✓CABCA ✗ BCC ✓ CACBBAB ✓
SEITE 17
Declare:Relation Constraint Templates
RespondedExistence(A, B)If A occurs in the process instance, then B occurs as wellCAC ✗ CAACB ✓BCAC ✓ BCC ✓Response(A, B)If A occurs in the process instance, then B occurs after ABCAAC ✗ CAACB ✓CAC ✗ BCC ✓AlternateResponse(A, B)Each time A occurs in the process instance, then B occurs afterwards, before A recursBCAAC ✗ CAACB ✗ CACB ✓CABCA ✗ BCC ✓ CACBBAB ✓ChainResponse(A, B)Each time A occurs in the process instance, then B occurs immediately afterwardsBCAAC ✗ BCAABC ✗ BCABABC ✓
Activation Target
Declare:Relation Constraint Templates
RespondedExistence(B, A)If B occurs in the process instance, then A occurs as wellCAC ✓ CAACB ✓BCAC ✓ BCC ✗Precedence(A, B)B occurs in the process instance only if preceded by ABCAAC ✗ CAACB ✓CAC ✓ BCC ✓AlternatePrecedence(A, B)Each time B occurs in the process instance, it is preceded by A and no other B can recur in betweenBCAAC ✗ CAACB ✓ CACB ✓CABCA ✓ BCC ✗ CACBAB ✓ChainPrecedence(A, B)Each time B occurs in the process instance, then B occurs immediately beforehandBCAAC ✗ BCAABC ✗ CABABCA ✓
Target Activation
Declare:Relation Constraint Templates
CoExistence(A, B) ≡ Resp’dEx.(A, B) Resp’dEx.(B, A)If B occurs in the process instance, then A occurs, and viceversaCAC ✗ CAACB ✓BCAC ✓ BCC ✗Succession(A, B) ≡ Response(A, B) Precedence(A, B)A occurs if and only if it is followed by B in the process instanceBCAAC ✗ CAACB ✓CAC ✗ BCC ✗AlternateSuccession(A, B) ≡ Alt.Resp.(A, B) Alt.Prec.(A, B)A and B occur in the process instance if and only if the latter follows the former, and they alternate each other in the traceBCAAC ✗ CAACB ✗ CACB ✓CABCA ✗ BCC ✗ CACBAB ✓ChainSuccession(A, B) ≡ ChainResp.(A, B) ChainPrec.(A, B)A and B occur in the process instance if and only if the latter immediately follows the formerBCAAC ✗ BCAABC ✗ CABABC ✓
Activation Target
Target Activation
Declare: NegativeRelation Constraint Templates
NotCoExistence(A, B)A and B never occur together in the process instanceCAC ✓ CAACB ✗BCAC ✗ BCC ✓NotSuccession(A, B)A can never occur before B in the process instanceBCAAC ✓ CAACB ✗CAC ✓ BCC ✓NotChainSuccession(A, B)A and B occur in the process instance if and only if the latter does not immediately follows the formerBCAAC ✓ BCAABC ✗ CBACBA ✓
Activation Target
Target Activation
Subsumption hierarchyof constraint templates
SEITE 22
Declarative processesDiscovery of DECLARE models
Constraints mining
?
Objective: understanding the constraints that best define the allowed behaviour of the process behind the event log
SEITE 24
Mining declarative processes:ingredients
“Submit abstract”,“Submit paper”,“Accept paper”,…
SEITE 25
A,B,C,…
Activities Process alphabet
Mining declarative processes
RespondedExistence(A, B) ?RespondedExistence(A, C) ?…Response(A, B) ?Response(A, C) ?…
SEITE 26
• Support:fraction of cases fulfilling the constraint
• Confidence:support scaled by fraction of traces in which the activation occurs
• Interest factor:confidence scaled by fraction of traces in which the target occurs
Support Conf. I.F.
Threshold
Threshold
Threshold
Mining declarative processes
RespondedExistence(A, B) ?RespondedExistence(A, C) ?…Response(A, B) ?Response(A, C) ?…
SEITE 27
Support Conf. I.F.
• Support:fraction of cases fulfilling the constraint
• Confidence:support scaled by fraction of traces in which the activation occurs
• Interest factor:confidence scaled by fraction of traces in which the target occurs
Mining declarative processes
RespondedExistence(A, B) RespondedExistence(A, C) …Response(A, B) ?Response(A, C) …
SEITE 28
Support Conf. I.F.
• Support:fraction of cases fulfilling the constraint
• Confidence:support scaled by fraction of traces in which the activation occurs
• Interest factor:confidence scaled by fraction of traces in which the target occurs
Mining declarative processes
RespondedExistence(A, B) RespondedExistence(A, C) …Response(A, B) Response(A, C) …
SEITE 29
Support Conf. I.F.
• Support:fraction of cases fulfilling the constraint
• Confidence:support scaled by fraction of traces in which the activation occurs
• Interest factor:confidence scaled by fraction of traces in which the target occurs
Constraints miningAn example
SEITE 30
A A B C A B C A C B C D
A C C A B B C B C A C B B D
C C C C C A A B C A A B A A B
A B B B D
C B A B D
A B B D
A A A C A C B D
A B C D
C A B A A C C B B D
B C C D
C A A C C C A A B C B C C B D
The role of Support(event-based)
A A B C A B C A C B C DA C C A B B C B C A C B B DC C C C C A A B C A A B A A BA B B B DC B A B DA B B DA A A C A C B DA B C DC A B A A C C B B DB C C DC A A C C C A A B C B C C B D
Support for Response(A, B)
1.0 Precedence(A, B)
0.926 (25/27) Pruning on the basis of a
support threshold E.g., 0.95
A threshold equal to 1.0 for a constraint means “always valid in the log”
ActivationTargetTarget
Activation
SEITE 31http://github.com/cdc08x/minerful
Prom Nightly Builds > “Declare MINERful” plug-in
The role of Support(trace-based)
A A B C A B C A C B C DA C C A B B C B C A C B B DC C C C C A A B C A A B A A BA B B B DC B A B DA B B DA A A C A C B DA B C DC A B A A C C B B DB C C DC A A C C C A A B C B C C B D
Support for Response(A, B)
1.0 Precedence(A, B)
0.818 (9/11) Pruning on the basis of a
support threshold E.g., 0.95
A threshold equal to 1.0 for a constraint means “always valid in the log”
SEITE 32 http://sourceforge.net/projects/prom/files/ProM/6.3
Declarative processesChallenges
SEITE 34
Objective
SEITE 35
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered
models Specify templates
SEITE 36
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered
models Specify templates
SEITE 37© Granger, NYC, source: https://www.granger.com/
Redundancies in discovered constraints
E.g., A trace likeA B A B C A B C Csatisfies (w.r.t. A and B): RespondedExistence(A, B), RespondedExistence(B, A),
Response(A, B), AlternateResponse(A, B), ChainResponse(A, B),Precedence(A, B), AlternatePrecedence(A, B), ChainPrecedence(A, B),CoExistence(A, B), CoExistence(B, A),Succession(A, B), AlternateSuccession(A, B), ChainSuccession(A, B)
Among them, the mining algorithm should return only the most restricting constraint – i.e., ChainSuccession(A, B)
In order to face this issue, returned constraints are pruned on the basis of the subsumption hierarchy of constraints
SEITE 38
Pruning redundant constraints
1.0
1.0
1.0
1.01.0
1.0
1.0
1.0
1.0
1.0
1.0
✖✖✖
✖✖✖✖
✖✖✖
SEITE 39
Pruning redundant constraints
0.8
0.8
0.9
0.90.9
0.9
0.7
0.7
0.9
0.9
0.9
✖ ✖??
✖
✖
✖
??
Support ✖
✖✖✖
SEITE 40
Mining declarative processes
RespondedExistence(A, B) RespondedExistence(A, C) ?…Response(A, B) Response(A, C) …
SEITE 41
Support Conf. I.F.
Mining declarative processes
RespondedExistence(A, B) RespondedExistence(A, C) ?…Response(A, B) Response(A, C) …
SEITE 42
Support Conf. I.F.
Mining declarative processes
RespondedExistence(A, B) RespondedExistence(A, C) …Response(A, B) Response(A, C) …
SEITE 43
Support Conf. I.F.
http://github.com/cdc08x/minerfulProm Nightly Builds > “Declare MINERful” plug-inhttp://sourceforge.net/projects/prom/files/ProM/6.3
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered
models Specify templates
SEITE 44
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered
models Specify templates
InterludeDECLARE & Regular Expressions
SEITE 45
Mining declarative processes
RespondedExistence(A, B)RespondedExistence(A, C)andResponse(A, B)Response(A, C)and…
SEITE 46
Semantics of Declare:Regular Expressions
SEITE 47
From constraints-based model to FSA
SEITE 48
[^a]*((a.*b.*)|(b.*a.*))*[^a]* [^a]*(a.*c)*[^a]*
RegularExpression
DeterministicFiniteState
Automaton
RespondedExistence(A, B)RespondedExistence(A, C)andResponse(A, B)Response(A, C)and…
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered
models
Reprise
SEITE 49
While mining a real-life log…
SEITE 50
Submit
draft
Write
deliverableOrganise
agenda
Time to challenge the X
SEITE 51
Submit
draft
Write
deliverableOrganise
agenda
The result
SEITE 52
The problem
When support threshold is lower than 100%,constraints can be valid through most of the log, though being in conflict
Example: an event log consists of two traces:1. <A, B, A, B, A, B, C>2. <A, B, A, B, A, C>
Support threshold: 0.7• a is always the first
Init(A)• c is always the last
End(C)• In 6 cases over 8 (75%), a and c do not directly follow each
other NotChainSuccession(A, C)
• In 5 cases over 7 (71.143%), b and c do not directly follow each other
NotChainSuccession(B, C)
SEITE 53
The problem
When support threshold is lower than 100%,constraints can be valid through most of the log, though being in conflict
Example: an event log consists of two traces:1. <A, B, A, B, A, B, C>2. <A, B, A, B, A, C>
Support threshold: 0.7• a is always the first
Init(A)• c is always the last
End(C)• In 6 cases over 8 (75%), a and c do not directly follow each
other NotChainSuccession(A, C)
• In 5 cases over 7 (71.143%), a and b do not directly follow each other
NotChainSuccession(B, C)
Question: what can be done right before C? inconsistency!
SEITE 54
The algorithm to detect inconsistencies
SEITE 55
Init(a)
Participation(b)AtMostOne(c)End(c) ChainPrecedence(a, b)…
ChainSuccession(a, b)Response(a, b)ChainResponse(b, a)NotChainSuccession(a, c) ChainSuccession(b, a)NotChainSuccession(b, c)…
…
…
1
Event log
Declare constraints
Automata
Safe constraints
(support 100%)
Unsafe
constr
aints
Product
automaton
The algorithm to detect inconsistencies
Init(a)
Participation(b)AtMostOne(c)End(c) ChainPrecedence(a, b)…
ChainSuccession(a, b)Response(a, b)ChainResponse(b, a)NotChainSuccession(a, c) ChainSuccession(b, a)NotChainSuccession(b, c)…
…
…
1
2
Event log
Declare constraints
Automata
Safe constraints
Unsafe
constr
aints
Product
automaton
SEITE 56 http://github.com/cdc08x/minerful
SEITE 57
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells Increase the expressiveness of discovered
models
SEITE 58© Granger, NYC, source: https://www.granger.com/
The application of the method to minimise the model
Rationale:1. How to find redundancies among constraints?
Use the automaton-model correspondence Same language recognised after the product?
Main difference with the inconsistency-checking algorithm
Constraints having support 100% are checked for redundancies
More details in the paper
SEITE 59 http://github.com/cdc08x/minerful
The application of the method to minimise the model
SEITE 60
BPIC 2012
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells
Inspect the imperative counterpart Simulate runs
Increase the expressiveness of discovered models
Specify templates
SEITE 61
From declarative to imperative models
SEITE 62
Event log
Declare
constraintsResponse(Queued, Accepted)NotChainSuccession(Queued, Completed)Response(Queued, Completed)NotChainSuccession(Queued, Unmatched)Response(Accepted, Completed)End(Completed)NotSuccession(Completed, Unmatched)AtMostOne(Unmatched)RespondedExistence(Unmatched, Accepted)AlternateResponse(Unmatched, Completed)
…
Automata
Product
automatonTheory ofRegions
Petri Net
Petrify
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells
Inspect the imperative counterpart Simulate runs
Increase the expressiveness of discovered models
Specify templates
SEITE 63
Generation of logs
SEITE 64
http://github.com/cdc08x/minerfulhttp://github.com/processmining/synthetic-log-generator
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells
Inspect the imperative counterpart Simulate runs
Increase the expressiveness of discovered models
Specify templates
SEITE 65
Target-Branched Declare
Support for Response(A, {B, C})
1.0
B A A B A B C DA A D B B A A D CB A D D A D D A B EC A B CD A D B DA E A CA A A D BD A D D B DA A A D C DA D D D A CA A BE D E B C E E C B E
SEITE 66
ActivationTarget
Branching factor = 2
Objective
SEITE 67
+ Expressive Power+ Conciseness
Evaluation:constraints pruning
SEITE 68
BPI Challenge 2012(0) 6,654,480 (1) 10,676 (2) 1446 (3) 12
Evaluation:performance
SEITE 69
BPI Challenge 2012(KB) 00:07.274 (Const’s) 25:51.678 (Total) 26:11.380
http://github.com/cdc08x/minerful
Make mined modelsintelligible
Prune irrelevant/redundant information Detect inconsistencies (yes, it happens) Picture what the model tells
Inspect the imperative counterpart Simulate runs
Increase the expressiveness of discovered models
Specify templates
SEITE 70
Semantics of Declare:LTL and LTLf
Linear Temporal Logic (LTL) initially was a specification language for the execution of (endless) concurrent programs (Pnueli, 1977) Syntax (let A be a propositional symbol):
DECLARE was initially based on LTL
SEITE 71
“Until”
“Eventually”“Always”
“Next”
Semantics of Declare:LTL
SEITE 72
Declarative process modelling
“Open model” Specify constraints for
permitted behaviour Every execution that
complies with them is acceptable
Works best with flexible processes
The set of DECLARE templates is extendible
SEITE 73
Extendibility of DECLARE:A clear business impact
SEITE 74Source: http://businessvalueexchange.com/
Semantics of Declare:LTL and LTLf
Linear Temporal Logic (LTL) initially was a specification language for the execution of (endless) concurrent programs (Pnueli, 1977) Syntax (let A be a propositional symbol):
Interpretation over infinite traces,i.e., an infinite sequence of consecutive instants of time
LTLf formulae are meant to be interpreted over finite traces
“Until”
“Eventually”“Always”
“Next”
SEITE 75
Semantics of Declare:LTLf
SEITE 76
Semantics of Declare:SCIFF
SEITE 77
Semantics of Declare:R/I-nets
SEITE 78
Semantics of Declare:FOL over finite traces
SEITE 79
Semantics of Declare:Regular expressions
SEITE 80
More alternatives for DECLARE spec.: A clear business impact
SEITE 81Source: http://businessconsultantsnewyork.blogspot.co.at/
My long-term objective
Establish an algebra of constraint templates, built on a basis of behavioural relations that are: orthogonal to one another (no entailments,
subsumptions…) covering multiple trace-based behavioural relations
definition languages by linear composition have clear semantics (last but for sure NOT least) ...
SEITE 82
Further reading
Presented: About MINERful:
Claudio Di Ciccio, Massimo Mecella: On the Discovery of Declarative Control Flows for Artful Processes. ACM Trans. Management Inf. Syst. 5(4): 24:1-24:37 (2015)
Simulation of DECLARE models: Claudio Di Ciccio, Mario Luca Bernardi, Marta Cimitile, Fabrizio Maria Maggi: Generating Event Logs Through the
Simulation of Declare Models. EOMAS@CAiSE 2015: 20-36 Discovery of target-branched DECLARE models:
Claudio Di Ciccio, Fabrizio Maria Maggi, Jan Mendling: Efficient discovery of Target-Branched Declare constraints. Inf. Syst. 56: 258-283 (2016)
Getting rid of redundancies and inconsistencies: Claudio Di Ciccio, Fabrizio Maria Maggi, Marco Montali, Jan Mendling: Ensuring Model Consistency in Declarative Process
Discovery. BPM 2015: 144-159 From DECLARE to Petri nets:
Johannes Prescher, Claudio Di Ciccio, Jan Mendling: From Declarative Processes to Imperative Models. SIMPDA 2014: 162-173
More: First steps towards data awareness of discovered DECLARE models:
Stefan Schönig, Claudio Di Ciccio, Fabrizio Maria Maggi, Jan Mendling: Discovery of Multi-perspective Declarative Process Models. ICSOC 2016: 87-103
Against the little learnibility of DECLARE: Michael Hanser, Claudio Di Ciccio, Jan Mendling: A New Notational Framework for Declarative Process Modeling.
Softwaretechnik-Trends 36(2) (2016) Guarantee of the significance of discovered DECLARE models:
Fabrizio Maria Maggi, Marco Montali, Claudio Di Ciccio, Jan Mendling: Semantical Vacuity Detection in Declarative Process Mining. BPM 2016: 158-175SEITE 83