Date post: | 12-Jan-2016 |
Category: |
Documents |
Upload: | catherine-ray |
View: | 218 times |
Download: | 1 times |
1
Sundar Gopalakrishnan, Guttorm Sindre, and John Krogstie:Adapting UML activity diagrams for mobile work process modelling: Experimental comparison of two notation alternatives
PoEM 2010 Delft, 9.-10. Sept.
2
Agenda
• Motivation• Background• Research method• Results• Discussion of threats to validity• Summary and further work
3
Traditional perspectives to modeling• Structural• Behavioral• Functional• Goal and rule-oriented• Object-oriented• Social communication• Actor/role-oriented
What about location/place/space... ?
4
Zachman Framework
21e.g. DATA
ENTERPRISE ARCHITECTURE - A FRAMEWORK
Builder
SCOPE(CONTEXTUAL)
MODEL(CONCEPTUAL)
ENTERPRISE
Designer
SYSTEMMODEL(LOGICAL)
TECHNOLOGYMODEL(PHYSICAL)
DETAILEDREPRESEN- TATIONS(OUT-OF- CONTEXT)
Sub-Contractor
FUNCTIONINGENTERPRISE
DATA FUNCTION NETWORK
e.g. Data Definition
Ent = FieldReln = Address
e.g. Physical Data Model
Ent = Segment/Table/etc.Reln = Pointer/Key/etc.
e.g. Logical Data Model
Ent = Data EntityReln = Data Relationship
e.g. Semantic Model
Ent = Business EntityReln = Business Relationship
List of Things Importantto the Business
ENTITY = Class ofBusiness Thing
List of Processes theBusiness Performs
Function = Class ofBusiness Process
e.g. Application Architecture
I/O = User ViewsProc .= Application Function
e.g. System Design
I/O = Data Elements/SetsProc.= Computer Function
e.g. Program
I/O = Control BlockProc.= Language Stmt
e.g. FUNCTION
e.g. Business Process Model
Proc. = Business ProcessI/O = Business Resources
List of Locations in which the Business Operates
Node = Major BusinessLocation
e.g. Business Logistics System
Node = Business LocationLink = Business Linkage
e.g. Distributed System
Node = I/S Function(Processor, Storage, etc)Link = Line Characteristics
e.g. Technology Architecture
Node = Hardware/SystemSoftware
Link = Line Specifications
e.g. Network Architecture
Node = AddressesLink = Protocols
e.g. NETWORK
Architecture
Planner
Owner
Builder
ENTERPRISEMODEL
(CONCEPTUAL)
Designer
SYSTEMMODEL
(LOGICAL)
TECHNOLOGYMODEL
(PHYSICAL)
DETAILEDREPRESEN-
TATIONS (OUT-OF
CONTEXT)
Sub-Contractor
FUNCTIONING
MOTIVATIONTIMEPEOPLE
e.g. Rule Specification
End = Sub-condition
Means = Step
e.g. Rule Design
End = ConditionMeans = Action
e.g., Business Rule Model
End = Structural AssertionMeans =Action Assertion
End = Business ObjectiveMeans = Business Strategy
List of Business Goals/Strat
Ends/Means=Major Bus. Goal/Critical Success Factor
List of Events Significant
Time = Major Business Event
e.g. Processing Structure
Cycle = Processing CycleTime = System Event
e.g. Control Structure
Cycle = Component CycleTime = Execute
e.g. Timing Definition
Cycle = Machine CycleTime = Interrupt
e.g. SCHEDULE
e.g. Master Schedule
Time = Business EventCycle = Business Cycle
List of Organizations
People = Major Organizations
e.g. Work Flow Model
People = Organization UnitWork = Work Product
e.g. Human Interface
People = RoleWork = Deliverable
e.g. Presentation Architecture
People = UserWork = Screen Format
e.g. Security Architecture
People = IdentityWork = Job
e.g. ORGANIZATION
Planner
Owner
to the BusinessImportant to the Business
What How Where Who When Why
John A. Zachman, Zachman International (810) 231-0531
SCOPE(CONTEXTUAL)
Architecture
e.g. STRATEGYENTERPRISE
e.g. Business Plan
TM
5
Motivation
• ’Where’ is increasingly relevant– Outsourcing– Supply Chain Management/Logistics– Virtual organization– Mobile applications and information systems
• And it is possible to utilize ’where’ to a larger degree (also real time) to know where users, equipment and goods should be, are or where at a certain time
– Tracking (RFID, UWB, GPS, GSM, WiFi, Ultrasound…) – Internet of Things (IoT)
• This paper: – Presents some different notation alternatives based on UML activity diagrams vs. Modeling mobile
information systems– Summarizes an analytical evaluation from a previous paper (I-ESA’10)– Makes an experimental comparison of the two most promising ones
6
Possible notations looked at(I-ESA’10)• Standard UML, using annotation boxes to indicate
context or location• Redefining swimlanes to indicate context / location• Using colour for context / location
• These three were compared analytically, using a home care case supported by a mobile IS as an example
7
Alt 1: Using annotation boxes
• Advantages– Smallest deviation from standard
UML AD– Enhanced understandability for those
who already know AD
• Disadvantages– Greatly increases # nodes in diagram
(poor expressive economy)– May be confusing if you also need to
use notes for something else in addition to context / location
8
Alt 2: Swimlanes for where• Advantages
– Shifting ”who” to stick figures: AD more uniform with UCD
– Swimlanes intuitively indicate location
• Disadvantages– Many lines from stick figures to
activities, poor readability (would be even worse with bigger example)
– New usage of swimlanes may confuse those already familiar with AD
9
Alt 3.: using colour• Advantages
– No increase in # nodes or lines, better readability
– With two process design alternatives beside each other, it is easy to spot differences in location
• Disadvantages– Larger deviation from
standard UML
– Must add legend
– Possible challenge for colour blind users
10
Analytical comparison
Analytical comparison (I-ESA’10):• Alt 1. (trad. UML with annotations) and Alt 3.
(Colours) came out as the two most promising• Proposed further work: Make experimental
comparison
Notation Minimal deviation from
standard
Expressiveness Intuitive / Easy to
read
Less Complexity
Simple Large Simple Large Simple Large Simple Large
Annotated + + + + - - - + - -
Location Swimlanes - - + + - -- + - -
Colours - - + + + + + + + + + +
11
Experimental design• Compare two diagram alternatives, annotation and colour• Controlled experiment looking at the participants’
– Performance using the notation
– Opinion about the notation
• Within-subjects design (Latin squares)– Controls better for selection bias
– ”Doubles” the N
• Measured variables:– Performance:
• Understanding: score on 12 True/False questions about a case after reading textual description and seeing diagram
• Error_detection: score on identifying 5 deliberately seeded errors in a diagram relative to textual description
– Opinion: • Perceived Ease of Use, Perceived Usefulness, and Intention to Use, measured by
answers to a TAM-inspired questionnaire w 14 questions
12
Hypotheses
• Since the colour notation was best in the analytical comparison, this was hypothesized to have advantages, i.e.:– H1: understanding scores will be better for the colour notation than
for the annotated notation
– H2: error detection scores will be better for the colour notation than for the annotated notation
– H3: participants’ opinion about the colour notation will be more positive than for the annotated notation
13
Experimental tasks1. Answering a pre-experiment questionnaire investigating
relevant competence
2. Reading tutorial about first diagram notation (annotatated or colour depending on group)
3. Reading text and diagram for case (home care or traffic control), answering 12 T/F questions
4. Answering post-task questionnaire giving opinion about the notation
5. Repeating steps 2-4 with the opposite case and notation
6. Repeating 2-6 with the error detection task
14
Latin squares designGroup Id (Understanding+ TAM
factor) Questionnaire onError Identification Questionnaire on
Group A Annotated Home Care +Colour Traffic Control
Annotated traffic Control + Colour Home Care
Group B Colour Traffic Control +Annotated Home Care
Colour Home Care +Annotated traffic Control
Group C Annotated traffic Control + Colour Home Care
Annotated Home Care +Colour Traffic Control
Group D Colour Home Care +Annotated traffic Control
Colour Traffic Control +Annotated Home Care
15
Results• 46 students participated, randomly assigned into the
four Latin squares groups• Clear advantage for colour notation in performance
– 3 students performed very poorly in error detection (not doing a serious job); might be considered outliers?
• Slight, but not significant advantage for colour when it comes to opinion
Compared variable (N=46)
Coloured diagram
Annotated diagram
Diffe-rence
EffectSize
Sign.? Y/N(p-value)Mean SD Mean SD
Understanding 0.960 0.057 0.926 0.078 0.0344 0.51 Y (0.01)Error detection 4.50 1.34 3.93 1.17 0.57 0.45 Y (0.001)Errors (w/o outliers, N=43)
4.77 0.53 4.16 1.04 0.60 0.77 Y (0.001)
16
Conclusion on hypotheses
• H1: coloured notation would be better than annotated for understanding (answering T/F questions): CONFIRMED
• H2: coloured notation would be better than annotated for detecting errors: CONFIRMED
• H3: participants’ opinion about the coloured notation would be more positive: REJECTED
17
Threats to validity
• Conclusion validity: significant results but small to moderate effects, should have had larger N to make a strong claim about results
• Construct validity: many other ways of understanding a model than answering T/F questions, and many other work tasks than identifying errors. But at least, this is a relevant task, and the ability to answer questions correctly should indicate to some extent whether a model has been understood
18
Threats to validity, cont.• Internal validity: Latin squares design and pre-exp.
questionnaire should control very well for any selection bias. Notations were presented in equal detail and style in tutorials, and no preferred or hypothesized outcome was signalled to the students
• External validity: The biggest challenge:– Students are not practitioners and motivation may be limited in an
experiment which has no impact on their job or a delivered product. But the comparative nature of the experiment should mean that performance with both notations are equally hurt by low competence or motivation.
– Small experimental tasks are not representative of the more complex tasks in ”real” mobile IS development.
19
Further work
• Also experiment with different pattern fills instead of colour (e.g. better for colour blind users)
• More experiments, possibly including practitioners and/or including collaboration among several persons instead of just individuals answering questions
• Larger industrial case studies, to try out alternative notations with larger and more realistic work tasks