Run-time Monitoring-based Evaluation and
Communication Integrity Validation of Software
Architectures
Ana Dragomir
02.12.2014
Motivation
State of the art vs. state of the practice
Goals
Approach
Evaluation
Summary and outlook
The ARAMIS Project
Motivation
“You don’t need architecture to build a dog kennel, but you’d better have some for a skyscraper”
Software architecture (SA) description is useful to understand
and meaningfully evolve a software system but…
“Is it just a shared hallucination?”
…the architecture drifts from its description!
The description:
is no longer as useful
can lead to misunderstandings
domino effect
Problem Statement
SHOULD IS
VS.
Stakeholders
Architects
Software Landscape Architects
Are the systems of the landscape interacting as prescribed?
Which systems are the hubs and sinks?
Software Architects
Is the system built according to its description?
Which architecture units are the hubs and sinks?
Which architecture units are “too complex”?
Software Developers
If I need to change a requirement, how can I quickly find out
how it is currently implemented?
What architecture/software units are interacting & how?
State of the Art
Related Work
State of the Art
Related Work
State of the Practice
Our Experience: Company 1
CMMI Level 3
More than 1000 IT Employees
Large, heterogeneous systems (Java EE, Cobol, …)
System architecture Decisions and architecture are documented, though separately
Code-quality monitors (e.g., Sonarqube) are used
Enterprise architecture Inter-system information flow diagrams
Manual proces
No architecture reconstruction tools! “Read the documentation, then start making phone calls”
Reconstruction is cumbersome and must be supplemented with support for evolution
State of the Practice
Our Experience: Company 2
Medium Enterprise
More than 500 IT Employees
Mainly Java-based systems
System architecture
“The Developer Handbook”
Low-level descriptions
Abstract view of the system missing
Joint attempt with SWC to reconstruct the architecture
Purchased Sonargraph Architect
Encountered terminology differences
State of the Art
Flaws
The reconstruction occurs on very different abstraction levels
Structural view: layers, modules, subsystems, etc.
Behavioral view: objects, methods, etc.
Heterogeneity of terminology is not addressed
Reconstruction tools have stiff meta-models
The architects expect results that conform to their terminology
Heterogeneity of systems is not properly addressed
The interplay of heterogeneous systems is important
Reconstruction by itself is not a goal
Are the units behaving as expected? What are the deviations?
Evolution support is needed!
Goals
Improve the traceability between usage scenarios, implementation
and architecture documentation
Develop/Use a minimally intrusive technical solution for run-time
monitoring and real-time visualization
Provide a means to evaluate if the predetermined architecture
rules have been respected
Easily extendable solution
Multi-level Behavior Monitoring
XX
X Y
MM
MTo
From
Operation Name
Parameters
Monitoring
Syste
ms T
o M
on
ito
rA
na
lysis
m2 m3
m4m6
AA
A Bm1
m10
m7
m8
m9
Interest: communication
between AA, XX, MM
AA XX MMm2m5
m6
m8
m9
Interest: communication
within AA
A Bm1
m10
m7
Interest: High level
violations
AA MMm8
m9
Architecture
ARAMIS for Communication Integrity Checking and Evaluation
Architecture Meta-model
Metrics @ ARAMIS
Goal: find weak points in the analyzed behavior
Step 1: Characterize behavior
Step 2: Compare previous result with the architects’ expectations
Metrics Categories
Behavioral coupling and cohesion metrics
Behavior hotspots
Violations-based metrics
ARAMIS
Coupling and Cohesion Metrics
Behavioral Coupling (BCo)
Behavioral Cohesion (BCh)
Scenario-based Unit Behavior Metric (SUB)
Cohesion vs. Coupling of an Architecture Unit
SUB = 𝐵𝐶ℎ
𝐵𝐶ℎ+𝐵𝐶𝑜
SUB Characterization (SUBC)
High coupling/low cohesion; SUB ∈ [0, 0.5)
Mid coupling/mid cohesion; SUB ∈ [0,5, 0.66)
Low coupling/high cohesion; SUB ∈ [0.66, 1]
Evaluation
Phase 1: validation of communication rules
The MosAIC software system
Incentive: an initial architecture description was available
interest in evaluating its conformance
111753 LOC
116 classes, 10 packages
Defined 4 top-level units & 16 inner units
20 allowed rules
>100000 monitored calls
3 distinct architecture violations (frequencies: 1, 2, 22 resp.)
The result was used to improve the architecture
Increased confidence in MosAIC’s quality
Evaluation
Phase 2: relevance of the SUB and SUBC metrics
The JHotDraw framework
Incentive: presumably, a good designed architecture
126068 LOC
529 classes, 38 packages
Defined 12 top-level units but only 7 were used at run-time
No rules
> 150000 monitored calls
Evaluation
JHotDraw results
Conclusions
ARAMIS aims to:
Support the understanding of software systems
Validate their communication integrity
Assess their behavior
The current results are promising
Evaluated two projects
MosAIC
JHotDraw
Outlook
Develop relevant multi-level visualizations
Experiment with monitoring the inter-play of heterogeneous systems
Model evolution scenarios
Simulation & Impact analysis
Explore further relevant metrics to depict the quality of the
architecture
Flexible quality model, tailorable according to the specific needs of the
architects
Summary
ARAMIS
Flexible approach towards
Monitoring architecture
Validating its communication integrity according to specified rules
Assessing its quality
Supporting a reasonable evolution