Which role might model-based engineering play in software certification?
Selo Sulistyo
2
1. Software Certification
2. Model-based Engineering (MBE)
3. Roles of MBE in Software Certification
4. Summary
Outline
3
Software certification is the term used to certify the quality of a software application/product against the functional and non-functional specifications before its release to production/ end-user. (Eushiuan Tran, 1999)
Definition
Software Certification
4
The software certification process establishes the level of confidence in a software system in the context of its functional and safety requirements.
Why is a certification needed?
Software Certification
5
Virtual Machine
byteCode
Generates
is executed by
What is being certified?Software Certification
Operating System
Programming Languages
Source Code
Is used to produce application
Running Application
Machine Codeis executed and managed byis executed and managed byis executed and managed by
Is compiled using
Compiler Generates
Assembly
Assembler
Development process
Computer Hardware
Product
6
Product basedExample: ISO IEC 14598. This international standard provides guidance for the practical implementation of software product evaluation when several parties need to understand, accept and trust evaluation results.
Process-based Example:IEC 61508 (Functional safety of electrical/electronic/
programmable electronic safety-related systems)
DO-178B (Software Considerations in Airborne Systems and Equipment Certification).
Certification Approaches
Software Certification
7
Verification and validation are part of the certification process for software system (Eushiuan Tran, 1999)
Certification process
Software Certification
Verification Validation Certification
Developer Functional Expert Certification Authority
8Development Testing
Was the product built right?
Was the right product built?
iteration
When and how the verification and validation are done?
Software Certification
artifactsrequirements
Requirement analysis
design
implementation
verification
deployment
Validation
Testing
Deve-lopment
9
The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase.
Was the product built right?
Verification does not ensure the application:– Solves an important problem– Correctly reflects the workings of a real world
process
Verification
Software Certification
10
The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements.
Was the right product built?
Validation
Software Certification
11
Verification and Validation Techniques
Software Certification
12
An example : DO-178B
Software Certification
13
An example : DO-178B
Software Certification
14
1. Software Certification
2. Model-based Engineering (MBE)
3. Role of MBE in Software Certification
4. Summary
Outline
15
An approach to engineering that uses models as an integral part of the technical baseline that includes
the requirements, analysis, design, implementation, and verification
of a capability, system, and/or product throughout the acquisition life cycle.
Definition
Model-based Engineering
Models
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
16
A physical, mathematical, or otherwise logical representation of a system, entity phenomenon or process
• Models are scoped to purpose• Models are appropriate to the context (e.g.,
application domain, life cycle phase)• Models integrated or interoperable across
domains and across lifecycle
What is a model?
Model-based Engineering
17
Software Model
Model-based Engineering
model
system
Modeling
language
is written in
describes
18
Jos B. Warmer and Anneke Kleppe defined Modeling Maturity levels (MML)
MML 0: No Specification MML 1: Textual Specification MML 2: Text with Models MML 3: Models with Text MML 4: Precise Models MML 5: Models only
The use of model in software development
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
Testing
Deve-lopment
19
Testing
Deve-lopment
MML 0 No Specification
My software
…
No specification
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
20
Testing
Deve-lopment
Specification of software is written down in one or more natural language documents.
MML 1 Textual Specification
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
21
Testing
Deve-lopment
Specification of software in one or more natural language documents plus several high-level diagrams to explain the overall architecture.
+
MML 2 Text with Models
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
22
Testing
Deve-lopment
Specification of software is written down in one or more models. Additional natural language text is used to explain the background and motivation of the models.
+
MML 3 Models with Text
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
23
Testing
Deve-lopment
Specification of software is written down in one or more models.
Natural language text is used to explain the background and motivation of the models.
The models are precise enough to have a direct link with the actual code.
MML 4 Precise Models
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
24
Testing
Development
The models are precise and detailed enough to allow complete code-generation.
The code is invisible (as assembler is today).
Modeling language High level programming language.
MML 5 Models only
Model-based Engineering
requirements
Requirement analysis
design
implementation
verification
deployment
Validation
25
MML 0: No Specification MML 1: Textual Specification MML 2: Text with Models MML 3: Models with Text MML 4: Precise Models MML 5: Models only
Modeling Maturity Levels (MML)
MBE !
Model-based Engineering
26
OMG’s Model-driven Architecture
Model-based Engineering
The current state of the art in MDE is much influenced by the ongoing standardisation activities around the OMG Model Driven Architecture® (MDA®).
MDA is a framework which defines a model-driven approach to software systems development.
Focus on the solution of the problem rather than the implementation problem
Raising the level of abstraction, dealing with complexity
An automatic code generation
27
Models in MDA (Model-Driven Architecture)
Model-based Engineering
CIM: Computational-independent
models
PIM: Platform-independent Models
PSM: Platform-specific Models
RequirementAnalysis
Implementation
Low-level design
High-level design
PIM
Code
PSM
TT
TT
Code
PSM
TT
TT
Code
PSM
TT
TT
CIM
TT
TT: Transformation Tool
28
1. Software Certification
2. Model-based Engineering (MBE)
3. Role of MBE in Software Certification
4. Summary
Outline
29
How does model-based engineering produce applications?
Modeling Language
Models
Tools and Production processes
Transformation tool
e.g Code Generator
??
Role of MBE in Software Certification
Operating System
Programming Languages
Source Code
Programming Languages
Source Code
I s used to produce application
Machine Codeis executed and managed byis executed and managed byis executed and managed by
Is compiled using
Compiler Generates
Assembly
Assembler
Machine Codeis executed and managed byis executed and managed byis executed and managed by
Machine Codeis executed and managed byis executed and managed byis executed and managed by
Is compiled using
Compiler Generates
Assembly
Assembler
I s compiled using
Compiler Generates
Assembly
Assembler
Compiler Generates
Assembly
Assembler
Computer Hardware
Virtual Machine
byteCode
Generates
is executed by
Virtual Machine
byteCode
Generates
is executed by
Virtual Machine
byteCode
Generates
Virtual Machine
byteCode
Generates
is executed byRunning Application
Products
30
Role of MBE in Software CertificationCertification
One central goal of MBE is to enable analysis of the system, thus ensuring the quality of the system already on the model level.
early verification and validation
31
requirements
Requirement analysis
design
implementation
Role of MBE in Software CertificationWhen and how the verification and validation are done?
How to verify these software
models?
Certification Authority
verification
deployment
Validation
Model Verification
Model Validation
Verified and Validated Software Models
…..
Models
Was the model built right?
Was the right model built?
Was the model built right?
32
Role of MBE in Software CertificationModel verification
Verification is done to ensure that:
– The model is developed correctly and conformance with standard
– The algorithms have been implemented properly
– The model does not contain errors or bugs
Verification does not ensure that the model:
– Solves an important problem
– Meets a specified set of model requirements
– Correctly reflects the workings of a real world process
33
Role of MBE in Software CertificationVerification techniques
Verification techniques that can be used:
– Model checking
– proof of correctness
– consistency checking
This will ensure that the models are
Consistent
Correct against to language syntax of the language
34
Role of MBE in Software CertificationModel validation
Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained. Compliance with requirements
The ultimate goal of model validation is to make the model useful in the sense that the model addresses the right problem, provides accurate information about the system being modeled, and to make the model actually used.
35
Role of MBE in Software Certification
B
B
BA
A
A
CIM
PIM
Code
PSM
TT
TT
TT
Model verification and validation
Code
PSM
TT
TT
B
Product familyProduct family
B
A
A: Verification
- Consistency
- Conformance with standard
B: Validation
- Compliance with requirements
- Traceability
TT: Transformation Tool
36
Models provides a view to a complex problem and its solutions.
Models are less risky, cheaper to develop and easier to understand than the implementation of a genuine target system
By assuming that the transformation tools is verified then the verification process can be done in model level, that obviously it saves time and cost.
It also improves the quality of design and software code by better assuring the traceability between artifacts.
Role of MBE in Software CertificationBenefits MBE in software certification
37
MBE can play role on:
Process-based certification.
Early verification process. Activities for verification, validation, and test with Model-Based Engineering can be applied at every stage of the development process.
The model traceability will minimize the certification efforts, especially for product families. reducing certification and re-certification times,
Saving time and cost since the verification is done on higher abstraction levels, but it may be a complicated process with the current MDE technology.
Summary
38
Thank You
39
1. Jos Warmer, The role of OCL in the MDA, Klasse Objecten, http://www.klasse.nl
2. Eushiuan Tran, Verification/Validation/Certification, Carnegie Mellon University, 18-849b Dependable Embedded Systems, Spring 1999
3. Rajwinder Kaur Panesar-Walawege, Mehrdad Sabetzadeh, Lionel Briand, Using Model-Driven Engineering for Managing Safety Evidence: Challenges, Vision and Experience, Simula Research Laboratory, University of Oslo, Lysaker, Norway
4. Ewen Denney and Bernd Fischer, Software Certification and Software Certificate Management Systems, USRA/RIACS, NASA Ames Research Center, Moffett Field, CA 94035, USA
5. OSMAN BALCI, A Methodology for Certification of Modeling and Simulation Applications, ACM Transactions on Modeling and Computer Simulation, Vol. 11, No. 4, October 2001, Pages 352–377.
6. Damian Dechev, Bjarne Stroustrup, Model-based Product-Oriented Certification, Texas A&M University College Station, TX 77843-3112
References