Applicability of MDD to
Software Factories
The FAST Experience
ISP – Industrialisation of Software Production
1- Industrialisation Strategies
Industrialisation of Software Production
ISP as key expertise area for Tecnalia/ESI with more than 15 years of experience on ISP related proyects (14 FP7-EUREKA and 30 market references) Objective: To support an organisation to increase applications production capacity through a software development faster, cheaper and with a better quality through the industrialisation process of software production
There are several ISP
strategies, and there is no
one-size-fits-all technology
nor method that supports the
wide range of potential
scenarios. We group all
strategies in 4 groups.
Industrialisation of Software Production
Product Lines for product-centric reuse strategies
Software Factories for multi domain-centric reuse
Architecture Driven for reference architecture
constrained development
Community Driven for fuzzy and less structured
organisations
We select strategies based on characteristics such as reuse opportunities and risks analysis, current and desired reuse practices, technical context, productivity control and measurement and important issues of the organizational context such as company size, customer base, metrics of projects and products, etc.
ISP – Community Driven
Community Driven: GMC by Tecnalia
GMC Platform
GMCCrawler
Enterprise EcoSystem
GMCClones
GMC4Migration
GMCDeveloperProfile
GMC IDE
GMCLicenceAnalyst
GMCPeople
GMCDojo
ISP – Architecture Driven
Architecture Driven, examples:
GEMDE – MDE toolbox for different architectures
• MDE for RT Embedded systems with MARTE/SysML
• MDE for legacy modernisation with X2U and custom made DSLs
ISP – Product Line
Generic tool for the design, implementation and management of Software Product Lines following a MDSD (Model Driven Software Development) approach.
Version 3.0 available. Selected for the European CESAR RTP. CVL support.
Ready for verticalisations and joint delivery with ICT companies. FAST powered.
ISP – Software Factory – Next Generation
IIn
du
stri
ali
sati
on
Co
ntr
ol
& M
on
ito
r
Pro
du
ctiv
ity
to
ols
Pro
cess
Org
an
isa
tio
n
SO
A –
OS
Gi
Ecl
ipse
ID
E
2 – MDA, MDD, MDE, MDSD…
Model Driven X
Let’s not focus on the conceptual differences between MDA-MDD, MDE, MDSD..
Let’s focus quickly on basic ideas and value
Models Abstraction of an object/phenomena in the real world
Simplified view of the complex reality
Conforms to metamodels!
Metamodels Defines what a model looks like
A kind of model grammar (-meta)
A collection of concepts of a certain domain
Formal description ensuring information consistency
End-user
Developer
Sales and field support
Development manager
System administrator
Functionality
Architect
Ease of use
Performance and throughput
Reliability
Ease of customization
Price
Development costs, skills
On time delivery
Stability and maintainability
Ease of integration
Ease of diagnosing problems
Ease of introducing modifications
Testability and tractability
Structure and dependencies between parts
Ease of installation
Complexity
Functionality
Technology
Skills
Process
Organization and culture
Qualities
Business Concerns
FOR COMMUNICATION…
FOR INTEGRATION…
Analyze
Requirem.
Design
Validate
& Verify
Build Code
Requirements Manag.
Code GenerationModel CheckingCASE Tool
Performance SimulationSchedulability Analysis WCET AnalysisReliability Analysis
Common
Repository
FOR MULTIDISCIPLINARY APPROACHES…
Domain in Embedded
Systems Development
Ab
str
act
ion
Le
ve
l
So
ftw
are
Ele
ctr
on
ics
Me
ch
an
ics
Co
ntr
ol
Systems Engineering
Legend:
Artifact Dependencies: Relations among domain-specific artifacts and system-level artifacts
…
Development Artifacts: Design artifacts Product artifacts
…
…
…
…
…
Business Process …
…
…
Artifacts of varying abstractions and engineering domains Original slide from Peak (GIT), SysML
Presentation 2005
MD VALUE: DESIGN-BUILD-VALIDATE-VERIFY
• Configurations, dependencies,redundancy…
Evaluate architectural design alternatives
• reliability, performance, cost,… Multiple points of
view
• refinement/abstraction, multi-perspective, layering…
Best software engineering practices
MD VALUE: DESIGN-BUILD-VALIDATE-VERIFY
3 – FAST
• Software Factory:
– An organisation for the disciplined development of
software with engineering methods
• The Software Factory leads to innovation in:
The organisational structure Technical Office
The methods and techniques of work Domain Engineering / Application Engineering
The production tools FAST, among others.
Objective
• Differentiate the production means from the actual software production
Strategy
• Creation and development of a specific organisation: Technical Office
Functions
• Domain Engineering
• Create, develop and manage development means
Control
• Reuse
• Productivity
• Performance
What’s the Technical Office in this context?
• Normalisation and standardisation are the principles, the techniques and organisation are the instruments and productivity and quality are the results.
The Paradigm
• Reach high levels (>60%) of code reuse
• Produce software almost independent of its author
• Increase productivity and quality in the production of software
Objectives
• Solve the dilemmas of model based software production
• Models with models, or without models
• Adapt the technology to the factory domain
Strategies
The maturity of the
organisation
The development
culture
The initial workload of
the Technical
Office
Technology stacks
What difficulties can we come across?
New Generation
In 2009 one of the major Core Banking organisations in Spain requests a new generation of Software Factory technologies to Tecnalia. FAST.
What’s FAST ?
FAST is an Eclipse based tool to build Software Factories and not a software factory itself.
Our approach is to build an extremely configurable framework so you can use it to generate industrialised software production platforms
This basically means that you should have or plan to build a family of similar products or developments.
What’s FAST ?
FAST uses models, but models are invisible to the developer.
The code and artifacts generated have no dependencies with the factory that generated them.
You can use our software to build your own domain specific software factories using the tools and guidelines provided with it.
Why use Eclipse at all?
The customer answered this for us by making it a non-functional requirement.
”We are already using tools integrated in eclipse and we want to be able to keep integrating these tools and the new software factory, and anything else we might come up in the future.”
So we needed to build an
And what better way to do it by using eclipse to make it
And since we are using Eclipse, why not make use of every technology available to us?
And many more
Model Driven = EMF?
Safe to think so if you are working in Eclipse
But...
What if the end user does not use EMF models?
What if the end user does not even know what EMF-like models are?
What if the end user does not want to use graphical CASE tools?
There are people who actually do not like them!
What if the end user has its data/models in many different formats like text files and excel sheets?
EMF and models are good but not all the users like EMF or models at all!
Model driven without models?
The fact that the user does not perceive the
existence or interact with anything that they consider models does not prevent that:
What they are using are actually models, they are just not aware of this (nor do they need/should to be).
The internal representation of data and processes within the tool use models.
Technical Office engineers should be able to transform whatever data format to an EMF representation.
FAST capabilities
Generates/Builds things, mostly code but not only. And put traces everywhere!
Allows end users to decide the representation in which that are going to give instructions and parameters to the factory.
Allows easy development of transformations/scripts without relying in any specific transformation/scripting technology.
Allows development of transformations/scripts for non-model savvy people (using plain normal code and algorithms and data)
FAST capabilities
Allows concurrent non-blocking generations
Allows distributed generations and deployments of their factory
Any kind of distributed, not client-server only.
Integrates with distributed repository systems
Allows each company to use the factory as they want
The factory must adapt to the company, not the other way around
Allows data/control/GUI integration with other Eclipse and if possible non-eclipse tools
Which are some of FAST’s self imposed requirements?
Modularity
So we can reuse features developed for this project in other developments.
Standards compliant
OSGi and Java standards are to be followed and never broken.
Reduce third party API level dependencies
The APIs are always presented as Java interfaces and dependencies to libraries outside OSGi and Java SE are avoided.
The experience
It’s possible to use MD for the Technical Office, not exposing models to the developers
Domain Engineering assisted by models and transformations
Application Engineering working on native coding
Providers developed for different languages and sources
In the Core banking particular case:
COBOL developers working 10% to 20% faster than before
60% to 80% of sustainable reuse
Integrated with several Eclipse based tools (e.g. lifecycle)
Thank you very much.
Jason Mansell Productivity technologies lead
ICT/European Software Institute
Tecnalia [email protected]
Joseba Laka Market Director
ICT/European Software Institute Tecnalia [email protected]
4 – Annex
What is this about OSLC?
• Open Services for Lifecycle Collaboration (OSLC) – OSLC is a new approach to solving the problem of
integrating lifecycle tools • Standardize the way that software lifecycle tools can share
data with one another:
– Requirements
– Defects
– Test cases
– Plans
– Code
– …
– See: http://open-services.net/about/
– Members: http://open-services.net/participate/
Is OSLC just another middleware?
• No, OSLC is not middleware – It does not really define kind of API, instead it reuses already
existing standards.
• Communication is done via HTTP1.1/REST
• Basic semantics are inherited from RDF/DCMI specifications
• OSLC defines extended semantics to allow semantic data transfer between tools.
• OSLC sometimes looks more like a convention rather than a full fledged specification. – Proposes how data should be accessible
– How to create catalogs of the exposed resources
– How to navigate these catalogs
• For an application to be OSLC compliant it does not need to adhere to any middleware or architecture.
What are OSLC domain specifications?
• Definitions of how tools working for a particular domain should share their resources.
• Some finalized OSLC domains: – Change Management – Quality management – Requirements Management – Architecture Management – Software Configuration Management – …
OSLC Core Specification How to define OSLC resources. How to offer services.
How to inform clients of resource shapes. How to offer delegate UIs. How to offer query capabilities. How to offer resource creation. What authentication is allowed. How specification versioning works. How to represent OSLC defined resources.
OSLC Domain Specification Defines OSLC resources. Offers services.
May offer resource shapes. May offer delegate UIs. May offer query capabilities. May offer resource creation. Provides examples of representations.
What is the relationship between IBM Jazz and OSLC?
• Jazz based/powered tools implement OSLC specifications
• Jazz is a middleware – Which implements OSLC services – Facilitates exposing other tools resources as OSLC
• Jazz Integration Architecture (JIA) extends OSLC further – JIA defines Jazz Foundation Services
• Jazz team Server provides an implementation of those.
• Both are IBM initiatives. – Jazz is more IBM centric
• Latest versions of IBM tools moving towards Jazz -> OSLC.
– OSLC is more community centric.
• More information at: http://www.slideshare.net/snoopdave/innovate-2010oslcjazz
Why would someone like to integrate an IDE with FAST?
• Whatever you can do with the tools that are already known you want to keep doing it with the same tools. – Diminishes resistance to change.
• If the GUI is consistent between both tools the learning curve diminishes a lot. – The HMI between the tools uses the same channels and conventions.
• If both tools are integrated into a single IDE the end user only sees one tool. – Keeps thing simpler
• Reduces end user annoyance and frustration. – No need to switch between tools
• If you are generating code, you would like to see it right away in your IDE. – Same for any other thing you are generating and is managed by the
IDE.
• Why not? – Integration is generally a good thing if done correctly.
How can I integrate an Eclipse IDE with FAST?
• Several ways to integrate: – Install FAST modules together with RAD in the same Eclipse
environment • IDE integration • GUI unification • Can create plug-ins based on OSGi services or Java APIs to add
data and control integration to the applications.
– Use OSLC • Example: RAD integrates with IBM Rational Team Concert which is
built upon Jazz – Enables to create RAD plug-ins which can use provided FAST OSLC
services and vice versa.
– No need to be installed in the same Eclipse (or even the same machine)
• Control integration • Data integration • IDE and GUI integration still an option but not achieved freely
– Do both or something else • FAST is extremely flexible and modular
What integration approach is better between an Eclipse IDE and FAST?
• Deploy in the same Eclipse – The IDE
• RAD for example
– FAST’s Generation Client Services • Host APIs for generation management
– FAST’s Generation Client GUI • Eclipse based
– Desired FAST’s Request Grammar Editors • Might be Eclipse Editors • Might be editors based on the installed IDE • Might be external tools
– FAST’s Provisioning services • Resources access and cache
– FAST’s Networking services • Seamless network interaction with at least one communication supported (OSLC)
• Deploy in the same or other Eclipse’s within the organization’s network – FAST’s Generation Server Services
• Generator implementation
– FAST’s Provisioning services – FAST’s Networking services – FAST’s Persistence services
• Automated distributed storage abstraction services (for generation results and requests)
• The IDE and FAST client do not communicate but share the same environment
• FAST clients can be networked via OSLC to other remote FAST servers
Can FAST be used as replacement or complement of cheat sheets for design pattern enforcement?
• Yes! For both questions.
• Replacement – If all the user information for the pattern can be obtained at
the beginning of the guidance
• FAST can be used to create a batch generation
• Complement – If the information cannot be compiled at the beginning of the
process
• Cheat sheet are still a better approach – The automation of each cheat sheet can be implemented via a FAST
generation.
• One of the advantages of software factories is: – Standardization (relative to the organization) of all the
generated code.
• For architectural patterns the benefit is even greater.
What does FAST offer in regards of team collaboration?
• Depends on the definition of Team Collaboration – User management
• Authentication (via JAAS and OSGi User Admin) • Authorization (via OSGi Conditional Permission Admin)
– Process management • FAST offers APIs and services that allow it to be automated by process management and automation
tools. • Deep integration with Mylyn is planned but not critical at the moment.
– OSGi User Admin integration with Mylyn
– Brisges to FAST’s editors to Mylyn.
– Traceability • FAST automatically creates traces of the generations, with complete information about the
inputs/outputs and requesters. • Exposes traces as OSLC links. See: http://open-
services.net/bin/view/Main/OslcCoreSpecAppendixLinks
– Versioning • FAST Provisioning features and modules are being designed to offer seamless management of
versioned resources. – It relies on third party backends
– The integration via Eclipse’s Team Provision is granted.
• FAST is tool designed to allow the implementation of a wide selection of services that a software factory based development process would need. – The tool does not impose any methodology of its own.
• Although it comes with off the shelf support to implement Tecnalia’s approach
What does FAST offer in regards of process enforcement?
• FAST is designed to help automate development processes. – Mainly code generation
– Documentation
– Configuration generation
• It does this by allowing each organization to code their own rules, processes and languages – In their selected coding languages
– In their selected representation models
• Finally it provides a distributed batch generation environment. – All the data is gathered before a generation from one more heterogeneous
sources
– The generation is requested, then processed, the output stored and indexed and the requester notified.
• This allows organizations to ensure the quality of their products by taking developers through semi-automated processes – Generate as much as possible
• If the generators are well built the results will be homogeneous, predictable and easily maintainable.
– Only code where allowed • Furthermore, the special code each program might need can be asked in advance of
the generation and allow it to place it where appropriate.
What integration can be achieved with process management tools/features?
• Defining Users and Roles – FAST uses the OSGi UserAdmin Retrieve/Set the users/roles
• Manage Users (create/update/query) • Manage Roles (create/update/query)
– Each project area in Process Management tools can usually define the set of roles that are available within that project. • Fast provides APIs to automatically synchronize the OSGi
repository with User and Role information Providers. – Event and notification based
• Automation – FAST provides several control integration interfaces to enable
the automation of its usage. • REST services. • OSGi services. • Open Java API. • OSGi Events. • …
What integration can be achieved with team awareness tools/features?
• Tracing each element to the user that
– Generated it.
– Contributed to it.
– Created a GenBot.
– Created a work order.
– …
• Traceability is handled internally by FAST
– Traceability is stored within the log of each generation
– OSLC external link representation is available
• All traces can be represented via links
• These traces can eventually be stored in other tools.
What integration can be achieved with source control tools/features?
• FAST Provisioning Services – Handle versioned resources – Manage local caches for remote resources – Notification management – Automatic synchronization of resources – Querying via URLs – …
• But it need to work against a backend – Eclipse Team Provider APIs ease backend
implementation/integration – Not restricted to it. – Open enough to adapt to any kind of backed and
version control model.
What integration can be achieved with administration tools/features?
• FAST provides administration features: – User/Role management – Authorization for Roles – Network management – Remote FAST configuration. – …
• Most administrative tools in the market provide web frontends for these things – User management and LDAP integration
• FAST authorization rule APIs use OSGi Conditional Permission for their enforcement – Provides Adaptors and APIs to integrate rule sources
and providers.
What is the benefit of integrating FAST with Software Development Process control tools?
• FAST provides services and tools to IMPLEMENT Software Factory related processes – FAST does not define those processes nor does it
manage them
• Software Development Process control tools – Define software processes – Allow automation of those processes
• Integration allows: – Defining and implementing software factory based
development processes. – Intagrating automated generation features to your
existing development processes.
What are the benefits of integrating FAST with testing automation tools?
• Depends on what is tested and how it is tested – There might be little benefit
• or there might be a lot.
• FAST creates software product lines / factories – Product lines create similar products
• One test for everything (little benefit) – Product line usage ensures product compliance to the test
definitions » Ensures relevancy of test across all products.
• Customized tests per customized product (lots of benefit) – Benefit comes from product lining the tests
» For each generated product the tests for that specific product should be generated.
– Need to create TWO product lines » One for the products » One for the tests
What does FAST’s Functional Architecture look like?
Is FAST Service oriented?
It follows one so technically yes, but architecturally not really
OSGi services and compliance is used. But direct calls to the methods of the services are avoided as
much as possible between service implementations.
Event/Command/Message oriented!
OSGi EventAdmin service is used as the main control mechanism.
Events and their respective handlers define the commands/functionality exposed and required by each module
Application is reactive and context aware Easily extensible thanks to the whiteboard pattern employed
by the EventAdmin service
Why the difference between client and server?
Peer 2 Peer (P2P) is the reason.
Do NOT confuse with with Point 2 Point.
In a P2P network/architecture each node can act
as:
a client/consumer of services/functionality
a server/provider of services/functionality
Both simultaneously.
None and act as simple relay
In order to fully decouple the design of the client
from the server we use a messaging infrastructure.
The interfaces of the messages need to be know by both the client and the server
But the client does not know that the server even exists and vice versa
If it is event based why use services at all?
OSGi standards compatibility
The fact that we recommend and only communicate via messages does not mean that there might not be an scenario in the future where the services need to be accessed directly
Division of responsibility
The classes and modules implementing OSGi services focus on providing the functionality of the domain.
Data
Calculus
The EventHandlers are the responsible of calling the appropriate services and their operations to perform a certain command
Security, communications and other horizontal issues are handled at event/message level or above.
Integration with third party tools
There are more tools providing services for control than tools relying on event or messages.
How does FAST know what providers are available?
FAST AUTOMATED FACTORY OSGi service
This is service that we have implemented taking what we liked from the technologies defined in the previous slide.
It basically allows a user of the AutomatedFactory service to request the creation of an instance of an object just knowing the interfaces it needs to implement and the parameters for its construction/initialization.
Also allows querying for how many
implementations are available for a certain interface and notifications.
How does FAST handle any/all data types?
FAST should be able to handle all Java Types
Should because the only way to prove it doesn't is to find a data type it cannot handle which has not happened yet.
Internally it makes heavy use of reflection so basically, as long as reflection is available for a type and the object can be created into memory there
should be no problem to pass it as a parameter to the transformations/bots and use it.
Runtime expression languages are the system that FAST
uses to transform a representation of an object into an
actual Java Object
Thus FAST can handle virtually any data type which has a
Java implementation to handle it.
What are runtime expression languages?
Basically they are plain text or strings which
represent an expression or an object.
They need to be evaluated and upon evaluation they return a Java Object
OCL
XPATH
FAST String Substitution Language (VAR)
FAST Java Constructor Language
Any other....
These languages need to be evaluated in
runtime and for that to happen there must
be plugins providing a FAST
EvaluationContext.
What is a FAST Evaluation Context?
It is the interface that classes need to implement in order to register a new language provider.
Registration takes place automatically using the Automated Factory.
This interface is not simple, and the number of methods that need an actual implementation instead of an empty one will vary depending solely on the nature of the language.
How do you use the expression languages?
Via FAST Expression service
With an ID of the language you are trying to interpret it will allow you to get an Evaluation context for it.
Once you have the evaluation context you can populate it with variables and other contextual information (if relevant).
Then you just need to call the method evaluate with the string you want to evaluate, and optionally the return type you expect and a placeholder for errors for feedback.
If you can specify in you DSLs, GenRequestProviders, or anywhere a method for extracting the ID of the expression language and the expression content as a string the you can use this system.
Which means that if a new expression language gets supported, it will be usable automatically by any user of the FAST Expression service.