1

The Ladder To Abstraction

Matthew Fowlermatthew.fowler@nte.co.uk

2

Agenda

• Introduction, Background• Techniques

– Pluggable Meta-models– Logical Modelling– Externalisation of Control– Model-to-model Transforms (Patterns)– Two-dimensional inheritance– Framework approach

• Conclusion

3

The Ladder Of Abstraction

• Chunking up– Higher levels of abstraction

• More power of thought and action

• Modelling - the more abstract the better/faster

• Removal of detail

• Relativity: logical and physical

• Chunking down: recreating the "lost" information– extract from super-classes – multiple outputs per concept

4

JeeWiz Introduction

• 2001: BEA: – "Stop customers messing up Architecture"

• Pre "MDA"/"MDD"• Goal: Express & automate

– the integration programmers do– best practice use of frameworks

• Large-scale end-to-end system generation– UI generation a difficult challenge– Challenges => new concepts and techniques

5

• Velocity as templating engine• Required Java model objects as "the context"

• Classes for Java Objects = meta-model• Problem: construction and maintenance

Background

ModelJava

model objects

Templates

Text

<entity name="Customer">

class Entityfield name="Customer"

1. Pluggable Meta-models

Velocity

6

Challenges

• 1. MaintenanceSize of meta-model (Java) codeDifficult to add new features

• 2. How to do .NET, new frameworks– But re-use surrounding pieces

Pluggable Meta-models

7

Solution 1. Meta-models

• Define in "Simple XML"• Generate Java classes for model obects

75% of meta-class code generated

• Very simple constructs: (cf EMF)– <meta-model>, <meta-class>– <property>, <list>, <validator>– c-data: Java impl, expressions, descriptions

• 320 meta-classes in all JeeWiz meta-models• (Sometimes we don't bother with MMs)

Pluggable Meta-models

8

ExamplePluggable Meta-models

<meta-class name="entity" extends="business-object" description="..." > <property name="table-name" /> <property name="generate-table-schema" type="boolean" default="true"/> <list name="query" description="Select and finder methods." > <validator message="An entity must have at least one attribute" expression="getAttributeList().size() > 0" />// getKeys gets either the autoKey or the primary key attributes listpublic List getKeys(){ initLists(); return ( autoKeyAttributeList.size() > 0 ) ? autoKeyAttributeList : keyAttributeList;}</meta-class>

9

Solution 2. Pluggability

• Multiple meta-models per build– Current record -

13 meta-models in a stack

• Late binding: configure stack at generate-time

• Modular• Abstract/Physical

separation Java

Bizobject

Screen

Spring

JSF

Company

Java

Bizobject

Screen

J2EE

WebSphere

Company

Pluggable Meta-models

10

Cross-MM Inheritance

Reference TypeClass

Business Object Entity

EJB Entity

JDO Entity

Entity

Java Meta-model

Business Object Meta-model

J2EE Meta-model

Pluggable Meta-models

Impl Class

11

Benefits

• Concept inheritance– building up the ladder of abstraction

• as far as you need to go

• go beyond classes (e.g. page components)

– inheritance through multiple meta-classes/models

• Modularity and reusability• Pattern for addressing new

technologies/processes

Pluggable Meta-models

12

Challenges

• Modellers & UML Tools– One set of concepts (stereotypes)

• Need to remove concepts (e.g. 'physical' detail)– e.g. public/protected/private in Java– fixed for business objects and business methods

• <business-method> => public access

• Fixups for modelling tool deficiencies– e.g. constraints in RSA/RSM

• Customer modelling requirements

2. Logical Modelling

13

Background

• Additional models

• Typically used as control information• Additional models normally "Simple XML"

ModelJava model object tree

Templates

Artifacts

AdditionalModel(s)

Added as second-levelobject in tree

Logical Modelling

14

Solution

Screen

Screen

Business Object

Java

CustomisedMeta-model,

ready for- modelling tool- customer envt.

UML Tool SpecificProfile

Business Object

Java

Aggregate Filter Convert

• Modular profile construction• No limit on # meta-models/meta-classes• Adaptation to UML tools and customer reqts.

Logical Modelling

Filter Spec

15

Benefits

• Separation of concerns– Aggregation– Filtering (& compensation for UML tool deficiencies)

• Filter customisation by end user• Easy to produce a "logical profile"

– Any level of abstraction– Reduction in number of concepts and detail

• General - any combinations of meta-models– separate adaptation to modelling environment

Logical Modelling

16

Interlude

• That was the easy bit - building the abstractions• Now for the interesting bit:

– Bridging the semantic gap to operational systems• Expansion: adding in the 'physical'

• Derivation: PIM1 → PIM2 concepts

– Variation: adapting to customer requirements– User Interface generation– Incorporating "Best Practice"

• Smart use of the technical platform

17

Challenges

• Component assembly from model objects• Build automation

– Move the workflow into the engine– "Externalisation of control"

• Number of configuration properties• (Let's face it)

We don't like build/configuration in Ant

3. Externalisation of Control

18

Solution

• A feature-rich framework for MDD• External workflow

– initialisation (overall and per model object)– model-to-model transforms and validation– " model-to-artifact"

(i.e. create various artifacts using Ant, deploy etc.)

• Pull in features by well-known names, e.g.– "component.properties" sets model object props– "build.xml" builds a model object using Ant– "uptodate.xml" and avoiding rewriting

Externalisation of Control

19

Benefits

• Model-to-model transforms– Created model objects produce output

• No additional config / selection required

– Automatic expansion of model object tree (x 30)• Revisit later

• Complete generation systems as components– use automatically in a larger build

• e.g. Java Jar build within overall application build

– e.g. model-driven deployment solution

Externalisation of Control

20

Challenges

• Building DotNet and J2EE– Too many similar M2Text transforms

• One for .NET, another for J2EE

– Only M2Text available, mixed together• intention (give me some data behind a page)

• implementation (here's a class generation)

– Aspects difficult to do • e.g. change class tracing at one point

• Jan 03

4. Model-to-model Transforms

21

Operation

• Transforms in templates with well-known names• Produce XML text as output

• Embellishes, or creates new, model objects• "Pattern" - practical application of design pattern

Model-to-model Transforms

ModelModel

objects

Model toText

Artifacts

Model toModel

+

<xml ...>

Velocity Scripts

22

Sample Pattern

• Entity: fix up for missing keys

• Various types of patterns:– "Fixup" - fix myself or my peers, before validation– Create other objects

#if( $keys.size() == 0 ) <this>

<attribute name="oid" autokey="true" type="Integer" />

</this>#end

Model-to-model Transforms

23

Details• Separation of concerns

– intention = original model objects– elaboration = generated model objects– implementation - downstream artifacts produced

• "<page>" different on Struts/JSP/Spring MVC

• Do it in place - enhance the input model• Completely automatic; regen every run

– No update by developers• This is where architecture is generated

• Tinkering by developers is a bad idea!

Model-to-model Transforms

24

Cross-Tier/MM Patterns

Entity

Relation

DAO

DAO reln

AttributeDAO attribute

MaintenanceSession

DAO

PagesSearch/Select/Edit

ButtonsCRUD

M2M Transforms

25

Benefits

• Implement higher concepts via lower ones?• Expansion

– half of the "fill in the physical part" solution

• Key to "brains in a box"– executable design patterns– create architecture, e.g. derived tiers, logically

• Understandable by developers– no new languages to learn; easy to change

• Intermixing of logic and output architecture

Model-to-model Transforms

26

Background

• Many existing features for variation– properties files for system init– properties files per model object– links to Java

• But for composability... we need more

2D Inheritance

27

The Challenge

• Reuse of "Renderings" – Higher-level concepts build on lower-level impl– Customer does too - naming, logging, security, L&F

• Variant Injection - easy insertion into lower layers• Java Inheritance is not enough• Variant generation at similar logic levels

– C#, Java– .NET, J2EE (some similarities)– Struts/JSF (different)

5. 2D Inheritance

28

Solution 1

• Generalise meta-model stack– "Domain-specific generation plug-in"

2D Inheritance

Java

Bizobject

Screen

J2EE

WebSphere

Companyper-level config, namingconventions

per-level macros

config, namingconventions

methods

Ant build.xml

transforms

Object Type

29

Details

• Operation– Search top-down– Merge - add new names (inheritance)– Override - existing names

• Externalisation of control – create well-known name-files at any level– only linkage is configuration of stack at build time

2D Inheritance

30

Per-object inheritance

• Second level of inheritance... groundwork:

Bus. Object Entity

EJB Entity

JDO Entity

Entity

Java Meta-model

Business Object Meta-model

J2EE Meta-model

Impl Class

2D Inheritance

Bean

Reference TypeClass

New

31

Bean model-to-model

<this access="private">#foreach( $field in $fieldList ) #if( $field.readable )

<method name="get${field.nameCapitalised}" return-type="${field.type}" access="public" > return ${field.name}; </method>

#end #if( $field.writeable )

<method name="set${field.nameCapitalised}" return-type="void" access="public" > <parameter name="${field.name}" type="${field.type}" /> this.${field.name} = ${field.name}; </method>

#end#end

</this>

2D Inheritance

32

Solution 2

• We need two dimensions of inheritance1. Model object type (bean → class)

2. Generation plug-in (BOM → Java MM)

• Details– Separate from meta-class (Java) inheritance– Under control of meta-programmer

• 'BOM:bean' says: "now try 'class'" object type

• Redirections search the whole stack of plug-ins– Override at multiple layers/concepts

2D Inheritance

33

Benefits

• Incremental creation of generation plug-ins• Use of "inherited" plug-ins

– Above M2M example works for Java or C#

• Plug-in replacements - Struts/JSF/SpringMVC• Variant injection

– Change small details in a lower-level plug-in• e.g. "is()" for beans

• Flexible approach to climbing down "the ladder"

2D Inheritance

34

Challenge

• Encapsulating is hard– Architecture, design patterns, best practice– Sheer volume

• 1000's of meta-programs

6. Framework Approach

35

Solution

• Many other features/refinements– Non-model "controls" for generating variant text

• In Java, Velocity or both

– OO in Velocity - methods, inheritance, overriding– 180 methods

• Framework (externalisation of control)

Framework Approach

36

Benefits

• Less volume in the meta-programs• Easier to

– Express best practice and architecture– See the wood for the trees– Change

Framework Approach

37

Conclusion• Feature Checklist

• Pluggable Meta-models, Logical Modelling

• Externalisation of Control

• Model-to-model Transforms (Patterns)

• Two-dimensional inheritance

• Framework approach

• Relevance– More powerful frameworks

• less need for simple codegen

• more need for – integration / best practice / architectural generation