Extension and EvolutionIN4308

Model-Driven Software DevelopmentLecture 12

Eelco Visser

Software Engineering Research GroupDelft University of Technology

Netherlands

May 25, 2010

GPLprogram

GPLprogram compilecompile 'machine'

code

'machine'code

'model''design'

'model''design' codecode

Conventional Software Development

Conventional Software Maintenance

GPLprogram

GPLprogram

understandunderstand

'machine'code

'machine'code

'model''design'

'model''design' compilecompile

modifymodify

abstractions encoded in programmaintenance at low level of abstraction

GPLprogram

GPLprogram compilecompile 'machine'

code

'machine'codegenerategenerate

DSLprogram(model)

DSLprogram(model)

raise the level of abstraction to a technical or application domainautomatically generate implementation code from model

Model-Driven Software Development

modifymodify

maintenance on models instead of implementationimplementation regenerated after modification of models

Model-Driven Software Evolution

What makes a good DSL?

Expressivity

expressive dsl requires (much) less code than gpl implementationfewer lines of code = less implementation effort

Coverage

dsl covers a subset of programs in the target languagegenerator is not surjective

Completeness

incomplete code generation produces programs with holesto be filled in by developer

Portability

dsl-to-csharpdsl-to-csharp

abstract over the target domaingenerate code for multiple platforms from same model

C#C#

JavaJavadsl-to-javadsl-to-java

Code Quality

does generator produce robust and correct code?are errors detected early in the translation pipeline?

model verification is preferable over debugging target code

DSL Engineering Dimensions

● expressivity

● coverage

● completeness

● portability

● code quality

● evolution

Coverage

dsl covers a subset of programs in the target languagegenerator is not surjective

not all target programs are reachable

Coverage of 'Interesting' Programs

targeting representatives of equivalence classescan we reach all equivalence classes?

Program Equivalence

when are two programs the same?

Many programs are the same if we abstract over 'implementation details'.

Program Equivalence

Two programs are the same if they have the same

- behaviour- userinterface- effects on the database- performance

What is the meaning of 'same'?

Abstract over non-important differences

Implementation details(e.g. order of statements does not always matter)

Look and feel

Reasons for Lack of Coverage

Completeness

only part of the domain is covered

Example: domain model vs business logic

80/20 rule : generate 80%, write 20%

Tuning

the generated code is not exactly what is desired

Customization of Generated Code

not all customizations can be realized in models generated code may need to be adapted

modify

Scaffolding (aka Fill in the Blanks)

incomplete code generation produces programs with holesto be filled in by developer

Scaffolding (aka Fill in the Blanks)

'boilerplate code' is generated'business methods' should be

filled in by developer

PublicationPublication

title : Stringauthors → List<Author>journal → Journal

title : Stringauthors → List<Author>journal → Journal

citation() : Stringcitation() : String

public class Publication { Private String _title; Public String getTitle() { Return _title; } Public void setTitle(x : String) { _title := x; } … getters, setters for authors, journal … Public String citation() { // fill in code }

}

public class Publication { Private String _title; Public String getTitle() { Return _title; } Public void setTitle(x : String) { _title := x; } … getters, setters for authors, journal … Public String citation() { // fill in code }

}

Scaffolding Breaks Evolution

if model and code are modified, how do we merge the results?

??

modifymodify

modifymodify ??

Protected Regions

generator preserves 'protected regions'does not support unanticipated tuning

breaks generator encapsulation by exposing implementation

modifymodifymodifymodify

modifymodify

modifymodify

copycopy

copycopy

Protected Regionspublic void openUserScreenShowPersons() { ShowPersonsScreenComposite showPersonsScreen = new ShowPersonsScreenComposite(mainController.getShowPersonsScreenContainer() .getScreenParent(), this); /*PROTECTED REGION ID(ShowPersons_game_behavior_usecases_persons_ScreenComposite) ENABLED START*/ showPersonsScreen.personList.add("Marcy"); showPersonsScreen.personList.add("Freddy"); showPersonsScreen.personList.add("Mary-Anne"); /*PROTECTED REGION END*/ mainController.getShowPersonsScreenContainer() .showScreen(showPersonsScreen); // next step called by gui to transition...() method}public void systemCallEnd() { /*PROTECTED REGION ID(systemCall_game_behavior_usecases_personsend) ENABLED START*/ // TODO: perform activity mainController.getShowPersonsScreenContainer().getShell().close(); /*PROTECTED REGION END*/ }

Source: http://public.tfh-berlin.de/~petrasch/uploads/media/TU_MDAPetrasch_110907.pdf

Roundtrip Engineering

extract model from modified codecan changes to code be reflected in model?

implies lack of abstraction

modifymodify

Roundtrip Engineering

Make changes to model and codecan changes to code be reflected in model?

implies model/code bijection

modifymodify

modifymodify

preserve?preserve?

Tuning Generated Code

default pretty-print rules can be generated; tuning needed to get it right general: skeleton for user interface from data model need tuning

Entity -- KW["entity"] _1 KW[":"] _2 KW["{"] _3 _4 KW["}"]

Entity -- V[V is=2[H[KW["entity"] _1 KW[":"] _2 KW["{"]] _3 _4] KW["}"]]

"entity" Id ":" Id "{" Property* Function* "}" -> Entity {cons("Entity")}

Customization 'From the Outside'

customization should never require direct modification of generated code customization code must modify/interact with generated code

what is the interface? avoid exposing generation scheme

Customization with Partial Classes

partial class extends generated class without modifying generated files

inputs

Customization with Partial Classes

public partial class Publication { // generated code; don't change private Author _author; public Author getAuthor () { return _author; } }

public partial class Publication { // generated code; don't change private Author _author; public Author getAuthor () { return _author; } }

public partial class Publication { public String citation() { ... if(this.getAuthor() != null) { c := c + getAuthor().getName() } ... } }

public partial class Publication { public String citation() { ... if(this.getAuthor() != null) { c := c + getAuthor().getName() } ... } }

entity Publication { Author → Author …}

entity Publication { Author → Author …}

Evolution with Partial Classes

examples: pp tables, styling

modifymodify

????

Customization with Inheritance

class GenPublication { // generated code; don't change private Author _author; public Author getAuthor () { return _author; } }

class GenPublication { // generated code; don't change private Author _author; public Author getAuthor () { return _author; } }

class Publication extends GenPublication { public String citation() { ... if(this.getAuthor() != null) { c := c + getAuthor().getName() } ... } }

class Publication extends GenPublication { public String citation() { ... if(this.getAuthor() != null) { c := c + getAuthor().getName() } ... } }

entity Publication { Author → Author …}

entity Publication { Author → Author …}

gen default

Generate Subsystem

generated code is called from regular code

Calling Stratego from C

module foobar

strategies

foo = innermost(Foo <+ Bar)

module foobar

strategies

foo = innermost(Foo <+ Bar)

#include “foobar.h”

void main() { ATerm t = parse(file); t = foo_0_0(NUL, t); ...}

#include <strinclude.h>

ATerm foo_0_0(StrSL sl, Aterm t) { t = innermost_1_0(...); return t;}

calling convention should be fixed, implementation is hidden

Model/Code Interaction

customization code should be considered as part of the generator inputshould interact with (interface of) models, not with generated code

Built-in Types

make external code available through type abstraction to modelbuilt-in types capture domain-specific functionality

Aimpl a;x.a.f()

Aimpl a;x.a.f()

a :: Af(x.a)

a :: Af(x.a)

class Aimpl { f() {…}}

class Aimpl { f() {…}}

Built-in Types: Patch

entity PageDiff { page -> Page next -> PageDiff title :: String patch :: Patch created :: Date previous -> PageDiff date :: Date author -> User version :: Int}

extend entity Page {function makeChange(.., newText : WikiText,...) : Page { PageDiff { ... patch := newText.makePatch(this.content) ... }}}extend entity PageDiff { function computeContent() : WikiText { if (next = null) { return patch.applyPatch(page.content); } else { return patch.applyPatch(next.content); } }

Foreign Function Interface

declare functions/types available in 'native' libraries

call fcall f

prim(f)prim(f)def fdef f

Foreign Function Interface

Primitives in Stratego allow calling (wrapped) native functions.

epoch2local-time = ?EpochTime(t) ; prim("SSL_epoch2localtime", t) ; prim-tuple-to-ComponentTime

epoch2local-time = ?EpochTime(t) ; prim("SSL_epoch2localtime", t) ; prim-tuple-to-ComponentTime

Multi Models

Multiple Models / Multiple DSLs

split model into several modelscombine DSLs to increase coverage

CC CC

YACCYACCLEXLEX

LEX: lexical analysis with regular grammarsYACC: context-free analysis with context-free grammars

Multiple Models / Multiple DSLs

Multiple Models / Multiple DSLs

ChecksChecksJPAJPA ServletsServlets

UIUIDMDM ACAC

DM: data modelUI: user interface

AC: access control

Model/Model Interaction

consider models as components / moduleswhat is interface of a model? what is the scope of model elements

model encapsulation; separate compilation

LEX & YACC

"PRINT" { return PRINT; }[0-9]+ { yylval = atoi(yytext); return NUMBER; }[a-z] { yylval = yytext[0] - 'a'; return NAME; } \ { ; }\n { nextline(); }\t { ; }"//".*\n { nextline(); }. { yyerror("illegal token"); }

"PRINT" { return PRINT; }[0-9]+ { yylval = atoi(yytext); return NUMBER; }[a-z] { yylval = yytext[0] - 'a'; return NAME; } \ { ; }\n { nextline(); }\t { ; }"//".*\n { nextline(); }. { yyerror("illegal token"); }

statement: designator ASSIGN expression | PRINT expression | IF expression THEN stmtseq ELSE stmtseq FI | IF expression THEN stmtseq FI | WHILE expression DO stmtseq OD ;

statement: designator ASSIGN expression | PRINT expression | IF expression THEN stmtseq ELSE stmtseq FI | IF expression THEN stmtseq FI | WHILE expression DO stmtseq OD ;

lexical syntax [a-zA-Z][a-zA-Z0-9\_]* -> Id [a-zA-Z0-9\-\_]+ -> FileName {FileName "/"}+ -> ModuleName ~[\n\r]* -> SectionName context-free syntax "{" Statement* "}" -> Block {cons("Block")} Block -> Statement "var" Id ":" Sort ";" -> Statement {cons("VarDecl")} Exp ";" -> Statement {cons("Stat")} "return" Exp ";" -> Statement {cons("Return")}

context-free syntax "module" ModuleName Section* -> Module {cons("Module")} "imports" ModuleName -> Definition {cons("Imports")}

lexical syntax [a-zA-Z][a-zA-Z0-9\_]* -> Id [a-zA-Z0-9\-\_]+ -> FileName {FileName "/"}+ -> ModuleName ~[\n\r]* -> SectionName context-free syntax "{" Statement* "}" -> Block {cons("Block")} Block -> Statement "var" Id ":" Sort ";" -> Statement {cons("VarDecl")} Exp ";" -> Statement {cons("Stat")} "return" Exp ";" -> Statement {cons("Return")}

context-free syntax "module" ModuleName Section* -> Module {cons("Module")} "imports" ModuleName -> Definition {cons("Imports")}

Language Integration and Separation of Concerns

SDF integrates lexical and contex-free syntax

Language Integration and Separation of Concerns

define page topic (topic : Topic) { … navigate(editTopic(topic)){…} …}define page editTopic(topic : Topic) { … }

define page topic (topic : Topic) { … navigate(editTopic(topic)){…} …}define page editTopic(topic : Topic) { … }

extend entity Topic { acl -> ACL }

extend entity Topic { acl -> ACL }

predicate mayViewWeb(w : Web) { ((w.acl.view.length = 0) || memberOf(w.acl.view))}predicate mayEditWeb(w : Web) { memberOf(w.acl.edit)}rules page topic(topic : Topic) { mayViewTopic(topic)}rules page editTopic(topic : Topic) { mayEditTopic(topic)}

predicate mayViewWeb(w : Web) { ((w.acl.view.length = 0) || memberOf(w.acl.view))}predicate mayEditWeb(w : Web) { memberOf(w.acl.edit)}rules page topic(topic : Topic) { mayViewTopic(topic)}rules page editTopic(topic : Topic) { mayEditTopic(topic)}

data model

userinterface

access control

WebDSL provides separate languages for different concerns. Language integration enables verified cross-language references.

Embedded Domain-Specific Languages

GPLprogram

GPLprogram

compilecompile 'machine'code

'machine'code

assimilateassimilateDSLprog

MetaBorg (OOPSLA'04)DSLs for abstraction over libraries/frameworks

fine-grained interaction with 'host' codelanguage conglomerates mix DSL and GPL code

Swing Userinterface Languageimport javax.swing.*;import java.awt.*; public class Test3 { public static void main(String[] ps) { JFrame frame = frame { title = "Welcome!" content = panel of border layout { center = label { text = "Hello World" } south = panel of grid layout { row = { button { text = "cancel" } button { text = "ok"} } } } }; frame.pack(); frame.setVisible(true); }}

DSL with embedded GPL Code

compilecompile 'machine'code

'machine'code

assimilateassimilate

provide GPL expressivity/coverage to complement DSL

DSL with embedded GPL Code

vexp : dexp { $$.hi = $$.lo = $1; } | '(' dexp ',' dexp ')' { $$.lo = $2; $$.hi = $4; if( $$.lo > $$.hi ){ printf( "interval out of order\n" ); YYERROR; } } | dexp '*' vexp { $$ = vmul($1, $1, $3); } | vexp '/' vexp { if(dcheck($3)) YYERROR; $$ = vdiv($1.lo, $1.hi, $3); }

Source: http://dinosaur.compilertools.net/yacc/index.html

Mixing DSLs: HQL in WebDSL

function sortedBlogEntries(b : Blog) : List<BlogEntry> { var entries : List<BlogEntry> := select distinct e from BlogEntry as e, Blog as b where (b = ~b) and (e member of b._entries) order by e._created descending; return entries;}

Evolution

evolution scenarios

DSLprogram(model)

DSLprogram(model)

DSLprog

regular evolution

DSLprogram(model)

DSLprogram(model)

DSLprog

DSLprogram(model)

DSLprogram(model)

DSLprogmodifymodify

regular evolution: adapt software to new requirementsimplementation simply regenerated after modification of models

language evolution

DSLprogram(model)

DSLprogram(model)

DSLprog

evolveevolve

language (syntax and/or transformations) evolve

model migration

DSLprogram(model)

DSLprogram(model)

DSLprog

DSLprogram(model)

DSLprogram(model)

DSLprog

evolveevolve

migratemigrate

language evolution requires migration of models

platform evolution

DSLprogram(model)

DSLprogram(model)

DSLprog

DSLprogram(model)

DSLprogram(model)

DSLprog

evolveevolve

changes in the platform requires evolution of transformations

evolveevolve

model extraction

DSLprogram(model)

DSLprogram(model)

DSLprog

DSLprogram(model)

DSLprogram(model)

DSLprog abstractabstract

derive DSL programs from (legacy) GPL programs

abstraction evolution

DSLprogram(model)

DSLprogram(model)

DSLprog

develop higher-level abstractions

DSLprogram(model)

DSLprogram(model)

DSLprog

abst

ract

Coupled Evolution

Datamodel

Datamodel

DataModel'

DataModel'

Data Data'Data'migratemigrateDataData

evolveevolve

Schedule

● Lab: implement your DSL

● Design 1: demonstrations Thursday and Friday

● Cases: grades coming up

● Lecture 13: Betsy Pepels from CapGemini

● Lecture 14: Johan den Haan & Michel Westrate from Mendix