PATTERNS FOR AUTOMATIC INTEGRATION OF THE DOMAIN-DATA LAYERS IN ENTERPRISE-SYSTEMS

1

PATTERNS FOR AUTOMATIC INTEGRATION OF THE DOMAIN-DATA LAYERS IN ENTERPRISE-SYSTEMS

Mira Balaban

Joint work with Lior Limonad

Ben Gurion UniversityBeer-Sheva, Israel

IBM Haifa, Oct 06 2

Example: an e-Mail client system

Persistency requirements:

IBM Haifa, Oct 06 3

Demo

Persistent class Persistent In-memory Persistent Persistent Persistent collection

Domain Layer

Storage

Data Access Layer

IBM Haifa, Oct 06 4

Modern Systems

Built into layers Typical layers are:

Presentation Domain Data

Grasp changing requirements as a norm Such as: persistency requirements

Adhering to the MDA spiritView DB as a natural extension of the

application

IBM Haifa, Oct 06 5

Persistency Motivation

A strict demand to keep an object state, when its data is important for the continuity of the system.

Objects are too big. Large amount of objects.

Too much for loading all together to memory (hardware limit).

Need to deal with objects one at the time.

Need to save historical states of the system. Sharing object states between multiple

applications.

IBM Haifa, Oct 06 6

Agenda

1. Demo Motivation

2. The problem of partial persistency The research goal

3. Partial solutions4. The DAL Patterns solution5. Application Example6. Evaluation

Correctness proof Merits and shortages

7. Conclusions and future work

IBM Haifa, Oct 06 7

Partial Consistency

Input: Mapping between application (Domain layer) and storage (Data layer). Based on a semantic linkage between layers.

Output: integrated layers.

Task: Introduce Partial persistency. Requires system restructuring (refactoring, transformation).

Requirements: Persistency of (only) marked objects Behavior preservation Transparency – minimal changes to the domain layer

IBM Haifa, Oct 06 8

Persistency of (only) marked objects complications (1)Co-existence of storage and in-memory objects Mixture – ingoing and outgoing references:

Ingoing: In-memory Persistent Availability

Outgoing: Persistent In-memory Reference preservation

IBM Haifa, Oct 06 9

Persistency of (only) marked objects complications (2) Duplication handling:

Consistency– Inter-layer –

Duplicated of objects Duplicated references

Intra-layer – preventing duplication of objects

IBM Haifa, Oct 06 10

Insertion of persistency requirements

Goal: Achieve automatic persistency insertion in a running application

Integration must fulfill Partial persistency Persistency of (only) marked objects Behavior preservation Transparent integration

Domain-layer stays intact


Agenda

1. Demo Motivation






Current Technologies (1)e.g. J2EE / Hibernate Support:

Mapping: may be generated automatically

Intra-layer Consistency Availability – handled by the environment

e.g. EJB container


Current Technologies (2)e.g. J2EE / Hibernate Necessary manual work:

Modification of persistent classes Adhering to standards (e.g. EntityBean interface)

Mixture handling Handle ingoing and outgoing references

e.g. serialization Interlayer consistency

Object consistency Manual invocation of, e.g., session.save(obj), session.load(class,id),

ejbLoad() / ejbStore() / ejbCreate() / db.bind(…) / db.deletePersistent(…)

Reference consistency e.g. between two stored objects

Still require writing tedious (bug infected…) code! Domain-Layer is modified – No Transparency!


Current Technologies (3)e.g. J2EE / Hibernate

Missing:

An algorithmic, correct, automated solution for

integration of the Domain and the Persistent layers


Partial solutions:Fowler’s Active Record Pattern

Message'<<persistent>>

subjectbodyid

addSignature()addAttachment()insert()update()delete()

EditoSession'style

checkSpelling()alignLeft()alignRight()

editing

Transparent Partial persistency

Preserving references between objects Preservation of objects availability

Behavior preserving Object structure preservation


Partial solutions:Fowler’s Data Mapper pattern

Transparent Partial persistency

Preserving references between objects Preservation of objects availability

Behavior preserving


Agenda

1. Demo Motivation






The DAL Patterns Solution

1. A small set of Data-Access Layer (DAL) patterns (refactorings, transformations):

I. Core pattern: Proxy-Data-Mapper: Context independent insertion of persistency

II. Context Patterns: Context tailored refactorings of:

Persistent-to-In-Memory interactions Persistent-to-Persistent interactions

2. DAL-Insertion algorithm A combined procedure for pattern application


Proxy Data Mapper Pattern

Messagebodysubject

print() : String<<static>> create() : Message

<<persistent>> Application precondition: message has a factory method

Automatic application:



Messagesubjectbody


<<persistent>>

MessageImp

Application precondition: message has a factory method

Automatic application: Extract subclass “MessageImp”



Message


<<persistent>>

MessageImpsubjectbody

print() : String


Automatic application: Extract subclass “MessageImp” Move implementation:

Message MessageImp



Message


<<persistent>>


print() : String

MessageDataMapper

load(id) : MessageImpupdate(imp : MessageImp)store(imp : MessageImp)delete(id)

instantiates



Message MessageImp Add a MessageDataMapper



Message


<<persistent,abstract>>


print() : String

MessageProxyid

print()MessageProxy()~MessageProxy/Finalize()load()update()

MessageDataMapper


instantiates

11dataMapper

imp1



Message MessageImp Add a MessageDataMapper Add a MessageProxy


Proxy Data Mapper Pattern Application precondition:

message has a factory method Automatic application:

Extract subclass “MessageImp” Move implementation:

Message MessageImp Add a MessageDataMapper Add a MessageProxy

Turn Message to be abstract Redirect the factory method

Message

print() : String<<static>> create() : MessageProxy

<<persistent,abstract>>


print() : String

MessageProxyid

print()MessageProxy()~MessageProxy/Finalize()load()update()

MessageDataMapper


instantiates

11dataMapper

imp1


Example of Proxy Data Mapper Usage

Client mes : Message

mes : MessageProxy

dataMapper : MessageDataMapper

imp : MessageImp

1: print( )2: load(id) 3: create( )

4: imp

5: print( )

6:

7: update(imp)

8:

9:


The Proxy Data Mapper Automatable Refactoring (1)Input: persistentClass, dbSchemaElement

1. Extract the implementation from the persistent class:persistentClassImp = extractSubClass(persistentClass);

2. Construct the data mapper class:dataMapper = addClass(concat(persistentClass.name+''DataMapper''));addMethod(dataMapper, “load”,constructLoadImpMethod(persistentClassImp, dbSchemaElement));addMethod(dataMapper, “store”,constructStoreImpMethod(persistentClassImp, dbSchemaElement));addMethod(dataMapper, “update”,constructUpdateImpMethod(persistentClassImp, dbSchemaElement));addMethod(dataMapper, “delete”,constructDeleteImpMethod(persistentClassImp, dbSchemaElement));


The Proxy Data Mapper Automatable Refactoring (2)3. Construct the proxy class:

proxy = addClass(concat(persistentClass.name+''Proxy''));addField(proxy, “_idField”, Integer);addField(proxy, “_mapperField”, dataMapper); Add finalize, load, update methods. add all constructors:

for each constructor c in constructors(persistentClassImp)addMethod(proxy, proxy.name,

constructConstructore(dataMapper, persistentClassImp)); add all delegated methods to the proxy class:

for each method m in instance_methods(persistentClassImp)addMethod(proxy, m.name,

constructDelegatedMethod(m, persistentClassImp);


The Proxy Data Mapper Automatable Refactoring (3)

4. Redirect factory methods:for each factoryMethod f in factories(persistentClass)redirectFactoryMethod(proxy);

5. Tag modifications:stereotype(persistentClass, ``abstract'');


Proxy Data Mapper – Example Applying PDM to the Message class:


Problems solved by the Proxy Data Mapper Pattern

Persistency of only marked classes Mixture

Availability Reference preservation

Duplication Consistency

Inter layer Object – handled by the proxy Reference

Intra layer – factory method uses Fowler’s “Identity Map”

Behavior preservation Full Transparency



1. A small set of Data-Access layer patterns:I. Core pattern: Proxy-Data-Mapper:

Context independent insertion of persistency

II. Context Patterns: Context tailored refactoring of:


2. DAL-Insertion algorithm A combined procedure for pattern

application


Context Patterns

Problem parameters: In/Out associations into/from the persistent

class Cardinality constraints Persistent/In-memory marking of the Context

class 5 context patterns

>>persistent<<Context Class

>>persistent<<fooClass

1**1


DAL Patterns


I. Persistent Collection Data Mapper

context


I. Persistent Collection Data Mapper – problems(1)

The in-memory collection realization problem:

The collection of all proxy objects may be too large to fit in-memory

m1:messageImp

p1:MessageProxy

:MessageDataMapper

Messagesm1..mn

vs:VirusScan

col:MessagesCollection

m1:messageImp

p1:MessageProxy

:MessageDataMapper

m3:messageImp

p3:MessageProxy

:MessageDataMapper

mn:messageImp

pn:MessageProxy

:MessageDataMapper



Solution to the collection realization problem: Introduce a collection that loads

corresponding proxy objects by demand

Messagesm1..mn

vs:VirusScan

col:MessagesCollectionMapper

mn:messageImp

pn:MessageProxy

:MessageDataMapper

getMsg(n)

id1..idn



I. Persistent Collection Data Mapper – problems(2) The deletion problem:

Collection of IDs A new implicit reference to persistent objects

col:MessageCollectionMapperid

p1:MessageProxyid



The deletion problem: When a proxy is deleted, how to prevent

message deletion from storage?

Messagesm1..mn

vs:VirusScan


mn:messageImp

pn:MessageProxy

:MessageDataMapper

getMsg(n)

id1..idn



I. Persistent Collection Data Mapper– problems (2)

Solution to the deletion problem: Extend the garbage collector to handle

persistent objects. References to persistent objects = clients

holding their ids. So far: In-memory proxy objects. Collections of persistent objects.


I. Persistent Collection Data Mapper– problems (2)

Solution to the deletion problem: Insert TableManager (object) for the

message – guards storage deletion by tracking reference count to it

All clients: proxys/collections, notify itabout reference changes

:MessageTableManager

Increase(id)

Decrease(id)

mn:messageImp

pn:MessageProxy

:MessageDataMapper


I. Persistent Collection Data Mapper

Solution structure:


I. Persistent Collection Data Mapper– problems (3) The collection itself is too large to reside in-

memory:


id1…………………………………………………………………..idn

<<Persistent>>


I. Persistent Collection Data Mapper– problems (3) Extended solution:

Make the collection persistent Apply the Proxy Data Mapper pattern

col:MessagesCollectionMapperImp

id1…………………………………………………………………..idn

MessagesCollectionMapper

proxy:MessagesCollectionMapperProxy

mapper:MessagesCollectionMapperMapper


I. Extended Persistent Collection Data Mapper


II. Memoization Data Mapper patterns context Memoization data mapper

Multi memoization data mapper


II. Memoization Data Mapper patterns problems The memoization/multi-memoization problems

When a message is stored, how to handle outgoing references to other in-memory objects or in-memory collections?

m:messageImp

:MessageProxy

:MessageDataMapper

Messagesm

e:EncryptionKey


II. Memoization Data Mapper patterns problems Solution:

Extend the proxy with a reference to the outgoing referenced object or collection

:MessageProxy

:MessageDataMapper

Messagesm

e:EncryptionKey

m:messageImp


II. Memoization data mapper pattern

Solution structure:


III. Multi memoization data mapper pattern Solution structure:


IV. Persistent to Persistent Patterns – context

Persistent to Persistent

Persistent to multi persistent


Main problem – Interlayer reference consistency

Example: element1 element3


IV. Persistent to Persistent Pattern – problems (1)

The load problem: Realization of an outgoing reference to

another persistent object

m:messageImp:MessageProxy

:MessageDataMapper

Messagesm

Attachmentsa



Solution to the load problem: A lightweight representative (proxy) is loaded

by the persistent data mapper


:MessageDataMapper

Messagesm

Attachmentsa

:AttachmentProxy



Update/create problems of the referenced object: If attachment is replaced, how to know that a new one

need to be stored. If attachment is updated, how to know that it should be

updated in storage.m:message

Imp:Message

Proxy

:MessageDataMapper

Messagesm

Attachmentsa

m:AttachImp.

:Attach.Proxy

:Attach,DataMapper



Solution to the referenced object update/create problem: Message data mapper handles reference replacement Attachment update handled by attachment data-mapper

(PDM)m:message

Imp:Message

Proxy

:MessageDataMapper

Messagesm

Attachmentsa

m:AttachImp.

:Attach.Proxy

:Attach,DataMapper



The deletion problem: When message is deleted from memory,

should attachment be deleted from storage?


:MessageDataMapper

Messagesm

Attachmentsa

m:AttachImp,

:Attach.Proxy

:Attach,DataMapper



Solution to the deletion problem: Use the extended garbage collection mechanism that

handles persistent objects. Insert TableManager (object) for the attachment –

As in the “persistent collection data mapper” pattern. m:Attach

Imp,:Attach.Proxy

:Attach,DataMapper

:Attach,TableManager

Increase(id)

Decrease(id)


V. Persistent to Multi Persistent Pattern – problem (4) Persistent collection realization problem

How to represent an in-memory 1 to Many relation in storage?

Domain Layer

Data Layer


V. Persistent to Multi Persistent Pattern – problem (4) Solution to the persistent collection

realization problem: Insert a collection of proxies (loaded by

demand)m:message

Imp:Message

Proxy

:MessageDataMapper

m:MailServerImp

:MailServerProxy

:MailServerDataMapper

:MessagesCollection

MapperImp

load

instantiates

DB

resolve

The collection is created and identified by the client data-mapper


IV. Persistent to Persistent Pattern

Solution structure:


V. Persistent to Multi Persistent Pattern Solution structure:


DAL Patterns characterizations

Realized as refactorings Can be automated Transparent insertion:

No modifications/deletions.

Introduce a new Data-Access Layer (DAL) New classes introduced.



1. A small set of Data-Access layer patterns:I. Core pattern: Proxy-Data-Mapper:

Context independent insertion of persistency

II. Context Patterns: Context tailored refactoring of:


2. DAL-Insertion algorithm A combined procedure for pattern

application


Dal-Insertion Algorithm

Combined procedure for patterns application:


Application Example

Input System: Analysis Model

EncryptionKey

UndoDataMailServer

VirusScan

Message

**

scanSet

11

encrypt

**

history

**

outgoingFolder

Attachment11attached

FileSearcher11

searcher

**

files


Application Example

Input System

VirusScan FileSearcher

Attachment

EncryptionKey

UndoData

Message

MailServer

**

scanSet

11

encrypt

**

history

**

outgoingFolder

11attached

11

searcher

**

files

ScanSet

OutgoingFolder

History

Files


Application Example

Input System

EncryptionKey

UndoData

Attachment

VirusScan

ScanSet11

scanSet

FileSearcher11

searcher Files

11

files

**

MailServer

OutgoingFolder

1outgoingFolder

1

Message

**

11

encrypt

**

11attached

History **

1

history

1


Application Example

Input System: persistency marking

FileSearcherVirusScan11

searcher

MailServer<<persistent>>

Files<<persistent-col>>

11

files

UndoData

ScanSet<<persistent-col>>

11scanSet

EncryptionKeyOutgoingFolder<<persistent-col>>

11 outgoingFolder

Attachment<<persistent>>

**

History**

Message<<persistent>>

**

11

encrypt

**

11attached

11history


Application Example

Input System: factory methods

FileSearcherVirusScan

11

searcher


create()


create()11

files

UndoData

ScanSetcreate()

<<persistent-col>>11 scanSet

EncryptionKeyOutgoingFolder<<persistent-col>>

create()

11outgoingFolder


create()

**

History**


create()

**

11

encrypt

**

11attached

11history


Application Example

Apply Proxy-Data-Mapper pattern to all persistent classes:


11

searcher Files<<persistent-col>>

create()11

files


<<MailServerProxy>> create()

ScanSet11

scanSet

EncryptionKey

OutgoingFolder<<persistent-col>>

create()

11outgoingFolder


<<AttachmentProxy>> create()

**

UndoDataHistory**


<<MessageProxy>> create()

**11

encrypt

**

11attached

11

history

MailServerImp

MailServerDataMapper

MailServerProxy

AttachmentImpAttachmentProxy

AttachmentDataMapper

MessageImpMessageProxy

MessageDataMapper


Application Example

Apply memoization-proxy-data-mapper pattern


11


create()11

files



ScanSet11

scanSet


create()

11outgoingFolder



**

UndoDataHistory**



**

**

11attached

11

history

MailServerImp


MailServerProxy



MessageImp

MessageDataMapper

MessageProxy

EncryptionKey

11

encrypt

encrypt


Application Example

Apply multi-memoization-proxy-data-mapper pattern


11


create()11

files



ScanSet11

scanSet


create()

11outgoingFolder



**

UndoData



**

**

11attached

MailServerImp


MailServerProxy



MessageImp

MessageDataMapper

EncryptionKey

11

encrypt

MessageProxy

encrypt

History**

11

history

history


Application Example

Apply persistent-to-persistent pattern


11


create()11

files



ScanSet11

scanSet


create()

11outgoingFolder



**

UndoData



**

**

11attached

MailServerImp


MailServerProxy

AttachmentImp

MessageImp

EncryptionKey

11

encrypt

MessageProxy

encrypt

History**

11

history

history

AttachmentProxy


MessageDataMapper

AttachmentTableManager<<singleton>>


Application Example

Apply persistent-to-multi-persistent pattern


11


create()11

files




11 scanSet


create()

11

outgoingFolder



**

UndoData



**

**

11attached

MailServerImp


MailServerProxy

AttachmentImp

MessageImp

EncryptionKey

11

encrypt

History**

11

history

AttachmentProxy



MessageDataMapper

OutgoingFolderColMapperImp

load

MessageProxy

encrypt

history

*

MessageTableManager

<<singleton>>


Application Example

Apply persistent-collection-data-mapper pattern


11

searcher




create()11

files


11 scanSet


create()

11

outgoingFolder



**

MailServerImp


MailServerProxy

AttachmentImp

MessageImp

UndoData



**

**

11attached

AttachmentProxy


EncryptionKey

11

encrypt

History**

11

historyMessageDataMapper


load

MessageTableManager

<<singleton>>

MessageProxy

encrypt

history

*

FilesColMapper

*



Application Example

Result


11

searcher




create()11

files


11 scanSet


create()

11

outgoingFolder



**

MailServerImp


MailServerProxy

AttachmentImp

MessageImp

UndoData



**

**

11attached

AttachmentProxy


EncryptionKey

11

encrypt

History**

11

historyMessageDataMapper


load

MessageTableManager

<<singleton>>

MessageProxy

encrypt

history

*

FilesColMapper

*



Evaluation – Integration Correctness

Persistency of all and only marked objects

Behavior preservation: Hard to quantify! Measurable observables:

Object structure preservation Preservation of references between objects Preservation of objects availability Method preservation

Transparent integration No domain-layer changes


DAL-Insertion algorithm – characterizations:

Requires only local information – no data flow analysis:

The system Marked persistent classes Marked persistent collections

Non deterministic Minimal – based on a minimal set of

constraints for DAL pattern combination Correct – satisfies all observables - solves the

problem of partial persistency


Overall Persistency Insertion Steps

1. Domain layer marking of: Persistent classes Persistent class collections

2. Data Layer: Mapping specification between domain-layer classes

and storage elements (i.e. Tables) Access specifications (i.e. DB driver)

3. Automatic Data Access Layer insertion Application of DAL-Insertion algorithm


Conclusion

The DAL patterns approach provides the missing link towards full automation of Domain-Data layers integration

Patterns can be implemented as independent software refactorings (in a compositional refactoring manner).

Experimental tool for static aspects was implemented

Restricted (toy) implementation of the patterns exists.


Future

Hierarchy related patterns should be finalized. Full implementation into an existing IDE (e.g.

Eclipse) – started. Extend the solution to a shared storage Extension to other paradigms: Networking,

distributed computing, web services Problems:

recovery handling, transactions (current patterns assume auto commit), performance – query optimization

Collection persistency

82

Thank You



Avoiding in memory duplications Handled by the implementation of factory method

e.g. using fowler’s “Identity Map”

Date post:	05-Jan-2016
Category:	Documents
Upload:	billy
View:	41 times
Download:	1 times

PATTERNS FOR AUTOMATIC INTEGRATION OF THE DOMAIN-DATA LAYERS IN ENTERPRISE-SYSTEMS

Documents