STREAM 8 - SOA IN FOCUS
© 2007 IBM Corporation
© 2008 IBM Corporation
Service Data Objects (SDO): Handling Data Consistently in a SOA Environment
Kelvin Goodson, IBM UK – [email protected] EngineerOpen Source SOA, Apache Tuscany
SDO Session
© 2008 IBM Corporation2
SDO 2.1 Spec Introduction
SDO Session
© 2008 IBM Corporation3 Page
What are Service Data Objects?
Service Data Objects (SDO) provide:
– Uniform access to data from heterogeneous sources
• XML, RDB, EJB, LDAP, SOAP etc...
– Has both static (strongly typed) and dynamic programming models
– Disconnected “data graph” capable of tracking changes
– Meta-data for easy introspection of data types
– Relationship integrity enforcement
– Language neutral APIs with implementations in Java, C++ and PHP
SDO Session
© 2008 IBM Corporation4
SDO for COBOL
SDO Session
© 2008 IBM Corporation5 Page
Instance and Model
SDO Session
© 2008 IBM Corporation6 Page
How do I store and retrieve SDOs ?
XML Documents
– SDO specification describes serialisation to and de-serialisation from XML documents.
SDO specification discusses a Data Access Service (DAS)
– DAS is a mediator between data sources and SDO applications
– Apache Tuscany currently implements an RDB DAS, and some work done on an LDAP DAS
SDO Session
© 2008 IBM Corporation7 Page
Combining SDO and DAS
SDO Client
ChangeSummary
Data GraphDataObject
: EJB: InvoiceEJB: Customer
JDBC
XPath / XQuery
Local
XML/HTTP
CCI / Proprietary
Data Access Service
RDB
XML DB
Web service
JCA
SDO Session
© 2008 IBM Corporation8 Page
Combining SDO with SCA
SCA is the component model Components may be wired together SDO DataObjects are data passed between and manipulated by Components DAS is a SCA component with a special implementation type (implementation:das)
Component
AccountServiceComponent
Component
AccountDataServiceComponent
Service
AccountService
Reference
StockQuoteService
Composite AccountComposite
WebServer
.NetWeb
Service
SCA
SCASCA
SDO
SDO
SDO
RDB
SDO Session
© 2008 IBM Corporation9 Page
SDO Components and Key Features
Generated data API: POJO beans
Dynamic data API: DataObject
Change summary API: ChangeSummary
Introspection API: Type and Property
Sequence API where instanceorder is preserved
Relationship Integrity
XML and XSD Integration– XPath Navigation through Graphs of Data
XML-based serialization for transferring data sets on the wire– Can conform to pre-defined XML Schema
– Can generate XML Schema
SDO Session
© 2008 IBM Corporation10
The Strongly Typed SDOs also Implement the Dynamic API
SDO Session
© 2008 IBM Corporation11 Page
Static (Generated) Data API
POJO interfaces/classes
Strong-typed accessors
public interface Person {
String getName();
void setName(String name);
Address getAddress();
void setAddress(Address address);
}
Person p = (Person) dataFactory.create(Person.class);
p.setName("John");
Address a = (Address) dataFactory.create(Address.class);
p.setAddress(a);
<complexType name="Person"> <attribute name="name" type="string"/> <element name=“address" type=“Address"/></complexType><complexType name=“Address”> …
SDO Session
© 2008 IBM Corporation12
Dynamic Data API (commonj.sdo.DataObject)
The Dynamic SDO API provides a reflective way to introspect and manipulate data– Recall: Static SDO can be cast to DataObject
Customer c = CustomerFactory.createCustomer();
((DataObject)c).setString("name", "John"); // Set name
<complexType name="Person"> <attribute name="name" type="string"/> <element name=“address" type=“Address"/></complexType><complexType name=“Address”> …
SDO Session
© 2008 IBM Corporation13 Page
Example 1 – Accessing DataObjects
// use SDO API to find the employee, at index 1 in the list. List departments = company.getList("departments");
DataObject department = (DataObject) departments.get(0); List employees = department.getList("employees");
DataObject employeeFromList = (DataObject) employees.get(1);
DataObject employee = company.getDataObject
("departments[1]/employees[2]");
// Get an employee using an SDO xpath expression DataObject employee = company.getDataObject (“departments[number=123]/employees[SN=0002]”);
SDO Session
© 2008 IBM Corporation14 Page
// create a new employee DataObject newEmployee = department.createDataObject("employees"); newEmployee.set("name", "Al Smith"); newEmployee.set("SN", "0004"); newEmployee.setBoolean("manager", true);
// Set employeeOfTheMonth company.set("employeeOfTheMonth", newEmployee);
Example 2 – Updating DataObjects
company.setString(“companyName", "ACME");
<company name="ACME" employeeOfTheMonth="0004"> <departments name="Advanced Technologies" location="NY" number="123"> <employees name="John Jones" SN="0001"/> <employees name="Jane Doe" SN="0003"/> <employees name="Al Smith" SN="0004" manager="true"/> </departments> </company>
SDO Session
© 2008 IBM Corporation15
SDO Meta-Model
SDO provides a simple, universal meta-model
– Used across JavaBeans, XML, or any data source
– Useful for tools and IDE’s (Model in MVC)
Meta-data Classes
– Type
• Has name, URI, instance class, and properties
– Property
• Has name, type, default value, etc.
DataObject obj = …;Type type = obj.getType();Collection c = type.getProperties();Iterator i = c.iterator();while (i.hasNext()) {
Property prop = (Property) i.next();System.out.println(prop.getName());
}
SDO Java XSD
Type Class Type
Property Field Element
Attribute
SDO Session
© 2008 IBM Corporation16
Type System Scopes
HelperContext
– Agent of scoping for Types
– provides access to
• TypeHelper• DataFactory• XSDHelper• XMLHelper• EqualityHelper• CopyHelper• DataHelper
– HelperProvider.getDefaultContext()
– Application can create new HelperContexts
SDO Session
© 2008 IBM Corporation17
XML/XSD integration
Key interfaces: XMLHelper and XSDHelper
Direct correspondence between XML and DataObjects
– commonj.sdo.XMLHelper
• Load and save DataObjects to XML streams
XSD mapping to and from SDO
– commonj.sdo.XSDHelper
• Get XML Schema specific information:
– isElement, isMixed, local name, appinfo
– Define Types and Properties from XSDs
• Annotations or XSLT for mapping control
– Generate XSDs from Types and Properties
SDO Session
© 2008 IBM Corporation18
Create SDO types from an XML schema
XSD model can be directly used to define SDO types
XSDHelper xsdHelper = …;
TypeHelper typeHelper = …;
// Populate the SDO types from the XSD
URL url = XMLSample.class.getResource(“/stockquote.xsd”);
InputStream inputStream = url.openStream();
xsdHelper.define(inputStream, url.toString());
inputStream.close();
// Look up the SDO Type
Type type = typeHelper.getType(“http://stock”, “Quote”);
// Get the XSD namespace URI for a SDO type
String ns = xsdHelper.getNamespaceURI(type);
SDO Session
© 2008 IBM Corporation19
Registering Type-System for SDO Runtime
Register types dynamically from XSD
Register types statically (strongly typed)
Instance is still of type Employee
→ Data Object instantiation:DataObject employee = dataFactory.create(NAMESPACE, “Employee”);
→ Data Object instantiation:Employee empl = (Employee)dataFactory.create(Employee.class);OrDataObject empl = dataFactory.create(NAMESPACE, “Employee”);
SDO Session
© 2008 IBM Corporation20
CreateCompany Example
SDO Session
© 2008 IBM Corporation21
SDO API for Programatic Type Creation
Uses DataObject API
Build data graph describig Typs system – then compile
Type TypeHelper.define(DataObject)
List<Type> TypeHelper.define(List<DataObject>)– Permits circular references in metadata (Properties of Type as yet not “defined”)
DataObject cType = scope.getDataFactory().create("commonj.sdo", "Type"); cType.set("uri", "http://example.com/customer"); cType.set("name", "Customer");
Type intType = typeHelper().getType("commonj.sdo", "Int"); DataObject cNumProperty = customerType.createDataObject("property"); cNumProperty.set("name", "custNum"); cNumProperty.set("type", intType); …… Type registeredType = typeHelper.define(cType);
SDO Session
© 2008 IBM Corporation22
Load/Save XML documents
XMLHelper can load SDO from XML or save SDO to XMLDataObject quote = …;
XMLHelper xmlHelper = …;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// Save the data object as an XML document
xmlHelper.save(quote, "http://www.example.com/simple", "stockQuote", baos);
byte[] xml = baos.toByteArray();
// Load the XML bytes as an XMLDocument
XMLDocument document = xmlHelper.load(new ByteArrayInputStream(xml));
// Get the root DataObject
quote = document.getRootObject();
SDO Session
© 2008 IBM Corporation23 Page
XML
<purchaseOrder orderDate="1999-10-20">
<shipTo country="US">
<name>Alice Smith</name>
<street>123 Maple Street</street>
<city>Mill Valley</city>
<state>PA</state>
<zip>90952</zip>
</shipTo>
...
</purchaseOrder>
country= USname= Alice Smithstreet= 123 Maple Streetcity= Mill Valleystate= PAzip= 90952
XML / SDO Mapping
Values
Properties
orderDate=1999-10-20
shipTo
SDOPurchaseOrder Type
USAddress Type
SDO Session
© 2008 IBM Corporation24
Use SDO to invoke a Web Service
Load the SOAP Envelope model from the XSD
// Create the helperContext
HelperContext helperContext = SDOUtil.createHelperContext();
// Load the soap envelope XSD
XSDHelper xsdHelper = helperContext.getXSDHelper();
URL xsd = SoapEnvelope.class.getClassLoader().getResource("soap-envelope.xsd");
InputStream xsdStream = xsd.openStream();
xsdHelper.define(xsdStream, null);
xsdStream.close();
// Load the WSDL model
URL wsdl = SoapEnvelope.class.getClassLoader().getResource("stockquote.wsdl");
InputStream wsdlStream = wsdl.openStream();
xsdHelper.define(wsdlStream, null);
wsdlStream.close();
SDO Session
© 2008 IBM Corporation25
Open Types
SDO “Open” Types
– Type.isOpen()
– Presence of <xsd:any/> implies Type.isOpen() = true
– TypeHelper.defineOpenContentProperty(String, DataObject);
– TypeHelper.getOpenContentProperty(“namespace”, “name”);
– XSDHelper.getGlobalProperty(“namespace”, “name”)
Property p = typeHelper.getOpenContentProperty( “myExtraInfoUri”, “usefulData”);DataObject usefulExtraInfo = dataFactory.create(p.getType);…openDataObject.getList(p).add(usefulExtraInfo);assert usefulExtraInfo.getInstanceProperties().size() == usefulExtraInfo.getType().getProperties().size() +1;
SDO Session
© 2008 IBM Corporation26
Create and populate the SOAP message
Construct the SOAP Envelope as a DataObject
DataFactory dataFactory = helperContext.getDataFactory();
DataObject envelope = dataFactory.create("http://www.w3.org/2003/05/soap-envelope", "Envelope");
DataObject header = envelope.createDataObject("Header");
DataObject body = envelope.createDataObject("Body");
Property prop = xsdHelper.getGlobalProperty("http://example.com/stockquote.xsd", "getLastTradePrice", true); // is element
DataObject request = dataFactory.create(prop.getType());
request.setString("tickerSymbol", "IBM");
body.getList(prop).add(request);
SDO Session
© 2008 IBM Corporation27
Producing the XML for the SOAP message
StringWriter writer = new StringWriter();
XMLHelper xmlHelper = helperContext.getXMLHelper();
XMLDocument doc = xmlHelper.createDocument(envelope, "http://www.w3.org/2003/05/soap-envelope", "Envelope");
xmlHelper.save(doc, writer, null);
// We can write to an HTTP output stream to invoke the Web Service
<?xml version="1.0" encoding="ASCII"?><tns:Envelope xmlns="http://ns1" xmlns:tns="http://www.w3.org/2003/05/soap-envelope" xmlns:xsd1="http://example.com/stockquote.xsd"> <tns:Header> <myHeader>001</myHeader> </tns:Header> <tns:Body> <xsd1:getLastTradePrice> <tickerSymbol>IBM</tickerSymbol> </xsd1:getLastTradePrice> </tns:Body></tns:Envelope>
SDO Session
© 2008 IBM Corporation28
Relationship Integrity
Containment
– By default Properties represent containment relationships
DataObject root = …DataObject originalChild = root.getDataObject(“child1”);DataObject parent = originalChild.getContainer();
assert parent == root;
DataObject newChild = dataFactory.create(ChildType.class);root.set(“child1”, newChild);
assert originalChild.getContainer() == null;assert newChild.getContainer() == root
SDO Session
© 2008 IBM Corporation29
Other Relationships
Non-containment
– Properties can be made to represent Non-containment relationships
• xsd:ID and xsd:IDREF in Schema translate to non-containment
– Without non-containment all data graphs would be single rooted hierarchies
Opposite
– Bidirectional relationships (other than parent-child)
– Requires non-containment (Property type = xsd:ID, xsd:IDREFS or anyURI)
– sdo:oppositePropertyType required
SDO Session
© 2008 IBM Corporation30
Sequenced Types
Type.isSequenced() for instance ordered DataObjects
Triggered by
– xsd:mixed=“true”
– repeating choice
– sdo:sequenced=“true”
– …
DataObjet.getSequence()
– Iterate over Lists of Property Value pairs
– Sequence.getProperty(int), Sequence.getValue(int)
SDO Session
© 2008 IBM Corporation31
Sequenced Example
SDO Session
© 2008 IBM Corporation32
Tracking changes for disconnected data
Service data is often disconnected from the source in SOA Service data can be changed in the flow of business processes
// Create an empty data graph and add a root object, an instance of type Quote
DataGraph dataGraph = SDOUtil.createDataGraph();
DataObject quote = dataGraph.createRootObject("http://www.example.com/simple", "Quote");
// Begin logging changes: usually DAS responsibility
ChangeSummary changeSummary = dataGraph.getChangeSummary();
changeSummary.beginLogging();
// Modify the data graph: client code
quote.setString("symbol", "fbnt");
…
quote.setDouble("change1", 1000);
changeSummary.endLogging(); // DAS code
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// Serialize with change summary data
SDOUtil.saveDataGraph(dataGraph, baos, null);
SDO Session
© 2008 IBM Corporation33 Page
Data Access Services - DAS
Data Access Services (DAS) provides the data access solution with two fundamental capabilities:
– Results of a data source query execution returned as a graph of Data Objects
– Graph modifications reflected back to data source as a series of Creates/Updates/Deletes
SDO Client
ChangeSummary
Data GraphDataObject
: EJB: InvoiceEJB:
JDBC
XPath / XQuery
XML/HTTP
CCI / Proprietary
Data Access Service
RDB
XML DB
Web service
JCA
SDO Session
© 2008 IBM Corporation34
DAS High Level Architecture
SCA component with DAS impl. type
SDO Session
© 2008 IBM Corporation35
Apply changes back to the data store
Change-aware DAS can retrieve the data and update the data by applying the deltaDAS das = DAS.FACTORY.createDAS(getConnection());
// Read customer
Command select = das.createCommand("Select * from CUSTOMER");
// ChangeSummary is enabled by DAS
DataObject root = select.executeQuery();
DataObject customer = (DataObject) root.get("CUSTOMER[1]");
// Modify customer
customer.set("LASTNAME", "Pavick");
// Build apply changes command
das.applyChanges(root);
SDO Session
© 2008 IBM Corporation36
DAS example
SDO Session
© 2008 IBM Corporation37
SDO Implementations
Open Source implementations of in context of SDO– The Tuscany project (incubating at Apache) for SCA, SDO and DAS– The Eclipse SOA Tools Platform Project– The SOA for PHP project
Commercial SDO Implementations– XIC from Xcalia– HydraSDO from Rogue Wave Software– FireStorm/SDO from CodeFutures Software– AquaLogic Data Services Platform 2.0 from BEA– Virtual XML Garden from IBM– IBM WebSphere Process Server V6.0.x– IBM WebSphere Application Server V6.1– IBM WebSphere Application Server V6.1 Feature Pack for SOA (beta)– SAP NetWeaver Application Server, Java(TM) EE 5 Edition
SDO Session
© 2008 IBM Corporation38
WID has a “Bo-Editor” to define XSD/SDO
SDO Session
© 2008 IBM Corporation39
WAS v6 EJB – RSA Tooling
SDO Session
© 2008 IBM Corporation40
WAS v6 JDBC DAS – RSA Tooling
SDO Session
© 2008 IBM Corporation41
WAS v6 JDBC DAS – RSA Tooling
SDO Session
© 2008 IBM Corporation42
Summary Unified data access to heterogeneous data sources
– Unified programming model for business data– DAS layers to take care of different data sources and format conversions on the edges
Unified support for both static and dynamic data APIs– Generated POJO interfaces/classes for strongly-typed APIs– Dynamic APIs: DataObject/Type/Property– Static SDOs can be mixed with Dynamic SDOs
Support for disconnected programming models– ChangeSummary to track changes– Change-aware DAS update– DAS specification mentioned in the SDO 3 objectives document
Multiple language support– Java– C++– PHP– …
SDO Session
© 2008 IBM Corporation43 Page
Ready to try?
Finalised specs -- OSOA collaboration
– http://www.osoa.org
Articles etc.
– http://www.osoa.org/display/Main/SDO+Resources
– http://java.sys-con.com/read/313547.htm
– http://java.sys-con.com/read/358059.htm
– http://incubator.apache.org/tuscany/das-overview.html
Open Source
– Apache Tuscany Project
• http://incubator.apache.org/tuscany– PHP & PECL
• http://www.php.net/• http://pecl.php.net/package/SCA_SDO