Date post: | 25-Dec-2014 |
Category: |
Presentations & Public Speaking |
Upload: | matthias-furrer |
View: | 196 times |
Download: | 3 times |
2011 © Trivadis
BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
Welcome Trivadis SOA Integration Blueprint @Work
Matthias Furrer
Principal Consultant Technologies
April 2012
April 2012TVD SOA Integration Blueprint @Work
1
2011 © Trivadis
Introduction
TVD SOA Integration Blueprint @Work2
• Matthias Furrer� Long-standing experience in SOA and ERP Area
� Architect, Consultant, Lead-Developer and Project Manager
� SOA Certified Professional
� Oracle SOA Blackbelt Consultant
� Trainer with Trivadis Oracle Middleware und SOA
� Reviewer of Technical Publications
� More than 20 years of software development experience
� Contact: [email protected]
April 2012
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
3
Initial Situation
Customer• 3 different companies – developing and running various applications operated in 3 different service centers
• Purchasing Oracle SOA Suite – including Oracle Service Bus
• Oracle Service Bus intended to become strategic enterprise-wide backbone for all integration tasks – externally and internally
• Trivadis engaged for PM in the ESB Platform building project and creating a customer-specific Integration Blueprint
Basis for the customer specific Integration Bluepri nt• Trivadis Book “Service Oriented Architecture: An Integration Blueprint” by Guido Schmutz,
Daniel Liebhart, Peter Welkenbach
Implementation Projects• First solutions developed and currently in testing phase
• Go-live date for first solutions: June 2012
• Trivadis designed and implemented Oracle SOA platform, CI Concept and first implementations (Java, OSB and SOA-Suite)
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
4
Trivadis SOA Integration BlueprintElements – Views, Domains and Layers
The customer-specific Integration Blueprint is built upon the Integration Architecture Blueprint from Trivadisand is focused on the integration-specific components in this specific view of the system architecture. The main focus lies on the technical domain “Integration” and describes the information flow between the systems while following the layered architecture principle.
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
5
Trivadis SOA Integration BlueprintNotation
The customer-specific Integration uses the same notation as introduced in the TrivadisIntegration Architecture Blueprint to visualize the scenarios.
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
6
Goals of the Integration Blueprint• Unified Implementations of Integration Requirements (Design Principles)
• Best -Practices for specific Tasks (Design Principles)
• Ensuring Interoperability of Services (Design Principles and Standards)
• Detailed Integration Scenarios as Design Patterns (Integration Scenarios)
• Templates for specific Tasks (Reference Implementations)
Answers in the Integration Blueprint• Which Design Pattern is best suited for a specific use case
• Which Integration scenario can be used for specific requirements
• Which tool should be used for a specific requirement
• How to ensure transaction reliability and consistency
• How to avoid and handle faults
• How to implement validation and transformation
• How to secure services
• How to integrate basic services (e.g. .NET services)
SOA Integration BlueprintWhy Considerung an Integration Blueprint
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
7
Infrastructure• Availability and Performance Requirements
• System Topology and Platform **
Architecture• Domain Model
• Service Models
• Service Categorization *
• Integration Scenarios *
Development• Design Guidelines and Standards **
• Tools Choice *
• Transaction Reliability *
• Error Handling *
Administration• Manage, Build, Test and Deploy (CI) **
• Versioning
Common Challenges and Considerations in a SOA ProjectWhat was covered in the Customer Integration Blueprint
• Security
• Service Granularity
• Message Exchange Patterns *
• Decoupling and Events *
• Validation and Transformation *
• Canonical Schema *
• Shared Artefacts and Metadata *
• Service Access Security *
• Transport/Message Protocols *
• Repository and Registry
*) covered in the customer-specific Integration Blueprint
**) covered in other, dedicated concepts or project documents by Trivadis
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
8
SOA Integration BlueprintGoals and Rules for Usage in the specific Customer Project
Content and Goals
The Integration Blueprint introduces concepts and usage scenarios as well as components for implementation in an abstract form, which will be replaced by application-specific modules in implementation projects.
Design Rules will provide best practice approaches and ensure a consistent implementation of integration projects. Executable Sample Implementations will illustrate the described scenarios and can be used for reference purposes in implementation projects.
Rules for usage in the project
• Variations in the solution design from solution scenarios or design principles as described in the Integration Blueprint are possible, however these must be reasonable, documented and approved by the Quality Gate.
• The notation for integration scenarios as used in the Integration Blueprint must also be used for documentation of integration projects.
• Described Design Rules apply to integration projects only and are not valid for application or service development.
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
9
Customer Integration BlueprintContent and Chapters
Chapter Title Description
1 Fundamentals Description of basic elements, terminology and notations in the Integration Blueprint
2 Implementation with Oracle SOA Suite and OSB
Implementation of the Blueprint with Oracle SOA Suite and Service Bus. Type Mapping to Frameworks and development components.
3 Composite Services Guidelines for development of Composite Services – particularly regarding development platform to be used (BPEL vs. OSB)
4 Error Handling and Preventing Guidelines and sample implementations for error handling and strategies to avoid errors for all service types
5 Validation and Canonical Schema Principles of Validation and Transformation Services and Canonical Schema in message exchanging
6 Security Implementation guidelines for access security(Draft Version)
7 Basic Services Integration Guidelines and scenarios for the development of .NET /Ruby/GWT Services and integration with Oracle Service Bus(Draft Version)
8 Integration Blueprint (Interface Type) Reference Implementation for specific Interface Type.(Open)
9 Design Principles (Summary) Overview and Summary of all Design Principles and Reference Implementations
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
10
Customer SOA PlatformOracle SOA Suite / OSB 11gPS4
• 3 Tier DMZ-Architecture
• Load-Balancing in front of HTTP Service
• Currently no Clustering on Integration Tier
• SSL Offload in Web Tier
• Dedicated Servers for Production Environment
• Dedicated WL Domains for OSB and SOA Suite
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
11
Customer SOA BlueprintAbstract SOA Blueprint and Service Interaction
Abstract SOA Blueprint
Multi-Layer Model for a holistic view of the IT landscape and partitioning of the entire system into logical units for Identification, borderline and definition of components and interactions within the system as well as establishing methods, standards and tools for development.
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
12
Customer SOA Service TypesCategorization into Service Types for Common Understanding
� Process Services: Implementation of the Business Process in an enterprise with application-comprehensive transactions – often including Human Workflow functionality as e.g. approval processes.
� Composite Services: Multiple Transactions or Queries – typically application-comprehensive, which are composed from multiple Basic Services – or other Composite Services.
� Basic Services: Logic- or data-centric Services, as the foundation of a SOA represent the basic functionality in an application domain..
� Logic Services: Implementation of business logic in non agnostic" context: Example: CalculatePrice
� Entity Services: Data operations on a specific entity. Example: GetCustomerAddress, AddCustomer
� Utility Services: Domain independent operations: Example: ValidateBirthDate, FormatSSCNo
� Decision Services: Decision Rules for controlling processes and workflows
� Virtualization Services: Service abstraction from logical and physical endpoints
Service Types used for this SOA Integration Blueprint
Introducing service types allows to divide the system into components with common characteristics in order to apply different rules to different types of services. A well defined terminology is crucial to establish a common understanding in the project.
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
13
Customer SOA Integration BlueprintMapping Integration Blueprint to Oracle FMW Products
Level Layer BuildingBlock
Developer Framework Developer Component Technology
Transport Level Communication Layer
Transporter - - HTTP, JMS, FTP, SMTP, IIOP, RMI, MQ etc.
Integration Domain Level
Collection/Distributïon Layer
Adapter - Oracle SOA Suite - Tech Adapter (File, FTP, DB, AQ, MQ etc.)- JMS Adapter- WS Adapter- HTTP/Socket Adapter- EJB Adapter
- JCA- JMS- SOAP- TCP- EJB
Mapper - Oracle SOA Suite (implicit) - Implicit Functionality in Adapter Framwork
-
Mediation Layer Translator - Oracle Service Bus- Oracle SOA Suite
Transformation (Action)Mediator (Service) *)
XSLT, XqueryXSLT
Router - Oracle Service Bus - Proxy Services proprietary
Application Level
Process Layer Orchestrator - Oracle SOA Suite BPEL PM WS-BPEL
The following components will be used in the first phase of the implementation project:
• Service Bus• Mediator• BPEL
Candidates for usage in the mid-term planning are:
• Human Workflow• Business Rules
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
14
Customer SOA Integration BlueprintMapping Service Types to Oracle FMW Products
Service Type Subtype BusinessLogic
CrossAppl
Reuse-abilityPotential
Sync/Async
WS SOAPInboundrequest
WS SOAPOutboundreply/callback
InstanceTracking / stateful
Development Framework
Process Service yes yes Low async yes yes yes BPEL
Composite Service yes yes Medium async *) yes *) yes yes *) - BPEL- (OSB)
Basic Service Logic yes no Medium sync yes **) yes **) no - Native (Java, .NET etc.)- (BPEL, Mediator)
Entity yes no Medium sync yes **) yes **) yes *) - BPEL- Native (Java, .NET etc.)
Utility no yes High sync yes yes no - Native (Java, .NET etc.)- (BPEL, Mediator)
Decision yes no Medium sync yes yes no Business Rules Engine
Virtualization no yes High sync *) possible possible no - OSB- (Mediator) ***
*) Typical characteristics, can vary
**) Existing Functions of Legacy Systems (e.g. PL/SQL) can be used and exposed via Adapters or Wrappers
***) Only for XML Validation and Data Transformation
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
15
Customer SOA Integration BlueprintDesign Principles - Usage of Oracle Service Bus (Extract)
• Grundsätze für den Einsatz von OSB als Mediationsla yer
� Sämtliche anwendungsübergreifenden Aufrufe von Webservices müssen immer über den Oracle Service Bus erfolgen. Das heisst wenn beispielsweise das Customer Center (Ruby Anwendung) einen Web-Service aus dem Samba Integration Layer konsumiert der mit .NET implementiert wurde, muss dieser über den OSB (Virtualisierungsservice) erfolgen [DP-IA-2-001].
� Kommunikation zwischen Webservices innerhalb der gl eichen Anwendung können direkt (point-to-point) implementiert werden und müssen nicht über einen Virtualisierungsservice erfolgen. Wenn beispielsweise ein Websservice aus dem Samba Integrationslayer als Consumer für einen anderen Webservice aus dem Samba Integrationslayer agiert, und beide sind mit WCF implementiert, ist kein Mediationslayer erforderlich [DP-IA-2-002].
� EJB Komponenten die aus einer Anwendung oder einem Service konsumiert werden , können direkt angesprochen werden und müssen nicht über einen Virtualisierungsservice bezogen werden [DP-IA-2-003].
� Processservices (BPEL) die Webservices konsumieren , müssen diese immer über den Mediationslayer beziehen. Als Protokoll wird dabei nicht SOAP, sondern das RMI-basierende «SOA direct» Protokoll eingesetzt [DP-IA-2-004].
� Anwendungen oder Services, die als SCA Komponenten implementierte Services konsumieren , müssen diese immer über den Mediatonslayer im OSB beziehen. Das bedeutet dass sämtliche Aufrufe von BPEL Processservices, Business Rules oder Transformationsservices die mit Mediator implementiert wurden, über einen Virtualisierungsservice erfolgen müssen. Ausnahmen sind EJB’sdie innerhalb eines SCA’s implementiert wurden oder SCA Komponenten die von anderen SCA Komponenten konsumiert werden[DP-IA-2-005].
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
16
Customer SOA Integration BlueprintImplementation of Orchestration Services – BPEL vs. OSB
SLA Monitoringrequired
- Instance Tracking required OR- Compensation required OR- Human Workflow included- Restart Capability required
ImplementationOSB
ImplementationBPEL
Implementationcombined
YES NO X - -
YES YES - - X
NO NO X - -
NO YES - X
OSB and BPEL developer environment both provide capabilities to implement Orchestration and Composite Services. However, based on exclusive features and restrictions in the respective frameworks the following rules can be defined for the usage of BPEL on the integration layer for those service types :
• Design Principle: [DP-IA-3-001]
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
17
Customer SOA Integration BlueprintIntegration Scenario and Sample Implementation OSB and BPEL Combined
Referenzimplementierung im Blueprint
C3-1 - AddCustomerAsyncWithBPEL
http://esb-test.ltvintra.ltv.ch:7011/Blueprint/AddCustomerAsyncWithBPEL?WSDL
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
18
Customer SOA Integration BlueprintDesign Principles - Fault Handling (Extract)
� Design Rules – Rückgabe von Fehlern der Service Prov ider� Technical Faults, die nicht in der entsprechenden Instanz selber behandelt werden, müssen mit HTTP Response Status «500» antworten[DP-
IA-4-001].
� Business Faults sind gültige SOAP Response - die entsprechende Serviceinstanz muss jedoch immer mit HTTP Response Status «500» antworten[DP-IA-4-002]. *)
� Business Faults, Validation Errors und Processing Errors sollten als SOAP Fehlercode <faultcode> immer den Wert «soapenv:Client» (SOAP 1.1) oder «soapenv:Sender (SOAP1.2) zurückgeben. Dies gilt generell als ein Indikator, dass kein Retry versucht werden sollte[DP-IA-4-003].
� System Errors und Security Errors sollten als SOAP Fehlercode <faultcode> immer den Wert «soapenv:Server» (SOAP 1.1) oder «soapenv:Receiver (SOAP1.2) zurückgeben. Dies gilt generell als ein Indikator, dass ein Retry ausgeführt werden kann [DP-IA-4-028].
� Business Faults und Validation Errors müssen die entsprechende Fehlerinformation immer über eine im WSDL des entsprechenden Services definierte Fault Message anzeigen. Eine ausschliessliche Anzeige über Elemente im Payload ist nicht ausreichend. Damit wird unter anderem auch die Fehlerbehandlung via Fault Policy in den SCA Komponenten ermöglicht[DP-IA-4-004].
� Bei reinen Virtualisierungsservices zur Serviceabstraktion im OSB muss die Fehlermeldung des Serviceproviders unverändert an den Client zurückgegeben werden. Als Virtualisierungsservices in diesem Kontext gelten [DP-IA-4-008]:• nur ein Service Provider im Flow• keine Veränderung (z.B. Transformation) der Request oder Response Message
• Bei unbehandelten Fehlern (mit Ausnahme von Business Faults), die unverändert an den Client zurückgegeben werden, muss die ursprüngliche Fehlermeldung des Service Providers in das Error-Log des OSB geschrieben werden. Bei kritischen Transaktionen kann dazu auch ein Pipeline-Alert verwendet werden[DP-IA-4-009].
Service Provider URI 2
Service Provider URI 1
Service Provider URI 3
Proxy Service
Business
Service
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
19
Customer SOA Integration BlueprintDesign Principles - CompensationBei Virtualisierungsservices ist der Service Provider für die Transaktionssicherheit zuständig. Auf der Integrationsebene wird lediglich der entsprechende Fehler an den Consumer zurückgegeben [DP-IA-4-024].
Bei Composite Services ist dieser selber für die Transaktionssicherheit zuständig. Dabei wird die Steuerung der Compensation von dieser Komponente übernommen. Für Process Services gelten die gleichen Regeln. [DP-IA-4-025].
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
20
Customer SOA Integration BlueprintDesign Principles – Error Prevention
Fehlerszenario Strategien
Nicht-Erreichbarkeit von Service Providern oder Ressourcen aufgrund geplanter oder ungeplantem Ausfall von Infrastrukturkomponenten
- Implementierung von Retry Verfahren- Service Pooling im OSB- Implementierung des Messaging über Queues anstelle synchronem Request/Reply
Messaging
Ausfall der ESB Service-Infrastruktur - Implementierung des Polling Consumer Patterns über Queues oder DB Persistierung
Überlastung der ESB-Infrastruktur - Implementierung von Split-Join Patterns in OSB- Work Manager in OSB
Überlastung von Service-Endpoints - Caching im OSB- Service Throttling in OSB zur Limitierung von gleichzeitigen Anfragen
Interoperability Issues für externe Consumer in Enterprise Services
- WS-I Compliance Validierung der Services- Implementierung von WS-I Checks im OSB
The Customer SOA Integration Blueprint contains Explanations, Design Principles, Usage Scenarios and a reference Implementation for the below listed error preventing strategies.
Proxy Service
Business Service
Legacy Service
Message Buffer
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
21
Customer SOA Integration BlueprintError Prevention – Polling Consumer
� Design Regeln
• Initiierung von geschäftskritischen Transaktionen ü ber Polling Consumer Pattern . Geschäftskritische Transaktionen müssen immer über einen Persistenzlayer mit passiven Polling Mechanismen implementiert werden. Die verhindert den Verlust von Transaktionen bei nicht verfügbarer Integrationsplattform. Falls möglich soll dazu eine JMS Queue verwendet werden. [DP-IA-4-010].
• Falls eine JMS Queue oder Topic für die Persisitierung von Nachrichten für Polling Consumer verwendet wird, darf der dazugehörige JMS Server nicht auf der gleichen physischen Plattform wie der OSB ausgeführt werden. Andernfalls könnten Nachrichten bei einem Ausfall der OSB Plattform ebenso wenig wie im synchronen Request/Reply Pattern zugestellt werden [DP-IA-4-027].
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
22
Customer SOA Integration BlueprintDesign Principles - Validation and Transformation
Action Type Action Subtype Standards Exposedas Service
Codingrequired
ConsumedBy
WS SOAPInterface
Sync/Async
Development Framework
Data Validation Technical Validation XML Schema no no - OSB- BPEL- Mediator
- - -
Functional Validation - Native- Schematron *)
yes yes **) - OSB- BPEL- Mediator- Legacy App
yes sync - Java, .NET, PL/SQL etc.- (BPEL, OSB)
Data Transformation ***) Data Transformation - XSLT should yes - OSB- BPEL
yes sync Oracle Mediator
Data Mapping - XSLT- XQuery
possible yes - OSB- BPEL- Mediator
(yes) (sync) - JDeveloper- Eclipse
Data Formatting - no yes - XSLT- XQuery
- - - JDeveloper- Eclipse
Data Value Mapping - possible no - XSLT (yes) (sync) - Oracle SOA Suite Domain Value Mapping
Validierung und Datenkonversion [DP-IS-5-006]
Untenstehende Abbildung zeigt eine Übersicht der einzelnen Aktionen, Services mit deren Ausprägungen, Eigenschaften und jeweilig bevorzugt einzusetzenden Development Framework.
*) Einsatz von Schematron offen**) Vorhandene Services aus Legacy Applikationen können über Wrapper angesprochen werden (wo möglich)***) Sollte immer in einer Operation implementiert werden
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
23
Customer SOA IntegrationPilot ProjectAddress Validation, Enrichment and Synchronization
2011 © Trivadis
TVD SOA Integration Blueprint @WorkApril 2012
24
Summary
Summary and lessons learned• The customer specific SOA Integration Blueprint currently contains approx. 60 Design Principles for
the implementation of integration projects with Oracle SOA Suite and Service Bus
• The customer specific SOA Integration Blueprint currently contains approx. 20 Reference Implementations implementing related patterns, design principles and integration scenarios. More integration scenarios are described but not implemented as executable implementations.
• Design Principles and patterns are based on common best-practices, implementation experiences and general architecture principles. Nevertheless, these rules and patterns can prove not to be completely applicable or suitable in practical usage. Reasons can be limitations or restrictions of the platform, too complex to implement while not providing expected benefit in some cases or changing requirements and needs. Therefore the establishment of such a Blue Print should be considered as an on-going process which has to be continuously reviewed, monitored and adjusted as needed.
• Each customer or customer project has individual requirements , constraints and environments. A generic Integration Blueprint can be used as a starting point to define architectural and conceptual rules and patterns, but should always be adjusted specifically.
2011 © Trivadis
BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
Thank you.Trivadis AG
Matthias Furrer
Europa-Strasse 5CH-8152 Glattbrugg
April 2012TVD SOA Integration Blueprint @Work
25