Date post: | 15-Dec-2015 |
Category: |
Documents |
Upload: | maia-voice |
View: | 214 times |
Download: | 0 times |
1 © 2005 Sonic Software Corporation
The BAA Terminal 5 Project – The reality of XML, ESB and Web services
2 © 2005 Sonic Software Corporation
The reality of serving 35 million people per year
37 million man hours to build T5
6.5 million cubic metres of earth works
15,000 cubic metres of concrete per week
16 major projects, 100 sub-projects
Sub projects cost between £30M and £150M
60,000 people involved in the build
The IT infrastructure must operate entirely new level of
speed, efficiency and availability (they land planes!)
3 © 2005 Sonic Software Corporation
The IT landscape – must be integrated
6000 display systems, 400 COTS apps,197 line of business apps, 35 operationalIT platforms, over 1000 servers
One hour server failure has Europe-wide impact on flights, more than one hour has global impact
There have been two failures of over 6 hours in the last 8 months – these even caused several mile tailbacks on major surrounding roads
£250M fine for latedelivery of T5 !!!
CONNECT EVERYTHING. ACHIEVE ANYTHING.™
Why not Web services alone?
If we asked you to solve this problem with WS-*, SOAP and XML – what would happen?
5 © 2005 Sonic Software Corporation
How would we get there?
CRM ERP
PARTNER SYSTEMS FINANCE
Today’s architecture rigid, costly and difficult to operate
Proprietary technologies and skill sets
Multiple communication infrastructures
High cost of license, consulting and operation
Lots of turf control and organizational issues
ORDERENTRY
6 © 2005 Sonic Software Corporation
J2EE™ APPLICATION
PACKAGED APPLICATION
& LEGACY SYSTEMS
.NET™APPLICATION
PARTNER SYSTEM
WEBSERVICE
In walks MOM – and Web services
Hiding implementation details enables reuse
XML-based data easily exchanged
Designed for remote access, across heterogeneous platforms
Can be easily passed over HTTP(S), JMS, CORBA, Sockets, MQ, RV and almost any other messaging layer
Standard Interfaces are Major Step Forward
TCP/IP
WEB SERVICESINTERFACE
XML
XML
7 © 2005 Sonic Software Corporation
WEB SERVICESINTERFACE
J2EE™ APPLICATION
PACKAGED APPLICATION
& LEGACY SYSTEMS
.NET™APPLICATION
PARTNER SYSTEM
WEBSERVICE
Web Services
Is it reliable, scalable and secure?
How do you change business processes?
How do you manage and monitor distributed services?
But Have We Solved The Whole Problem?
Web services are interoperable communications stacks and don’t offer routing, service deployment, management, format transformation, guaranteed delivery, etc.
You are building standards based spaghetti !
TCP/IP
8 © 2005 Sonic Software Corporation
Why not Web Services alone?
WS-Reliable Messaging creates reliable point-to-point connections– But still 100’s or 1,000’s of them – where’s the manageability?– How do you configure reliability to suit your needs?
WS-Security creates security “Swiss cheese”…– Each secure Web service needs to authenticate incoming
messages– All accessing corporate security server? At the same time?– Creates 100’s of security holes? Can you run your Web services in
the DMZ?– Many customers we work with simply do not allow external Web
services
WS still lacks federated enterprise features
9 © 2005 Sonic Software Corporation
Where is the Strategic Inflection Point?
When the balance of forces shifts from the old structure, from the old ways of doing business and the old ways of competing, to the new.
Before the strategic inflection point, the industry simply was more like the old.
After it, it is more like the new. It is a point where the curve has subtly but profoundly changed, never to change back again.
- Andy Grove, Only the Paranoid Survive, 1996
By 2008, Gartner predicts that SOA will be a prevailing software-engineering practice, ending the 40-year domination of monolithic software architecture.
11 © 2005 Sonic Software Corporation
BAA – The Strategy
“… our strategy is to minimize the interdependencies between products, using open standards to increase operational flexibility and make sure that applications are responsive to change. Therefore a Service Oriented Architecture approach is inevitable.
Our first challenge was to find a platform that would
work well in our very demanding environment, and could orchestrate the services that will drive T5 operations. Sonic Enterprise Service Bus is a very natural fit."
Nick GainsHead of ITBAA
12 © 2005 Sonic Software Corporation
Why did BAA choose SOA?
New technology, regulations, re-organizations, and market demands
For years, change velocity has outstripped IT capacity
Legacy integration approaches failed Costs -- license and services -- exceeded plans Broker / platform stacks: costly, closed, complex Infrastructure never scaled to the extent of the enterprise For example, not one single terminal has opened successfully in over 25 years The last attempt was Seoul with CORBA – it was a complete failure
Very high development costs, particularly with integration Time delays where projects are always behind the business needs Lack of visibility and understanding of systems
Typical Result:
13 © 2005 Sonic Software Corporation
Service Oriented Architectures lead to:
Reduced costs — Simplifies the integration process by making application interoperability "plug-and-play". By utilising open standards, there is less software infrastructure to purchase and maintain.
Faster time to market — An extended enterprise will be able to respond more quickly to market changes than its competitors, as its business is more agile.
Greater operating efficiencies — Companies will be able to reuse existing application components and utilise new services, rather then manually duplicating them in-house.
Increased customer satisfaction — Through tighter integration of the business value chain and less manual intervention in business processes, suppliers and customers will have greater reuse of data, and more reliable and timely information.
14 © 2005 Sonic Software Corporation
INCREMENTAL DEPLOYMENT
BAA’s Enterprise SOA Vision
BROAD-SCALE INTEROPERABILITY
MODULARITY / REUSE
FLEXIBILITY
APPLICATION SERVER
USER-DEFINED SERVICE
LEGACY APPLICATION
PROCESS SERVER
RELATIONAL DATABASE
BATCH SYSTEM
PORTAL SERVICE
Benefits address long-standing IT dilemmas
What most people concentrate on are the endpointsBut the problem area is the “white-space” of SOA
15 © 2005 Sonic Software Corporation
Architectural Challenges of SOA platform
Dependability – they land planes!– Reliable, high-performance communications between services– High Availability = Business continuity– Security
Flexibility – this is a changing environment– Mechanism to orchestrate process through the network– Ability to dynamically re-configure services to new uses– Ability to normalise in-flight documents between services– Bridge multiple low-level middleware technologies
Reach and scale– Connect any resources regardless of where they are deployed– Scale from initial phases to arbitrarily large deployment– Retain visibility and control of distributed infrastructure
16 © 2005 Sonic Software Corporation
INCREMENTAL DEPLOYMENT
Enterprise SOA
BROAD-SCALE INTEROPERABILITY
MODULARITY / REUSE
FLEXIBILITY
APPLICATION SERVER
USER-DEFINED SERVICE
LEGACY APPLICATION
PROCESS SERVER
RELATIONAL DATABASE
BATCH SYSTEM
PORTAL SERVICE
BAA picked an ESB as their SOA framework
BAA chose an ESB to address this SOA “white-space”
HIGH AVAILABILITY
17 © 2005 Sonic Software Corporation
WITHOUT AN ESB
The Purpose of an ESB
WITH AN ESB
Connect, Mediate and Control
Connect Control
Mediate
18 © 2005 Sonic Software Corporation
Connect applications and servicesMultiple on-ramps, dependable communications
Connect old and new– Legacy applications, RDBMS– J2EE, .Net– Web services– B2B protocols
Link services and processes across the extended enterprise
Establish robust, scalable and secure communications
Connect
All connected resources are first-class citizens
Examples:– Securely link internal processes
with those of business partners.– Portal integration
19 © 2005 Sonic Software Corporation
Mediate servicesBridge and extend incompatible technologies
Mediate
Reconcile system incompatibilities– Communication Protocol– Interaction model
Transform and enrich data– Map between data formats– Split, aggregate and enrich data
Provide flexible routing and process flow
– Decoupled, event-driven services– Intelligent routing– Support stateful process management
Eliminate service interdependencies
Examples:– Aggregate data from multiple SAP systems– Regulation compliance logging
20 © 2005 Sonic Software Corporation
Control service interactionDeploy, configure, manage
Dynamically configure, deploy and upgrade hosted services
Establish and alter process flows, routing, Quality of Service
Gain control and visibility over services and their interaction
Control
Configured, not coded
Examples:– Deploy and upgrade 1000s of end-points
from a single location.– Detect faults and diagnose problems in
complex deployment.
21 © 2005 Sonic Software Corporation
Global reach, global scalabilityEnd-to-end SOA
ESB spans clusters and security infrastructure to form federated environment
Bus topology obviates hub-and-spoke bottlenecks Deploy what you need, where and when you need it
23 © 2005 Sonic Software Corporation
How do you manage a project this big?
How do they leverage their existing IT portfolio?
What will this cost?
What would be the impact of– Changes?
– Expansion?
– New security threats?
– Regulation changes?
How will they accommodate future requirements?
Business Process Definitions
24 © 2005 Sonic Software Corporation
Imagine Project Managing the Internet
…would you scope the project?
…would you consider all future needs
…would you handle training?
…would you manage change?
…would ensure interoperability?
…would you manage the technical differences?
…would you manage the scale?
…would you manage the risk?
How…
25 © 2005 Sonic Software Corporation
So how was Internet Successful?
Built around a few sacred principles
Evolved from selected technology standards
Deployment abstracted from design
Incremental deployment
Tactical execution
What made it work?
26 © 2005 Sonic Software Corporation
Strategy Versus Tactics
This is how BAA are making T5 a success
Everything is broken down into manageable tasks
Matrix management
Evolutionary project management
– NOT Waterfall project management
An IT back-bone and architecture from
the very start
Industry patterns are being exploited
What makes some ideas work where others fail?
27 © 2005 Sonic Software Corporation
Design patterns
I encourage you to work with patterns – check out:– http://www.enterpriseintegrationpatterns.com/
– http://www.oreilly.com/catalog/esb/index.html
Working with patterns…– Some examples…
31 © 2005 Sonic Software Corporation
Micro Patterns as Services
Print Req.
ESB Infrastructure
PDS
JMS
Web
JCA
MDB
EJB
SSB
Servlet
Portlet
P2P
P2P
P2PCITP
MQ
1
2 3 4
5
5
5
1. Print Request arrives at CITP2. Request crosses the MQ Series Bridge3. Print Token is resolved in PDS4. Request is routed via CBR5. Request is consumed in Terminal
32 © 2005 Sonic Software Corporation
Reservation Applications
Using a re-factoring pattern
Agent Central CRM Finance Applications
SOAPB
row
se
r
XML SOAP
DB
Registration AppAudit
Adapter
Integration Broker
eP
oS
Cli
en
t
Siebel
Adapter
Mainframe
MQ
S
erie
s
33 © 2005 Sonic Software Corporation
Incremental Staged Deployment
Registration App
Bro
ws
er
DB
ESB
Enterprise SOA – one step at a time
Service Containers
eP
oS
SOAPHTTP
WS
SOAP
Integration Broker
Audit
Adapter
COM
Cli
en
t
Siebel
Adapter
MQ
S
erie
s
Mainframe
Reservation ApplicationsAgent Central CRM Finance Applications
34 © 2005 Sonic Software Corporation
Widely Distributed Enterprises
Registration App
Bro
ws
er
ESBESB
eP
oS
SOAP
Integration Broker
Audit
Adapter
SOAPHTTP
WS
Cli
en
t
Siebel
Adapter
MQ
S
erie
s
Mainframe
DB
Reservation ApplicationsAgent Central CRM Finance Applications
35 © 2005 Sonic Software Corporation
Leave and Layer
Registration App
Bro
ws
er
DB
ESBESB
eP
oS
SOAPHTTP
WS
Integration Broker
Cli
en
t
Siebel
Adapter
Audit
Adapter
ESB
MQ
S
erie
s
Mainframe
Reservation ApplicationsAgent Central CRM Finance Applications
36 © 2005 Sonic Software Corporation
Partner Integration
Registration App
Bro
ws
er
DB
ESBESB
eP
oS
SOAP
MQ
S
erie
s
Siebel
Adapter
Audit
Adapter
SOAPHTTP
WSESB
Integration Broker
Mainframe
Cli
en
t
Reservation ApplicationsAgent Central CRM Finance Applications
37 © 2005 Sonic Software Corporation
Incremental Adoption
I like to say: “Think Strategically, Act Tactically”
Registration App
Bro
ws
er
DB
Mainframe
Cli
en
t
ESBESB
Phase 1Phase 2
Phase n
MQ
S
erie
s
ESB
eP
oS
Siebel
Adapter
Audit
Adapter
Partner ESB
Integration Broker
Reservation ApplicationsAgent Central CRM Finance Applications
CONNECT EVERYTHING. ACHIEVE ANYTHING.™
Some architecture tricks to be aware of…
– Dealing with: Trapped messages Out of Order messages
39 © 2005 Sonic Software Corporation
Machine failure causes trapped messages
Two machines, a brokereach and clustered
Server
Client
1 2
3 4
Machine fails!
40 © 2005 Sonic Software Corporation
Clustering means system carries on straight away
Two machines, a brokereach and clustered
Server
Client1 2
Machine fails!
3 4
41 © 2005 Sonic Software Corporation
Machine recovers and messages delivered
Two machines, a brokereach and clustered
Server
Client1 2
Machine fails!
3 4
Recovery of messages takes:1. Machine reboot2. OS restart3. Software reload/restart4. Database recovery5. Re-send of messages
= several minutes…
42 © 2005 Sonic Software Corporation
Can you wait several minutes?
Trader trying to execute a buy
Retailer trying to process your credit card at a till
Bank trying to process mortgages before the end of the day
Airport trying to route baggage
Telco network usage just before billing run at end of month
Many other situations…
It gets worse…
Now add a requirement for guaranteed message ordering
43 © 2005 Sonic Software Corporation
Typical scenario – an airport display board
Flight Information(four messages)
1 2
3 4
Messages:1 = BMI256 last call gate 32 = BA35 to gate 193 = BA35 to gate 23 (correction)
4 = BA765 last call gate 15
Machine fails!
44 © 2005 Sonic Software Corporation
Messages:1 = BMI256 last call gate 32 = BA35 to gate 193 = BA35 to gate 23 (correction)
4 = BA765 last call gate 15
Message ordering causes “pile-up” in other broker
Flight Information(four messages)
1 2
3 4
Machine fails!
Where are mymessages!
Due to guaranteed message ordering,can’t deliver other messages!
And remember, recoverytakes several minutes…
(I can see potential for missed flights!)
Think of complexity if you add in XA transactions
45 © 2005 Sonic Software Corporation
Turn to Continuous Availability Architecture
CAA
Conventional hardware Seconds to recover Simple and flexible Maintains transactional integrity Easy to use with message
ordering
CAAHOT-HOT deployment
Increases overall throughput
• Each broker has a hot backup broker on another machine
• Transparent to client• Transactional integrity during failover• Once and only once delivery ensured• Fully scalable as we can also add clustering!
46 © 2005 Sonic Software Corporation
P
Remember our airport?
Flight Information(four messages)
11 22
33 44
Machine fails!
Secondary becomes primary and sends messages
47 © 2005 Sonic Software Corporation
Sonic ESBSonic ESB
Overview architecture for BAA T5
Sonic Cluster
BrokerBroker A’A’
BrokerBroker BB
BrokerBroker AA
BrokerBroker B’B’
Sonic Cluster
BrokerBroker A’A’
BrokerBroker BB
BrokerBroker AA
BrokerBroker B’B’
Geographical Site 1 Geographical Site 2
AODB
Sonic ESBcustom JavaservicesAODB
Sonic ESBcustom Javaservices
Clie
nt
Ap
p
Subscribe
Adapter
Clie
nt
Ap
p
Subscribe
Adapter
48 © 2005 Sonic Software Corporation
Summary – moving to “One Architecture” I suggest you look into the following:
– Evolutionary project management Things like extreme programming techniques
– Design and architecture patterns www.enterpriseintegrationpatterns.com
– Service Oriented Architecture This is one of the most sought after roles in IT right now
– (There were 280 architect roles on monster yesterday) Enterprise Service Bus concepts – read the ESB book!
SOA has gone past the critical inflection point– Learn SOA, understand the nuances (the detail)– Serious money is being spent on this new technology– Web services is important, but is only one part of the hybrid
architecture that companies are moving toward