Date post: | 16-Jul-2015 |
Category: |
Technology |
Upload: | datacenters |
View: | 169 times |
Download: | 2 times |
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
ProActive Integration with HP Software in the AGOS project
Towards the integration of business services within a grid environment supported by a virtualized infrastructure
Grids@Work 2008Sophia Antipolis – October 21st, 2008
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
2 June 14, 2010
Project contextPartner use casesDraft architectureProducts mappingHP integrationProActive features
Agenda
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Project context
Putting together experts from grids, services, resources and data to define a simple, coherent and realistic solution
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProject context
Selected by the “Pôle de compétitivité Solutions Communicantes Sécurisées”
List of partners: HP, Oracle, Inria, Activeeon Defined scope: integration project
- Definition of an integration methodology- Definition of new software components- Integration of business and service dashboards- Integration of services with virtualized resources- Contribution to standards
Objective: “Integrate GRID technology into a business oriented SOA stack running over a virtualized infrastructure”
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProposed solution
A reference architecture putting together:- business intelligence- service oriented components- grid environment- virtualized systems
A library of process and service templates based on standards
A set of tools to easily build such applications A real-time monitoring of business processes,
services and resourcesA methodology to migrate to the architecture
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Applied onto water operator use case
Simulation application to better handle water distribution network and provision adequate hydraulic meshing
Focus is on software which allows the study, the planning and the control of irrigation
- hydraulic simulation software for meshed networks for distribution of raw water or drinking water under pressure
- full network is too large to be simulated at once; split into sub-network parts that can be studied independently
Main interest:- Parallelization of the application (SPMD approach)- Overlap multiple runs (parameters sweeping) on subnetworks- Better integrate with other applications- Better use of resources to speed up computation
Measured in terms of speed increasing of digital processing provided by the grid architecture
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sApplied onto global distribution system’s usage use case
Parsing and analysis of web session logs from target applications Focus is on providing massive data analysis:
- Customer session graph analysis or other data mining services- Know how groups of customers behave: the usage / navigation in websites
Main interest- Handle the size and the volume of data generated by core applications- Parsing of logs done massively in parallel- Results need to be stored in a distributed database- Better understand how the products are used- Help to improve the products continuously through the feedback loop
Expectations- Enable an advanced workflow beyond current standards to support processes
and sub processes and/or to compose hybrid processes with services.- Enable streaming results from any parallel activity on a grid to feed a
distributed database- Support operational aspects like monitoring, clustering, scalability, security,
load balancing and failover thus enabling high availability SLA compliance agreements to be respected.
- Enables one button push start, stop and resume on any running grid process
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
From building blocks to functional architecture proposal
Draft architecture
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
9 June 14, 2010
Base line…
Staged pyramid- Adaptive infrastructure of heterogeneous resources- Service delivery with quality added-value- Business intelligence based on standardized processes
Based on standards- As defined in the related standards documents- Existing implementations (proprietary / open-source)
With grid middleware- Grid environment: parallelism, distribution- Web interfaces: thin web clients access- Modeling facilities: composition, adaptive, configurable- Efficient execution: design & execution patterns
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Grid
Grid Grid
Grid
BI
SA
AI
…of a Service Grid
BusinessIntelligence
ServiceArchitecture
AdaptiveInfrastructure
BPELETL
SCA / GCMSOA / WS
VM / XENP2P / GTK
Standards Functional
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sMain building blocks
Systems Virtualization Grid Utility Interface Resource Manager Tasks & Services
Scheduler Enterprise Service Bus Distributed Shared
Memory Database Interface Application Server
Interface Web Service Interface
SOA Registry / Repository SOA Governance Component Architecture Load Balancer Fault Tolerance Reconfiguration Process Business Process Execution Real-time Monitoring Tools Service Level Management Business Intelligence Reporting Tools
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sDefine the business processes
Processes defined in terms of business items Cope with business processes workflow Focus on business interactions between human and applications
- How are used the applications- How are they chained in any given scenario- What is expected as an output
Not a programmatic task flow- Do not cope with service implementation details- Far from resource allocation concerns
Handle specific concepts:- Long transactions (suspended time, recurring patterns…)- Requests approval (decision making, validation…)- Human interactions (subjective perception, biased usage…)- Business ecosystem (management, billing, visualization…)
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Business Intelligence
Business Process
Service Level Management
ReportingBusiness
Intelligence
ServiceArchitecture
AdaptiveInfrastructure
- Delivered results analysis
- Scenario validation
- Service & Service Level monitoring
- Reporting on key aspects by category
- Validation of performance gain, result precision, time to deliver…
- Comparison with reference benchmarks
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sCalling value-added services
Services stored in a registry with governance applied Manipulating proxies to available web services Services as composite components With predefined characteristics and features
- Interface contract- Policy enforcement- Delivered quality of service
Mapped onto:- Existing web services (black-box)- Composite web services (functional composition)- Skeleton web services (parallel pattern)- Subordinate processes (time composition)- Simple sequential services
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sImplemented as grid services
Components executed on a set of resourcesSkeletons providing parallel patterns for efficient
execution on the grid Transformed as jobs delegated to the grid
schedulerOptimized active objects exposed as web
services on the grid Jobs scheduled on the grid with direct access to
the grid schedulerSpecialized grid schedulers dedicated for specific
usage
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Service Architecture
Service CompositionService Component Architecture
Parallel ProgrammingDistributed shared memory
SOA EnvironmentRepository, Registry, Orchestration
BusinessIntelligence
ServiceArchitecture
AdaptiveInfrastructure
- Distributed parallel services accessing data in distributed shared memory
- Functional composition of service components with dynamic reconfiguration
- Exposed as web services in registry with a policy enforcement contract
- Executed through proxies & orchestrated by a workflow engine
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sExecuting on heterogeneous resources
Code execution on well-known resources or over a set of dynamically discovered resources- As tasks through a scheduler managing load-balancing, fault
tolerance, high availability…- As active objects with associated features such as migration, file
transfer…- As uncontrolled legacy application- As tasks delegated to another grid utility environment
Evolving set of resources (join / leave support) With specific or adaptive characteristics (virtualized or
clustered) On a set of various operating systems Protected or not, intrusive or not
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Resource Virtualization
Grid Utility interface
Resource Manager
Task Scheduling
BusinessIntelligence
ServiceArchitecture
AdaptiveInfrastructure
Adaptive Infrastructure
- Virtual resources managed dynamically- Tasks executed on the set of evolving resources- Requests of tasks execution received via the bus
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Quality Testing Real-Time Monitoring
Parallel Computing
Adaptive Infrastructure
Grid Middleware
Business Services
Business Intelligence
Service Level Agreement
Management
Business Processes Workflow
Data Warehouse Reporting
Service Components
Parallel ServicesEnd-to-end Service Testing
End-to-end Monitoring
Resource Manager
Tasks Scheduler
Distributed Shared Memory
SOA Environment
GRID Environment
AI Environment
Policy Enforcement
Registry
Grid Components
Skeletons
Native Runtime Virtual Machine Hypervisor Grid Utility
Proposed architecture
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProducts integration mapping
Global contribution- ActiveEon products from the ProActive Suite- Oracle products from the Business Services Suite- HP products from the Virtualization, Quality and SOA
Centers
Alternatives from the open-source among others- ActiveBPEL, Cockpit- Sequoia- Tuscany- Xen VM
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProposal
OS, HW
OS Virtualization Grid Utility interface
Enterprise Service Bus
Resource Manager
Task Scheduling
Service CompositionService Component Architecture
Parallel ProgrammingDistributed shared memory
Business Process
Qua
lity
Test
ing
SLA Mgt
ReportingReal-Tim
e Monitoring
BusinessIntelligence
ServiceArchitecture
AdaptiveInfrastructure
SOA EnvironmentRepository, Registry, Orchestration
BPEL EngineBPEL Process Manager
Integrity VM System Insight
BAC D
ashboardBAC
Qua
lity
Cent
er
Enterprise Service Bus
BIEE
BPA Suite
JDeveloper
DSM Coherence J2EE AS
DB + RAC
Resource Manager
Tasks Scheduler
Grid Utility
IC2D
GCM components
SOA ManagerSOA Systinet
Skeletons
ProActive Hewlett Packard Oracle
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sOpen Source
OS, HW
OS Virtualization Grid Utility interface
Enterprise Service Bus
Resource Manager
Task Scheduling
Service CompositionService Component Architecture
Parallel ProgrammingDistributed shared memory
Qua
lity
Test
ing
Real-Time M
onitoring
BusinessIntelligence
ServiceArchitecture
AdaptiveInfrastructure
SOA EnvironmentRepository, Registry, Orchestration
BPEL EngineActive BPEL
Xen VM
IC2D
Apache ServiceMix
Eclipse
Sequoia Jboss / Jonas
MySQL
Resource Manager
Tasks Scheduler
Gtk
Skeletons
Timit
Apache jUDDI
Cockpit
RUDDI
Tuscany
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
HP Integration Points
Products & Methodology
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sHP contribution
Adaptive Infrastructure lab for the projectArchitecture blueprint draftConsulting & Integration methodologyHP Software products from:
- SOA Center (Service Oriented Architecture)- BSM Center (Business Service Management)- VSE Center (Virtual Server Environment)- Quality Center (Quality Test Framework)
Objective: Virtualization of services and resources of the parallel / distributed grid in a business context
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sAGOS blueprint draft
Parallel Computing
Parallel ServicesDistributed SharedMemory Services
Grid Monitoring
AGOS blueprint
Adaptive Infrastructure
Grid Middleware
Business Services
Business IntelligenceQuality Testing Real-Time Monitoring
Native Operating Systems
Virtualized Operating Systems
Hardware Resource
Virtual Machines
Hypervisor
Resource Monitoring
Grid Utility Interface
Resource Manager
Task / Grid Service
Scheduler
Component Runtime
Enterprise Service Bus
Parallel Programming Distributed Shared Memory
Policy Enforcement
Service Registry
Service Component
Business Process Engine
ReportingDataware
HouseService Level Management
Consolidated End-to-End Monitoring
Grid Monitoring
Task / Grid Service
Monitoring
End-to-End Service Testing
Business Workflow
Data Mining
Service Monitoring
9a
8b 8a
11g1
11b
11c
9b
10
11g
MessageMonitoring
5c 5d
5a
16
2b
3a3d
1b
5b
11a
6c
1a
11c1
11c3
4b
11c2
11b3
11b2
11b1
2a
11g2 11g3 11g4
6a
6b
5e
3c3b
1c
4a 11f
7a7b
11c4
12a 12b 12c
13
14a
14b
Benchmarking
14c
15a
4c
15b12d 14d
14d
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProducts from HP SOA Center
HP Systinet- UDDI Registry- As the de-facto SOA reference repository
HP Policy Enforcement- Policy Enforcement- As the smart SOA governance runtime
HP BAC for SOA- Services Monitoring- As the integrated SOA services monitoring- Combining business, service and resource monitoring
HP Quality for SOA- Services Testing- As the best-in-class testing environment
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sIntegrated specifically within AGOS
Provide a library of business processes templates including both functional aspects (gather input data, run simulation, perform pre-processing, analyze output data…) and business aspects (warn user, check execution time, request for more resources…)
Define the library of predefined SOA services Declare all the SOA services to be used by the business
processes in the SOA registry Define and declare all SOA policy enforcements (as proxy
of services) on top of existing SOA services
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProducts from HP BSM Center
HP Business Availability Center- Integrated business dashboard- Reporting for business metrics at every layer
• Business processes• Services execution• Resource consumption
HP System Insight Manager- Collect raw metrics from adaptive infrastructure- Feeder of HP BAC systems for real or virtualized
hardware problems
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sIntegrated specifically within AGOS
Define or discover the business service models to get monitored
Get in SOA registry all the services to get metrics on the architecture components
Push the indicator values into the BAC monitoring system (source, metrics name + value)
Define the SLA based on customer’s expected results
Define the reports to deliver to the business managers
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProducts from HP VSE Center
HP VM- Virtualization of resources and operating systems- Adaptive deployment of resources with hypervisors- Support enterprise hardware and configurations
• Storage Area Network• Blade Servers (Itanium, Pa-Risc, x86)• Hardware & Software High Availability
HP RDP- Resource deployment protocol- Manage VM and installed software transparently
Integration expertise with Xen environment Integration expertise with VMWare environment Thursday 23rd 10:30: HP VSE by Francois Donze
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sIntegrated specifically within AGOS
Deploy hardware (cpu, ram, disk) on demandDeploy operating system / virtual machine or
dedicated application through web services Integrate with ProActive resource manager
- Add, get or list available nodes- Define constraints of usage or quality of service
• Deploy within the hour• Schedule the deployment of x VM• Get node with x ram + y cpu
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
ProActive used features
Framework in action
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sProActive Suite
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sIntegration with ProActive ProActiveAdapter
X
OS, HW
OS Virtualization Grid Utility interface
Enterprise Service Bus
Resource Manager
Task Scheduling
Service CompositionService Component Architecture
Parallel ProgrammingDistributed shared memory
Qua
lity
Test
ing
Real-Time M
onitoring
Business Intelligence
Service Architecture
AdaptiveInfrastructure
SOA EnvironmentRepository, Registry, Orchestration
BPEL EngineBPEL Process Manager
Integrity VM System Insight
BAC D
ashboard
BAC
Qua
lity
Cent
er
Enterprise Service Bus
BIEE
BPA Suite
JDeveloper
Coherence J2EE AS
DB + RAC
Resource Manager
Tasks Scheduler
Grid Utility
IC2D
Skeletons
SOA ManagerSOA Systinet
1 42
5
96
3
78
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
sIntegration points with ProActive
1) Generate SCA component as a GCM component2) Expose grid components in SOA Repository3) Provide distributed shared memory’s data to tasks &
components4) Publish metrics from components, skeletons, tasks and
resources5) Make tasks scheduler accessible as a service6) Expose virtual machines as resource nodes7) Expose external grid services (like scheduler as a node
resource hosting a tasks scheduler)8) Provide IC2D aggregated metrics towards monitoring system9) Provide introspection agent to test a particular service,
component, task or object
Arc
hite
ctur
e G
rill
e O
rien
tée
Ser
vice
s
Thank youQuestions / Answers