04/12/23 Gargi shankar verma 1

The Open Grid ServicesArchitecture

Open Grid System Architecture By Gargishankar Verma

Reader – RCET Bhilai Dept- Information Technology

04/12/23 Gargi shankar verma 2

OGSAThe `Open Grid Services Architecture` (`OGSA`)

describes an architecture for a service-oriented grid computing environment for business and scientific use, developed within the Global Grid Forum (GGF).

OGSA is based on several other Web service technologies, notably WSDL and SOAP.

OGSA is a distributed interaction and computing architecture based around services, assuring interoperability on heterogeneous systems so that different types of resources can communicate and share information.

OGSA has been described as a refinement of the emerging Web Services architecture, specifically designed to support Grid requirements.

OGSA has been adopted as a grid architecture by a number of grid projects including the Globus Alliance.

04/12/23 Gargi shankar verma 3

Gargi shankar verma 4 04/12/23

The Open Grid Services Architecture

• An open, service-oriented architecture (SOA)– Resources as first-class entities– Dynamic service/resource creation and destruction

• Built on a Web services infrastructure• Resource virtualization at the core• Build grids from small number of standards-based

components– Replaceable, coarse-grained– e.g. brokers

• Customizable– Support for dynamic, domain-specific content…– …within the same standardized framework

Gargi shankar verma 5 04/12/23

• Logical view of capabilities• Relatively coarse-grained functions • Reusable and composable behaviors• Encapsulation of complex operations• Naturally extendable framework• Platform-neutral

– machine and OS

Why Use an SOA?

Gargi shankar verma 6 04/12/23

SOA & Web Services: Key BenefitsSOA

•Flexible− Locate services on any server− Relocate as necessary− Prospective clients find services using

registries

•Scalable− Add & remove services as demand varies

•Replaceable− Update implementations without disruption

to users

•Fault-tolerant− On failure, clients query registry for

alternate services

SOA•Flexible− Locate services on any server− Relocate as necessary− Prospective clients find services using

registries

•Scalable− Add & remove services as demand varies

•Replaceable− Update implementations without disruption

to users

•Fault-tolerant− On failure, clients query registry for

alternate services

Web Services• Interoperable− Growing number of industry standards

• Strong industry support• Reduce time-to-value− Harness robust development tools for

Web services− Decrease learning & implementation time

• Embrace and extend− Leverage effort in developing and driving

consensus on standards− Focus limited resources on augmenting &

adding standards as needed

Web Services• Interoperable− Growing number of industry standards

• Strong industry support• Reduce time-to-value− Harness robust development tools for

Web services− Decrease learning & implementation time

• Embrace and extend− Leverage effort in developing and driving

consensus on standards− Focus limited resources on augmenting &

adding standards as needed

Gargi shankar verma 7 04/12/23

Virtualizing Resources

Resources

Webservices

AccessAccess

StorageStorage SensorsSensors ApplicationsApplications InformationInformationComputersComputers

Resource-specific InterfacesResource-specific Interfaces

Common Interfaces

Type-specific interfaces

Gargi shankar verma 8 04/12/23

A Service-Oriented Grid

Virtualized resources

Grid middleware

servicesBrokering Service

Brokering Service

Registry Service

Registry Service

DataService

DataService

CPU ResourceCPU ResourcePrinter ServicePrinter Service

Job-Submit Service

Job-Submit Service

ComputeService

ComputeService

Notify

Advertise

ApplicationService

ApplicationService

04/12/23 Gargi shankar verma 9

A Closer Look at OGSA

Gargi shankar verma 10 04/12/23

OGSA Capabilities

Security• Cross-organizational users• Trust nobody• Authorized access only

Security• Cross-organizational users• Trust nobody• Authorized access only

Information Services• Registry• Notification• Logging/auditing

Information Services• Registry• Notification• Logging/auditing

Execution Management• Job description & submission• Scheduling• Resource provisioning

Execution Management• Job description & submission• Scheduling• Resource provisioning

Data Services• Common access facilities• Efficient & reliable transport• Replication services

Data Services• Common access facilities• Efficient & reliable transport• Replication services

Self-Management• Self-configuration• Self-optimization• Self-healing

Self-Management• Self-configuration• Self-optimization• Self-healing

Resource Management• Discovery• Monitoring• Control

Resource Management• Discovery• Monitoring• Control

OGSAOGSA

OGSA “profiles”OGSA “profiles”

Web services foundation Web services foundation

Gargi shankar verma 11 04/12/23

CDLCDL

3. Select from or deployrequired resources

3. Select from or deployrequired resources

Execution Management• The basic problem

– Execute and manage jobs/services in the grid– Select from or provision required resources

• The basic problem– Execute and manage jobs/services in the grid– Select from or provision required resources

2. Submit the job2. Submit the job

1. Describe the job1. Describe the job

JSDLJSDL

Job

4. Manage the job4. Manage the job

Gargi shankar verma 12 04/12/23

Describing a Job Submission: JSDL• Job Submission Description Language (JSDL)

– A language for describing the requirements of jobs for submission– Declarative description

• A JSDL document describes the job requirements – Job identification information – Application (e.g., executable, arguments)– Required resources (e.g., CPUs, memory)– Input/output files

JobJob

IT InfrastructureIT Infrastructure

JSDLJSDL

Gargi shankar verma 13 04/12/23

Configuration & Deployment: CDL• Prepare the infrastructure so that the job can execute

– Provide a right-shaped slot to fit the job

• Main parts:– Configuration Description Language (CDL) provides declarative definition of

system configuration– Deployment service carries out configuration requests to deploy and configure the

system

IT InfrastructureIT Infrastructure

CDLCDL

Prepare

Gargi shankar verma 14 04/12/23

IssuesIssues

Find Describe

Access

DataDataDataData

Formats

ProtocolsProtocols

Use casesUse cases

DataDataDataData

DataData

DataDataMove/Copy/ReplicateMove/Copy/Replicate

Metadata Metadata

DataData

ManageManage

Common

accessCommon

access

Data ServicesThe basic problem• Manage, transfer and access distributed data services and resources

The basic problem• Manage, transfer and access distributed data services and resources

Derived dataCatalog

Sensor Data stream

Text file

Relational database

Gargi shankar verma 15 04/12/23

Basic Data Services

Data ResourcesData Resources

Managed StorageManaged Storage

Data ResourcesData ResourcesTransfer ProtocolsTransfer Protocols

Storage Management

Storage Management

Data Management

Data Management

Other Data Services

Other Data Services

TransferTransfer RegistriesRegistries

Non-OGSA client APIs & other servicesNon-OGSA client APIs & other services

Serviceinterface

Resourceinterface

Gargi shankar verma 1604/12/23

Data Services

Data Service n

Data Service n

Data Service 1

Data Service 1

Data Service 2

Data Service 2

Composite Data Services

ReplicationReplication

CacheCache

FederationFederation

Gargi shankar verma 17 04/12/23

Basic Data Interfaces

• Storage Management− e.g. Storage Resource

Management (SRM)

• Storage Management− e.g. Storage Resource

Management (SRM)

• Data Access− ByteIO− Data Access & Integration

(DAI)

• Data Access− ByteIO− Data Access & Integration

(DAI)

• Data Transfer− Data Movement Interface

Specification (DMIS)− Protocols (e.g. GridFTP)

• Data Transfer− Data Movement Interface

Specification (DMIS)− Protocols (e.g. GridFTP)

• Replica management• Metadata catalog• Cache management

• Replica management• Metadata catalog• Cache management

Gargi shankar verma 18 04/12/23

Resource Management• Provides a framework to integrate resource management functions

− interfaces, services, information models, etc.• Enables integrated discovery, monitoring, control, etc.

• Provides a framework to integrate resource management functions− interfaces, services, information models, etc.

• Enables integrated discovery, monitoring, control, etc.

High-levelmanagementservices(GGF)

Domain-specific capabilitiesDomain-specific capabilities

OGSA

Access tomanageability(OASIS, DMTF)

Informationmodels (DMTF,SNIA, etc.)

Resources

WSDM, WS-ManagementWSDM, WS-Management

WSRF/WSN, WS-Transfer/EventingWSRF/WSN, WS-Transfer/Eventing

Dataservices

Securityservices

ExecutionManagement

services

Application-specific

Gargi shankar verma 1904/12/23

Self-Management

Self-

Management

Self-

Management

MonitoringMonitoring

ProjectionProjection

AnalysisAnalysisActionAction

PolicyPolicyPolicyPolicyPolicyPolicy

PolicyPolicyPolicyPolicySLASLA

• Self-configuration: Automatically adapt to changes in the environment:− e.g. Deploy/undeploy resources as load changes

• Self-configuration: Automatically adapt to changes in the environment:− e.g. Deploy/undeploy resources as load changes

• Self-optimization: Automatically tune system to best meet user or business needs− Uses service-level agreements (SLAs)

• Self-optimization: Automatically tune system to best meet user or business needs− Uses service-level agreements (SLAs)

• Self-healing: Automatically detect & correct problems− Component failures− Security violations− etc.

• Self-healing: Automatically detect & correct problems− Component failures− Security violations− etc.

Gargi shankar verma 20 04/12/23

Information Services

Executionmanagement

Resourcereservation

Problemdetermination

Accounting

Applicationmonitoring

Loadbalancing

Servicediscovery

Consumers ConsumersInformationServices

InformationServices

• Reliable• Secure• Efficient

Provide management and access facilities for information about applications and resources in the grid environment

Provide management and access facilities for information about applications and resources in the grid environment

ProducersProducers

Asynchronous notification

Retrieval

RegistryRegistry

LoggerLogger

Gargi shankar verma 2104/12/23

Security Services• Authorization, roles, and access privileges– Locally (site) managed– Based on SAML and XACML security standards– Implementations provide credential mapping

• Working with GGF Security Area groups– Authorization attributes for grids

• Developing OGSA basic security profiles

PKIcertificate

PKIcertificate

WS-SecurityWS-Security

WS-AddressingWS-Addressing

OGSA

security profilesOGSA

security profiles

Gargi shankar verma 22 04/12/23

OGSA Profiles• The normative definition of OGSA

– Styled on WS-I profiles to promote interoperability– Define specific usage patterns

• e.g. execution management

– Basis for claims of conformance• “My scheduler conforms to the OGSA Execution Management Profile…”

• Include specifications developed by GGF and by other bodies– Issue: How mature and widely adopted?– OGSA Profile Definition document provides guidelines

In the futureEarly stagesIn the pipeline

Data ProfileData Profile

OGSA WSRF Basic Profile OGSA WSRF Basic Profile

OGSA Basic Security Profile – CoreOGSA Basic Security Profile – Core

OGSA Basic Security Profile – Secure ChannelOGSA Basic Security Profile – Secure Channel

Execution Mgmt ProfileExecution Mgmt Profile

Others…Others…

HPC ProfileHPC Profile

OGSA Services:Virtual OrganizationComposition, Grouping, Orchestration, Workflow TransactionsMetering, Accounting, and BillingInstallation, Deployment, ProvisioningApplication ContentsInformation and MonitoringLoggingMessagingSecurityPolicyDataProgram Execution

04/12/23 Gargi shankar verma 23

Virtual OrganizationSpecific services will dynamically create and destroy

a VO, as well as manage its members and policies.Virtual Organization (VO) refers to a dynamic set of

individual and/or institutions defined around a set of resource-sharing rules and conditions.

All these virtual organizations share some commonality among them, including common concerns and requirements, but may vary in size, scope, duration, sociology, and structure.

04/12/23 Gargi shankar verma 24

04/12/23 Gargi shankar verma 25

Composition, Grouping, Orchestration, WorkflowThese exotic terms designate a group of

services that deal with coordination of other services that cooperate on a common task.

This includes defining workflows and addressing their scheduling, execution, and monitoring.

Workflows may model business processes as well as perform computational tasks.

04/12/23 Gargi shankar verma 26

TransactionsTransaction services repesent transactions and

transaction management.Because of the heterogeneous and distributed

nature of the Grid, various solutions may be suitable for different environments such as financial application domain or data centers.

04/12/23 Gargi shankar verma 27

Metering, Accounting, and Billing• Effective service sharing and trading between

commercial institutions need to be based on commonly accepted Grid economy.

• Metering services measure the service consumption by the applications and users.

• Rating services apply the pricing information to this consumption and translate the usage from Grid service terms into financial terms.

• Accounting service applies it to the user account and manages the invoicing. Finally, payment is received by the billing service.

• Data generated by the metering and rating procedures may also be cached for the purpose of audit.

04/12/23 Gargi shankar verma 28

Installation, Deployment, Provisioning

• Specialized services handle deployment of new services and their provision to users, so that various types of service capabilities can be delivered to the consumer in uniform fashion.

04/12/23 Gargi shankar verma 29

Application Contents• Grid applications are expected to be broken into at least three tiers:

application, service, and resource.• The application developers write very little code; their main task is

assembling the off-the-shelf services to perform tasks in a coordinated fashion.

• The highest application tier is composed of very little code and a lot of meta-data that defines how the jobs should be performed by the underlying services.

• This meta-data defines the logical structure of the application and the desired runtime configuration of resources.

• It describes deployment parameters for the execution units. It may also contain configuration files for various entities engaged in the application runtime, such as hosting environments.

• In OGSA, such meta-data is called the application content.• OGSA is proposing a model for uniform application content storage

and management, in which application contents service plays a central role.

04/12/23 Gargi shankar verma 30

Information and Monitoring• In a dynamic on-demand environment it cannot be

assumed that applications have static, permanent knowledge of the available services and resources.

• Rather, at some reasonable time before executing particular tasks, the available services need to be interactively discovered, reserved, and subsequently provided to the application.

• The discovery process is based on the information system, which monitors the services available in the Grid and stores the service meta-data in a system of registries.

04/12/23 Gargi shankar verma 31

• In the simplest model, service providers constantly publish their data to the registry, and service consumers query the registry to find out what is available.

• Various enhancements to this model are possible, such as organizing registries into a hierarchical directory, push information flow (subscription), or differentiate confidentiality levels in the published information.

• Apart from a service discovery, the information system would also be used by the real-time monitoring services, as well as the optimizing frameworks that constantly collect and store the information on the state of services for later off-line analysis.

04/12/23 Gargi shankar verma 32

Logging• Distributed logging framework is similar to the

information system. • Rather than the meta-data describing the state of

services, it is concerned about the diagnostic information produced by the applications.

• The logged data would be streamed to the receiver.• OGSA proposes common infrastructure for handling

and delivery of logs in the Grid environment. • Clients of this framework include various diagnostic

tools, real-time monitoring, debugging, problem determination, and performance optimization tools.

04/12/23 Gargi shankar verma 33

Messaging• OGSA frameworks will provide the hosted services

with extended communication mechanisms suitable for various information flow models.

• Services can notify each other in real time about the events that occur during their execution.

• Such notifications would normally be coupled with the subscription mechanism, in which the receiver would signal an interest in the topic.

04/12/23 Gargi shankar verma 34

Security• We have already described the security requirements in

the Grid environment. • In an OGSA context, part of this functionality may be

implemented as services.• Actions such as trust establishment, authentication, or

authorization can each be performed by a specialized service.

• Isolating responsible functions such as authentication in a service can simplify the overall system design and maintenance, as, for example, when security patches need to be applied.

• Another example of a security service is a bridging service, which may be necessary in secure communication between administrative domains, to translate between the security credentials belonging to various domains using incompatible security technology.

04/12/23 Gargi shankar verma 35

Policy• The policy framework will enforce regulations of

service behavior. Policies will specify the rules for services in the areas such as security and resource allocation.

• Policies may have various relations to each other. For instance, the organization structure may be represented by a hierarchy of policies imposed by different levels of administration.

• Policies are related to agreements (contracts between the service requestor and the provider, further restricting the rules for job execution).

04/12/23 Gargi shankar verma 36

• OGSA proposes a unified approach to policy management.

• Policy Service Manager controls access to the policy repository for the policy providers, while another service, Policy Service Agent, is communicating with the policy consumers

• The participating services implement the Policy Enforcement Point interface for effective cooperation with the framework.

04/12/23 Gargi shankar verma 37

Data• Data storage systems and database management

systems (DBMS) are natural candidates for resource virtualization.

• They will be represented by gateway services allowing data access.

• Data caching, replication, and transportation services will take over the function of optimizing data location in relation to the executing environments.

• Data transport services will provide abstraction over low-level transport protocols.

• Data transformation services will be available to perform common operations on data sets such as filtering, search, or format conversion

04/12/23 Gargi shankar verma 38

Program Execution• Program execution environments, such as

computational clusters, are typically managed by scheduler services that ensure optimal usage patterns.

• Among well-known examples of scheduler families are Platform LSF, Sun N1 Grid Engine, and PBS from Altair Engineering.

• Broker services live on top of schedulers. • Brokers cooperate with several schedulers and

submit jobs to those that match the requested criteria.

• An example of a broker is Community Scheduler Framework (CSF), present in the Globus Toolkit 4.

04/12/23 Gargi shankar verma 39