EMI is partially funded by the European Commission under Grant Agreement RI-261611
Discovering Infrastructures with EMI Registry (EMIR)
Emidio Giorgio – INFN - IGIOn behalf of Shiraz Memon, Jülich Supercomputing Center
ISGC 2012, 29.02.2012
EMI I
NFS
O-R
I-261
611
• Motivation• Overview• High level architecture
– Registry Hierarchies– Global level replication– Information Model
• Integration with EMI Services – UNICORE, ARC• Conclusions
Outline
19/04/23 ISGC 2012 2
EMI I
NFS
O-R
I-261
611
• EMI Registry (EMIR)
Motivation
19/04/23 ISGC 2012 3
• Lack of a common service index; within EMI– UNICORE (Global) Registry– ARC ISIS– Site / Top Level BDII
• Middleware specific Discovery • Various Information Models• Not Federation Aware
Unfortunately, nothing to do with oil
motivationmotivation
EMI I
NFS
O-R
I-261
611
• Cross Middleware Service Discovery• Supports Hierarchies (Theoretically infinite)• Common Access Point and Unified Method
to service registrations and discovery• Lightweight REST-full API• Standards based GLUE 2.0 Information
Model
EMIR: A Unified Service Registry
19/04/23 ISGC 2012 4motivationmotivation
EMI I
NFS
O-R
I-261
611
• Domain Service Registry (DSR)– A registry node at any level of the hierarchy– Registration and Rich Query Interface– PKI based Authentication– XACML or simplified ACL for Authorization– Synchronization with a parent registry node– Filtered Registrations
• Global Service Registry (GSR)– Similar to DSR– Replication with other GSR using P2P techniques– Bootstrap from publicly available Global List
Main Components
19/04/23 ISGC 2012 5overviewoverview
EMI I
NFS
O-R
I-261
611
High Level Architecture
19/04/23 ISGC 2012 6High level architectureHigh level architecture
EMI I
NFS
O-R
I-261
611
Communication of DSRs (1/2)
19/04/23 ISGC 2012 7
<<Child>>
DSR
<<Parent>>
DSR
Service
publish
<<Global>>GSR
Domain
Computing Center
State/Region
Country
Publish
Publish
Liveness at the GSR ∞ Level of Hierarchy
High level architectureHigh level architecture
EMI I
NFS
O-R
I-261
611
• Predefined Parent ref. at the Child DSR• Synchronization is done through events• Active Sync.
– Parent notified as soon as the changes occurred in the child DSR
• Passive Sync.– Parent notified as soon as it joins the hierarchy
(fresh startup or recovered from a failure)– Temporary caching
Communication among DSRs (2/2)
19/04/23 ISGC 2012 8High level architectureHigh level architecture
EMI I
NFS
O-R
I-261
611
Global Level Replication
19/04/23 ISGC 2012 9
• Replication of Registries based on (structured) P2P
• Data Synchronization at the Global level
• Why? • Federation level Service
Discovery• Failover• Robustness• Scalability• Load Balancing
• Same query and registration interface• Where to find the GSRs?
• Publicly known list
<<Global>>
GSR
<<Global>>
GSR
<<Global>>
GSR
https://gsr1,https://gsr2,https://
gsr3 Bootstrap
<<Global>>
GSR
GSR List
High level architectureHigh level architecture
EMI I
NFS
O-R
I-261
611
• Schema free rendering• Adopts GLUE 2.0’s generic service entities• Supports standard XML as a response on
query functions• JSON for message exchange (query, de-
registration) and service record• Expiration time for every service record• Constraining service URL as “unique”
identifier
Information Model
19/04/23 ISGC 2012 10High level architectureHigh level architecture
EMI I
NFS
O-R
I-261
611
• Service Name• Service Type• Endpoint URL• Service Capabilities• Downtime Info.
JSON Rendering
19/04/23 ISGC 2012 11
[{ "Service_Name": “EMI Execution Service", "Service_CreationTime": {"$date": "2011-10-16T11:47:24Z"}, "Service_Type": “eu.emi.es", "Service_Capability": [“activity submission", “activity creation"], "Service_QualityLevel": "production", "Service_Endpoint_URL": "http://eu-emi.eu/emi-es", "Service_Endpoint_Technology": “SOAP over HTTP", "Service_Endpoint_InterfaceName": “ActivityManagement", "Service_Endpoint_InterfaceVersion": ["1.0“ ], "Service_Endpoint_WSDL": "http://eu-emi.eu/emi-es ?wsdl", "Service_Endpoint_HealthState": "ok", "Service_Endpoint_HealthStateInfo": “it is OK, I can see it", "Service_Endpoint_ServingState": "production", "Service_Endpoint_StartTime": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_DowntimeAnnounce": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_DowntimeStart": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_DowntimeEnd": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_QualityLevel": "production", "Service_ExpireOn": {"$date": "2011-07-21T11:47:24Z"}, "Service_UpdateOn": {"$date": "2011-07-21T11:47:24Z"}, "Service_OwnerDN": “CN=Shiraz Memon,O=FZJ,OU=JSC” }]
High level architectureHigh level architecture
EMI I
NFS
O-R
I-261
611
Integration with EMI Services
19/04/23 ISGC 2012 12
Services•TargetSystemFactory •BESFactory•EMI-ES•StorageFactory
UNICORE
Services•AREX Compute•gridftpd•Chelonia Storage
ARC
Register
Register
ARC Client
UNICORE Command Line Client (UCC)
Lookup & Discover EMI Services
Integration with EMI
EMI I
NFS
O-R
I-261
611
• Integration with all the EMI Services• Automatic (de)commissioning of a Registry
node in the hierarchical network• Handling synchronization and failures while
replicating the large number of service entries
Future Work
19/04/23 ISGC 2012 13Conclusions
EMI I
NFS
O-R
I-261
611
• Common Service Index for EMI• Support for federation• Architecture• Information Model• Integration Scenario
Summary
19/04/23 ISGC 2012 14Conclusions
EMI I
NFS
O-R
I-261
611
• Documentation • (REST API (WADL Document)
• Wiki• Code• Packages• Contacts => EMI Forum
Links
19/04/23 ISGC 2012 15Conclusions