Date post: | 13-Jan-2016 |
Category: |
Documents |
Upload: | garry-robbins |
View: | 213 times |
Download: | 0 times |
perfSONAR Information DiscoveryperfSONAR Information Discovery
February 11th 2010, APAN 29 – perfSONAR WorkshopJeff Boote, Assistant Director R&D
• Motivation• Information Services Architecture
– Global Lookup Service– Home Lookup Service– Topology
• LS Operations– Registration– Discovery– Synchronization
• Protocol
2 – 04/21/23, © 2009 Internet2
Outline
• Network measurements are useful for a local domain, but are also pivotal in solving end to end problems.– Many network problems are found to be at the demarcation of a
network– Most forms of communication must cross outside of the home
domain• Each domain should strive to make historical data available
– À la carte operation frees network staff from special requests– Data availability will drive the production of automatic tools
• Finding specific information can be challenging– Finding a specific type of measurement– Finding data for a given domain or IP subnet– Finding data classified by a VO or community
3 – 04/21/23, © 2009 Internet2
Motivation
• The perfSONAR information services were designed to record the location of data categorized on the following criteria:– Data Type, e.g. by classification such as latency or bandwidth. Or
by tool such as ping or iperf– Domain name– Topology reference (For L3: IP Address (v4 or v6))– Keyword, e.g. “tagging”. Users may assign arbitrary metadata to
measurement data to signify membership in a VO (USATLAS, eVLBI) or other forms of association (Internet2, DOE).
• Services are autonomous – they know to register to some (any close) information service.
• Information services are autonomous as well – they are self organizing to distribute information.
4 – 04/21/23, © 2009 Internet2
Motivation
• The architecture of the perfSONAR Information Services consists of the following components:– Global Lookup Service– Home Lookup Service– Topology– Clients and Services
• Each component is designed in the SOA fashion to do one specific job.
• All components speak the same XML protocols for interoperability
5 – 04/21/23, © 2009 Internet2
Information Services Architecture
6 – 04/21/23, © 2009 Internet2
Information Services Architecture
7 – 04/21/23, © 2009 Internet2
Information Services Architecture
8 – 04/21/23, © 2009 Internet2
Information Services Architecture
9 – 04/21/23, © 2009 Internet2
Information Services Architecture
10 – 04/21/23, © 2009 Internet2
Information Services Architecture
11 – 04/21/23, © 2009 Internet2
Information Services Architecture
12 – 04/21/23, © 2009 Internet2
Information Services Architecture
• The Global Lookup Service is a federation of individual instances– Recommended that the total number be kept small and
geographically placed– Form an ‘information cloud’– Each will synchronize with others to exchange information– Each server is a peer, and within a set amount of time knows what
other servers know• To “bootstrap” where to find the gLS services, a ‘hints’ file is
used– Similar to DNS– Current location: http://www.perfsonar.net/gls.root.hints– Each hints file = a new cloud, can be many at any given time
13 – 04/21/23, © 2009 Internet2
Global Lookup Service
• The gLS performs periodic operations:– Synchronizing with others– Summarizing internal data– Cleaning the databases of expired data
• Data has a TTL – it is automatically cleaned if not renewed• The gLS maintains 2 databases:
– ‘Registered’ information – this comes from hLSs– ‘Summarized’ information – this is a combination of everything
registered• Query Types are structured, or ad-hoc
– XQuery interface to arbitrarily query the databases– Well formed query description for ‘discovery’ need
• Domains• IP Addresses• Data Type• Keywords
14 – 04/21/23, © 2009 Internet2
Global Lookup Service
• gLS servers only accept registrations from hLS servers– MA/MP etc. services do not register to the gLS
• gLS servers respond to the following messages:– LSRegistrationRequest
• Register data from an hLS
– LSDeRegistrationRequest• Deregister data for an hLS
– LSKeyRequest• Retrieve the registration key for an hLS
– LSKeepaliveRequest• Renew a registration for an hLS
– LSQueryRequest• Query the database of the gLS
15 – 04/21/23, © 2009 Internet2
Global Lookup Service
• The Home Lookup Service manages the registration information of the services– Services register their name/address– Services register all of the measurement data they maintain
• Each domain should deploy an hLS– Use multiple hLSs if many services are required (e.g. load on hLS
will dictate splitting – recommended < 10 services per hLS)• hLSs will contact a ‘close’ gLS by using the hints file• hLSs may contact multiple gLSs if desired (gLS synchronization
does not make this a requirement)
16 – 04/21/23, © 2009 Internet2
Home Lookup Service
• The hLS performs periodic operations:– Synchronizing with others– Summarizing internal data– Cleaning the databases of expired data
• Data has a TTL – it is automatically cleaned if not renewed• The hLS maintains 2 databases:
– ‘Registered’ information – this comes from services– ‘Summarized’ information – this is a combination of everything
registered• Query Types are structured, or ad-hoc
– XQuery interface to arbitrarily query the databases– Well formed query description for ‘discovery’ need
• Domains• IP Addresses• Data Type• Keywords
17 – 04/21/23, © 2009 Internet2
Home Lookup Service
• hLS servers only accept registrations from services• hLS servers respond to the following messages:
– LSRegistrationRequest• Register data from a service
– LSDeRegistrationRequest• Deregister data for a service
– LSKeyRequest• Retrieve the registration key for a service
– LSKeepaliveRequest• Renew a registration for a service
– LSQueryRequest• Query the database of the hLS
18 – 04/21/23, © 2009 Internet2
Home Lookup Service
perfSONAR Information DiscoveryperfSONAR Information DiscoveryFebruary 11th 2010, APAN 29 – perfSONAR WorkshopJeff Boote, Assistant Director R&D
For more information, visit psps.perfsonar.net
19 – 04/21/23, © 2009 Internet2
• The topology service stores and normalizes network topology– XML format – similar to measurements– Each topology element features IDs and IDrefs to link state– Supported elements:
• domain• network• node• port• link• path• service
20 – 04/21/23, © 2009 Internet2
Topology Service
• Topology descriptions can be entered in two ways:– By hand via configuration files– Individual elements registered by services
• Queries are supported:– XQuery interface to answer arbitrary topology questions
• The TS registers the domains it has control over to the LS infrastructure.
21 – 04/21/23, © 2009 Internet2
Topology Service
• hLS servers respond to the following messages:– TSQueryRequest
• Query the TS for information
– TSAddRequest• Add new information to the TS
– TSUpdateRequest• Update the time to live for existing information in the TS
– TSReplaceRequest• Replace existing information in the TS
22 – 04/21/23, © 2009 Internet2
Topology Service
• The following operations will be described– Registration– Discovery– Syncronization
• Each operation may involve different actors– hLSs– gLSs– Services– Clients
• There is a time sensitivity to each operation, and it may be performed periodically.
23 – 04/21/23, © 2009 Internet2
LS Operations
• There are 2 Types of Registration:– Service to hLS registration– hLS to gLS registration
• Services will register:– Service name, contact information– All measurement “Metadata”, e.g. the description of each
measurement data set. • hLSs will register:
– Service name, contact information– Summarized data for each registered service (e.g. all “Metadata”
for a given service will be summarized into a set format)
24 – 04/21/23, © 2009 Internet2
LS Operations - Registration
25 – 04/21/23, © 2009 Internet2
LS Operations – Service Registration
26 – 04/21/23, © 2009 Internet2
LS Operations – Service Registration
27 – 04/21/23, © 2009 Internet2
LS Operations – hLS Registration
28 – 04/21/23, © 2009 Internet2
LS Operations – hLS Registration
• Registration accomplishes 2 goals:– Services identify themselves into the LS infrastructure– Services ‘renew’ the registration, e.g. establish themselves as
being active• Other messages go hand in hand with registration:
– DeRegistration• Remove an entire registration• Remove data from the registered data set
– Keepalive• Renew a registration
– KeyRequest• Request a key for a registered data set
29 – 04/21/23, © 2009 Internet2
LS Operations - Registration
• Discovery is the process of locating registered information• Discovery is normally a two part process:
– Find who to ask the question– Ask the question
• The first layer in the discovery process is the gLS:– gLS maintains a list of all hLSs and their summaries– Can answer which hLS has more information for a given discovery
query• The second layer is the hLS:
– hLS maintains a list of some services and their summaries– Can answer which service has more information for a given
discovery query
30 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
• Typical data request:– “I want SNMP data for the Chicago -> New York link on the
Internet2 backbone”• Breaking down the components:
– SNMP Data– Internet2 Domain– Specific interfaces (OC192 between Chicago and New York)
• How to ask the question to the hLS/gLS:– Who has SNMP Data for the Internet2 Network?
• The answer from each will be different:– From gLS: which hLS(s) to ask– From the hLS: which service(s) to ask
31 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
32 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
33 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
34 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
35 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
36 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
37 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
• hLSs and gLSs accept the same kinds of query message– XQuery
• Allows complex use of this language to request almost anything about the data
– Discovery• Structured to limit the querying to specific elements
– Domains– IP Ranges– Data Types– Keywords
• Full recursion is possible with the query model, but not currently supported.
38 – 04/21/23, © 2009 Internet2
LS Operations - Discovery
• gLS servers are independent yet federated– Any domain can deploy one– Services may register to any gLS– gLSs exchange data sets with each other, and ‘register’ things they
may not know about• Synchronization relies on:
– Hints file containing which gLSs participate in a “cloud”– gLSs maintaining a database of the services that have registered
directly with them (e.g. “authoritative” registration source for a given service)
– gLSs running a periodic “broadcast” to all other gLSs containing the hLS instances they are authoritative for. Other gLSs will then choose to either register or ignore this information.
39 – 04/21/23, © 2009 Internet2
LS Operations – Synchronization
40 – 04/21/23, © 2009 Internet2
LS Operations – Synchronization
• There are 5 Major messages in the gLS/hLS Protocol– LSRegistrationRequest– LSDeregistrationRequest– LSKeepaliveRequest– LSQueyRequest– LSKeyRequest
• Each is based on the standard perfSONAR message format:– Message container– At least one metadata element– At least one data element, linked to the metadata element
41 – 04/21/23, © 2009 Internet2
Protocol
• LSRegister(Request/Response)– Register content into the hLS (or gLS)– Metadata(s) = Service Structure– Data(s) = Metadata associated with service– Different operations based on context
• ‘New’ – If the service is not in the LS, it is added to the LS as a new entity
• ‘Update’ – Use the ‘key’ to add new items to the LS• ‘Clobber’ – Send the key and changes to the service element (N.B.
this nukes existing dataset)
Protocol - LSRegister
Protocol - LSRegister
Protocol - LSRegister
Protocol - LSRegister
• LSDeregister(Request/Response)– Remove content from the hLS (or gLS)– Metadata = Key for a service– Data = nothing, or specific service metadata• ‘nothing’ case is a data trigger• ‘Selective’ case is a pS-PS addition (EU *LS code does
not support this)
Protocol - LSDeregister
Protocol - LSDeregister
Protocol - LSDeregister
• LSKeepalive(Request/Response)– Updates the time in the control structure to allow
this service to still live in the hLS (or gLS)– Metadata = key– Data = nothing (trigger)
Protocol - LSKeepalive
Protocol - LSKeepalive
• LSQuery(Request/Response) has different types (varies by eventType):– XQuery (one for each ‘database’ the LS maintains)
• http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/xquery/2.0
• http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/query/xquery/2.0
– Discovery• http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/
discovery/summary/2.0
Protocol - LSQuery
• LSQuery(Request/Response) - XQuery– Query the database in a ‘raw’ format (need to
know a little about how the data is structured)– Metadata = XQuery/XPath statement– Data = nothing (trigger)
Protocol - LSQuery
Protocol - LSQuery
• LSQuery(Request/Response) - Discovery– Similar to query, search discovery set for specific items– Metadata = Discovery Items
• Domains• Keywords• Addresses• EventTypes
– Data = nothing (trigger)– More valuable in gLS case (N.B. discovery on an hLS works
only on the hLS dataset)
Protocol - LSQuery
Protocol - LSQuery
• LSKey(Request/Response)– Request a key (normally given at registration time)
for a service– Metadata = Service description– Data = nothing (trigger)– Currently not ‘safe’ due to no AA infrastructure
Protocol - LSKey
Protocol - LSKey
perfSONAR Information DiscoveryperfSONAR Information DiscoveryFebruary 11th 2010, APAN 29 – perfSONAR WorkshopJeff Boote, Assistant Director R&D
For more information, visit psps.perfsonar.net
58 – 04/21/23, © 2009 Internet2