Post on 12-Jan-2016
transcript
OBIS Portal Architecture Conceptsplus potential for utilization as a basis for Regional
OBIS Nodes
Tony Rees, CSIRO Marine Research, Hobart (and OBIS Technical Subcommittee)
for: OBIS Nodes meeting, Halifax, September 2004
OBIS Architecture – Version 1 (2002)
data provider 1
data provider 2
data provider 3
(etc.)
www user 2
www user 3
(etc.)
Mapping tool 3
Mapping tool 2
all queries
OBIS Portal
C-squares mapper
= custom database wrapper
www user 1
Typical search sequence – OBIS version 1
Typical search result – OBIS version 1
OBIS Architecture – Version 1 (2002)
data provider 1
data provider 2
data provider 3
(etc.)
www user 2
www user 3
(etc.)
Mapping tool 3
Mapping tool 2
all queries
OBIS Portal
C-squares mapper
= custom database wrapper
www user 1
many searches return no data !!!(user does not know what is a viable query)
What is in the Index ...
OBIS Index
name index
• scientific “names with data” from data providers
• “names without data” from Catalogue of Life
• common names from Cat. of Life
• “near match” versions of scientific names
• allocation of all taxa to OBIS taxonomic categories
• synonyms transferred to current names
• metadata on each species (how many records, etc.)
• filters on marine vs. non-marine species, etc.
spatial index
• list of geographic units (squares) in which each species occurs – see next slide
New concept (1) developed in 2003/4 – the “OBIS Index”
Spatial indexing units used for OBIS: 0.5 x 0.5º squares (approx. 50 km resolution) – stored in “c-squares” notation (www.marine.csiro.au/csquares)
OBIS Index
75007500:47500:499
7500:499:4
Part of the spatial index (actually from another database, which employs finer resolution squares):
OBIS Index
New search options – current OBIS version
“Stage 1” searches – return lists of names, with metadata (nos. of records, etc.), taxonomic groups, common names, and “quick maps”
Other search options – scientific name + category search
Other search options – browse by taxonomic category
Other search options – clickable maps by taxonomic category
Example “Stage 1” search result – e.g. genus = “Lutjanus”
For each species ... (NB, all this information is from the Index) – example row:
from Catalogue of Life information
allocation to OBIS taxonomic category
metadata (from parsing the Cache content)
from the Spatial Index – species distribution by 0.5 x 0.5 degree squares
from Cat. of Life info
pre-formatted link to “Stage 2 search”
What is in the Cache ...
• One record per OBIS data point, with copy of key information fields for that point
• Cache content is built by crawling the data providers (and refreshing at intervals)
• Purpose is to provide insulation from providers being off line at any time, and to improve data retrieval speed
• Also as by-product, makes the task of keeping the Index up-to-date relatively simple (crawling of the provider content is already done).
New concept (2) developed in 2003/4 – the “OBIS Cache”
OBIS Cache
Pre-formatted link to “Stage 2” search (get OBIS data) ...
Typical search result – “Stage 2” search (2004)
OBIS Architecture – version 2
www user 1
www user 2
www user 3
(etc.)
Mapping tool 3
Mapping tool 2
“Quick maps”
OBIS Portal (search
application)
OBIS Cache
OBIS Index
“Stage 1”(get info) queries
C-squares mapper
“Stage 2” (get data) queries data provider 1
data provider 2
data provider 3
= DiGIR translation software
global names list (partially complete)
Cat. of Life
metadata refresh
(etc.)provider crawling
Index building
Implications ...
• Index is a self-contained guide to what data are available in the OBIS system (metadata layer) – makes the Portal “intelligent” (content aware)
• Can use the Index as a standalone tool to answer:
– For what species does OBIS have data (and how much), at any particular time
– What is the distribution of species “X” (by 0.5 x 0.5 degree squares) – displayed as “Quick Map” using the c-squares mapper
– Which species occur in region “Y” (0.5 degree square or larger – default query from main map entry point is currently set to 10 x 10 degrees)
– Browse OBIS content, e.g. by category, genus, alphabetical, plus show summary statistics (e.g. numbers of records by category)
– Auto-complete scientific names, correct misspellings, etc.
(above are “Stage 1” [Index] searches)
– Provide 2 entry points to “Get OBIS Data” (stage 2 search):
• Pre-formatted hyperlink to retrieve all data for a species (1-40,000 records), no additional typing required
• Click on any “Quick Map” to retrieve spatially filtered subset for a species.
Options for OBIS Nodes (under this model)...
Possible OBIS node – minimalist configuration
www user 1
www user 2
www user 3
(etc.)
(at remote location)
Mapping tool 3
Mapping tool 2
“Quick maps”
“Stage 2” (get data) queries
OBIS Portal (search
application)
OBIS Cache
OBIS Index
“Stage 1”(get info) queries
C-squares mapper
Possible OBIS node – expanded configuration
www user 1
www user 2
www user 3
(etc.)
Mapping tool 3
Mapping tool 2
“Quick maps”
“Stage 2” (get data) queries
OBIS Portal (search
application)
OBIS Cache
OBIS Index
“Stage 1”(get info) queries
C-squares mapper
The big picture ...
Common OBIS Data Set
OBIS search application – local entry point
www user
Regional Node (Standard
Configuration)
www user
Independent data provider 1
(etc.)
provider crawling
Index building
OBIS search application
(master)
OBIS Cache (master)
OBIS Index (master)
Independent data provider 2
Independent data provider 3
Catalogue of Life
“Quick maps”C-squares mapper
Mapping tool 2
Common OBIS / 3rd party tools (web accessible)
“OBIS Central” – Rutgers
OBIS search application
(copy)
OBIS Cache (copy)
OBIS Index (copy)
www user
Regional Node + full mirror
function
RON sourced data
Mapping tool 3
OBIS search application – local entry point
www user
Regional Node (Standard
Configuration)
OBIS search application – local entry point
www user
Regional Node (Standard
Configuration)