Date post: | 17-Dec-2015 |
Category: |
Documents |
Upload: | noah-dennis |
View: | 215 times |
Download: | 0 times |
The Color of Ocean Data, Brussels, Belgium, 25-27 Nov, 2002The Color of Ocean Data, Brussels, Belgium, 25-27 Nov, 2002
Experience of XML use in Russian NODC and perspectives - MedBlackDODS and other
Nicolay Mikhailov, Evgeny Vjazilov, Alexander Besprozvanykh, Natalja Puzova
RIHMI-WDC/Russian NODCObninsk, Russian Federation
CONTENTCONTENT
1, Two words about XML1, Two words about XML
2, Russian NODC experience2, Russian NODC experience
3, Marine XML 3, Marine XML
4, View on Distributed Ocean Data System4, View on Distributed Ocean Data System
What is XML (What is XML (eXtensible Markup Language) ?- eXtensible Markup Language) ?- classicclassic
XML XML is a component of Web-technologies to describe is a component of Web-technologies to describe information resources for exchange and distribution information resources for exchange and distribution
under Webunder Web
Independence from computer platformsIndependence from computer platforms
Support by the developersSupport by the developers – – W3C standard for HTTPW3C standard for HTTP
Self-describingSelf-describing - - readablereadable and allows and allows to automate data processingto automate data processing
Hierarchy-Hierarchy-allows easily to describe allows easily to describe complex structures of the datacomplex structures of the data
This is sintactic tool onlyThis is sintactic tool only
It is required to It is required to rewrite rewrite existing existing software software - - partiallypartially
Data volume and processing Data volume and processing timetime
More complicated technologyMore complicated technology
New level of marine DM demands the New level of marine DM demands the use use of new of new information information technologiestechnologies
Why XML?Why XML?Computer network
Web midleware
Web DBMS/GIS
Web DM applications
XML roleXML role
Driving forcesDriving forces – national research programme World Ocean – national research programme World Ocean
10 ministries/agencies, 40 partners
1999-2002 – 1-st stage, 2003-2007 – 2-nd stage, 2008-2012 – 3-d stage
Unified System of Information on World Ocean ConditionsUnified System of Information on World Ocean Conditions
Marine Marine environmentenvironmentMeteo, Physic Ocean,Meteo, Physic Ocean,
Ice, Chemistry, Ice, Chemistry, Pollution, Pollution,
Bioresources, Bioresources, Geology-geophysic Geology-geophysic ….….Marine activityMarine activity IInfrastructurenfrastructure
Lows and Lows and specificationsspecifications
Social economic ,...Social economic ,...
Users:Users:
Federal levelFederal level
Regional level Regional level
Organization levelOrganization level
ESIMO ESIMO ProductsProducts
•MetadataMetadata
•Observ. dataObserv. data
•Climat, Climat, analisys, analisys, forecastsforecasts
•Analytical Analytical informationinformation
•forfor •requestsrequests•““pushing”pushing” •special services special services
•In In – – operational operational and delay-mode and delay-mode
Collection
Accumulation Processing
Dissemination
Integration and services
Storage
1. Historical data - Russian NODC (1920-1984)
2. Historical data - ICES (1985-1986)
3. GTSPP data (Aug -Sept 2000)
Web-serverApp server
5. GTS Data • SHIP• BATHY• TESAC• BOUYfor Aug-Sept. 2000
DBMSRIHMI-WDC Obninsk
Web/XML technology prototype – IODE XVI Demonstrator/2000
Web-serverApp server
Lisbon, DM 1
Lisbon, DM 2
Lisbon, DM 3Application
• Caspian Sea, 20022002/2003/2003
ГосНИНГИГосНИНГИ
ЦКБ ГМПЦКБ ГМП
НИЦ НИЦ ПланетаПланета
ВНИИГМИ-МЦДВНИИГМИ-МЦД
ГРМЦГРМЦ
• Black Sea,, 2000 г. 2000 г.
RTS
Integrated DBHist. DB
Sat. DBSat. DBClimat DB
GTSGTS
GTS DB
ESIMO integration module on Web/XML technology prototype – 2000/2002
Web/XML “live” applications/2002Web/XML “live” applications/2002
ESIMO Integration Module – ESIMO Integration Module – metadata and information metadata and information management ( as information resource objects)management ( as information resource objects)
Cruise and ocean stations metCruise and ocean stations metааdatadata
On-line GTS data base – ocean and marine On-line GTS data base – ocean and marine hydro/meteo ship/coastal data hydro/meteo ship/coastal data
•MEDAR/MEDATLAS II InventoryMEDAR/MEDATLAS II Inventory•Russian NODC/WDC-B on-line inventoryRussian NODC/WDC-B on-line inventory
ESIMO Integration Module – Web-portal /2002ESIMO Integration Module – Web-portal /2002
ESIMO centres
Web - портал Главная страница - список ИП
Global Inventory of information Global Inventory of information resources resources
HtmlHtml GIFGIF ApplicationsApplicationsDocDoc DB/DB/filesfiles
ftpftp
• Information resourcesInformation resources
•Web-sites with Web-sites with metadata, data and metadata, data and productionproduction
•Independed Independed applicationsapplications
• Resource descriptionsResource descriptions
•Identification and Identification and typetype
•ParametersParameters
•Time and geoTime and geo
• AuthorsAuthors
•Links to XML docs and Links to XML docs and etc. etc.
ServicesServices
•User access User access passwords passwords
•Check of reCheck of resourcesource
readiness readiness
•User User WP WP Adjustments Adjustments
•......
Testing: 20.09.2002 – 30.11.2002Testing: 20.09.2002 – 30.11.2002
Launching: 20-25.01.2003Launching: 20-25.01.2003
Specialized Specialized ApplicationsApplications
Object name Object amount
XML-file name
Data Provider Descriptions Marine organizations 100 orgs.xml
Projects and Programmes 90 project.xml Experts 110 experts.xml
Observation Networks 15 observ.xml Observation Methods 25 Metods.xml
Research Vessels 800 ships.xml Satellites 35 satel.xml
Observ. Equipments 150 prib_CCB.xml Coastal Stations 700 Coastel.xml
Information Resource Descriptions Data sets and databases 400 Data_set.xml
R/V ocean cruises 33500 Cruise.XML Marine maps 160 maps.xml
Web-resources 20 web_res.xml DM tool descriptions
Data Formats 25 format_n.xml Software 20 soft.xml
Parameter Dictionary 400 UDOP.xml
Metadata on ESIMO Web-pagesMetadata on ESIMO Web-pages
< ST_LIST REF = "TN89196300001"OBS = "bottle" xmlns:dt="urn:uuid:C2F41010-65B3-11d1-A29F-00AA00C14882/">< ST NUM = "00091" ><LAT> 41.5 </LAT><LON> 30.8333 </LON>< DAT dt:dt = "date" > 1963-04-11 10:00 </DAT><GOR> 1233 </GOR><LIST_PAR><PAR> PRES </PAR><PAR> TEMP </PAR><PAR> PSAL </PAR><PAR> DOX1 </PAR></LIST_PAR></ST>
Cruise and ocean metadataCruise and ocean metadata - - MEDAR/MEDATLAS Inventory and on-line MEDAR/MEDATLAS Inventory and on-line Russian NODC/WDC-B inventoryRussian NODC/WDC-B inventory
<CR_LIST xmlns:dt="urn:uuid:C2F41010-65B3-11d1-A29F-00AA00C14882/"><CR><CNT> 06 </CNT><SHIP> 06MT </SHIP><REF> AD06199325003 </REF><DC> FI </DC><FILES>< F OBS = "ctd" INV = "st.mete25c.ctd.xml" name ="/Ctd/new/mete25c.ctd " ><IHB> 28B </IHB>< ST dt:dt = "date" > 1993-07-03 </ST>< FN dt:dt = "date" > 1993-07-16 </FN></F></FILES>< R / ></CR><CR>
DBMS+Data files
XML filesXML files Java Java
MEDAR/MEDATLAS IIMEDAR/MEDATLAS IICDsCDs
Web-serverWeb-server
on-line Russian on-line Russian NODC/WDC-B NODC/WDC-B
inventoryinventory32,8 thou. cruisec32,8 thou. cruisec
COD Conference, Brussels, 25-27 Nov, 2002
MEDAR/MEDATLAS Cruise InventoryMEDAR/MEDATLAS Cruise Inventory
On-line GTS Data Base - HYDROMETEO DTDOn-line GTS Data Base - HYDROMETEO DTD
<DataBlock dbI="dbOcean"> <Table TbN="Buoy_Lv"> <Data StI="21533" Lat="36.2" Lon="130.47"> <D T="1999-12-06 22:22:00.0" Lev="0.0"> <P N="TYPE_OBS" V="74" /> <P N="MARS10" V="131" /> <P N="CENTR" V="WKARS" /> <P N="LEVTYPE" V="100" /> <P N="TEMPWAT" V="16.36" Q="0" /> </D> <D T="1999-12-06 22:22:00.0" Lev="3.0"> <P N="TYPE_OBS" V="74" /> <P N="MARS10" V="131" /> <P N="CENTR" V="WKARS" /> <P N="LEVTYPE" V="100" /> <P N="TEMPWAT" V="16.68" Q="0" /> </D> </Data>
</Table></DataBlock>
Four forms are used for parameter definition: Form 1 - pointer to DBMS table name and column name
<ElT TbN="Buoy_Lv" ClN="TEMPWAT"/>Form 2 – code parameters:
<ElC Cod="2102"/>, Cod – from Parameter DictionaryForm 3 - function to produce parameter:
<ElF FnN="Function_Name"/>Form 4 - defining the process of calculation.
Data are placed in DataBlock Key values: Station ID, Latitude and LongitudeD node – data segment (profile)P node – data string, where N - temporary name, V - value, Q -quality control flag, D - additional definition.
DBMS
App Server MetDD DOM App Server MetDD DOM
Web-serverWeb-server
GTS
Generalization of XML useGeneralization of XML use
Common data model – structured presentation of application domain objects and relations
SemanticSemantic SyntaxSyntax
Common Parameter Dictionary – Terms and parameters definitionefinitions, s, Other attributes Other attributes
Common Codes – Platforms, Organizations, Geoareas, Equipments, QC sighs, Units, ...
Request model – structured presentation of data and services requests
Tags, attributes -start and end of element, hierarhy definition, ...
Entity references - start and end of element, hierarhy definition, ..
Processing instruction - specific information for applications, ..
Character data sections (CDATA) - character information
Document type declaration - DTD construction
DTD XML Scheme
New is well well forgotten oldNew is well well forgotten old - GF-3 - XML syntax, GF-3 subsets - - GF-3 - XML syntax, GF-3 subsets - DTD, GF-3+GF-3 Proc - DOM DTD, GF-3+GF-3 Proc - DOM
Format Name
Obs. type
Meta data
Codes
Data Struc-turing
Soft-ware
Self-defin-ing
ICES H, CTD CI WMO, IOC,
ICES FPAR Ocean
PC No
ICES Blueprint '86
H, CTD CG GF3 VPAR OceanPC
Yes/GF3
GF3 (Subsets) M, H, MBT/XBT, CTD,…
CR, CG, OM WMO, GF3, IOC VPAR GF-3 Proc
Yes/GF3
JGOFS M, H, MBT/ XBT, CTD, Bl, PL
CR, CG, OM WMO, GF3, added
VPAR W/R Yes/GF3
GTSPP M, H, MBT, XBT, CTD CI, OM, SQ, HQ, HM
WMO, IOC, GF3, MEDS, NODC
FPAR W/R No
BUFR (subsets) Bathy, Tesac CG WMO VPAR W/R Yes/BUFR
NODC (SD2) M, H CR, CG, OM WMO, IOC, NODC
FPAR ? No
NODC (BT) MBT, XBT
CR, CG, OM NODC FPAR ? No
NODC (h/r STD/CTD)
CTD CR, CG, OM WMO, IOC, NODC
FPAR ? No
OCL (WAOD’98) H, MBT/XBT, CTD, BL CI, OM, HM WMO, NODC/OCL
VPAR R Yes/?
TOGA M, H, XBT, CTD, … CR, CG, OM WMO, IOC, NODC, added
FPAR ? No
WOCE M, H, XBT CTD,… CR, CG, OM WMO, IOC, NODC, added
FPAR ? No
GLOSS Sea level CR, CG, OM added FPAR R No MEDS M, H, MBT/XBT CTD CG, OM, SQ,
HQ IOC, WMO, GF3, MEDS
VPAR W/R No
MEDATLAS H, MBT/XBT, CTD CR, CG, OM, HQ
IOC, WMO, GF3, ICES
VPAR QC Medar
Yes/GF3
P-3 (NODC OPDB)
M, H, MBT/XBT CTD CR, CG, OM, HQ
WMO, NODC VPAR DBMS Yes/?
NODC Russia M,H, MBT/XBT, CTD, PL
CR, CG, OM WMO, IOC, National
FPAR
HDDL System
Yes/HDDL
NODC Argentina /RNODC
M, H, MBT/XBT, CTD CG, OM WMO, IOC, National
FPAR
W/R No
NODC Spain M, H, MBT/XBT, CTD CG, OM WMO, IOC, National
FPAR
W/R No
OceanAtlas H, CTD CR No FPAR OA No TSV-O(DVO) H, CTD CR No FPAR DVO No
Ocean Data Management
XML1
XML2
XML9
XML8
XML13
XML
XML10
XML21
XML14
XML22
XML15
XML100
XMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXMLXML
Stages of Marine XML development – Russian NODC viewStages of Marine XML development – Russian NODC view
Blocks or Bricks (Bob Keely, MEDS) development
Common Data model
Relations (int. and ext.) development
Combining and normalization
Common Parameter Dictionary adoption
Data model description using common Dictionary
Request model
Common code tables adoption
Mapping on XML syntax - Marine XML Scheme
DODS technical issues
Applications
Integrators
Navigator (s)
Data Sources
Marine XML:-common data and request model - Common dictionary and Code-full XML Scheme
Middleware:- Web-server,- App Server- local DM systems
Applied software- single user entry- user interface for criteria and results- “puch” and “requiest/replay” modes
System software:- Navigator (user requiest parsing, requiest navigation, data source monitoring, duplication, …) - Integrator (DBMS, File system, Web-sources converting in Marine XML
Communications:- Internet based- routes and providers
W3C standards
Web server
Web-portalNavigation/Processing
DODS functional issues – Mediterranean/Black seas case
Discipline Discipline XML XML filefile
DBDB11, DB2, DB3,…, DB2, DB3,…
DS1DS1-SISMER-SISMER
PartnersPartners
DS5 -DS5 - RNODC RNODC
DS2DS2-IEO-IEO
DS4 -DS4 - HNODC HNODC
Web serverWeb server
Integrator
DB
Web serverWeb server
Integrator
DB
Web serverWeb server
Integrator
DB
Web serverWeb server
Integrator
DB
DS3 -DS3 - OGS OGS
Poly-discipline Poly-discipline XML fileXML filess
DBDB11, DB2, DB3,…, DB2, DB3,…
Operational data Operational data XML fileXML file
DBDB11, DB2,…, DB2,…
Web serverWeb server
Integrator
DB
Collection Accumulation Processing
MFSSMFSSArgo/Caeiolis/Argo/Caeiolis/PaseidonPaseidon GTSGTS Obs.dataObs.data ClimatClimat
Storage
The Color of Ocean Data, Brussels, Belgium, 25-27 Nov, 2002The Color of Ocean Data, Brussels, Belgium, 25-27 Nov, 2002
Experience of XML use in Russian NODC and perspectives - MedBlackDODS and other
Nicolay Mikhailov, Evgeny Vjazilov, Alexander Besprozvanykh, Natalja Puzova
RIHMI-WDC/Russian NODCObninsk, Russian Federation
History of new technologies (XML and etc.) use History of new technologies (XML and etc.) use in Russian NODC/RIHMI-WDCin Russian NODC/RIHMI-WDC
1996-1998 - 1996-1998 - aaccumulation of knowledge/experience and development ccumulation of knowledge/experience and development of the principal of the principal decisions on client-server DM technologydecisions on client-server DM technology
1990-1995 - 1990-1995 - DBMS/GIS use on personal/organization level DBMS/GIS use on personal/organization level
1999-2001 – 1999-2001 – development of technology prototypes anddevelopment of technology prototypes and design design decisions on distributed marine data system as unified decisions on distributed marine data system as unified information space/data warehouseinformation space/data warehouse
2002-….. – 2002-….. – development of “live” DM applications under development of “live” DM applications under Web/XML technology andWeb/XML technology and design decisions on components of design decisions on components of distributed marine data systemdistributed marine data system
Driving forcesDriving forces – national research programme World Ocean – national research programme World Ocean