+ All Categories
Home > Documents > IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and...

IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and...

Date post: 29-Jan-2016
Category:
Upload: hannah-malone
View: 215 times
Download: 0 times
Share this document with a friend
Popular Tags:
of 23 /23
IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC technical commision for oceanography and marine meteorology (JCOMM) Marine Hydrophysical Institute (MHI) Vlaams Instituut voor de Zee (VLIZ)
Transcript
Page 1: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Virtual Lab project

International oceanographic data and information exchange programme of IOC (IODE)

Joint WMO/IOC technical commision for oceanography and marine meteorology (JCOMM)

Marine Hydrophysical Institute (MHI)

Vlaams Instituut voor de Zee (VLIZ)

Page 2: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

What is VLIZ ?

• VLIZ : Flanders Marine Institute

• Oostende, Belgium

• Supports Marine Research in Flanders

• Support IODE, IOC, UNESCO

• Flanders Marine Datacenter

Page 3: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Located : Oostende

Page 4: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

What is the virtual lab ?

• The Virtual Lab will be a computing environment collecting the data, models and interfaces needed to run hydrodynamic models for training and research.

Page 5: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Hydrodynamic models

• Hydrodynamic models like POM predict water currents, wave heights and ocean circulations, so basically the movement of watermasses, and all that is carried by it.

• The water movements are caused by wind, temperature and atmospheric pressure. These meteo parameters are called the forcing data because they make the waterccolumn move. These movements are then deviated by land masses and shallow waters.

• The models are used in forecasting, operational mode when using predicted meteorological parameters. Important for safety at sea, stormsurge warning etc.

• They can also be used in hindcasting using corrected meteo parameters. Mainly for research or training.

Page 6: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Running models

• Modelling software eg. POM • Forcing data eg.NCEP • Bathymetry eg.ETOPO• Powerfull computers• Visualization software eg GRADS• Expertise & training

Page 7: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Known problems

• Difficult software– Runs on Linux, command line mode– Edit fortran code, and recompile to run the model.– Install viewers, learn to operate them

• Preparing data, lots of work– Download forcing data (>100Mb : bandwidth ?)– Extract bathymetry

• Very computer intensive– Reformatting data– Running the model

• Possible solution : Virtual Lab

Page 8: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

The virtual lab

• Experts will install the models• Install powerful hardware• Make use of GRID computing• Make software user friendly• Develop easier interfaces• Batch jobs download & reformat the data

automatically• Make VL available through internet

Page 9: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

System diagram

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients

Page 10: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

System diagram:providers

Data repositories

GRID network

Data collectors Interfaces

Data providers ClientsData providers are the met-offices, oceanographic datacentres, basically any organisation that has relevant data

Page 11: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients•User authentication based on name/password and/or certificates

•Can be Windows, Mac or Unix machines•Software needed : at least an internet browser (IE, firefox,mozilla…)•Optionally SSH , FTP , VNC, Xwindows software

System diagram:clients

Page 12: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

System diagram:GRID

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients

•GRID network is the EGEE computing GRID, specially the BEGRID component of it.•IODE computers from trainingroom

Page 13: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

System diagram:collectors

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients•Scripts written in perl, php, python, csh•Run several times a day, unattended•Download data from providers•Extract and reformat data to the formats needed by the different models.•Where appropriate the jobs can be submitted to run on the GRID network.

Page 14: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

System diagram:repositories

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients•Computers running NFS, FTP, SMB(samba/windows)•Database servers running MySQL•GIS databases•SAN coupled (fibre disks with Terabytes storage capacity)

Page 15: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

System diagram:interfaces

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients

•Linux systems•Access protocols : SSH, (s)FTP, HTTP,VNC,Xwindows•SSH and FTP access will allow full control of the models•WebPages can simplify the process of running the models, and allow to submit jobs on the GRID, and be used to visualise the results.•VNC and Xwindows access provided the network bandwidth is available

Page 16: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Internet

System diagram:flow

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients

SSH, VNC, Xwindows, FTPHTTP, HTTPS

FTP,HTTP

NFS, SMB, SQL

edg-job-submit

NFS

edg-job-submit

NFS, SMB

Page 17: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Candidate models

• Selected for the JCOMM – IODE jamboree– Extra tropical storm surge (Ø.Sætra, NMI)

– WAM wave model from (NMI)

– Tropical storm surge (S. Dube, IIT)

– Circulation model for the Black Sea (Korotaev,MHI)

Page 18: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

First steps : Black sea circulation Model

• Project VLIZ – MHI Ukraine• Based on POM• Running high resolution model for Blacksea is very

computer intensive.– 5km resolution, 1 month : hours– 1km resolution, 1 month : days

• Current approach : nested model– Low resolution model for whole BlackSea– Use this as input for high resolution model for smaller regions

• Ideal for GRID computing– 1 job for whole black sea– each region = 1 job, independant of others

Page 19: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

28.6 28.8 29.0 29.2 29.4 29.6 29.8 30.0 30.2

Long itude, E

43.5

43.7

43.9

44.1

44.3

44.5

44.7

44.9

45.1

45.3

45.5

45.7

La

titu

de

, N

B o tto m re lie f, m

Basin-Scale Circulation Model

Regional Atmospheric Model (ALADIN Family)

Sea Surface Elevation

Current Velocity

Sea Water Temperature and Salinity

Regional Models Family

Kalamita BayRegional Model

Nowcasting system

32.6 32.8 33.0 33.2 33.4 33.644.6

44.8

45.0

45.2

45.4K a l a m i t a

B a y

Romanian Coastal ZoneRegional Model

36.5 37 37.5 38 38.5 3944

44.5

45

Russian Coastal ZoneRegional Model

0 1 2 3 4 5 6 70

1

2

3

4

5

6

7

8

9

17.6

17.8

18

18.2

18.4

18.6

18.8

19

19.2

19.4

19.6

19.8

20

20.2

20.4

NWSRegional Model

Burgas Bay Georgian Zone

Forecasting System of the Black Sea Circulation

Page 20: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

low resolution Blacksea model

• Lineair scalable till about 10 – 16 CPU

• Will run on 2x4core CPU system with 8Gb in VL

Page 21: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

regional models

• Kalamita Bay• GRID cluster with

5x2x4 CPU will be used for running the regional models in parrallel

Page 22: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Planning

• mid 2007 – install equipment : Blades with 5x2xQuadCores, 8Gb– install gLite middleware – install existing models

• end 2007– make models ‘GRID’ enabled– investigate MPI – POM

• beginning 2008– run model in ‘semi-operational’ forecasting mode

• in parallel– start same exercise for other models

Page 23: IODE/JCOMM/VLIZ/MHI – Virtual lab project Virtual Lab project International oceanographic data and information exchange programme of IOC (IODE) Joint WMO/IOC.

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Thank you


Recommended