IMRCruisetoolboxa technical presentation

Trond Westgård

Senior engineer

Norwegian Marine Datacenter (NMD)

Purpose

• Establish a framework to store, retrieve and distribute data sampled from research vessel surveys.

Principles• All data stored in relational databases with

the OGC spatial extension• Modern feature-rich end-user dialogs• Multiuser access to databases• Open source software if possible• Object oriented design with reuse of

components• “Smooth” distribution and sharing of data• A “family” of databases where each db deals

with a specific research area

Databases so far

• IMRReference – (species, platforms, persons, roles, gears, categories, ……)

• IMRHub – (cruise data: track, crew, tasks, purpose, stations, weather,….)

• IMRBiotic – (prototype.biological samples of fish and other taxonomic groups)

• IMREchosounder – (interpreted echo values for different acoustic categories)

• IMRSurvey – (estimated indexes for different stocks/surveys)

• More to come ….

Quick overview - DB

• Rules for establishing new databases (GUID, original, last edited,…)

• Copies of data should be easy to distribute in as many copies as needed (but only one database keeps the original/master value stored in a row in a table).

• Mechanisms for: ”transfer ownership”, ”update/sync copies”

• Central log on to access rights

• The databases are not always in a network (e.g.: field work in remote places)

• “Foreign keys” may refer ACROSS databases/schema

• PostgreSQL/PostGIS as database engine.

Quick overview - Architecture• 5 layer architecture (GUI, …., DB)

• Use of ORM and business logic with business objects

• Mechanisms for: ”transfer ownership”, ”update/sync copies”

• “Central” log on to access rights

• The databases are not always in a network (e.g.: field work in remote places)

• “Foreign keys” may refer ACROSS databases/schema

• UDP, list, limit value, area, ….

• Open Source tools if possible

• Java as primary language

• NetBeans IDE as main development tool

ArchitectureGUI

Data and Storage Management

Data Access

Business Logic

UI

Presentation

Tools used

JAVA / IDE NetBeans 6.8

DB – JDBC, PostgreSQL, PostGIS

Data Access – ORM iBATIS

Business Logic – JavaBeans VO, Java

GUI – OpenSwing/Swing/JMapPane(gt)NetBeans Platform

Libraries and applications

Imrcore.jar

Guid, sync, BaseVO,DBList,VisualBaseVO

Imrreflib.jarLogon,Species,Platform,Role

Imrbioticlib.jarSample,

Individual

IMRFrontend,Application

IMR-hub

IMR-reference

IMR-biotic

Imrhublib.jarCruise,Track,

Station

Imrechosounderlib.jarSA,Distance,Frequency

IMR-echosounder

Data distribution

HUBREF

BIOECHO

HUBREF

BIOECHO

HUBREF

BIOECHO

Sync

S

ync

Sy

nc

HR

BE

Example application - IMRfrontend

Whats left ???

• Rock solid framework for RCA (help,language,toolbar,docking,plugins..)

• Framework for web services and web based applications (Spring?)

Ownership – Klausul / Clause

• Institution (that owns some dataset and wants a clause on it)• Person (that is responsible for the clause, could be empty). • Description (text explaining what the clause is about)• Sampledatestart (start field sample date the clause is about)• Sampledateend (end field sample date that the clause is about)• Expiredate (The date when the clause could be disregarded)• Inherit (0,1- Underlying objects are also restricted by this clause)• Severity

• 0 = open access,• 1 = signed on user with read access,• 2 = signed on user with read/write permission to the database,• 3 = signed on user must have explicit permission to bypass the clause, i.e. it must exist an entry in the Clause_person_group table.

Ownership – “relations”

?IMR-biotic

Person/Group

Institution Clause

Report

Data

Ownership – imrreference

Clause Clause_person_group

Clause_vobject

Person

vobject Group Person_group

Ownership any database – applies to all nmd database tables with observations or derived data elements that contains business object id’s

• id• original• last_edited• id_r_institution_owner• sampledate• somedatacolumn_one• somedatacolumn_two• etc ….

• NOTE: When id_r_institution_owner and sampledate is NULL no clause applies to the datarow

Ownership – some imr data database clause_object table

• id• original• last_edited• id_r_vobject• id_r_clause• id_object

(id_r_vobject,id_r_clause,id_object) unique constraint

0 = GEOMETRY1 = POINT2 = CURVE3 = LINESTRING4 = SURFACE5 = POLYGON6 = COLLECTION7 = MULTIPOINT8 = MULTICURVE9 = MULTILINESTRING10 = MULTISURFACE11 = MULTIPOLYGON

WHY ?

Every ”framework” has it’s own way

• Java based and inherits from Swing and is true to Java core principles• Helpsets and multilanguage build in• Plug in architecture – freely adaptable also for end-user• Subscribe on updates for modules• Floating windows/toolbars/DnD/Trees/Tables/Properties….• Context sensitive pop-up menus• Remembers last user session• Well documented with courses• Open Source• Only one real opponent – Eclipse• Context events – One toolbar many functions – Subscribe to events• A rich set of API’s• MODULARITY• API “discipline” (publish classes in folders)• Oracle states continued support (Jbuilder, Eclipse Plugin, NetBeans)• more ….

Some facts about NetBeans Platform

• Helpsets and multilanguage user interfaces• Best practices for logging of messages and errors (log4J ?)• Migrate OpenSwing MDI to NetBeans Platform RCA

• toolbar – grid – form – module(s)• Config.xml handled by NetBeans Platform• Geotools and JMapPane as modules• From TopComponent to BusinessAO to BusinessVO objects• Persistence of objects and wrapping of data access into modules• Lazy loading using NetBeans PlatformTrees and Ibatis• Services to access business objects both for RCA and Web applications

Workshop items