Data Product Suite

Problem: Design, Implementation, and Assimilation of

new data products is expensive.

Constraints: Efficiency

15K+ daily images, 1GB+ data files Maintainability

changing grids: quad cells changing file formats: removed null values changing resource constraints: new machines

Data

Independence

Algebraic

Optimization

Data Independence

Different algorithms, each dependent on data characteristics. Changes to data characteristics mean changes to recipeLogical equivalences are obscured…

vtkExtractGeometryvtkThresholdvtkExtractGridvtkExtractVOIvtkThresholdPoints

We want:

VTK:

Exposing Equivalences for Algebraic Optimization

result(B) = result(A), but cost(B) < cost(A) Implies ‘Cross’ and ‘Restrict’ should be commutativeDifficult to determine which library routines, if any, have this property (“Documentational Semantics”)

cross restrict

crossrestrict

A:

B:result(B)

result(A)

3D Data Products

H(x,y,b)

V(z)

r(z>b) b(s) r(region)

F

H(x,y,b)

V(z)

r(z>b) b(s)

r(x,y)

r(z)

G

F

0

5

10

15

20

25

30

35

40

45

0 0.2 0.4 0.6 0.8 1

selectivity

tim

e (

se

c)

unopt

opt

vtk

rdbms

Horizontal Slice

H(x,y,b)

V(z)

r(z>b) b(s) slice

F

H(x,y,b)

<depth>

r(z>b) b(s)apply

H

Vertical Slice

H(x,y,b)

V(z)

r(z>b) b(s) search

F

P

P P V

P V

Vertical Slice

V(z)P

H(x,y,b)search b(s) interp

V(z)P

H(x,y,b)search b(s)first

051015202530354045

H(x,y,b)

V(z)

r(z>b) b(s)

r(x,y)

r(z)

V(z)P

search

interp

Plume Volume

H(x,y,b)

V(z)

apply(*) r(ocean)

area

length

sum

r(salt<26)

H(x,y,b)

V(z)

r(z>b) b(s)

F

merge

r(region)

Time dependent

Another Optimization

r(ocean)

r(salt<26)merge

B

A

r(ocean)r(salt<26)merge

B

A

r(ocean)

r(salt<26)merge

B

A

B’

C

C

C

A=B

Software APIs

API

Data Independence

vtkExtractGeometryvtkExtractStructuredGridvtkExtractThresholdvtkExtractGridvtkExtractVOIvtkThresholdPoints0

We want:

Custom: makeSmallGrid()

VTK: