ASTRO-WISE Science“A new approach to astronomical

archiving & researching for the data-flooded era”

Gijs Verdoes KleijnOmegaCEN,

Kapteyn Astronomical Institute, University of Groningen

The OmegaCEN team at Kapteyn

• Edwin Valentijn (lead)• Kor Begeman• Danny Boxhoorn • Erik Deul (Leiden)• Ewout Helmich• Philippe Heraudeau• John MacFarland• Michiel Tempelaar • Gijs Verdoes Kleijn• Ronald Vermeij• Willem Jan Vriend (Lofar)• Kovac, Schneider, Sikkema (PhDs)

Archiving your vision• Camera in front of eye:

– resolution ~5/60 degree (my eye)– Field of view 902 degree– 2x2 pixel sampling per resolution element – #pixels= 21602

– Dynamic range: ~103-5: take 216

– One image ~9Mbyte– One image/sec for 70years: 18Pbyte

• Storage cost (at 0.33euro/Gbyte)~ 6million euro

– ~1010-11 neurons

Brain does something smart…

……intensive linking is a key ingredient.

If vision archive were implanted for re-analysis• Applying improved analysis

– (Raw data: improved calibrations as well)

• Analysis of previously deemed uninteresting parts of data

• Variability analysis• if archives from different persons are combined

– Denser coverage in space+time– 3D construction of 2D view

• If fly’s UV eye archives are added:– Same object at different wavelengths

Aim: exploit data for purposes or in ways not yet conceived ...... flexible, internally linked, information system required

Key properties for intelligent information system

• Adaption: facilitate changes1. Changes due to improved encoded methods2. True physical changes of parameter values (e.g., change in instrument/atmospheric

properties)3. Improved insight in 2 or 3

• Learning: “more and better”– take advantage of existing results (from you or others)– Fast (re-)reduction and (re-)analysis – From quick-look to high-quality results– Not only more but also better data over time: ‘accumulation of knowledge

• Anybody– Small Individual research projects– Large projects with many collaborators

• Everywhere: federation– federation of storage and computing capabilities

• Scalability– No limits due to design for storage, databse power/storage processing power,…

A federated ‘brain’ interacting with many users

The Astro-Wise Environment a new archive/research tool for astronomical wide-field imaging

Status•environment is working

•First paper using Astro-Wise based results is out•Expansions and improvements on-going

The Astro-Wise Environment

UserPython prompt (awe>)

Web interfaces

CPUs +algorithms

Database•metadata of images•derived data from images (source lists)•=‘spider spinning AWE web’•Contains ALL input/output

Data server: images calibration & science From raw to reduced

“Archiving”

“Interpreting”

“Processing”

Key properties for intelligent information system

• Adaption: facilitate changes1. Changes due to improved encoded methods2. True physical changes of parameter values (e.g., change in instrument/atmospheric

properties)3. Improved insight in 2 or 3

• Learning– take advantage of existing results (from you or others)– Outdated not-yet existing data– Fast (re-)reduction and (re-)analysis– Not only more but also better data over time: ‘accumulation of knowledge ‘

• Anybody: – Each piece of information carries tag of ‘ownership’– Small Individual research projects – Large projects with many collaborators

• Anywhere: federation: – Own developed compute-grid and storage-grid– Nodes:active={Kapteyn, Bonn} almost={Munich, Paris,Naples}, in

progress={Nijmegen, Leiden, Santiago})• Scalability: performance proportional to I/O speed and processing power.

A federated ‘brain’ interacting with many users

Paradigm shiftAWE “Classical”

Information system

Releases

Dynamic archive Static archive

‘Continous’ releases (VO)

Fixed releases

End result ‘pulled’ via linking of all processing input/output

Raw data ‘pushed’ through pipeline to end-product

Science projects with AWE• PhDs Sikkema, Kovac,

Schneider: galaxy surveys with WFI, WFC, MDM

• Test projects– Variable sources around

CenA• light curves (Δmag~0.03) for

2x104 objects around Centaurus A • Valentijn

– Asteroid detection• Detections in WFI image from

catalog of ~105 numbered asteroids

• Jeffrey Bout (student), myself

Cen A

2dF

Asteroids

Large public data projects with AWE using OmegaCAM

at VST

• KIDS (PI: Kuijken): ESO Public Survey– Weak lensing; high-z QSOs;

galaxy/cluster evolution; baryon oscillations;

– 5000 deg2 u,g,r,i, ~500 nightsKIDS North KIDS South

GTO science with AWE using OmegaCAM at VST

• OmegaWhite (PI: Groot): – discover Galactic Population of

ultracompact binaries from periodic (<2hour) light curves

– 150 deg2 @ b=±5o; Sloan g’ 40sec exposures & additional ugriz’ coverage.

GTO science with AWEusing OmegaCAM at VST

• OmegaTranS (Saglia; Snellen; Alcala)– Searching for planet transits (15-20

new transits expected in first year)– ~1 order more powerful than OGLE-III

GTO science with AWEusing OmegaCAM at VST

• VESUVIO (PIs: Valentijn, Capaccioli) – Galaxy/cluster evolution– Horologium Supercluster: 100 deg2

medium deep (r'<25mag); ugriz– Hercules Supercluster: 12 deg2 deep

ugriz+Hα

Key ingredients to achieve the Astro-Wise environment

– Strict global data acquisition and processing model

– data model -> object model

– storing all I/O in single (distributed) database

– Database environment exploits• OOP inheritance (Python)• Complete linking (associations, references)

Conclusions

• New analysis environment for wide-field imaging in operation by OmegaCEN– Key ingredient: dynamic database

• Containing all I/O• Fully linked data

• Versatile: could be used for other kinds of data– Collaboration with LOFAR on-going

• Large science projects with AWE when OmegaCAM starts operations (early 2007)

• To find out more– Visit : www.astro-wise.org; – download/get now 2 page overview article

Contribution to LOFAR project