11 July 2002

NVO EPO Workshop 1

The National Virtual The National Virtual ObservatoryObservatory

Robert HanischSpace Telescope Science Institute

Project ManagerNSF NVO Project

(with liberal borrowing from NVO Project collaborators)

11 July 2002

NVO EPO Workshop 2

The National Virtual Observatory

• National Academy of Sciences “Decadal Survey” recommended NVO as highest priority small (<$100M) project“Several small initiatives recommended by the committee span

both ground and space. The first among them—the National Virtual Observatory (NVO)—is the committee’s top priority among the small initiatives. The NVO will provide a “virtual sky” based on the enormous data sets being created now and the even larger ones proposed for the future. It will enable a new mode of research for professional astronomers and will provide to the public an unparalleled opportunity for education and discovery.” (p.14)

11 July 2002

NVO EPO Workshop 3

What is the Virtual Observatory?

Real Virtualsky archives, catalogs, published literaturetelescope Internetdetectors, computer programs instrumentssite the user’s desktoptelescope control, instrument control,

data acquisition, data processing, data storage

astronomers, technicians, engineers, programmers, support staff, …

11 July 2002

NVO EPO Workshop 4

The Virtual Observatory…

• Provides “observers” with access to all archived astronomical data as if it were stored on the local computer

• Provides tools to locate and retrieve data of interest, regardless of where it is stored

• Provides tools to compare data from different real telescopes and instruments

• Provides computational services and data management services on a supercomputer scale

11 July 2002

NVO EPO Workshop 5

The Exponential Growth of Information in Astronomy

Total area of 3m+ telescopes in the world in m2, total number of CCD pixels in Megapix, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels.

• Moore’s Law growth in CCD capabilities/size

• Gigapixel arrays are on the horizon

• Improvements in computing and storage will track the growth in data volume

• Data growth rate implies that data storage cannot be centralizedData volume and complexity are increasing!

11 July 2002

NVO EPO Workshop 6

The Exponential Growth of Information in Astronomy

Total area of 3m+ telescopes in the world in m2, total number of CCD pixels in Megapix, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels.

• Moore’s Law growth in CCD capabilities/size

• Gigapixel arrays are on the horizon

• Improvements in computing and storage will track the growth in data volume

• Data growth rate implies that data storage cannot be centralizedData volume and complexity are increasing!

11 July 2002

NVO EPO Workshop 7

Discoveries

• When and where are discoveries made?– Always at the edges and boundaries– Going deeper, using more colors….– Physicists make many measurements and discard most;

Astronomers make many measurements and find wealth in their entirety and combination (J. Ostriker, 6/14/02)

• Metcalfe’s law– Utility of computer networks grows as the

number of possible connections: O(N2)

• VO: Federation of N archives– Possibilities for new discoveries grow as O(N2)

• Current sky surveys have proven this– Very early discoveries from SDSS, 2MASS, DPOSS

11 July 2002

NVO EPO Workshop 8

New Science

• Rare and exotic objects– Very high redshift quasars– Brown dwarfs– Time-variable objects,

transient events: distant supernovae and microlensing

– Dark matter in the galactic halo

– Variable stars– Asteroids

11 July 2002

NVO EPO Workshop 9

New Science

An example of a possible new type of a phenomenon, which can be discovered through a systematic exploration of the Time Domain:

A normal, main-sequence star which underwent an outburst by a factor of > 300. There is some anecdotal evidence for such megaflares in normal stars.

The cause, duration, and frequency of these outbursts is currently unknown. Will our Sun do it?

A new generation of synoptic sky surveys may provide the answers -- and uncover other new kinds of objects or phenomena.

11 July 2002

NVO EPO Workshop 10

New Science• Structure and evolution of the universe

– Proper statistical comparison between local and distant samples

– Cluster surveys as tracer of large-scale structures– Automated detection of arc-shaped objects to locate

gravitational lenses

11 July 2002

NVO EPO Workshop 11

New Science

• A fully digital Galaxy– Star catalogs (stellar

evolution, stellar dynamics)

– Interstellar medium– Role of close

encounters and influence on star formation

– Comparison with theoretical models and simulations

11 July 2002

NVO EPO Workshop 12

New Science• Census of active galactic nuclei

– Systematic searches for black holes– Panchromatic approach to

circumvent obscuration problems

• Search for extra-solar planets– Search for planet transits in much

larger data samples (Large Synoptic Survey Telescope), bolstered by supporting astrometric data

• Theoretical astrophysics– Globular cluster modeling– Galaxy mergers– Evolution of large-scale structure

11 July 2002

NVO EPO Workshop 13

New Technologies

• Standardizing access to distributed data– Web Services:

• XML: Extensible Markup Language• SOAP: Simple Object Access Protocol• WSDL: Web Services Description Language• RDF: Resource Description Framework• UDDI: Universal Description, Discovery

and Integration

• Standardizing distributed computing– Grid Services

• Custom configure remote computing dynamically• Build your own remote computer, and discard• Virtual Data: new data sets on demand

The Grid

the toolsfor buildingthe semanticweb

"The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation." -- Tim Berners-Lee, James Hendler, Ora Lassila, The Semantic WebScientific American, May 2001

11 July 2002

NVO EPO Workshop 14

New Technologies

• Standardizing access to distributed data– Web Services:

• XML: Extensible Markup Language• SOAP: Simple Object Access Protocol• WSDL: Web Services Description Language• RDF: Resource Description Framework• UDDI: Universal Description, Discovery

and Integration

• Standardizing distributed computing– Grid Services

• Custom configure remote computing dynamically• Build your own remote computer, and discard• Virtual Data: new data sets on demand

The Grid

the toolsfor buildingthe semanticweb

"The computational grid is analogous to the electric power grid. Grid computing allows to couple geographically distributed resources and offers consistent and inexpensive access to resources irrespective of their physical location or access point. It enables sharing, selection, and aggregation of a wide variety of geographically distributed computational resources (such as supercomputers, compute clusters, storage systems, data sources, instruments, people), thus allowing them to be used a single, unified resource for solving large-scale compute and data intensive computing applications.”-- www.gridcomputing.com

11 July 2002

NVO EPO Workshop 15

New Technologies

• Standardizing access to distributed data– Web Services:

• XML: Extensible Markup Language• SOAP: Simple Object Access Protocol• WSDL: Web Services Description Language• RDF: Resource Description Framework• UDDI: Universal Description, Discovery

and Integration

• Standardizing distributed computing– Grid Services

• Custom configure remote computing dynamically• Build your own remote computer, and discard• Virtual Data: new data sets on demand

The Grid

the toolsfor buildingthe semanticweb

11 July 2002

NVO EPO Workshop 16

How Will It Work?

• Define commonly used `atomic’ services• Build higher level toolboxes/portals on top• We do not build `everything for everybody’• Use the 90-10 rule:

– Define the standards and interfaces– Build the framework– Build the 10% of services that are used by 90%– Let the users build the rest from the components

11 July 2002

NVO EPO Workshop 17

NSF NVO Project

• NSF ITR project, “Building the Framework for the National Virtual Observatory” is a collaboration of 17 funded and 3 unfunded organizations– Astronomy data centers– National observatories– Supercomputer centers– University departments– Computer science/information technology specialists

• PI and project director: Alex Szalay (JHU)• CoPI: Roy Williams (Caltech/CACR)• $10M award for five-year period, beginning 1 Nov 01

– 4-5% for EPO coordination, plus in-kind contributions

11 July 2002

NVO EPO Workshop 18

Project ManagementNSF

CISE + AST

ExternalReview Committee

PI/Project Director: SzalayCo-PI/Chief Architect: Williams

ExecutiveCommittee

Data Centers

ProjectScientist

ProjectManager

SystemArchitect

E&OCoordinator

Te

chn

ical W

orkin

g

Gro

up

Sci

en

ce W

ork

ing

G

rou

p

InfrastructureActivities

Local/Distant Universe

Digital Milky Way

Rare/Exotic Objects

AGN Census

Extra-Solar Planets

Sci

ence

Pro

toty

pes

Theoretical Astrophysics

Portals/Workbenches

Metadata Standards

Grid Services/Testbed

Data Models

DataAccess/Resources

Data Providers

E&OCoordinator

ProjectManager

11 July 2002

NVO EPO Workshop 19

Education & Outreach

• Integral part of project• Emphasis is on development of partnerships• Kick-start with this workshop

– Understand requirements on NVO services from perspective of formal education, informal education, commercial/corporate, and public outreach content developers

11 July 2002

NVO EPO Workshop 20

Education/Outreach Partners

Association of Science-Technology Centers

International Planetarium

Society

National Air and Space Museum Silicon Graphics (Digital Planetarium)

Spitz (Electric Sky) Maryland Space Grant Consortium

Gettysburg College (Project CLEA)

UC Berkeley (CSE@SSL)

American Museum of Natural History

11 July 2002

NVO EPO Workshop 21

Milestones

• Nov 2001 – Jan 2002: Established project structure• April 15, 2002: VOTable V1.0• May 1, 2002: 50+ ConeSearch services registered• May 8, 2002: Defined initial science demos• June 13, 2002: Formed International VO Alliance• July 11-12, 2002: EPO Workshop• Nov 15, 2002: Internal testing of science demos• January 2003: Initial science demonstrations (AAS)• August 2003: Intermediate NVO science demos (IAU)

11 July 2002

NVO EPO Workshop 22

Critical Path

• Science demonstrations– Identified, scoped, and scheduled

• Service registry issues– International coordination

• User interface issues– Retrofit existing portals

• EPO requirements– Impact on metadata standards

11 July 2002

NVO EPO Workshop 23

Role of Science Prototypes

• Keep focus on user and science needs• Identify most common services• Verify standardization efforts• Encourage data providers to participate• Demonstrate to community that NVO tools will

– arrive soon– will be useful for everybody– can evolve incrementally

• First science demos planned for January 2003

11 July 2002

NVO EPO Workshop 24

Initial Science Prototypes

• Brown-Dwarf search– Distributed query across several archives– Correlations with non-detections– Example of typical NVO search

• Gamma-Ray burst– Event follow up service– Exercise in standards compliance/interoperabilty

• Galaxy evolution in clusters– On-the-fly image analysis and pattern recognition– Exercise in grid computing

11 July 2002

NVO EPO Workshop 25

SkyQuery

• Distributed Query tool using a set of services• Feasibility study, built in 6 weeks from scratch

– Tanu Malik (JHU CS grad student) – Tamas Budavari (JHU astro postdoc)

• Won 2nd prize in Microsoft .NET Contest• Allows queries like:

SELECT o.objId, o.ra, o.r, o.type, t.objId FROM SDSS:PhotoPrimary o,

TWOMASS:PhotoPrimary t WHERE XMATCH(o,t)<3.5

AND AREA(181.3,-0.76,6.5) AND o.type=3

11 July 2002

NVO EPO Workshop 26

http://contest.eraserver.net/SkyQuery/

11 July 2002

NVO EPO Workshop 27

ConeSearch

• Search for catalog objects around a point• Returns data in VOTable format• Requires a registered profile• Point of the exercise

– A learning experience– Existing archives test and implement VOTable– Understand service description issues

• Automated test and verification• In less than two weeks 7 groups, 50 services• Cross-Identification service built on top

11 July 2002

NVO EPO Workshop 28

http://skyserver.pha.jhu.edu/VOconeprofile/

11 July 2002

NVO EPO Workshop 30

Summary

• NSF ITR NVO project is one of four major and numerous other small VO-related initiatives now underway world-wide

• NVO is adopting, adapting, or developing necessary technology as derived from science requirements

• NVO project is dealing with many of the management challenges that will face the ultimate VO organization

• NVO (and only NVO) has emphasis on EPO as integral component of project

http://us-vo.org