EVLA Computing

Software Overview

Contents

• Organization and staffing

• Funding Profile

• Requirements & testing, status and plans

• Selected subsystems – current status

• Future plans

• Major risks

Organization and Staffing

• EVLA computing division– Head – 0.8 FTE

– M&C – 9 FTEs• 8 FTEs on EVLA (4.3 EVLA, 3.7 Contributed Effort)

• 1 FTE on VLA/VLBA operations

• In original EVLA budget

– E2e – 6 FTEs• 4.3 FTEs on EVLA (3.2 EVLA, 1.1 Contributed Effort)

• Soon to decrease by one to 3.3 (2.2 EVLA, 1.1 Contributed effort)

• 1.7 other

• Not in original EVLA budget

– In March, 2006: M&C and E2e groups combined

Organization and Staffing (continued)

• Systems Engineer for Software (0.5 FTE)– Reports to EVLA project manager

• Project Scientist for Software (0.5 FTE)– Reports to EVLA project scientist

• Science Software Group– Managed together with ALMA through Interferometry Software

Division

– 8 FTEs, 2.65 FTEs EVLA

Organization and Staffing (continued)

• Staffing developments in 2005/2006– Butler replaces Clark as EVLA Project Engineer for Software

– Frail steps down as EVLA Project Scientist for Software

– Ye (e2e/Proposal Tool) moves to ALMA, position filled by Loveland

– Morgan (M&C/Correlator Backend) leaves NRAO, position filled by Pokorny

– Two new e2e positions created out of EVA contingency• Harland – High Level Architecture

• Truitt – Observation Preparation Tool

– Waters (e2e/HLA) to move to Science Software in May 2006; will not be replaced for now due to uncertainty in funding

Funding profile (1) M&C

EVLA M&C funding profile

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

2005 2006 2007 2008 2009 2010 2011

FY

FT

Es

CE

EVLA

Funding Profile (2)E2e

EVLA e2e funding profile

0.0

1.0

2.0

3.0

4.0

5.0

2005 2006 2007 2008 2009 2010 2011

FY

FT

Es

CE

EVLA

Science Domain/e2e requirements

• Scientific requirements for all subsystems developed in 2003 – 2004, reprioritized in 2005– Each requirements is assigned priority (1, 2, 3) …

• 1: essential, 2: important, 3: desirable

– … and timescale (A – E)• A: prototype WIDAR, B: Limited Production WIDAR, C:

development of more user-friendly tools, D: WIDAR installed, major modes available, E: all antennas converted, full science operations

– Requirements were reorganized according to timescale– For each priority 1 requirement, time to complete was estimated– Plan to deliver all priority 1 requirements by 2010 developed

Required e2e staffing

required e2e staffing

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

2006 2007 2008 2009 2010

FY

FT

Es

Quicklook/Telcal

Testing/Integration

Archive

Scheduling

ObservationPreparation

Proposal

Infrastructure

Overall Design

E2e staffing – required vs. available

• FTE-years required 25.8

• FTE-years budgeted: 12.7 (EVLA) + 4.9 (contributed effort) = 17.6

• Shortfall 8.2 FTE-years, or ~2 FTEs through remainder of project– Assumes contributed effort funding continues at current level

• New NRAO e2e operations project has the potential to alleviate this shortfall

• Further de-scope would require reprioritizing priority 1 requirements– Barriers to non-radio astronomers remain

– Limit products archived; affects ease of access

Management/target tracking

• M&C – weekly general EVLA coordination meeting– With Electronics, Engineering Services, etc

• E2e – weekly Science Domain meeting– High-level Architecture– Subsystems

• Proposal preparation/submission• Observation preparation• Scheduling• Archive• Pipeline

• Post-processing – bi-weekly meetings– NAUG – NRAO AIPS++ Users Group– NAWG – NRAO Algorithms Working Group

• Monthly Division Meeting during which a topic of interest is presented• Weekly/bi-weekly code reviews/walk-throughs

Subsystems – High level architecture

• Builds on Overall Design presented in 2004, after which effort stalled because of lack of manpower

• Activity resumed as soon as two new e2e positions were filled

• Largely concurrent with Observation Preparation– Project data model

– Scheduling block definition

• M&C design document developed in parallel

Subsystems: M&C

• 3 EVLA antennas in array– MIB module programming

– interim executor

– operator interface

– Control/Monitor processor – archives EVLA monitor data, will soon control VLA antennas

– monitor data archive

• WIDAR– Correlator board drivers complete and ready for testing

– Station and baseline board test screen GUIs developed

– Correlator backend ready to support prototype testing

Subsystems: Proposal Preparation

• Goal: one proposal tool for all NRAO instruments• Developed in parallel with NRAO-wide user database• May 10, 2005 1st release, June GBT deadline• September 10, 2005 2nd release, October GBT deadline, VLA tests• January 10, 2006 3rd release, February GBT and VLA deadline

– Optional for VLA; 67% opts for new tool• May 10, 2006

– Uses new code base allowing code sharing with other subsystems– Will be only way to submit proposals for both GBT and VLA

• September 10, 2006: planned release for October deadline– bug fixing, improve handling tools– No support for other instruments

• EVLA support will be implemented much later; when required

Subsystems: Observation Preparation

• Started as soon as two new e2e positions were filled• Shares objects with proposal preparation tool• Timeline

– Apr06: framework with minimal functionality– Jul06: Add VLA calibrator database access, initial spectral setup– Oct06: Full calibrator setup, more observation setup– Jan07: VLA mostly supported. Some validation/PB creation– Apr07: Beginning prototype WIDAR support– Jul07: VLA fully supported; prototype WIDAR mostly supported– Oct07: Prototype WIDAR fully supported

Subsystems: Archive

• Monitor data archive – functional and in active further development; now a M&C item

• Visibility archive– Will be using ALMA NGAS software/hardware

• NGAS Software to be delivered by ALMA archive IPT

• NGAS Hardware has been purchased

– Plan to have rudimentary archiving in place for WIDAR prototype

– Agreement on a common Science Data Model between ALMA and EVLA in progress

Post-processing

• Science Software group co-managed with ALMA• CASA/AIPS++ Infrastructure and most (2/3) of applications in

common• Main differences:

– Wide field, wide-bandwidth imaging– Calibration– RFI

• EVLA testing:– Summer 2005 – wide field images; w-projection– Spring 2006 – User Interface working group (with ALMA)– Fall 2006 – UI, basic calibration, full beam, full Stokes imaging– Q2 2007 – RFI, Auto-flagging

Collaboration with ALMA

• Subsystems:– Proposal preparation – EVLA has working version, agreement on

common project model with ALMA in progress– Observation Preparation. More difficult because of more

instrument dependencies; also requires agreement on common project model

– Visibility data archive – sharing will require agreement on common science data model, which we are actively working on

– Pipeline – Not among priority 1 items for EVLA; we expect heavy borrowing from ALMA

– Post-processing – both ALMA and EVLA have adopted CASA as their default data post-processing system

Plans for next 12 months

• New release proposal tool every four months

• Complete HLA, e2e PDR early fall 2006

• EVLA tests CASA fall 2006

• M&C CDR late fall 2006

• Modcomp replacement 1st half 2007

• Observation preparation tool in time for WIDAR prototype

• Visibility archive in time for WIDAR prototype

• Start Observation Scheduling

Risks (1)

• M&C– Unpredictable hardware redesigns add extra cycles of development

and testing. Mitigate by contracting work out, which does not come cheap

– Key responsibilities shared by a few expert staff members who are oversubscribed. Mitigate by training other staff where possible

• E2e– Insufficient staffing. Mitigate by requesting staffing increase or

further de-scope– Reaching agreement with ALMA on data models in order to

leverage parallel developments. Negotiate with much bigger partner with different priorities and deadlines can be a challenge

Risks (2)

• Post-processing– Present acceptable user interface to user. Mitigate by ongoing user

interface testing

– Acceptance of CASA by user community. Mitigate by substantial internal NRAO testing and documentation before external release

– Reliance on historic increases in hardware capability. Mitigate by looking into parallel processing using e.g. clusters, and increase code efficiency

– Algorithmic development hard to plan ahead; as some of this is more research than software development. We attempt to mitigate this by employing students/postdocs