SDS Presentation Jonathan McDowell CXC 1 Chandra Users Committee Meeting, 25 January 2005 Science...

SDS Presentation Jonathan McDowell

CXC

1

Chandra Users Committee Meeting, 25 January 2005

Science Data Systems (SDS)

CUC Presentation

Jan 2005


CXC

1


Science Data Systems - supporting user science with software and data products

• Work closely with Data Systems to provide science overview of software and detailed specs

• CIAO - the software package (and documentation!) for user data analysis

• Pipelines - ensure data products created with correct algortihms and contain the needed metadata

• CALDB - figure out how raw calibration results from CAL team must be packaged to support user analysis and data processing

• From CUC point of view, we mediate between users and Data Systems teams to deliver improved user software


CXC

1


Science Data Systems - under new management

• Jonathan McDowell, SAO - group leader: [email protected]

• Mike Wise, MIT - deputy leader: [email protected]• SDS is about 50/50 SAO and MIT: an integrated team• Close coordination with Pepi Fabbiano's Data Systems groups,

especially Janet DePonte Evans' software development teams: CIAO and the pipelines are joint SDS/DS products

• Let us know how we can improve your ability to do science with CIAO and with CXC data products


CXC

1


Science Data Systems: summary• Recent releases

– CIAO 3.2– APPHOT and DS9 interaction

• Pileup Report– Documentation– Estimation– Correction

• What else is coming– Improvements– Enhancements


CXC

1


CIAO 3.2 Highlights

• New 'How does the new release affect your analysis' web page• New mkacisrmf tool released• Spatial model of the ACIS contamination• New ACIS hot pixel tools• CALDB 3.0


CXC

1


CIAO 3.2 Web Pages

•


CXC

1


mkacisrmf infile=acisD2000-01-29p2_respN0001.fits outfile=source_pi.wrmf \ energy=0.1:10.0:0.05 channel=1:1024:1 chantype=pi \ wmap=spectrum_pi.fits gain=acisD2000-01-29p2_gain_ctiN0001.fits

• New tool in CIAO 3.2 to support changing ACIS calibration • Implements algorithm of A. Vikhlinin (calcrmf2)• Separates RMF calculation into position dependent and independent pieces• “Ideal” response computed at readout prior to the effects of CTI (one per CCD)• Scatter matrix computed automatically from simulations including CTI• Faster execution, better interpolation, and retains all existing mkrmf functionality

mkacisrmf

Ideal Response Total ResponseScatter matrix

=Ene

rgy

PHA


CXC

1


• New RMFs provide small improvements over mkrmf products

• 10-15% more accurate at energies below 1 keV

• Currently calibrated for ACIS chips 0-3, 5, 6, and 7 at -120 C • In the future, only mkacisrmf Cal. products will be produced

Released mkrmf RMF New mkacisrmf RMF


CXC

1


Spatial Model for the ACIS Contaminant

The ACIS QE degradation model has been enhanced to account for spatial variations in thecontamination on the ACIS optical blocking filters. The contamination is now expressed asa function of time, energy, and ACIS chip coordinates.

Temperature structure of ACIS OBF

Instrument maps (mkinstmap)

ACIS-I2 ACIS-S3


CXC

1


Spatial model implemented in all CIAO 3.2 tools

• Default model in CIAO 3.2 / CALDB 3. 0

• mkarf, mkgarf, mkwarf, mkinstmap

• ~20% variation across chip

• Extended or off-axis sources most effected

• Affects grating ARFs as well


CXC

1


Status of APPHOT Package

• Frank Primini's X-ray aperture photometry scripts demoed at last CUC as an example of scripting

• You asked for more details on their availability• They are now real software and users can have them!• And you asked how we did those cool things with ds9• I'll tell you how, and we'll soon put this up as a thread for

users• Working with Data Systems to formalize how we develop

and release interactive scripts as we increase their use. Stay tuned.


CXC

1



• Version 1.1.0 available for interested users athttp://hea-www.harvard.edu/~fap/APPHOT/. Will be added to CIAO contributed software after additional testing.

– Installs under CIAO 3.2; no additional installation of XPA or SLxpa modules required

– Can now determine actual ECF of aperture defined by user

– Faster computation of Theta, Phi (now uses S-Lang pixlib module)

• Version 1.2.0 in development.– Choice of method for accessing ECF data file

● Interpolation in theta, phi, energy● Nearest theta, phi, energy in ECF data cube

– Plots of ECF vs. aperture radius


CXC

1



• Additional development suspended pending evaluation of Penn State's ACIS Extract S/W package

– ACIS Extract richer, more sophisticated. We are considering porting its IDL-based functionality to CIAO's open source environment

– APPHOT could still be used to provide quick assessment of aperture sizes, e.g., for proposal planning


CXC

1


Launching CIAO Tools from ds9The ds9 image display program can be used to launch user-defined analysis tasks which can obtain input parameters from and display results to ds9. This capability is accomplished by means of ASCII “analysis commands files”, loaded into ds9 via the Analysis Menu, and is described in detail in the ds9 Reference Manual.

Typically, this makes sense when one is examining an image and wishes to define regions or image locations at which, say, counts or PI spectrum are desired. These can be obtained and displayed quickly using CIAO tools.

In the following pages we present some examples of analysis command files that launch CIAO tools.


CXC

1


A Simple Analysis Commands FileThe analysis commands file (ACF) is a structured ASCII file that contains command lines plus keywords that tell ds9 how and when to run the command. The command file can take advantage of a number of ds9 internal variables, or macros, that expand to arguments containing information on the current ds9 display, prior to command execution. The full list of macros is provided in the ds9 Reference Manual. Of particular importance are the following:

● $filename expands to the filename of the displayed image or event list;● $regions expands to the list of regions (in various formats) defined in the display;● $text displays any text output from the command to a text window.

The ACF can also define parameters, which are set in a parameter window, and which may be passed as arguments to the command line.

The following page contains a simple ACF that can be used to display some ds9 macros and run the CIAO tool dmlist to display header or data. Results are displayed on subsequent pages.


CXC

1


The ds9_ciao.ans ACF


CXC

1


Loading the ACFClicking the Analysis button allows users to load ACFs via the Load Analysis Commands submenu


CXC

1


Running dmlist

Clicking dmlist invokes the parameter windowClicking on dmlist invokes

the parameter window, in which values of the “option” parameter may be set

Clicking OK runs the dmlist command line


CXC

1


Results for dmlist




When dmlist finishes, the results are displayed in a pop-up text window


CXC

1


A More Complicated (but still simple) Example

It may be difficult to build complicated CIAO command lines that can be successfully parsed by ds9's command line parser. Rather, the user may wish to pass the arguments obtained from parameter windows or ds9 macros to a shell script with sufficient string handling capabilities to build and execute a valid CIAO command line.

The following page illustrates how that can be done in a script that runs the CIAO tool dmcoords to determine the off-axis angle of a point selected in a ds9 display.


CXC

1


Launching dmcoords from ds9_ciao.ans


CXC

1


Running dmcoords from a Script


CXC

1


Results for dmcoords




When dmlist finishes, the results are displayed in a pop-up text window

Clicking dmcoords runs the program to determine off-axis angle for the image location specified by this region. Results are displayed in the pop-up dmcoords window.


CXC

1


Advanced Use

The previous examples were only intended to illustrate the basics of setting up an ACF, defining parameters, and passing arguments to CIAO tools. In practice, the most flexible way to launch CIAO tools from ds9 is via scripts that communicate directly with ds9 via the XPA Messaging System. This can be accomplished either by running the various XPA programs (xpaget, xpaset, etc.) from the command line within the script, and then parsing the results for the desired input, or by using a scripting language, such as S-Lang, which provides bindings to the XPA library.

Users who wish to use these advanced features should examine the scripts in the APPHOT package.


CXC

1


SDS Efforts for Pileup

• Helping users diagnose and deal with ACIS pileup is an SDS top priority for 2005

• I'll present the current status


CXC

1


SDS Efforts for Pileup

• Comprehensive ABC Guide (status: draft in work)– One Stop Shopping for Information and Advice

• New Diagnostic Tool For Detecting Piled Sources (research)– Concept: Detects Piled PSFs– Potentially can Help Calibrate MARX’s Pileup Module

• Tools for Dealing with Piled Gratings Spectra– Simple Model Soon Available– Sophisticated Model in Development

• Future Work– Models of Piled Lightcurves


CXC

1


“The Chandra ABC Guide to Pileup”: Outline

• Introduction– Basic Definitions, Pileup Fractions and Their Effects, Methods for

Avoiding Pileup, When Not to Avoid Pileup

• Estimation– PIMMS, Simulation with Spectral Fitting Packages (Sherpa, ISIS,

XSPEC), MARX

• Detection– Simple Methods, Future New Detection Tool

• Mitigation– Spectral Fitting Packages (Sherpa, ISIS, XSPEC), Simple Gratings

Models, (Future) Advanced Gratings Models


CXC

1


“The Chandra ABC Guide to Pileup”: Goals

• Consolidate Information:– Proposers Guide, PIMMS Ahelp, Sherpa Ahelp, Threads,

Journal Articles, etc.

• Provide ‘Practical’ Advice:– Pros & Cons of Avoidance Strategies (Greater Detail than Proposer’s

Guide)– Subtleties of Detection Methods– Subtleties of Spectral Fitting Mitigation Techniques


CXC

1


New Detection Tool in Research/Development

• Compares PSF from 2-pixel split events to other events• Provides Statistical Assessment of Whether the Two

Distributions are Consistent (i.e., Not Piled)• Does not Directly Rely Upon Spectra• Potentially sensitive to Even Mild Pileup• Side Benefit: Statistics Module for CIAO (e.g., K-S Test,

Mann-Whitney Test, etc.)


CXC

1


PSFs Distort Under Pileup (In Complex Ways!)

Working Concept: ‘Split’ Events PSF Distorts Differently than that for Other Events


CXC

1


Low (Unpiled) Count Rate: PSF from Split Events and Other Grades Appear the Same

p-value > 0.05 Indicates Normalized PSF Distributions are Indistinguishable

(Off Axis Source)


CXC

1


Example: Radio Galaxy Nuclear Emission,Difficult to Assess Pileup via Spectra

Observation of Gambrill et al. 2003, A&A, 401Source was Thought to be Unpiled


CXC

1


Normalized Grade Selected PSFs Differ

p-value < 0.05 Indicates Differences are Statistically Significant: Good Indication of Mild Pileup?


CXC

1


PSF Can be Assessed with the MARX Pileup Module: Consistent with Mild Pileup


CXC

1


Spectral Fit Function, Based on Observed Counts/Frame/Pixel, for Simple Correction

10% Pileup Correction

Simple Spectral Model, Only Applicable to Sources with Peak PileupApproximately < 10%. (Here Applied to an X-ray Binary Spectrum)


CXC

1


More Sophisticated Correction in Development

Will Incorporate Detector Efficiency Variations, Deadtime Effects, Energy Dependence of PSF/LSF, etc.


CXC

1


Spectra, with Peak Pileup of ~20%, Corrected


CXC

1


Ongoing Work

• Better Afterglow Correction: acis_run_hotpix

• The Most Recent Gratings LSF: mkgrmf

• In Cooperation with Calibration, Need to Incorporate Better Models of:– Cosmic Ray Dead Area Correction

– Backside/Frontside Correction

– Iridium Edge Correction

• Exploring Possibility of Creating a Chandra Lightcurve Simulation, which Would Include Pileup Effects

– Adapt Existing XEUS Lightcurve Simulator for Chandra Use– MARX Provides Accurate Mean (White Noise) Piled Lightcurve, this Could

Provide Piled Lightcurves with Realistic (Red Noise, QPO) Variability Features

Requirements for Gratings Pileup Correction:

Future Pileup Work:


CXC

1


What’s coming in CIAO 4.0

• Infrastructure improvements: must keep up with users' environment– Increased platform support (under review - newer Linux builds)– 64 bit and large file support– Performance improvements (pixlib ~ 10 times faster)– Ability to build from source

• Enhancements: filling gaps in user science capabilities– New CHIPS plotting package; Sherpa improvements– Improved Backgrounds (ABC guide, scripts, tools)– Multi-observation support (mosaicing)

• Schedule and other enhancements still under review


CXC

1


Sherpa status

• Already in CIAO 3.2: bug fixes for improved stability, projection error method improvements, and revised manual

• We are now conducting a major review• Reworking the optimization methods (including evaluating

adding the LMDIF algorithm) for faster and more accurate convergence

• Expect to rework Sherpa to modularize it - make separate libraries available for models, optimization methods, etc. with defined interfaces to let us write small tools and scripts using parts of the Sherpa functionality. Will also greatly improve Sherpa code maintainability and stability

• Must add support for new CHIPS plotting

Date post:	18-Dec-2015
Category:	Documents
View:	218 times
Download:	0 times

SDS Presentation Jonathan McDowell CXC 1 Chandra Users Committee Meeting, 25 January 2005 Science...

Documents