A walk through XAS data analysis

A fictitious XAS session is illustrated in the following. The main XAS commands, their usage and their effect, are breifly explained.
A full description of XAS, its funcionalities and all its commands can be accessed starting from the URL:https://www.ssdc.asi.it/bepposax/software/xas/.

Most of the commands are the same for all instruments. Specific scripts have been set up to produce standard scientific products in an automatic way. Data screening and selection must be done with instrument-specific housekeeping parameters and ratemeters.

The raw telemetry SAX instrument data consist in a series of compact files collected in the Final Observation Tape (FOT). All XAS programs to reduce MECS, HPGSPC and PDS data requires the FOT files to be correctly dumped from the tape to a disk area. By reduce one means the preparation of data for scientific analysis. A standard XAS reduction process consists in:

setting FOT directory and XAS session
FOT reading (filing of tape data to disk)
environment variables setting
observation concatenation
data screaning and selection
accumulation of corrected scientific products (images, spectra, light curves or event files).

XAS uses both binary and ASCII files. Scientific products (spectra, light curves, images, housekeeping and ratemeters time profiles, event lists) are generally written in a specific XAS binary format.

XAS package includes however a set of tools to convert binary XAS files in standard FITS/OGIP format, which is suitable to be analyzed with X-ray software packages such as Xspec, Ximage, and Xronos or home-grown software.

Setting the FOT directory and the XAS session

Two cautions are advisable to SAX data observers before they start a reduction session:

the directory to dump the FOT file on disk should be kept separate from the work directory. For a single pointing the observer is likely to have hundreds of files for each instruments. The observer is not requested to know what such files are in details to analyze SAX data. But mixing FOT and product files can be really a great mess at last!
the observers should ensure to have enough disk space before starting to dump FOT files on disk. Up to several hundreds of MBytes may be necessary for a 100 ks observation of a tenthCrab source. If you think that such requirement is fulfilled, you can create your own SAX FOT directory:

mkdir /mydisk/mysubdisk/myfotdir

In order to use XAS package, one needs to have the environment variable ${XASTOP} set:

setenv XASTOP /mydisk/mysubdisk/xastop

where /mydisk/mysubdisk/xastop points to the top of the XASTOP tree (see the XAS installation guide for reference). One must also add to one's own path the following: ${XASTOP}/bin and ${XASTOP}/scripts

FOT data dump from tape to disk

The first operation using XAS software is to "file" the FOT. This can be achieved using the programs fotfile :

xasset rootdir
xasset fotdir mydisk/mysubdisk/myfotdir
fotfile tape instrument inslewobs targetobs outslewobs datatypes
(e:g: fotfile /dev/nrmt0h me 0 1-99 0 ALL reads all the MECS files of the normal pointing (no initial or final slew) from the unit /dev/nrmt0h)

fotfile must be run on only one instrument in the same time. However, file from different instruments can be kept in the same directrory, since their names are unique.

Running fotfile all target observations and all datatypes will generate in the datadir directory a list of files such as:


for each FOT:

saxfot.tapedir		directory file, list of files in a FOT
saxfot.obt_utc          table for OBT-UTC conversion
saxfot.attitude		attitude data
saxfot.ephemeris	ephemeris

me.instdir		directory file, list of observations in a FOT
hk.instdir
pd.instdir
le.instdir

	for each observation:

	nme001.m1dir002		MECS 1 scientific data 
	nme001.obsdir		directory file, list of files in an observation
	nme001.m1eng000		MECS 1 engeneering file
	nme001.mehkd000		MECS 1, 2, 3, housekeeping file
           .
           .
           .
	npd001.obsdir		directory file, list of files in an observation
	npd001.pddir001		PDS scientific data
	npd001.pdhkd000		PDS housekeeping file
	npd001.pdeng000		PDS engeneering file
           .
           .
           .

Environment variables setting

The first operation using XAS software is to set the environmental variables (or "global variables") appropriate to the data one would like to reduce and to the operation one would like to perform. The xasset program allows to assign values to global variables within XAS

The basic global variables to set starting a XAS section are: rootdir, datadir, fotdir, flight, slew, and instrument , which are rather self-explanatory. This can be done typing:

xasset rootdir
xasset datadir mydisk/mysubdisk/myfotdir
xasset fotdir mydisk/mysubdisk/mydatadir
xasset slew no
xasset flight yes
xasset instrument mecs

XAS searches FOT files in the ${ROOTDIR}${FOTDIR} directory (/mydisk/mysubdisk/myfotdir) and generally searches/handles/write files in the ${ROOTDIR}${DATADIR} directory (/mydisk/mysubdisk/mydatadir)

The second operation is to concatenate the observations performed with a similar experiment configuration. This can be done using the program concatenate :

concatenate 1+3+11

  Select observation ranges to analyse as follows :
     5              to analyse obs. 5 only
     1+3+7          to analyse obs. 1,3 and 7
     2-5            to analyse obs. from 2 to 5
     1+3-6+9        to combine ranges and single obs.
     0              to reset the chain to NONE
  Your selection? 1+3+11

A help to perform the above outlined operations can be given by running the script xasexp in the directory where the FOT files are present:

xasexp inst mode datadir

For instance: xasdef mecs dir002 /mydisk/mysubdisk/mydir will automatically:

set all the relevant environment variables to analyize MECS data of the DIR002 mode and the datadir to /mydisk/mysubdisk/mydir
return output strings (suitable to be given as an input to concatenate) like:

The following observation can be concatenated toghether:

1+3+4+5+7+8+10+11+12+15+17+18+20

Data screening

Basic data screening (exclusion of Earth occultations) can be performed generating a "time windows" set (the same concept as the Good Time Intervals, GTI, in FTOOLS environment) by using the programs saxauxcalc and iwindow.

saxauxcalc

will generate the files:

earth.time
bright.time
sunangle.time
rigidity.time
pointtype.time

containing the Earth elevation angle, the bright Earth angle, the Sun angle, the Pointing type and the Cut-off Rigidity of the Geomagnetic field as a function of the time.

iwindow no_occultation.window earth.time Above 5

will generate the time window file no_occultation.window corresponding to intervals when Earth elevation angle is higher than 5 degrees. The SAX Team suggests a set of screening criteria which maximize the particle background rejection:

Bright Earth Angle > 5 degrees
Earth Elevation Angle > 5 degrees
Sun Angle > 60 degrees
Geomagnetic Cut-off rigidity > 6 GeV/c²

Alternatively, saxauxcalc produces also a file pointype.time which contains the "light curve" of a pointing flag. Its values are:

0 = nominal pointing
1 = Bright Earth occultation
2 = Dark Earth occultation

An automatic screening to select only the on-source data can be then obtained with the command:

iwindow no_occultation.window pointtype.time Below 0.5

All accumulations will use time window if the appropriate XAS variable is set:

xasset timewindow no_occultation.window

Data screening and selection can also be performed plotting appropriate housekeeping data and ratemeters. In such a case, one must:

accumulate a time profile of the relevant housekeeping or ratemeter
run:
iwindow outfile infile mode threshold
(e.g.: iwindow hv.window hv.time B 1500 will produce a time window selecting all the intervals where the hv parameter is below 1500)

It is also possible to generate "time windows" graphically from a previously accumulated time profile with the program twindow.

Time windows can be merged using the program mergewindow:

mergewindow w_fileout w_filein1 AND w_filein2

Merging more than two time windows with a single command is currently not supported.

Product accumulation

Product accumulation can be performed using the program accumulate:

It is possible to "accumulate" the following quantities:

spectrum	PHA or BL or RiseTIme spectra
image	X-Y, PHA-BL, etc images
time	time profiles, light curves
hk	housekeeping time profiles
photon	photon list, event files

The following is an example of a run to accumulate a spectrum from PDS scientific direct mode data. The information provided by the user are typed in bold.

accumulate spectrum

  setting default spacecraft to SAX
  processing observation   1
  accessing directory npd001.obsdir
   spectral file name  test1
  output file is 
 /dat1/op_627/xas/test1.spectrum

  processing instrument pds 
  processing observation   1
  accessing directory npd001.obsdir
  packets p1cal000 are eligible for accumulation
  packets p1lat000 are eligible for accumulation
  packets p1psa000 are eligible for accumulation
  packets p2cal000 are eligible for accumulation
  packets p2lat000 are eligible for accumulation
  packets p2psa000 are eligible for accumulation
  packets p3cal000 are eligible for accumulation
  packets p3lat000 are eligible for accumulation
  packets p3psa000 are eligible for accumulation
  packets p4cal000 are eligible for accumulation
  packets p4lat000 are eligible for accumulation
  packets p4psa000 are eligible for accumulation
  packets pddir001 are eligible for accumulation
  select one of the above  13 packet types
   enter wished packet type  pddir001         
  packetcap file name is $XASTOP/calib/sax/pds/saxpdsdir.packetcap
   select the quantity on spectrum X-axis  PHA
   range for PHA      inclusive of extrema : df [0       1023]<return>
   give binning factor for PHA              : df [ 1]   <return>
  Start TIME as y m d h m s : df [1996  7  9 18 35 42]  <return>
  End   TIME as y m d h m s : df [1996  7  9 10  0 24]  <return>
   range for RiseTime inclusive of extrema : df [0        511]  20 120
   range for Unit     inclusive of extrema : df [0          3]  0 3     
   range for Flags    inclusive of extrema : df [0          7]  0 7     
  processing observation   1
  accessing directory npd001.obsdir
  file selected 
 npd001.pddir001                                                                 
   706
  Opening telemetry file npd001.pddir001
  packet data will be read and byte-swapped as needed

All the scientific products made by accumulate are in binary XAS format. They can be converted in FITS/OGIP standard through the command:

toogip infile.ext

which produces an output file infile_ext.fits

MECS REDUCTION PIPELINE

In the MECS data reduction, all the required data corrections/selections are performed during the accumulation loop. The required actions are:

position linearization
spatial gain correction
temporal gain correction
spatial selection

The 1. and 2. must be done before the 3. and 4. In order any correction to be performed one must:

xasset correction enable

Position linearization

It is automatically performed during the accumulation loop; but currently the user is interactively required:

either the number of linearized pixel in the field of view or the pixel size in millimeter
the seed for a number generator routine

Temporal gain correction

It must be performed for each detector separately. As a first step one has to create a gain history file with the commands:

accumulate gain filename

It produces a light curve file filename.time in standard XAS format which contains the relative gain as a function of time. Any subsequent accumulation will perform the relevant time correction, provided the following xasset is given:

xasset gainhistory filename

Spatial selection

XAS interprets correctly SAOIMAGE-style-like region files, with a standard syntax. They can be created either via SAOIMAGE or with an editor. As usual, the user who intends to use a selection region file during an accumulation loop has to xasset the relevant global variables:

xasset selectmethod region
xasset selectfile filename(.region)

HPGSPC REDUCTION PIPELINE

HPGSPC accumulation of background subtracted spectra can be performed using the script hp_dir, which performs the following operations:

accumulates a time profile of the collimator position
generates time window for the collimator positions ("on", "off-" and "off+")
merges collimator time windows with the no Earth occultation time window (file no_occultation.window, see above)
accumulates separate spectra for different collimator positions
subtracts "off-" from "on" spectrum and applies the relevant needed corrections

hp_dir requires a file no_occultation.window to be already existent in ${ROOTDIR}${DATADIR}, which contains the time windows for the intervals of no Earth occultation.

A special caution must be set in choosing the set of observations to be concatenated. The HPGSPC electronic includes an Automatic Gain Control (AGC) system to correct on-board for any observation-by-observation gain variation. Such a system is set every time the satellite exits the South Atlantic Anomaly and requires few hundreds of seconds before the post-SAGA gain is correctly estimated. All the observations when the AGC is "on" should not be used for scientific analysis due to gain instabilities. This can be understood by looking at the ${ROOTDIR}${FOTDIR}/nhp???.hpexpconf files or by running xasexp.

xasunset timewindow

hp_dir observations hp_dirmode BL_low BL_high datadir [verbose]
(e.g.hp_dir 3+5+7 1 85 115 /mydisk/mysubdisk/mydatadir will analyze the HPGSPC DIR001 mode data of the observations 3+5+7, applying Burst Length boundaries from 85 to 115). Such values are suggested by the HPGSPC hardware team and the on-line response matrices were calculated with such thresholds. The option [verbose] allows the scripts to be run in a rather chatty way. The output of hp_dir consists of a single background subtracted spectrum (file on_bs.spectrum) with 512 energy channels. It is produced in XAS format and can be converted in OGIP standard format through the command toogip

toogip on_bs.spectrum

PDS REDUCTION PIPELINE

PDS data reduction pipeline requires the following steps:

generation of time windows for intervals where the units High Voltage (HV) are within the nominal expectation values
merging of the HV with the no Earth occultation time windows
generation of time windows for each position of the two PDS collimators ("off-", "on", and "off+");
merging of the collimators time windows with the no Earth occultation+HV time windows
accumulation of spectra for each of the four PDS units in each of the collimator positions;
addition of the spectra of each units accumulated in the positions "off-" and "off+";
subtraction of the spectra from the collimators "off" positions from the spectra accumulated in the collimator "on" position, for each units.
rebin of the background subtracted spectra
gain linearization of the spectra
sum of the rebinned, linearized, background subtracted

A number of scripts are provided to perform such task in an automatic way. The standard pipeline therefore consists of the following commands:

xasunset timewindows (reset the usage of time windows)

xasset countaxis c (spectra are to be currently accumulated in counts)

mv no_occultation.window no_occultation_tmp.window (temporary rename of the no Earth occultation file)

pds_hv_windows [verbose] (generates the hv.window)

mergewindow no_occultation.window no_occultation_tmp.window AND hv.window (merging of no Earth occultation with HV time window files)

pds_dir observations pds_dirmode [verbose]
(e.g.pds_dir 3+5+7 1 60 120 will analyze the PDS DIR001 mode data of the observations 3+5+7, applying Rise Time boundaries from 60 to 120)

pds_logsum outchan startenergy endenergy
(e.g.pds_logsum 60 12 300 will produce a PDS total spectrum with 60 PI channels from 12 to 300 keV> such values were employed to calculate the currently on-line responsa matrix

toogip totalS_grp.spectrum

The option [verbose] allows the scripts to be run in a rather chatty way.

The pipeline requires requires a file no_occultation.window to be already existent in ${ROOTDIR}${DATADIR}, which contains the time windows for the intervals of no Earth occultation.

The output of pds_dir consists of background subtracted spectra, one for each PDS unit, in XAS format. pds_logsum generates a file totalS_grp.spectrum with a single PDS rebinned, linearized, background subtracted spectrum. This file is in XAS format. toogip translates it in OGIP standard format.

Questions, comments or suggestions concerning the XAS software can be addressed to helpdesk@sax.sdc.asi.it

This page is maintained by Fabrizio Fiore and Matteo Guainazzi