A walk through SAXDAS data reduction


We run through two fictitious SAXDAS sessions (one for the LECS and one for the MECS instruments) to illustrate some of the commands, their usage and their effect. We start with the FOT and run the programs to reduce LECS and MECS data. By reduce we mean the preparation of data for scientific analysis, a process which consists in:

Scientific Data analysis will then be performed with X-ray software packages such as Xspec, Ximage, and Xronos, or any other packages or home-grown software.

The raw telemetry SAX instrument data consists in a series of compact files collected in the Final Observation Tapes (FOTs).

The first step of the processing is the conversion of FOT telemetry files to portable, machine independent FITS files.This is achieved using the program foting .

foting disk=yes indir=./ instrument=ME+LE outdir=./test216

At the moment foting is rather chatty, so don't be afraid by the number of warnings.
foting has now generated in the outdir directory a list of FITS files such as:

        fot_000491_LE_001_n_01_rev.fits         LECS raw event file
        fot_000491_LE_001_n_01_rhk.fits         LECS raw housekeeping file

	fot_000491_M1_001_n_rev.fits		MECS raw event file
	fot_000491_M1_001_n_rhk.fits		MECS raw housekeeping file

	fot_pppppp_mm_qqq_r_sss.fits

	pppppp = FOT number
	mm     = instrument
	qqq    = observation 
	r      = flag, n = normal, i = initial slew, f = final slew
	sss    = file identificator


LECS WALK THROUGH


Go into the directory containing the FITS files.

The operations of concatenation, correction and screening can be performed automatically by using the script lecspipe, which will run in sequence the programs that perform each single operations. Each program can of course be run independently. The same script can be invoked by running the script saxpipe instr=LE firstevent=fot_000232_LE_001_n_01_rev.fits.

lecspipe fot_000232_LE_001_n_01_rev.fits logfile=myfile.log baryconv=no exitstage=3

Find the observations with the same experiment configuration and concatenate them:

Executing command: catobs nfiles=fot_000232_LE_001_n_01_rev.fits indirpath=. stemoutfile=fot_000232_LE outdir=. hkconcat=yes

Conversion of housekeeping data from digital to physical units:

Executing command: eng2phys infile=fot_000232_LE_rhk.fits

Accumulation of calibration source spectra as a function of the time, fit of the centroids and storing of the fit results in the hosekeeping data:

Executing command: calcgain infile=fot_000232_LE_rev.fits

Event correction and data screening. We can perform the following corrections:

spatial gain correction
temporal gain correction
linearization
dead time correction
conversion from detector to sky coordinates
barycentric correction (to be implemented)

And the following screenings:

Removal of events detected outside the detector window
selections of events in a given burstlenght-PHA region

All operations are executed by the program evelin

Executing command: evelin infiles=fot_000232_LE_rev.fits

Data selection. We reject data from periods of high backgroud and then select Good Time Intervals.

hkgfile=./fot_000232_LE_01_hkg.fits stdhkints=yes verbosity=3

Executing: maketime infile=./fot_000232_LE_01_hkg.fits+2 outfile=./fot_000094_M1_01_gti00_2.fits expr='HV_POWER && HV_ENBL && RUNNING_MODE && CELL_HIGH_VOLTAGE.GT.-20200.0 && CELL_HIGH_VOLTAGE.LT.-19800 && PMT_HIGH_VOLTAGE.GT.1090.0 && PMT_HIGH_VOLTAGE.LT.1120.0 && EARTH_ANG.GT.4.0' name="" value="" time=TIME compact=no

The backgound is a strong function of the satellite position in the orbit, so periods of high background can be identified using attitude data. Data selection is then performed accepting data in intervals of the following quantities:

		HV_POWER 		T
		HV_ENBL 		T
		RUNNING_MODE 		T
		CELL_HIGH_VOLTAGE 	-20200.0,-19800
		PMT_HIGH_VOLTAGE 	1090.0,1120.0
		EARTH_ANG 		4.0,INDEF
These defaults settings can be modified by the user. This generates a fits file with GTI (e.g. fot_000232_LE_01_gti00.fits) which can then be applied in XSELECT.

lecspipe is divided in four stages which can be run independently:

--------------------------------------------------------------------------
STAGE I : Event Files Concatenation, HK data conversion & gain computation
--------------------------------------------------------------------------

------------------------------------------------
STAGE II : events correction
------------------------------------------------

------------------------------------------------
STAGE III : events selection and GTI creation
------------------------------------------------

----------------------------------------------------------
STAGE IV : accumulation of final products (using XSELECT)
----------------------------------------------------------

To run, for example, stages II and III only type:

lecspipe datadir=./ baryconv=no entrystage=2 exitstage=3

Plotting of housekeeping data, ratemeters, attitude data,and ephemeridis data, and selection based on these plots, can be performed interactively using the program hkplot A number of housekeeping parameters and ratemeters should be checked routinely:

E_MIN_RATEMETER counts above lower energy threshold
E_MAX_RATEMETER counts above higher energy threshold
PHA1 centroid of cal. source 1
PHA2 centroid of cal. source 2
GAIN_AVG Average gain
RA_SCZ RA Z axis
DEC_SCZ DEC Z axis
ROLL_AN Roll angle
EARTH_ANG Elevation angle
BREARTH_ANG Bright Earth angle
SUN_ANG Sun angle
COR_MIN Geomagnetic cut-off rigidity


MECS WALK THROUGH


Go into the directory containing the FITS files.

The operations of concatenation, correction and screening can be performed automatically by using the script saxpipe, which will run in sequence the programs that perform each single operations. Each program can of course be run independently. saxpipe instr=ME will run the programs for each MECS unit, in the following example we use MECS 1.

saxpipe instr=ME indir=./raw outdir=./clean

Find the observations with the same experiment configuration and concatenate them:

Executing command: mksaxobs instr=ME indir=test outdir=clean slewobs=no

mksaxobs executes the following command:

Executing command: meselobs ins=ME filelist=clean/32243.ls active=none | mecatobs filelist=stdin outdir=clean stem=-

Conversion of housekeeping data from digital to physical units:

Executing command: meeng2phys infile=clean/fot_000491_M1_01_rhk.fits outdir=clean

Accumulation of calibration source spectra as a function of the time, fit of the centroids and storing of the fit results in the hosekeeping data:

Executing command: mecalcgain infile=clean/fot_000491_M1_01_rev.fits plotspec=0,1

Event correction and data screening. We can perform the following corrections:

spatial gain correction
temporal gain correction
linearization
dead time correction (to be implemented)
conversion from detector to sky coordinates
barycentric correction (to be implemented)

And the following screenings:

Removal of events detected outside the detector window
selections of events in a given burstlenght interval

All operations are executed by the program meevelin

Executing command: meevelin meevelin infiles=fot_000491_M1_01_rev.fits indirpath=clean rmevents=yes blsel=yes tgain=yes sgain=yes lincrd=yes sky=yes

Data selection. We reject data from periods of high backgroud and then select Good Time Intervals.

Executing: memkgti hkgfile=./fot_000491_M1_01_hkg.fits stdhkints=yes verbosity=3

Executing: maketime infile=clean/fot_000491_M1_01_hkg.fits+2 outfile=clean/fot_000491_M1_01_gti00_2.fits expr='GAIN_AVG.GT.0.5 && GAIN_AVG.LT.1.5 && FOV_FLAG==0 && BREARTH_ANG>=5.0 && EARTH_ANG>=5.0 && !SAA_FLAG && COR_MIN>=6.0 && SUN_ANG>=60.0' name="" value="" time=TIME compact=no

Periods of Earth occultation can be identified using attitude and ephemeridis data. Data selection is then performed accepting data in intervals of the following quantities:

		GAIN_AVG 	0.5,1.5
		SAA_FLAG 	F
		FOV_FLAG 	0
		EARTH_ANG 	5.0,INDEF
		BREARTH_ANG 	5.0,INDEF
		SUN_ANG 	60.0,INDEF 
		COR_MIN 	6.0,INDEF
These defaults settings can be modified by the user. This generates a fits file with GTI (e.g. fot_000491_M1_01_gti00.fits) which is then applied by fltime or that can be applied in XSELECT.

Executing command: fltime infile=clean/fot_000491_M1_01_lev.fits+2 gtifile=clean/fot_000491_M1_01_gti00.fits outfile=clean/fot_000491_M1_01_lev_flt.fits column=TIME copyall=yes

This has generated a filtered and linearized event file fot_000491_M3_01_lev_flt.fits.

saxspipe instr=ME is divided in four stages which can be run independently:

------------------------------------------------
STAGE I : Event Files Concatenation
------------------------------------------------

------------------------------------------------
STAGE II : HK data conversion & gain computation
------------------------------------------------

------------------------------------------------
STAGE III : events correction
------------------------------------------------

------------------------------------------------
STAGE IV : events selection and GTI creation
------------------------------------------------


To run, for example, stages II and III only type:

saxpipe proddir=clean entrystage=2 exitstage=3

Plotting of housekeeping data, attitude data, ephemeridis data and ratemeters , and data selection based on these plots, can be performed interactively using the program hkplot A number of housekeeping parameters and ratemeters should be checked routinely:

VALEMIN (or CSELO) counts above lower energy threshold
REJEMAX counts above higher energy threshold
PHA1 centroid of cal. source 1
PHA2 centroid of cal. source 2
GAIN_AVG Average gain
RA_SCZ RA Z axis
DEC_SCZ DEC Z axis
ROLL_AN Roll angle
EARTH_ANG Elevation angle
BREARTH_ANG Bright Earth angle
SUN_ANG Sun angle
COR_MIN Geomagnetic cut-off rigidity
Geomagnetic cut-off rigidity

Final producs extraction. saxpipe, lecspipe or the interactive analysis have generated a cleaned and corrected event file (eg. fot_000491_M1_01_lev.fits) which can then by processed by XSELECT to extract images, spectra and light curves. It is of course possible to perform further data selections with XSELECT before the final products accumulation. At this point the observer should be ready for the scientific analysis of SAX LECS and MECS data.


Questions, comments or suggestions concerning the SAXDAS software can be addressed to helpdesk@sax.sdc.asi.it

This page is maintained by Fabrizio Fiore