enrico_folded_lc fails to select correct time intervals

[pmunar]

Hi,

I am trying to produce a folded LC. After setting the config file with the period and phase 0 time, enrico generates the correct number of config files and the Timeselection files that contain the pairs of MJD_start MJD_end to be used in the gtmktime function.

When enrico starts to run, it produces two event files, one finished with _EvtCoarse.fits and another one finished with _Evt.fits. Of these two files, only the _EvtCoarse.fits has the correct selection of times (the other one has the whole time interval given in the original config file). Then, in the next steps only the _Evt.fits file is used to produce the rest of files needed (CountsMap, ExpCube, etc.).

The fact is that before (now I am using another computer) it did not create the _EvtCoarse.fits file

Did something change in the enrico_folded_lc function?
By now, I have managed to continue by creating the _EvtCoarse.fits files, rename them and set clobber to "no" in the config files, so the program is using these event files.

Thanks.

As an example, here I put the output for the first time-bin:

enrico_foldedlc mysource_full_dataset_23nov17_LC_20d_bins.conf

************************************************************

*** SUMMARY: ***

************************************************************

Source = mysource
RA = 138.929 degrees
Dec = 11.1901 degrees
Start = 239557417.0 MET (s)
Stop = 531589417.0 MET (s)
ROI = 25.0 degrees
E min = 100.0 MeV
E max = 300000.0 MeV
IRFs = CALDB
evclass = 128
evtype = 3
Corresponding IRFs = ('P8R2_SOURCE_V6', ['BACK', 'FRONT'])

************************************************************

*** 1 gtselect --- Select data from library, coarse cut

************************************************************

time -p /home/software/Software/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10/x86_64-unknown-linux-gnu-libc2.19-10/bin/gtselect infile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/L1711231038193C92CE6134_PH00.fits outfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_EvtCoarse.fits ra=138.929 dec=11.1901 rad=25.0 tmin=239557417.0 tmax=531589417.0 emin=100.0 emax=300000.0 zmin=0.0 zmax=90.0 evclass=128 evclsmin=0 evclsmax=10 evtype="INDEF" convtype=-1 phasemin=0.0 phasemax=1.0 evtable="EVENTS" chatter=2 clobber=yes debug=no gui=no mode="ql"
Done.
real 71.71
user 69.04
sys 2.66

************************************************************

*** 2 gtselect --- Select data from library, fine cut

************************************************************

time -p /home/software/Software/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10/x86_64-unknown-linux-gnu-libc2.19-10/bin/gtselect infile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_EvtCoarse.fits outfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt.fits ra=138.929 dec=11.1901 rad=25.0 tmin=239557417.0 tmax=531589417.0 emin=100.0 emax=300000.0 zmin=0.0 zmax=90.0 evclass=128 evclsmin=0 evclsmax=10 evtype=3 convtype=-1 phasemin=0.0 phasemax=1.0 evtable="EVENTS" chatter=2 clobber=yes debug=no gui=no mode="ql"
Done.
real 13.71
user 12.64
sys 1.06

************************************************************

*** 3 gtmktime --- Update the GTI and cut data based on ROI

************************************************************

time -p /home/software/Software/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10/x86_64-unknown-linux-gnu-libc2.19-10/bin/gtmktime scfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/L1711231038193C92CE6134_SC00.fits sctable="SC_DATA" filter="(((START>239557417)&&(STOP<248273017))||((START>309282217)&&(STOP<317997817))||((START>379007017)&&(STOP<387722617))||((START>448731817)&&(STOP<457447417))||((START>518456617)&&(STOP<527172217))||((START>588181417)&&(STOP<596897017)))" roicut=no evfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt.fits evtable="EVENTS" outfile="/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt_1" apply_filter=yes overwrite=no header_obstimes=yes tstart=239557417.0 tstop=531589417.0 gtifile="default" chatter=2 clobber=yes debug=no gui=no mode="ql"
real 6.78
user 6.03
sys 0.74
time -p /home/software/Software/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10/x86_64-unknown-linux-gnu-libc2.19-10/bin/gtmktime scfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/L1711231038193C92CE6134_SC00.fits sctable="SC_DATA" filter="(((START>239557417)&&(STOP<248273017))||((START>309282217)&&(STOP<317997817))||((START>379007017)&&(STOP<387722617))||((START>448731817)&&(STOP<457447417))||((START>518456617)&&(STOP<527172217))||((START>588181417)&&(STOP<596897017)))" roicut=no evfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt.fits evtable="EVENTS" outfile="/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt_1" apply_filter=yes overwrite=no header_obstimes=yes tstart=239557417.0 tstop=531589417.0 gtifile="default" chatter=2 clobber=yes debug=no gui=no mode="ql"
real 6.77
user 5.97
sys 0.80
time -p /home/software/Software/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10/x86_64-unknown-linux-gnu-libc2.19-10/bin/gtselect infile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt.list outfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_EvtCoarse.fits ra=138.929 dec=11.1901 rad=25.0 tmin=239557417.0 tmax=531589417.0 emin=100.0 emax=300000.0 zmin=0.0 zmax=90.0 evclass=128 evclsmin=0 evclsmax=10 evtype="INDEF" convtype=-1 phasemin=0.0 phasemax=1.0 evtable="EVENTS" chatter=2 clobber=yes debug=no gui=no mode="ql"
Done.
real 2.19
user 1.95
sys 0.23

************************************************************

*** 4 gtbin --- Create a count map

************************************************************

time -p /home/software/Software/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10/x86_64-unknown-linux-gnu-libc2.19-10/bin/gtbin evfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_Evt.fits scfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/L1711231038193C92CE6134_SC00.fits outfile=/home/munar/MUNAR/fermi/my_source/full_dataset_START_NOV23_2017/mysource_folded_LC_20_bins_period_807_d_LC_0_CountMap.fits algorithm="CMAP" ebinalg="LOG" emin=100.0 emax=300000.0 ebinfile=NONE tbinalg="LIN" tbinfile=NONE nxpix=353 nypix=353 binsz=0.1 coordsys="CEL" xref=138.929 yref=11.1901 axisrot=0.0 rafield="RA" decfield="DEC" proj="AIT" hpx_ordering_scheme="RING" hpx_order=3 hpx_ebin=yes evtable="EVENTS" sctable="SC_DATA" efield="ENERGY" tfield="TIME" chatter=2 clobber=yes debug=no gui=no mode="ql"
This is gtbin version ScienceTools-v10r0p5-fssc-20150518
real 12.59
user 12.29
sys 0.30

As you can see, in the gtbin (and the next tasks) the program is using as event file the one ended with _Evt.fits, which has the events of the whole time period, instead of the selected bins.

[davidsanchez]

Hi
thanks for reporting. I'll try to fix it soon
david

[mireianievas]

I will have a look at it too, I introduced EvtCoarse files in order to be able to re-use them (for instance for SummedLikelihood analyses) without needing to go over the original data again.

[mireianievas]

I am not sure why, when it tries to consolidate the files into a single event file, it generates the EvtCoarse.fits instead of the Evt.fits

In the time_selection() function from gtfunction.py it basically executes SelectEvents (which calls gtselect) using as input the filename list containing all the time slices and sets as output self.eventfile (_Evt.fits). I am not fully sure about the particulaties of foldedLC methods, but in the standard LCs it does it correctly as far as I am aware.