Modify specsim / quickquasars to mimic how the real pipeline estimates the noise variance

[andreufont]

Moving here a discussion at the DESI meeting, and on Slack, with @julienguy.

• in specsim, the amount of noise added has a contribution from the quasar flux in a given pixel
• the value of IVAR (inverse noise variance) stored in the synthetic spectra is therefore correlated with the delta_flux in the pixel, because pixel with a lot of absorption (low delta_flux) will have smaller noise variance (larger IVAR)
• in Picca we use IVAR to compute weights in picca analyses, so the weights are correlated with delta_flux.

As explained by @julienguy (see below), the DESI pipeline tries to avoid this by using a model instead of the real flux to compute the noise estimate.

From Julien:
We use a model of the signal to compute the Poisson noise term.

1. we extract rapidly the spectra from the CCD (with row by row boxcar extraction, not spectro-perf)
2. we compute a sky model using the data and the flat
3. we convolve the sky subtracted spectrum with a gaussian kernel
4. we add back the sky and flat field, giving a spectral model:
   spectral_model = (((spectral_data/flat-sky) x gaussian_kernel ) + sky )*flat
5. we project back this spectral_model on the CCD using a cross-dispersion profile to estimate the Poisson noise in the pixels (edited)

So, yes, the ivar is less correlated with the delta_flux . it is correlated with the delta_flux convolved with a gaussian of sigma = 10 A. The code is here:
https://github.com/desihub/desispec/blob/37847d35bf6ff557097831063f3e69b5ec57b988/py/desispec/image_model.py#L51

[andreufont]

It would be good if someone could modify specsim to add the option to compute an alternative noise variance that uses a similar model.