OptoDAS channel numbering ignores spatial decimation of original file

[speleos]

Description

When attempting to read an OptoDAS file that is spatially decimated (only every 5th channel was saved), DAScore crashes, as it expects that the number of channels in the file matches the channel numbers.

dascore.exceptions.CoordDataError: Data array has a shape of (5000, 11380) which doesnt match the coordinate manager shape of (5000, 56896).

Expected behavior

DAScore should read the '/demodSpec/roiDec' field in the HDF5 file and calculate the new shape of the data block.

Versions

• OS Ubuntu 20.04
• DAScore Version 0.1.2
• Python Version 3.11

[d-chambers]

Hey @speleos,

Do you by chance have a small OptoDAS file which uses spatial decimation you can share? I implemented a potential fix in the branch fix_419 you can try out, but it would be nice to include a more formal test in the test suite.

There might also be some improvements we can make to how coordinates can represent decimated values but I will need a bit more time to think about it.

[speleos]

I have a ten second file directly from the interrogator, but even compressed it is too large to send through here.
Can I send it through wetransfer to your email?

[speleos]

I also have post-processed files using the simpledas library from ASN that might be missing a few header words. These break dascore in different ways.

[d-chambers]

Ok, send me both if you can. chambers.ja.derrick (a gmail address).

[speleos]

Done

[d-chambers]

Hey @speleos, #420 should fix your issue. It reads both files now, but I had to change the smaller file's decimation factor from 5 to 50 to match the array size. I suspect this was overlooked when creating the smaller file, but I could be wrong.

Can you take the fix_419 branch for a spin and see if it reads your DAS files? If you haven't installed a non-release version of DASCore before You should be able to install it by cloning DASCore, switching to branch fix_419 and installing in dev mode (see dev install docs).

[speleos]

The second file had been further spatially decimated.
I'll test the new branch.

[speleos]

For the original file it works now.
For the decimated file, I was expecting that the ASN library adjusted the roiDec parameter, but it just copies it over from the original.

I'm not sure if the proper way to handle this should rely solely on roiDec, as one can simply remove a few channels from a complete dataset. The roiDec parameter might apply for some, it will certainly not apply for all situations.

A simple count the number of elements in ['/header/channels'][:] seems more elegant and will apply to all situations.

import h5py
with h5py.File("/mnt/D2A/DAS/OptoDAS_SN44/26OCT-02NOV/events/27OCT23_derrick.hdf5", "r") as hf_in:
r=hf_in['/demodSpec/roiDec'][0]
chans=hf_in['/header/channels'][:]

chans
array([32500, 32550, 32600, 32650, 32700, 32750, 32800, 32850, 32900,
32950, 33000, 33050, 33100, 33150, 33200, 33250, 33300, 33350,
33400, 33450, 33500, 33550, 33600, 33650, 33700, 33750, 33800,
33850, 33900, 33950, 34000, 34050, 34100, 34150, 34200, 34250,
34300, 34350, 34400, 34450, 34500, 34550, 34600, 34650, 34700,
34750, 34800, 34850, 34900, 34950, 35000], dtype=int32)
len(chans)
51

[d-chambers]

Thanks, good suggestion. I have updated the code to use channels instead of the decimation factor.