How to use debugger to troubleshoot HTTPFile.read

[itcarroll]

The HTTPFileSystem uses async, and I have no idea how to use breakpoints the way I normally would to troubleshoot. Here is the code that's giving me trouble (you would need a NASA Earthdata credentials to run it, sorry, but the thing I'm troubleshooting is a specific service that requires auth).

import logging
import urllib3

# Set up logging to console at DEBUG level
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger('urllib3')
logger.setLevel(logging.DEBUG)

import earthaccess
import h5py
from fsspec.implementations.http import HTTPFile

earthaccess.login()
results = earthaccess.search_data(
    short_name="VIIRSJ1_L2_OC",
    count=1,
)
files = earthaccess.open(results)
h5py.File(files[0])

Here's the traceback.

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[3], line 1
----> 1 h5py.File(files[0])

File /opt/conda/envs/python3.10/lib/python3.10/site-packages/h5py/_hl/files.py:562, in File.__init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, fs_strategy, fs_persist, fs_threshold, fs_page_size, page_buf_size, min_meta_keep, min_raw_keep, locking, alignment_threshold, alignment_interval, meta_block_size, **kwds)
    553     fapl = make_fapl(driver, libver, rdcc_nslots, rdcc_nbytes, rdcc_w0,
    554                      locking, page_buf_size, min_meta_keep, min_raw_keep,
    555                      alignment_threshold=alignment_threshold,
    556                      alignment_interval=alignment_interval,
    557                      meta_block_size=meta_block_size,
    558                      **kwds)
    559     fcpl = make_fcpl(track_order=track_order, fs_strategy=fs_strategy,
    560                      fs_persist=fs_persist, fs_threshold=fs_threshold,
    561                      fs_page_size=fs_page_size)
--> 562     fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
    564 if isinstance(libver, tuple):
    565     self._libver = libver

File /opt/conda/envs/python3.10/lib/python3.10/site-packages/h5py/_hl/files.py:235, in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    233     if swmr and swmr_support:
    234         flags |= h5f.ACC_SWMR_READ
--> 235     fid = h5f.open(name, flags, fapl=fapl)
    236 elif mode == 'r+':
    237     fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File h5py/h5f.pyx:102, in h5py.h5f.open()

OSError: Unable to synchronously open file (file signature not found)

The DEBUG output includes DEBUG:fsspec.http:Fetch range for <File-like object HTTPFileSystem, https://oceandata.sci.gsfc.nasa.gov/cmr/getfile/JPSS1_VIIRS.20171129T213001.L2.OC.nc>: 0-5242888, and I think range requests are the source of the problem. I would like to drop a breakpoint at:

filesystem_spec/fsspec/implementations/http.py

Line 646 in 8be9763

r = await self.session.get(

But it does not hit because it's in a different thread?

How can I see what fsspec is actually getting as a response to that get request?

[martindurant]

You are correct, when fsspec is called from sync code, it runs an async event loop on a dedicated thread. However, because python has a GIL, you are totally able to place a breakpoint in the fetch method (with pdb, your favourite IDE or even editing http.py to add a breakpoint()). You would want the breakpoint to be beyond the async with r line, because that's when the request is actually sent.

[itcarroll]

Thanks, I dug deeper with your encouragement. My bad, sort of. (Turns out breakpoint() is broken in some combination of jupyterlab and ipykernel: ipython/ipykernel#897. Or the h5py extensions make traceback difficult? Dunno, but I can get what I need with breakpoints in VS Code and calling files[0].read(8))

[martindurant]

Or the h5py extensions make traceback difficult?

That may be, I think perhaps this is C code calling back into the python interpreter.