Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot read coinsdata created according to document #339

Open
hkthirano opened this issue Dec 5, 2023 · 2 comments
Open

Cannot read coinsdata created according to document #339

hkthirano opened this issue Dec 5, 2023 · 2 comments

Comments

@hkthirano
Copy link

I have created coinsdata as per the instructions in the documentation.

ome_zarr create coinsdata

However, I encounter an error when attempting to read coinsdata using this library.

code

url = "coinsdata"
store = parse_url(url, mode="r").store
reader = Reader(parse_url(url))
nodes = list(reader())

error

RecursionError: maximum recursion depth exceeded in comparison

@will-moore
Copy link
Member

Thanks for the feedback.

I tried the same code myself...

>>> url = "coinsdata"
>>> from ome_zarr.io import parse_url
>>> from ome_zarr.reader import Reader
>>> store = parse_url(url, mode="r").store
>>> reader = Reader(parse_url(url))
>>> nodes = list(reader())
Traceback (most recent call last):
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/zarr/storage.py", line 1428, in __getitem__
    return self.map[key]
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/mapping.py", line 151, in __getitem__
    result = self.fs.cat(k)
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/spec.py", line 877, in cat
    return self.cat_file(paths[0], **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/spec.py", line 766, in cat_file
    with self.open(path, "rb", **kwargs) as f:
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/spec.py", line 1295, in open
    f = self._open(
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/implementations/local.py", line 180, in _open
    return LocalFileOpener(path, mode, fs=self, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/implementations/local.py", line 302, in __init__
    self._open()
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/fsspec/implementations/local.py", line 307, in _open
    self.f = open(self.path, mode=self.mode)
OSError: [Errno 63] File name too long: '/Users/wmoore/Desktop/ZARR/data/temp/coinsdata/labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../labels/coins/../../0/.zarray'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/zarr/core.py", line 252, in _load_metadata_nosync
    meta_bytes = self._store[mkey]
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/zarr/storage.py", line 1430, in __getitem__
    raise KeyError(key) from e
KeyError: '0/.zarray'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/ome_zarr/reader.py", line 621, in __call__
    node = Node(self.zarr, self)
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/ome_zarr/reader.py", line 52, in __init__
    self.specs.append(Multiscales(self))

...

  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/ome_zarr/io.py", line 112, in load
    return da.from_zarr(self.__store, subpath)
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/dask/array/core.py", line 3601, in from_zarr
    z = zarr.Array(url, read_only=True, path=component, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/zarr/core.py", line 224, in __init__
    self._load_metadata()
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/zarr/core.py", line 243, in _load_metadata
    self._load_metadata_nosync()
  File "/Users/wmoore/opt/anaconda3/envs/omeropy39/lib/python3.9/site-packages/zarr/core.py", line 254, in _load_metadata_nosync
    raise ArrayNotFoundError(self._path)
zarr.errors.ArrayNotFoundError: array not found at path %r' '0'

This certainly looks like recursion, but with a different error

@will-moore
Copy link
Member

I found that I could fix the error I have above by reverting to older versions of s3fs and fsspec.
I previously had

fsspec==2023.12.1
s3fs==2023.12.1

After uninstalling and installing...

$ pip uninstall fsspec
$ pip uninstall s3fs
$ pip install "s3fs==2023.6.0"

I now have

$ pip freeze | grep fs
fsspec==2023.6.0
s3fs==2023.6.0

and this works with the code above.
Haven't looked into what the change is yet...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants