auto_mkdir not propagated resulting in FileNotFoundError during write_image

[npeschke]

I am trying to save a multidimensional image using ome-zarr, however saving fails with the following error:

FileNotFoundError: [Errno 2] No such file or directory: '/path/to/input_image.ome.zarr/0/0/0/0'

I am using the following code to save the image:

loc = parse_url(path, mode="w").store
group = zarr.group(store = loc)

n_channels = input_image.shape[0]
channels = [f"Channel{i}" for i in range(n_channels)]

group.attrs["omero"] = {
    "name": self.DEFAULT_INPUT_IMAGE_NAME,
    "channels": [{"label":channel, "color":self.channel_colors[i], "active":True} for i, channel in enumerate(channels)]
}

write_image(input_image, group = group, axes = "cyx", storage_options=dict(chunks=(1, 1024, 1024)))

The following directory structure is generated:

input_image.ome.zarr/
├── 0
│   └── .zarray
├── .zattrs
└── .zgroup

I've narrowed down the issue to auto_mkdir not being propagated to the LocalFileSystem object in the store

Is this intended behaviour?

[will-moore]

Hi, thanks for the feedback.
I was just trying to reproduce this issue locally, and failing until I was notified of #302, which I believe is the same issue.

On updating my local version of fsspec-2022.2.0 to fsspec-2023.9.0 also (s3fs-2022.2.0 to s3fs-2023.9.0) I now see the same issue.
So it looks like the short-term solution is to install fsspec-2022.2.0 (assuming you have a more recent version) while we try to fix the issue.

[will-moore]

Seems the commit at fsspec/filesystem_spec@285094f is involved in this issue: fsspec/filesystem_spec@285094f

[will-moore]

Same issue reported at #305 (with stack trace) and also causes test failures at #302.

See discussion on this issue at fsspec/filesystem_spec@285094f

[davidpross]

I got the same issue running the example here. I had fsspec 2023.9.0, running pip install fsspec=="2022.2.0" resolved the issue. Thanks.