Code

Zarr reader#271

Code

Open

norlandrhagen wants to merge 119 commits intomainzarr-developers/VirtualiZarr:mainfrom zarr_readerzarr-developers/VirtualiZarr:zarr_readerCopy head branch name to clipboard

+571-452

Commits

Commits on Oct 24, 2024

• wip toward zarr v2 reader

  norlandrhagen

  committed
  26a94df

  Copy full SHA for 26a94df

• removed _ARRAY_DIMENSIONS and trimmed down attrs

  norlandrhagen

  committed
  cfb7b8d

  Copy full SHA for cfb7b8d

• WIP for zarr reader

  norlandrhagen

  committed
  2f26f03

  Copy full SHA for 2f26f03

• adding in the key piece, the reader

  norlandrhagen

  committed
  eab87a6

  Copy full SHA for eab87a6

Commits on Oct 31, 2024

• virtual dataset is returned! Now to deal with fill_value

  norlandrhagen

  committed
  13db375

  Copy full SHA for 13db375

Commits on Nov 12, 2024

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  cc30ad7

  Copy full SHA for cc30ad7

• Update virtualizarr/readers/zarr.py

  Show description for a047ff9
  

  norlandrhagen

  and

  TomNicholas

  authored
  a047ff9

  Copy full SHA for a047ff9

• Merge branch 'zarr_reader' of https://github.com/zarr-developers/VirtualiZarr into zarr_reader

  norlandrhagen

  committed
  072bead

  Copy full SHA for 072bead

Commits on Nov 15, 2024

• replace fsspec ls with zarr.getsize

  norlandrhagen

  committed
  f7c9a3f

  Copy full SHA for f7c9a3f

• lint

  norlandrhagen

  committed
  2024606

  Copy full SHA for 2024606

• wip test_zarr

  norlandrhagen

  committed
  443435b

  Copy full SHA for 443435b

• removed pdb

  norlandrhagen

  committed
  50fd8b5

  Copy full SHA for 50fd8b5

Commits on Nov 19, 2024

• zarr import in type checking

  norlandrhagen

  committed
  d93c932

  Copy full SHA for d93c932

• moved get_chunk_paths & get_chunk_size async funcs outside of construct_chunk_key_mapping func

  norlandrhagen

  committed
  39be1c5

  Copy full SHA for 39be1c5

• added a few notes from PR review.

  norlandrhagen

  committed
  e718240

  Copy full SHA for e718240

• removed array encoding

  norlandrhagen

  committed
  bbcd473

  Copy full SHA for bbcd473

• v2 passing, v3 skipped for now

  norlandrhagen

  committed
  ed9f2b4

  Copy full SHA for ed9f2b4

• added missed staged files

  norlandrhagen

  committed
  db89da7

  Copy full SHA for db89da7

• fixed merge conflicts with main

  norlandrhagen

  committed
  e3d4318

  Copy full SHA for e3d4318

• missing return

  norlandrhagen

  committed
  410b2a3

  Copy full SHA for 410b2a3

• add network

  norlandrhagen

  committed
  8a69963

  Copy full SHA for 8a69963

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3fca8e6
  pre-commit-ci[bot]

  committed
  3fca8e6

  Copy full SHA for 3fca8e6

• conftest fix

  norlandrhagen

  committed
  34053b0

  Copy full SHA for 34053b0

• naming

  norlandrhagen

  committed
  5c26b1f

  Copy full SHA for 5c26b1f

• comment out integration test for now

  norlandrhagen

  committed
  fb784dc

  Copy full SHA for fb784dc

Commits on Nov 20, 2024

• refactored test_dataset_from_zarr ZArray tests

  norlandrhagen

  committed
  0444fd4

  Copy full SHA for 0444fd4

• adds zarr v3 req opt

  norlandrhagen

  committed
  66fd456

  Copy full SHA for 66fd456

• zarr_v3 decorator

  norlandrhagen

  committed
  13fce09

  Copy full SHA for 13fce09

• add more tests

  norlandrhagen

  committed
  c36962d

  Copy full SHA for c36962d

Commits on Nov 21, 2024

• wip

  norlandrhagen

  committed
  4be4906

  Copy full SHA for 4be4906

• adds missing await

  norlandrhagen

  committed
  ca5ff32

  Copy full SHA for ca5ff32

• more tests

  norlandrhagen

  committed
  88cbeca

  Copy full SHA for 88cbeca

• wip

  norlandrhagen

  committed
  1fbdc9c

  Copy full SHA for 1fbdc9c

• wip on v3

  norlandrhagen

  committed
  370621f

  Copy full SHA for 370621f

• add note + xfail v3

  norlandrhagen

  committed
  9bb0653

  Copy full SHA for 9bb0653

• tmp run network

  norlandrhagen

  committed
  7e03ea5

  Copy full SHA for 7e03ea5

• revert

  norlandrhagen

  committed
  5c1e331

  Copy full SHA for 5c1e331

Commits on Nov 22, 2024

• update construct_virtual_array ordering

  norlandrhagen

  committed
  9404625

  Copy full SHA for 9404625

Commits on Dec 3, 2024

• merge

  norlandrhagen

  committed
  1a5a960

  Copy full SHA for 1a5a960

• updated ABC after merge

  norlandrhagen

  committed
  cc7d68c

  Copy full SHA for cc7d68c

Commits on Dec 9, 2024

• wip

  norlandrhagen

  committed
  ac105ea

  Copy full SHA for ac105ea

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  7b57bd0

  Copy full SHA for 7b57bd0

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for ff01c92
  pre-commit-ci[bot]

  committed
  ff01c92

  Copy full SHA for ff01c92

Commits on Dec 10, 2024

• working for v2 and v3, but only local

  norlandrhagen

  committed
  4f2470a

  Copy full SHA for 4f2470a

• merge

  norlandrhagen

  committed
  0c1ff82

  Copy full SHA for 0c1ff82

Commits on Dec 11, 2024

• cleanup test_zarr reader test

  norlandrhagen

  committed
  05d4050

  Copy full SHA for 05d4050

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for f40ba28
  pre-commit-ci[bot]

  committed
  f40ba28

  Copy full SHA for f40ba28

Commits on Dec 12, 2024

• cleanup after zarr-python issue report

  norlandrhagen

  committed
  b5fb802

  Copy full SHA for b5fb802

• Merge branch 'zarr_reader' of https://github.com/zarr-developers/VirtualiZarr into zarr_reader

  norlandrhagen

  committed
  be5280f

  Copy full SHA for be5280f

Commits on Dec 16, 2024

• temp disabled validate_and_normalize_path_to_uri due to issue in zarr-python v3: zarr-developers/zarr-python#2554

  norlandrhagen

  committed
  690ffee

  Copy full SHA for 690ffee

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  98600e7

  Copy full SHA for 98600e7

• marked zarr integration test skipped b/c of zarr-v3 and kerchunk incompatability

  norlandrhagen

  committed
  31a1b94

  Copy full SHA for 31a1b94

Commits on Dec 17, 2024

• fixes some async behavior, reading from s3 seems to work

  norlandrhagen

  committed
  795c428

  Copy full SHA for 795c428

• lint + uri_fmt

  norlandrhagen

  committed
  c0004c6

  Copy full SHA for c0004c6

• adds to releases.rst

  norlandrhagen

  committed
  60b8912

  Copy full SHA for 60b8912

• nit

  norlandrhagen

  committed
  8240997

  Copy full SHA for 8240997

• cleanup, comments and nits

  norlandrhagen

  committed
  816e696

  Copy full SHA for 816e696

• progress on mypy

  norlandrhagen

  committed
  31aacf9

  Copy full SHA for 31aacf9

• make mypy happy

  norlandrhagen

  committed
  5d14b20

  Copy full SHA for 5d14b20

Commits on Dec 18, 2024

• adds option for AsyncArray to _is_zarr_array

  norlandrhagen

  committed
  fb844b6

  Copy full SHA for fb844b6

Commits on Dec 19, 2024

• big async rewrite

  norlandrhagen

  committed
  421f53f

  Copy full SHA for 421f53f

• merge w/ main

  norlandrhagen

  committed
  cedad11

  Copy full SHA for cedad11

• fixes merge conflict

  norlandrhagen

  committed
  1c5e42d

  Copy full SHA for 1c5e42d

• bit of restructure

  norlandrhagen

  committed
  89d8555

  Copy full SHA for 89d8555

• nit

  norlandrhagen

  committed
  c1a5218

  Copy full SHA for c1a5218

Commits on Dec 20, 2024

• WIP on ChunkManifest.from_arrays

  norlandrhagen

  committed
  6af84b4

  Copy full SHA for 6af84b4

Commits on Dec 21, 2024

• v2/v3 c chunk fix + build ChunkManifest from numpy arrays

  norlandrhagen

  committed
  349386f

  Copy full SHA for 349386f

• removed method of creating ChunkManifests from dicts

  norlandrhagen

  committed
  c776ab9

  Copy full SHA for c776ab9

• cleanup

  norlandrhagen

  committed
  fb6fff7

  Copy full SHA for fb6fff7

• adds xfails to TestOpenVirtualDatasetZarr due to local filesystem zarr issue

  norlandrhagen

  committed
  87c74d4

  Copy full SHA for 87c74d4

Commits on Jan 9, 2025

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  9e44a8a

  Copy full SHA for 9e44a8a

• some nits after merging w/ main

  norlandrhagen

  committed
  87dbdae

  Copy full SHA for 87dbdae

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 855fb5a
  pre-commit-ci[bot]

  committed
  855fb5a

  Copy full SHA for 855fb5a

• updates zarr v3 req

  norlandrhagen

  committed
  29434f1

  Copy full SHA for 29434f1

• Merge branch 'zarr_reader' of https://github.com/zarr-developers/VirtualiZarr into zarr_reader

  norlandrhagen

  committed
  0dcfc91

  Copy full SHA for 0dcfc91

• lint

  norlandrhagen

  committed
  5f7040c

  Copy full SHA for 5f7040c

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for d3b0a92
  pre-commit-ci[bot]

  committed
  d3b0a92

  Copy full SHA for d3b0a92

• remove build_chunk_manifest_from_dict_mapping function since manifest are build from np.ndarrays

  norlandrhagen

  committed
  4716114

  Copy full SHA for 4716114

• Merge branch 'zarr_reader' of https://github.com/zarr-developers/VirtualiZarr into zarr_reader

  norlandrhagen

  committed
  32f7060

  Copy full SHA for 32f7060

• tmp ignore lint

  norlandrhagen

  committed
  dc6e6f8

  Copy full SHA for dc6e6f8

• remove zarr fill_value skip

  norlandrhagen

  committed
  9db4339

  Copy full SHA for 9db4339

• fixes network req import in test_integration

  norlandrhagen

  committed
  4e0fb99

  Copy full SHA for 4e0fb99

• bump xarray to 2025.1.1 and icechunk to 0.1.0a10 in upstream

  norlandrhagen

  committed
  72ae8b0

  Copy full SHA for 72ae8b0

• merge w/ dep bump

  norlandrhagen

  committed
  177f2cf

  Copy full SHA for 177f2cf

• move zarr import into type checking

  norlandrhagen

  committed
  d61e593

  Copy full SHA for d61e593

• move zarr import in test_zarr

  norlandrhagen

  committed
  9edf706

  Copy full SHA for 9edf706

• adding back in missing nbytes property

  norlandrhagen

  committed
  3e68537

  Copy full SHA for 3e68537

• typing

  norlandrhagen

  committed
  594d4a8

  Copy full SHA for 594d4a8

Commits on Jan 10, 2025

• tmp testing & removing old xfail

  norlandrhagen

  committed
  3c6dc54

  Copy full SHA for 3c6dc54

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for dd20c8a
  pre-commit-ci[bot]

  committed
  dd20c8a

  Copy full SHA for dd20c8a

Commits on Jan 27, 2025

• merge w/ main

  norlandrhagen

  committed
  9be9455

  Copy full SHA for 9be9455

• adds back in validate_and_normalize_path_to_uri after upstream zarr fix & vendors concurrent map from zarr-python

  norlandrhagen

  committed
  dac6c77

  Copy full SHA for dac6c77

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3d230dc
  pre-commit-ci[bot]

  committed
  3d230dc

  Copy full SHA for 3d230dc

Commits on Jan 28, 2025

• removing kerchunk from zarr integration test

  norlandrhagen

  committed
  62ffe02

  Copy full SHA for 62ffe02

• Merge branch 'zarr_reader' of https://github.com/zarr-developers/VirtualiZarr into zarr_reader

  norlandrhagen

  committed
  843c286

  Copy full SHA for 843c286

• removed zarr manifest + lint

  norlandrhagen

  committed
  231b703

  Copy full SHA for 231b703

Commits on Jan 29, 2025

• wip on testing

  norlandrhagen

  committed
  0d4d653

  Copy full SHA for 0d4d653

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 7724969
  pre-commit-ci[bot]

  committed
  7724969

  Copy full SHA for 7724969

• revert min-deps change

  norlandrhagen

  committed
  a8c498a

  Copy full SHA for a8c498a

• merge

  norlandrhagen

  committed
  9684dd3

  Copy full SHA for 9684dd3

• Merge branch 'zarr_reader' of https://github.com/zarr-developers/VirtualiZarr into zarr_reader

  norlandrhagen

  committed
  9fb201d

  Copy full SHA for 9fb201d

• revert environment.yaml

  norlandrhagen

  committed
  e310f04

  Copy full SHA for e310f04

• removed zarr manifest writing

  norlandrhagen

  committed
  800a165

  Copy full SHA for 800a165

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  6609ae3

  Copy full SHA for 6609ae3

• cleanup and consolidation in zarr reader

  norlandrhagen

  committed
  ac538c7

  Copy full SHA for ac538c7

• typing

  norlandrhagen

  committed
  3be76ff

  Copy full SHA for 3be76ff

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  a74047c

  Copy full SHA for a74047c

• test_unsupported_zarr_python to zarr v3

  norlandrhagen

  committed
  1e60835

  Copy full SHA for 1e60835

• rel path issue?

  norlandrhagen

  committed
  5d91679

  Copy full SHA for 5d91679

• revert accidental icechunk commit

  norlandrhagen

  committed
  5084adf

  Copy full SHA for 5084adf

Commits on Feb 3, 2025

• merge w/ main

  norlandrhagen

  committed
  136cc2f

  Copy full SHA for 136cc2f

Commits on Feb 4, 2025

• wip on fixing codecs

  norlandrhagen

  committed
  e7a36d7

  Copy full SHA for e7a36d7

• cleaup of tests + codecs

  norlandrhagen

  committed
  df5a19e

  Copy full SHA for df5a19e

• Merge branch 'main' into zarr_reader

  norlandrhagen

  committed
  5f88589

  Copy full SHA for 5f88589

• renived test_zarr writer

  norlandrhagen

  committed
  efd0064

  Copy full SHA for efd0064

• bumping icechunk for now

  norlandrhagen

  committed
  442e519

  Copy full SHA for 442e519

• typing lint

  norlandrhagen

  committed
  54308fb

  Copy full SHA for 54308fb

• remove zarr writer test

  norlandrhagen

  committed
  973f6b0

  Copy full SHA for 973f6b0

• merge w/ main

  norlandrhagen

  committed
  f1c6c7d

  Copy full SHA for f1c6c7d