Skip to content

Commit

Permalink
fix: N5MetadataUtils make parseWholeTree arg do something
Browse files Browse the repository at this point in the history
  • Loading branch information
@bogovicj
bogovicj committed May 29, 2024
1 parent 6e5826e commit 75a5866
Showing 1 changed file with 13 additions and 10 deletions.
23 changes: 13 additions & 10 deletions src/main/java/org/janelia/saalfeldlab/n5/universe/N5MetadataUtils.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ public static void resetGroupParsers() {

/**
* Adds a parser to the current default list.
*
* @param parser the parser to add
*
* @param parser the parser to add
* @param append appends if true, prepends otherwise
*/
public static void addDefaultParser(N5MetadataParser<?> parser, boolean append) {
Expand All @@ -74,8 +74,8 @@ public static void addDefaultParser(N5MetadataParser<?> parser) {

/**
* Adds a group parser to the default list.
*
* @param groupParser the parser to add
*
* @param groupParser the parser to add
* @param append appends if true, prepends otherwise
*/
public static void addDefaultGroupParser(N5MetadataParser<?> groupParser, boolean append) {
Expand All @@ -88,7 +88,7 @@ public static void addDefaultGroupParser(N5MetadataParser<?> groupParser, boolea

/**
* Prepends to the default group parser list.
*
*
* @param groupParser the group parser to add
*/
public static void addDefaultGroupParser(N5MetadataParser<?> groupParser) {
Expand All @@ -109,11 +109,14 @@ public static N5Metadata parseMetadata(final N5Reader n5, final String group, fi
public static N5Metadata parseMetadata(final N5Reader n5, final String group, final boolean parseWholeTree,
final ExecutorService exec, final List<N5MetadataParser<?>> parsers, final List<N5MetadataParser<?>> groupParsers) {

final N5TreeNode root = parseMetadataTree(n5, group, exec, parsers, groupParsers);
if (root == null)
final String groupToParse = parseWholeTree ? "" : group;
final N5TreeNode node = parseMetadataTree(n5, groupToParse, exec, parsers, groupParsers);

This comment has been minimized.

Sign in to view

Copy link
@tischi

tischi May 29, 2024 •
edited
Loading

Hi @bogovicj, could you explain this?

To me this reads as if still the whole metadata tree below group is parsed... Is that true? For the OME-Zarr use case one sometimes only need to parse the .zattrs directly at the group but doesn't want to parse into any subfolders.

This comment has been minimized.

Sign in to view

Copy link
@tischi

tischi May 29, 2024 •
edited
Loading

For example there can be:

image/.zattrs
image/0/.zattrs
image/1/.zattrs
image/2/.zattrs
image/labels/.zattrs

Where I am only interested in the image/.zattrs

This comment has been minimized.

Sign in to view

Copy link
@bogovicj

bogovicj May 30, 2024 •
edited
Loading

Author Collaborator

To me this reads as if still the whole metadata tree below group is parsed... Is that true?

Yes. that's what it does.

For the OME-Zarr use case one sometimes only need to parse the .zattrs directly at the group but doesn't want to parse into any subfolders.

I agree, but to make that parsing pattern play nicely with other infrastructure would take a big refactoring that I don't have time for at the moment.

The discoverer is for situations where the application doesn't know what is in the container and what metadata type it may run into. Situations that know more (like if you know the data is ome-zarr specifically) could make an instance of the correct parser and use that directly.

This comment has been minimized.

Sign in to view

Copy link
@tischi

tischi May 30, 2024

I don't understand why you can't use...

            N5DatasetDiscoverer n5DatasetDiscoverer = new N5DatasetDiscoverer(
                    n5,
                    Executors.newCachedThreadPool(),
                    Arrays.asList( DEFAULT_PARSERS ),
                    Arrays.asList( DEFAULT_GROUP_PARSERS )
            );
            N5Metadata metadata = n5DatasetDiscoverer.parse( group ).getMetadata();

...for parseWholeTree == false ?

I tested it and it seems to work. However, for me, only for the single scale metadata, I don't get it to work for the "root" multi-scale metadata.

Am I doing something wrong?

Here's example code:

        String uri = "https://s3.embl.de/i2k-2020/platy-raw.ome.zarr";
        N5URI n5URI = new N5URI( uri );
        String containerPath = n5URI.getContainerPath();
        N5Factory n5Factory = new N5Factory();
        n5Factory.s3UseCredentials( new AnonymousAWSCredentials() );
        N5Reader n5 = n5Factory.openReader( containerPath );

        N5DatasetDiscoverer n5DatasetDiscoverer = new N5DatasetDiscoverer(
                n5,
                Executors.newCachedThreadPool(),
                Arrays.asList( DEFAULT_PARSERS ),
                Arrays.asList( DEFAULT_GROUP_PARSERS )
        );
        // works:
        N5Metadata metadata = n5DatasetDiscoverer.parse( "s3" ).getMetadata();
        System.out.println( metadata );

        // returns null:
        metadata = n5DatasetDiscoverer.parse( "/" ).getMetadata();
        System.out.println( metadata );
        metadata = n5DatasetDiscoverer.parse( "labels/cells" ).getMetadata();
        System.out.println( metadata );
if (node == null)
return null;
else if (parseWholeTree) // the node returned is the root, so get the correct descendant
return node.getDescendant(group).map(N5TreeNode::getMetadata).orElse(null);
else
return root.getDescendant(group).map(N5TreeNode::getMetadata).orElse(null);
return node.getMetadata();
}

public static N5TreeNode parseMetadataTree(final N5Reader n5) {
Expand All @@ -123,7 +126,7 @@ public static N5TreeNode parseMetadataTree(final N5Reader n5) {

public static N5TreeNode parseMetadataTree(final N5Reader n5, final String group) {

return parseMetadataTree(n5, Executors.newCachedThreadPool(), PARSERS, GROUP_PARSERS);
return parseMetadataTree(n5, group, Executors.newCachedThreadPool(), PARSERS, GROUP_PARSERS);
}

public static N5TreeNode parseMetadataTree(final N5Reader n5, final ExecutorService exec) {
Expand Down Expand Up @@ -158,7 +161,7 @@ public static N5TreeNode parseMetadataTree(final N5Reader n5, final String group
final N5DatasetDiscoverer disc = new N5DatasetDiscoverer(n5, exec, parsers, groupParsers);
try {
return disc.discoverAndParseRecursive(group);
} catch (IOException e) {}
} catch (final IOException e) {}

return null;
}
Expand Down

0 comments on commit 75a5866

Please sign in to comment.