Differentiate plugin types in the search

[haesleinhuepf]

Hi all,

first of all congratulations to the napari hub launch! I have a feature request: Would it be possible to differentiate plugin types? E.g. I would like to only see processing plugins in the result list only, and no reader/writer plugins. Maybe that is already possible? :-)

Thanks!

Best,
Robert

[neuromusic]

Hi Robert! Thank you and thanks for reaching out with one of our first feature requests 😄

This is not currently possible, but we are planning to support categories & tags for plugins by the end of the year and this is a use case that I'd love for us to be able to support. @liaprins-czi is leading the information design work here.

As a plugin developer yourself, how do you envision specifying whether your plugin is a processing plugin vs a reader/writer plugin? Would you prefer to be explicit and specify this in an additional metadata field in .napari/config.yml? or would you prefer if the hub tagged plugins by inspecting which hooks are implemented? Or something else?

[haesleinhuepf]

would you prefer if the hub tagged plugins by inspecting which hooks are implemented

If this is technically possible, I would prefer that way. In very general, meta-data has the highest quality if it can be determined automatically. I'm looking forward to this Christmas present ;-)

[neuromusic]

In the coming weeks, we want to add this feature, letting users filter plugins by the type of plugin (i.e. which hook specs they've implemented). One challenge is that some terms (like "dock widget") aren't intuitive to potential users (this came up in our UXR work around adding more metadata fields)

The current proposal would offer the following Plugin Types

• Extension (for napari_experimental_provide_dock_widget hooks, currently called "GUI hooks" in plugin docs)
• Analysis (for napari_experimental_provide_function hooks, currently called "Analysis" in plugin docs)
• File Reader (napari_get_reader)
• File Writer (napari_get_writer,napari_write_*)
• Sample Data (napari_provide_sample_data)
• Theme (napari_provide_theme)

Curious what people think about this breakdown, especially re: calling dock widgets "Extensions" and whether magicgui-enabled "Analysis" dock widgets should get their own checkbox (or should these just be "Extensions", too?)

Note: Later, we're going to add more categories for processing steps, like "Segmentation" and "Registration", so presumably either an "Extension" or "Analysis" plugin type could fall under the "Segmentation" category.

[haesleinhuepf]

That's great news @neuromusic ! Can you maybe share a preliminary list of plugins and in which category they would fall? I speculate that the current "analysis" and "extensions" provide similar functions to the end user. It may make sense to put them together in one category (e.g. "Processing / analysis") and make it more fine granular later when plugins can have something like tags... Again, a prelim. list would be nice to take a look at

[neuromusic]

Can you maybe share a preliminary list of plugins and in which category they would fall?

That's a good idea!

It may make sense to put them together in one category (e.g. "Processing / analysis") and make it more fine granular later when plugins can have something like tags

These would be independent "dimensions": Plugin Type, which would be mapped to the hook specs, plus other dimensions (subsets of EDAM) to capture the details of what the plugin does.

[tlambert03]

as mentioned in zulip, I'm curious how you're going to determine plugin type?

Id very much rather this not go in .napari/config.yml (in general, I'd like to see that file disappear: what it offers is almost entirely duplicated elsewhere, outside of ORCID id).

This was indeed a major part of the inspiration for the plugin manifests, so it would be nice not to implement another spec that we'll need to deprecate and monitor very soon

[neuromusic]

(and later, the manifest, but we'll probably want to maintain backwards compatibility for some time)

[tlambert03]

how? will you install the package statically for all decorators? or will you actually install all of these packages into a napari environment? keep in mind that this is very dynamic. So this would absolutely work as a GUI plugin:

# my_package/some_module.py
from napari_plugin_engine import napari_hook_implementation as nhi

@nhi(specname='napari_experimental_provide_dock_widget')
def get_widget():
    return QWidget

#my_package/__init__.py
from some_module import get_widget

# setup.cfg
[options.entry_points]
napari.plugins = 
    my_plugin = my_package 

but this would be very hard to determine by inspecting the package.

problems here:
A) you can't just look at the module that the entry point is pointing to, because it may import specs from elsewhere
B) even if you statically analyze the entire package, names are a problem because the hook doesn't need to be named after the hookspec, nor does the napari_hook_implementation decorator need to have that exact name.

edit: one thing that might work is to just parse the full source code for the strings napari_experimental_provide_dock_widget, etc... It's not a guarantee, but it's probably safe(ish).

[tlambert03]

so basically, you'd need to download and install every package, then trigger discovery (which happens lazily depending on the type of hookspec), and then ask the plugin manager what got loaded.

[tlambert03]

so I really think it'd be worth waiting for the manifest on this one

[neuromusic]

the "how" is TBD (we're focused on clarifying the design and requirements for this feature right now), but my assumption is that we'll do something akin to the dynamic method you outlined: when we see a new plugin on PyPI, we pull it down, install it, and determine the hookspecs dynamically. but the "how" will be determined once we start the engineering work for this feature.

[tlambert03]

Analysis (for napari_experimental_provide_function hooks, currently called "Analysis" in plugin docs) ... whether magicgui-enabled "Analysis" dock widgets should get their own checkbox (or should these just be "Extensions", too?)

this one is so poorly used and defined... so I'd definitely agree with just wrapping this into whatever you call dock widgets for now. (It currently does literally nothing more than the dock widget)

[sofroniewn]

+1 here too

[tlambert03]

in general i'm very supportive of this goal (this was a major impetus for the manifest in the first place). I'd definitely prefer that this wait until we have a manifest, and I'd really like for the "design requirements" here to take direct inspiration (and have bearing on) the wording used in the manifest.

I don't see it as urgent, and it will get significantly easier (and less error-prone) in the not too distant future. My only real concern though is that we don't create a protocol and/or terminology that plugin developers have to learn and then unlearn.

So if you want to play with the concept go for it. Please ping us before implementing something public facing.

[neuromusic]

My only real concern though is that we don't create a protocol and/or terminology that plugin developers have to learn and then unlearn.

Agreed

[neuromusic]

Its not clear to me why this is blocked by the manifest: the hub's needs are a bit different from napari's, which means that we can start inferring plugin "type" dynamically, add support for the manifest file once it is defined, then drop the old-fashioned dynamic approach once most/all plugins have updated to use manifest files.

[tlambert03]

yep, agreed. It's not "blocked" by the manifest... that's why I said go for it. I just think it's a decent amount of effort to put in to something that's going to have a very easy and less error-prone solution in the not-too-distant future. But again, as long as plugin developers don't have to do one thing now (i.e. put something in .napari/config.yml) and then a different thing later (i.e. move that info into the manifest), then it's all good by me :)

Basically, if there's possibility for confusing end users based on trying to interpret how what the hub is doing differs from what napari is doing, that concerns me. Beyond that, not at all.

[neuromusic]

ok, that makes sense. part of the reason I don't necessarily want to wait for the manifest is that it will still take time for the community to adopt it.

[tlambert03]

it will still take time for the community to adopt it.

all the more of a carrot! :)

[neuromusic]

I like carrots :D

[sofroniewn]

One option is that we don't include hookspecs currently called experimental, i.e. napari_experimental_provide_dock_widget, napari_experimental_provide_function, and napari_eperimental_provide_theme.

If we did I might do it as

• Extension (for napari_experimental_provide_dock_widget hooks, currently called "GUI hooks" in plugin docs and napari_experimental_provide_function hooks, currently called "Analysis" in plugin docs)
• Theme (napari_eperimental_provide_theme)

But we could also hold off on those or decide right before feature launch what to toggle on/ off

As for the other three I think

• File Reader (napari_get_reader)
• File Writer (napari_get_writer,napari_write_*)
• Sample Data (napari_provide_sample_data)

Are pretty reasonable and unlikely to dramatically change. Not sure if there is any concern about File * as are these things sometimes reading things more complicated that files, like folders/ urls/ remote datastores. Exact name could also get decided later.

As the to technical difficulties of automating/ inspecting the package, we'll just see how it goes. I think if we get 90% of things right then it's ok. If folks are doing something really different that we don't catch then that's fine. If this ends up being a complete nightmare and we're only getting 50% then maybe it would make sense to delay role out to manifest, but we'll have at least made a lot of progress on the rest of the feature implementation before then.

As to deprecation and migration I think that'll have to happen anyway - this actually might be really helpful during that process as it will let us know exactly which plugins need to be migrated in which ways!

[neuromusic]

Not sure if there is any concern about File * as are these things sometimes reading things more complicated that files, like folders/ urls/ remote datastores.

I wondered about this, too... "Data *" might be better?

[neuromusic]

Also just to note: from the users perspective, these things then show up in the "Open..." and "Save..." menus, so "Read" and "Write" might not be very intuitive. But we can change that on the frontend in the future without burdening plugin devs, so I think its low risk to go with terminology that's close to the napari plugin API for now.