Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BioImage.IO Meeting Minutes #28

Open
oeway opened this issue Jun 19, 2020 · 165 comments
Open

BioImage.IO Meeting Minutes #28

oeway opened this issue Jun 19, 2020 · 165 comments
Assignees
@cfusterbarcelo

Comments

@oeway
Copy link
Collaborator

oeway commented Jun 19, 2020 •
edited
Loading

Hi,

The BioImage.IO team will meet every Wednesday 4PM CET, check the next meeting in your time zone.

Meeting Format

As discussed during in the AI4Life general assembly meeting (Oct. 10th, 2023), we will make the meeting more structured. For the first 15 minutes, we will take turns and having one presentation every week from one of the AI4Life partners, or guest speakers from community partners, developer etc.

The aim of the first 15-minute presentation is to provide community-friendly (especial for users and developers outside the AI4Life consortium) updates on new features, changes, announcements etc.

And the later part, we will have regular internal discussions or other relevant topics. Everyone is welcome to join the meeting via zoom: https://kth-se.zoom.us/j/65777152331, feel free to add your name and topic to the meeting agenda.
@oeway oeway mentioned this issue Jun 19, 2020
Closed
@oeway
Copy link
Collaborator Author

oeway commented Jun 26, 2020 •
edited
Loading

June 26th, 2020

Attendees

Name affiliation username
Wei KTH @oeway
Esti UC3M @esgomezm
Hao KTH @haoxusci
Arrate UC3M @ArrateMunoz
Anna EMBL @akreshuk
Fynn EMBL @FynnBe
Maxim EMBL @m-novikov

Agenda + Minutes

  • All: general intro and your plans for the zoo in the near future

  • [name=Wei]:

    • updates on the new bioimage.io website
      • rebuild the website with a build system and CI
      • New UI design powered by Bulma
      • Support models, applications, notebooks, datasets
      • Support badges
      • Share the site template with other projects: covid19ai.io, shareloc.xyz etc.
      • Building backend (imjoy-ai-engine) for supporting uploading datasets and online inference service

  • [name=Anna] Technical question: single model with multiple weights, trained in different ways

  • [name=Anna] Other potential collaborators: ZeroCostDL4Mic

  • Discussion

    • How to support linking datasets with model weights and describe the training
    • Extend the model description spec, also for describing datasets, notebooks, applications

Note

  • How to group different cards
    • if only weights changed == same card
    • If postprocessing changed == different card

@constantinpape constantinpape mentioned this issue Jul 7, 2020
Closed
@oeway
Copy link
Collaborator Author

oeway commented Jul 10, 2020 •
edited
Loading

July 10th, 2020

Attendees

Name affiliation username
Wei KTH @oeway
Esti UC3M @esgomezm
Daniel EPFL @dasv74
Anna EMBL @akreshuk
Carlos UC3M @carlosuc3m
Maksim Novikov EMBL @m-novikov
Ricardo Henriques IGC/UCL @paxcalpt

Agenda + Minutes

  • [name=Wei]

    • Updates for the website
      • resource description file
      • support links between items (apps to notebooks, dataset etc.)
      • support pagination
      • add ZeroCostDL4Mic and HPA as a community partner
      • comment feature powered by github
      • community partner
        • what's the criterion and procedure to join as a community partner?
          • Should we allow their own model zoo? (Think image.sc)
          • Open source? comercial partners?

            • Anna: accessible dataset, source of the model

            • Esti: published model without close dataset

            • Wei: sensitive medical data

            • Ricardo & Wei: Use badges to indicate and encourage

            • conclusion: give recommendation and show badges

          • Allow at least one admin of bioimage.io to access the repo?
            • conclusion: agreed
        • what kind of role and responsibility does the community partner have?
          • Contribute and maintain models/applications/notebooks
          • Accept third-party contribution if relevant? Say ilastik will in charge of all PRs for the ilastik models?
            • Anna: for long term, we should see more interoperatable models and that should go to BioImage.IO
            • for dedicated models that works for only one community partner, contribute to the partner instead of BioImage.IO

  • [name=Anna]

    • What examples do we need to add to make it easy for people to understand how to run models?

    • We should add a general "About" page describing what and why

      • Wei: Will do, this already been added for the community partners

  • [name=Daniel]

    • Demo of Model manager in DeepImageJ
    • Suggestions by Wei:
      • Add bug report button to link the same github issue as on BioImage.IO
      • Show linked items

  • [name=Esti]

    • Good practices uploading models; i.e. authorship & responsabilities
      • Anna & Ricardo: cite and acknowledge properly, typically it's fine to upload models that are derived from others' work
      • Wei & Ricardo: follow the license, most license will include terms about attribution

  • Discussion

    • What kind of users we are targeting with BioImage.IO

    • What's scope and focus of the bioimage.io platform? Should we only provide models, or including also the downstream/upstream utility tools for building workflow. The layout of the website.

    • How to upload a model compatible with different software? Same card / id besides proper reference?

      • Wei: use CI to automatically test the software compatibility, using BioEngine api
      • Anna: These tests should be done automatically

  • Suggestions

    • Daniel: Allow users to post images to the github issue for each model
    • Esti: support linking to image.sc
    • Ricardo: Network view for navigation

This was referenced Jul 10, 2020
Open
Open
@esgomezm
Copy link
Contributor

esgomezm commented Aug 6, 2020

July 24th, 2020

Attendees

Name affiliation username
Esti UC3M @esgomezm
Maksim Novikov EMBL @m-novikov
Arrate UC3M @amunoz
Constantin EMBL @constantinpape
Anna EMBL @akreshuk

Minutes

Discussion

  • What’s the distinction between models and notebooks
    -- DeepImageJ: Notebooks and Models are actually separate
    -- Notebook is used for training the model
    -- models are then only the train
    -- more general, how do we link models, notebooks (e.g. used to train the model) and datasets (e.g. where model was trained on)

  • datasets:
    -- how do we reflect “small” changes to the data-sets, e.g. fixing issues in the ground-truth
    -- public datasets can be “easily” added
    -- what about the private datasets?
    -- change the name? maybe "working data", "training data"

  • easier way to explore the models apart from search, ideas:
    -- “structured tag cloud”: selecting multiple tags to find a model that fits the “segmentation”, “electron-microscopy”, “membranes”, …

  • Discussion about the CZI Open Software call
    https://chanzuckerberg.com/rfa/essential-open-source-software-for-science/
    DeepImageJ is considering to apply; question is if/how to integrate the BioImage.io in there.
    Including Demo resources.
    Ilastik is also applying for the call, but for a different purpose?
    Are other community partners applying?
    If so, it would be nice if all could mention the model zoo.
    bioimage.io is not yet mature enough to apply on its own

  • once the website is more stable, we could start to write user facing docs
    a good example would be integration of the different community partners, e.g. train model in a ZeroCost notebook and run it with another community partner software

  • started discussion on interoperability between tools by community partners; big issue is (still :)) the different pre and post-processing steps that are implemented in a language specific manner. For now, porting between tools will probably be done manual by a developer of that tool.

  • When to change bioimage.io as landing page / reference model page for all community partners (e.g. for DeepImageJ, CSBDeep, ZeroCost home pages)? Then we could announce on image.sc etc. E.g. First of November. Discuss with Wei. Need to discuss with other community partners.

  • Citation discussion; how many citations to put in

Suggestions for the website

How do we make the website more intuitive for users?

Name-changes:

  • Change the name "Datasets" to "Training Data"; this should be clear for users.
  • Change the name "Models" to "Preconfigured Models"
  • Change the name "Notebooks" to "Code"

Could also go with more actionable names:

  • "Models" to "Find Models"
  • "Applications" to "Preview" or "Try on your data"?
  • "Notebook" to "Find Code"
  • "Datasets" to "Find Data"
  • Get rid of the "Application" button and just provide "Run" or "Demo" button on the model windows

Icons on the Model and Notebook Cards are different for the each Model / Network. This is confusing.

@oeway
Copy link
Collaborator Author

oeway commented Aug 21, 2020

BioImage.IO Meeting Minutes

August 21st, 2020

Attendees

Name affiliation username
Wei Ouyang KTH @oeway
Emma Lundberg KTH emmalundberg
Ricardo Henriques UCL / IGC @paxcalpt
Guillaume Jacquemet ÅAU/UTU @guijacquemet
Anna EMBL @akreshuk

Minutes

  • planning for the release of Bioimage.IO
    • release date: November 1st
    • features to offer by then:
      • display plain model cards for download
      • support interoperatability? @emma high priority
        • a list of useful/popular model, make sure it compatible with a few software. (show complementary with software rather than competition)
          • pytorch model (segmentation): ilastik, imjoy, zero
          • tensorflow model (denoising): zero, fiji, deepimagej, imjoy
          • classification: different types
          • starting from a training dataset? task?
      • use outline color encoding for types @anna
      • show text description for tags
      • online inference (single model, save it for the release)
      • model searching application (lower priority)
      • integration with colab, binder
      • community parties
        • guidelines for the community parter
        • short recommendation from all the community partner
        • feature models, sorting, popularity, model searching (lower priority)
      • Group the software by three types of tasks: 1) preprocessing/ annotation 2) training / transfer learning 3) prediction. Click a model, show a list of software grouped by tasks types
      • Improve 2h rendering interval @ricardo
      • parser script should validate the link, valide the fields, cover image size, add badges

@oeway oeway mentioned this issue Sep 4, 2020
Merged
1 task
@oeway
Copy link
Collaborator Author

oeway commented Sep 4, 2020

September 4th, 2020

Attendees

Name affiliation username
Wei Ouyang KTH @oeway
Constantin Pape EMBL @constantinpape
Anna Kreshuk EMBL @akreshuk
Maksim Novikov EMBL @m-novikov
Fynn Beuttenmüller EMBL @FynnBe
Ricardo Henriques IGC/UCL @HenriquesLab
Estibaliz Gómez UC3M @esgomezm

Agenda

  • Wei
    • Outline color encoding
    • Zarr dataset and Vizarr
    • ImageJ.JS
    • discussion
      • Run DeepImageJ models
      • Model conversion service
        • BioEngine Application running on Binder
        • Tensorflow to Tensorflow.JS
        • PyTorch to ONNX to Tensorflow?
      • RPC link between the website and software
        • From the website: Send model from the website to Ilastik/Fiji etc.
        • From the software: launch model browser
        • Example PycroManager
  • Ilastik
    • Specification of training, Application specific?
    • Website header too large?

Minutes

  • Add underline color LUT
  • What models to use as show case
    • 2D Unet / Nuclei Broad
    • 3D Unet
  • Token (url) for linking model across software
  • Roadmap general issue
  • Training:
    • descriptive training procedure is too ambitious
    • instead: partners can support their own training procedures; training APIs can be shared
    • example: ilastik will have a training API that can also be exposed through imjoy RPC so it could be used from the browser

@oeway
Copy link
Collaborator Author

oeway commented Sep 18, 2020

September 18th, 2020

Attendees

Name affiliation username
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Carlos García UC3M @carlosuc3m
Estibaliz Gómez UC3M @esgomezm
Novikov Maksim EMBL @m-novikov
Florian Jug CSBD + HT @fjug
Fynn Beuttenmüller EMBL @FynnBe
Maksim Novikov EMBL @m-novikov

Agenda

  • Roadmap general issue
    • Model runners
    • Improve Landing
    • Documentation
    • animal icon?
    • Run button for each model
    • (Simple) models which can be run in multiple applications. E.g. train a model in a notebook in ZeroCost, run in both DeepImageJ and CSBDeep (or ilastik) (note: deal with output error produced by different framework)
  • Cross-compatibility of models - how much can we guarantee for a model following the specs?
  • Generic model runner (two runners, PyTorch and Tensorflow) to have the same as HPA model runner for all models?
  • What should really be in the "Applications" part?
  • New applications: CellPose, ImageJ.JS, CellPose-ImageJ.

@oeway oeway mentioned this issue Sep 20, 2020
Merged
@oeway
Copy link
Collaborator Author

oeway commented Oct 2, 2020 •
edited
Loading

October 2nd, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Florian Jug CSBD + HT @fjug
Dominik Kutra EMBL @k-dominik
Jan Funke Janelia @funkey
Carlos García UC3M @carlosuc3m
Estibaliz Gómez UC3M @esgomezm
Arrate Muñoz UC3M @arratemunoz

Agenda

  • Florian

    • Pitch the model zoo project

  • Wei

    • Updates on the bioimage.io website

    • BioEngine API for model runners

      api.registerService({
   "type": "model-runner",
   "init": init, // initialize the runner
   "predict": predict, // perform prediction
   "fit": fit // perform training (optional)
})

      Alternatively, we can use scikit-learn style with fit and transform function.

      • ilastik
      • deepimagej
        • support via pyimagej
        • support via imagej.js

    • Demo for a new model: Learn2Sofie

  • Jan

    • Introduction about the live supervised learning platform at Janelia

  • Anna and Wei

    • plan for the November release, implementation of the model runners.

@oeway
Copy link
Collaborator Author

oeway commented Nov 5, 2020

October 30th, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Tom Burke CSBD @tomburke-rse
Florian Jug CSBD + HT @fjug
Deborah Schmidt CSBD @frauzufall
Arrate Muñoz UC3M @arratemunoz
Anna Kreshuk EMBL @akreshuk
Novikov Maksim EMBL @m-novikov
Fynn Beuttenmüller EMBL @FynnBe
Dominik Kutra EMBL @k-dominik
Estibaliz Gómez UC3M @esgomezm

Agenda

  • Discuss the test_inputs and test_outputs format: will be feeded as a list and use npy as the only formats, this will be specifically used for CI testing, but combined with converters, for the users, we can use another field, e.g. test_samples
  • Change normalization to preprocessing, and agreed on several types of preprocessing functions
  • The model file can contain multiple formats of the same weights, the weights is a dictionary which uses a unique format string as the key

See also in the project: https://github.com/orgs/bioimage-io/projects/1

@oeway
Copy link
Collaborator Author

oeway commented Nov 16, 2020

November 13th, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Tom Burke CSBD @tomburke-rse
Florian Jug CSBD + HT @fjug
Deborah Schmidt CSBD @frauzufall
Arrate Muñoz UC3M @arratemunoz
Anna Kreshuk EMBL @akreshuk
Novikov Maksim EMBL @m-novikov
Fynn Beuttenmüller EMBL @FynnBe
Dominik Kutra EMBL @k-dominik
Estibaliz Gómez UC3M @esgomezm
Constantin Pape EMBL @constantinpape
Carlos García UC3M @carlosuc3m

Agenda

  • 0.3.0 project: https://github.com/orgs/bioimage-io/projects/1

  • Authors in weights vs. authors in model

    • 3 parties to give credit to:
      • Who is the creator of the bioimage.io.model.zip?
        The creator in the yaml points that out. The top-level description describes the training setup, purpose, etc.
      • Who is the creator of the type of model?
        The existing list of citations is good enough!
      • Who trained the linked weights?
        If not the same as the creator of the model, names can be given here. Description field is superfluous and needs to be removed here!

  • Weights-formats we support:

    • tensorflow_saved_model_bundle: Zip the weights and the folder with the variables and link to this zip from the yaml-file. (File name: *.zip)
    • keras_hdf5:
    • tensorflow_js: a json file + binary weights files
    • pytorch_state_dict:
    • pytorch_script:

  • Naming convention for the model bundle: *.bioimage.io.model.zip

  • show a table with features (weights format, pre/post processing types) that are implemented by consumers

@FynnBe FynnBe mentioned this issue Nov 20, 2020
Closed
@oeway
Copy link
Collaborator Author

oeway commented Nov 20, 2020

November 20th, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Florian Jug CSBD + HT @fjug
Deborah Schmidt CSBD @frauzufall
Arrate Muñoz UC3M @arratemunoz
Anna Kreshuk EMBL @akreshuk
Novikov Maksim EMBL @m-novikov
Fynn Beuttenmüller EMBL @FynnBe
Dominik Kutra EMBL @k-dominik
Estibaliz Gómez UC3M @esgomezm
Constantin Pape EMBL @constantinpape
Carlos García UC3M @carlosuc3m
Ricardo Henriques IGC & UCL @HenriquesLab

Project: https://github.com/orgs/bioimage-io/projects/1

Agenda

  • Suggested Agenda Points from Dresden
    • I2K - 0.3.0 releases? Our own dialects? What are our goals?
      • Let's make it 0.3.0 and not any of our dialects!
    • More generally: when and how would 0.3.0 count as 'released'? How can we ensure that the released state leaves no margin for interpretation? (Example: how would I find out what parts are optional?)
      • For now, the readme in the repo is the full spec. We will soon release on GitHub the 0.3.0 version and that's it!
    • (Minor?) Pre/-Postprocessing: could axis-concept be more ubiquitously supported? (Example: Pre::zero_mean_unit_variance vs. Post::scale_linear)
      • We will add axis to scale_linear (and maybe more?) after I2K and freeze 0.3.0 in its current form.
    • Input/Output types or class? Example: if you have a model that gets/produces an image and a table of datapoints. How would I know what is what? Where would I specify this best? (Wei: Hints for rendering of inputs/outputs tensors for UI, proposal)
      • Let's start a 0.3.1 branch and get going. Plenum discussion after I2K in December...
  • How do we distinguish between the models that do not need pre/post-processing and the ones that cannot be described with the defined steps?
    • DD thinks: In that case we use the 'source'-tag... (and we do that already for N2V).
  • Metadata of the images and numpy arrays?

Decided:

  • blow up if config key contains anything unfamiliar
  • custom pre / postprocessing should also go into the config

TODO befor releasing 0.3.0:

@oeway
Copy link
Collaborator Author

oeway commented Dec 4, 2020

November 27th, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Deborah Schmidt CSBD @frauzufall
Tom Burke CSBD @tomburke-rse
Arrate Muñoz UC3M @arratemunoz
Fynn Beuttenmüller EMBL @FynnBe
Estibaliz Gómez UC3M @esgomezm
Constantin Pape EMBL @constantinpape
Novikov Maksim EMBL @m-novikov
Carlos García UC3M @carlosuc3m
Daniel Sage EPFL @dasv74

Project: https://github.com/orgs/bioimage-io/projects/1

Agenda

  • What's left to do before releasing 0.3.0?

Outcome

More user-friendly website

  • add a introductory tour with several steps (easy option would be a tutorial video)

Dev documentation

  • improve md based spec documentation

Run torchscript (e.g. models from deepImageJ) in ilastik, (after ilastik can parse 0.3.0)

Autogenerating the spec documents from yamls or spec parser

@oeway
Copy link
Collaborator Author

oeway commented Dec 11, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Tom Burke CSBD @tomburke-rse
Arrate Muñoz UC3M @arratemunoz
Fynn Beuttenmüller EMBL @FynnBe
Estibaliz Gómez UC3M @esgomezm
Constantin Pape EMBL @constantinpape
Novikov Maksim EMBL @m-novikov
Daniel Sage EPFL @dasv74
Florian Jug CSBD + HT @fjug

Agenda

@oeway
Copy link
Collaborator Author

oeway commented Dec 18, 2020

December 18th, 2020

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Anna Kreshuk EMBL @akreshuk
Fynn Beuttenmüller EMBL @FynnBe
Constantin Pape EMBL @constantinpape
Florian Jug CSBD + HT @fjug
Carlos García UC3M @carlosuc3m

Project: https://github.com/orgs/bioimage-io/projects/1

Agenda

We have a very short meeting today, but still quite fruitful for finalising the last pieces left for v0.3.0!

Update to configuration: bioimage-io/spec-bioimage-io#59
Update to update: bioimage-io/spec-bioimage-io#61

Data type and data range: bioimage-io/spec-bioimage-io#55 (comment)

@oeway oeway mentioned this issue Dec 18, 2020
Open
@m-novikov m-novikov pinned this issue Jan 15, 2021
@oeway
Copy link
Collaborator Author

oeway commented Jan 15, 2021

January 15th, 2021

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Daniel Sage EPFL @dasv74
Tom Burke CSBD + MPI-CBG @tomburke-rse
Constantin Pape EMBL @constantinpape
Ricardo Henriques IGC & UCL @HenriquesLab
Estibaliz Gómez UC3M @esgomezm
Arrate Muñoz-Barrutia UC3M @arratemunoz
Jan Philipp Albrecht MDC @jpalbrecht
Deborah Schmidt MDC @frauzufall
Carlos García UC3M @carlosuc3m
Anna Kreshuk EMBL @akreshuk
Fynn Beuttenmüller EMBL @FynnBe
Maksim Novikov EMBL @m-novikov

Agenda

  • News: BioImage Model Zoo featured on Nature news https://www.nature.com/articles/d41586-021-00023-0

  • Documentation

    • User Guide
      • Overview
        • Add the cross-compatibility table
      • Tutorials
        • including videos
      • How-tos Guides
        • How to choose a model for my data
        • How to run this model in community partners' tools
        • How to search by tags
      • Explanations
        • The risk of model mismatch
    • Model Developers
      • Tutorial
      • How-tos Guides
        • How to make a model package
    • Application Developer
      • Tutorial
      • How-tos Guides
        • How to become a community partner
        • How to build a BioEngine App
      • Explainations
        • Responsibilites for community partners
      • API reference

  • Vision

    • Advanced AI models in one-click

  • Roadmap

    • Runners support 0.3.0; tests for runners and models; CI
    • Each model should be associated with at least one model runners
    • Support type info for guide the rendering of inputs/outputs tensors
    • A new tab for hosting scripts? (e.g. Fiji/ImageJ macro, similar to notebooks)
    • RPC interface
      • Support making RPC calls to local software
      • Connect model API servers for support cloud based training and inference (e.g.: cellpose)

  • Timeline / meetings

    • 3 weeks from now (Feb. 5th): compatibility hackathon
      • Ilastik, DeepImageJ, ImJoy, Zero?, Fiji
    • 5 weeks from now (Feb. 23rd): documentation polish
    • end of February: guinea pig users and contributors experiment
    • early March: release!

@oeway
Copy link
Collaborator Author

oeway commented Jan 22, 2021

January 22nd, 2021

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Tom Burke CSBD + MPI-CBG @tomburke-rse
Constantin Pape EMBL @constantinpape
Estibaliz Gómez UC3M @esgomezm
Arrate Muñoz-Barrutia UC3M @arratemunoz
Jan Philipp Albrecht MDC @jpalbrecht
Fynn Beuttenmüller EMBL @FynnBe
Maksim Novikov EMBL @m-novikov

upcoming Hackathon project: https://github.com/orgs/bioimage-io/projects/2

Agenda

  • TODOs before the hackathon

  • Hackathon planning
    Main goal: convert to different formats (e.g.: ONNX)

    • Currently supported model formats:

      • tiktorch (pytorch, torch script, onnx)
      • deepimagej (torch script, tensorflow saved model, keras h5)
      • Fiji (tensorflow saved model)
      • ImJoy (support all types of models with different plugins, including onnx, onnx.js, tensorflow.js)

    • Model cross-compatibility aims:

      • tiktorch <-> deepimagej (torch script)
      • tiktorch -> ImJoy (onnx)
      • ImJoy -> deepimagej (tensorflow saved model)
        Idea: all the consumer convert their model into onnx, and use it as a neutral model representation, which can be further converted into other formats.

    Date: Feb. 5th, 9am - 6pm

left for future agenda

@oeway
Copy link
Collaborator Author

oeway commented Jan 29, 2021

January 29th, 2021

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Tom Burke CSBD + MPI-CBG @tomburke-rse
Constantin Pape EMBL @constantinpape
Estibaliz Gómez UC3M @esgomezm
Arrate Muñoz-Barrutia UC3M @arratemunoz
Jan Philipp Albrecht MDC @jpalbrecht
Fynn Beuttenmüller EMBL @FynnBe
Maksim Novikov EMBL @m-novikov
Constantin Pape EMBL @constantinpape
Carlos García UC3M @carlosuc3m

Project: https://github.com/orgs/bioimage-io/projects/1

upcoming Hackathon: https://github.com/orgs/bioimage-io/projects/2

Agenda

  • Generate CI error report for Models

The idea is to provide a link to the CI status as an additional field in the manifest.bioimage.io.yaml file: {"ci_status": "url"}

Then the link point to a json file contains CI status for the models:

{
    "name": "Ilastik CI",
    "timestamp": "2021-xsssxxx",
    "status": "",
    "error": "",
    "models": {
        "<Model ID>": {"status": "pass", "error": ""},
        
    }
}

The <Model ID> is the same as here: https://github.com/deepimagej/models/blob/master/manifest.bioimage.io.yaml#L38

This json file can be uploaded to a separate branch of the model repo, and this is an example which contains Github Actions configurations for uploading artefacts to another branch: https://github.com/bioimage-io/bioimage-io-models/blob/master/.github/workflows/compile-manifest.yml

The CI status json file will be loaded from the website and rendered as badges for model status.

@oeway
Copy link
Collaborator Author

oeway commented Feb 5, 2021

BioImage.IO Hackathon on Model Cross-compatibility

Feburary 5th, 2021

Attendees

Name affiliation username (github)
Tom Burke CSBD + MPI-CBG @tomburke-rse
Wei Ouyang KTH @oeway
Estibaliz Gómez UC3M @esgomezm
Constantin Pape EMBL @constantinpape
Joran Deschamps MPI-CBG @jdeschamps
Maksim Novikov EMBL @m-novikov
Romain F Laine MRC-LMCB, UCL @Romain-Laine
Guillaume Jacquemet ÅAU @guijacquemet
Carlos García UC3M @carlosuc3m
Florian Jug (dropped by) CSBD + MPI-CBG @fjug

Project: https://github.com/orgs/bioimage-io/projects/1

upcoming Hackathon: https://github.com/orgs/bioimage-io/projects/2

Agenda

  1. Brainstorming on what model/approach to work with
    Please write the potential model/apporach you can potentially work on here:
  • Pytorch - TorchScript compatibility
  • ONNX weight format
  • bundling TensorFlow models directly from python (under development https://github.com/deepimagej/pydeepimagej/tree/bioimage-yaml)
  • linking material from other community partners in manifest.yaml (e.g. ZeroCostDL4Mic notebooks with bundled models)
  • update CSBDeep python and java parts to 0.3.0 + model runner integration into fiji
  • List the versions we are compatible with. e.g. Fiji supports TF until 1.15
  • Import ONNX models to ZeroCostDL4Mic for transfer learning and export models using ONXX for using models in other tools.
  • numpy reader for ImageJ/Fiji(?)

@wei - ImJoy

  • Loading DeepImageJ/FIji models via PyImageJ

  • ONNX.js models

  • Tensorflow.js models

  • tried ONNX.js multiple version: didn't work with the Unet model

@constantinpape - Ilastik

  • onnx models
  • torchscript models

@esti - DeepImageJ

  • tensorflow saved_model.pb
  • Pytorch - torchscript

@tom - Fiji/CSBDeep

  • Tensorflow1 saved models

@joran

  • ONNX version of imagej-tensorflow

@m-novikov

  • running tensorflow models (e.g. U-Net pancreatic cell segmentation)
  1. Work on the models
  1. Debriefing

  • Demo for running ONNX runtime

    • TODO: make docs for loading yaml and do inference using ONNX runtime

  • Changes in the spec 0.3.1 to support multiple version of the same weight format
    Proposal:

    • onnx:
      - sha256: 5bf14c4e65e8601a
      source: ./weights_opset11.onnx
      opset_version: 11
      parent:
      - sha256: 5bf14c4e65e86
      source: ./weights_opset12.onnx
      opset_version: 12
      test_outputs: ["test_output_onnx_opset_12.npy"]
      parent:
    • Dimension ordering for different formats? (e.g. Pytorch vs Tensorflow)
      Proposal: Add transpose_axis for different dimension ordering

  • ONNX version

Framework ONNX opset_version comment
ONNX runtime (Python)
ONNX runtime (Java) 12 version: 1.6.0
ONNX.js 9
onnx2keras
keras2onnx
  • Support Unet in different consumers
Consumers Works? comment
pytorch -> onnx(ilastik) yes except the output border
pytorch -> torchscript(deepimagej) yes except the output border (it works without tiling)
pytorch -> onnx(ONNX.js) No blocked
onnx -> keras (zero) No blocked
keras -> onnx (zero) ?
Fiji ?

@oeway
Copy link
Collaborator Author

oeway commented Feb 12, 2021

Feburary 12th, 2021

Attendees

Name affiliation username (github)
Tom Burke CSBD + MPI-CBG @tomburke-rse
Wei Ouyang KTH @oeway
Estibaliz Gómez UC3M @esgomezm
Constantin Pape EMBL @constantinpape
Joran Deschamps MPI-CBG @jdeschamps
Maksim Novikov EMBL @m-novikov

Project: https://github.com/orgs/bioimage-io/projects/1

Hackathon: https://github.com/orgs/bioimage-io/projects/2

Agenda

@oeway
Copy link
Collaborator Author

oeway commented Feb 19, 2021 •
edited
Loading

Feburary 19th, 2021

Attendees

Name affiliation username (github)
Wei Ouyang KTH @oeway
Estibaliz Gómez UC3M @esgomezm
Anna Kreshuk EMBL @akreshuk
Maksim Novikov EMBL @m-novikov
Jan Philipp Albrecht MDC @jpalbrecht

Project: https://github.com/orgs/bioimage-io/projects/1

Hackathon: https://github.com/orgs/bioimage-io/projects/2

Agenda

  • Plan for the upcoming hackathon on documentation

    • Discuss the priority for the release
    • How to contribute models
    • Model Documentation Templates with Runner
    • Focus more on the developers

  • Website updates:

    • Running pybio spec verification in the root model repo and add error information if a model fails
    • Displaying a spec badge with passing and failing status for all the models, please fix all the failing models!
    • Resource Item ID rule change: they will be all prefixed with the collection/partner ID, e.g.: deepimagej/WidefieldFitcSuperResolution

  • Upcoming changes:

    • Add a new tab about scripts
    • Improve the BioEngine, Model runner interface change
      • Support Ilastik/Tiktorch model runner
      • Support DeepImageJ.JS

@oeway
Copy link
Collaborator Author

oeway commented Feb 23, 2021

BioImage.IO Hackathon on Documentation

Feburary 23rd, 2021

Attendees

Name affiliation username (github)
Tom Burke CSBD + MPI-CBG @tomburke-rse
Wei Ouyang KTH @oeway
Estibaliz Gómez UC3M @esgomezm
Arrate Muñoz-Barrutia UC3M @arratemunoz
Anna Kreshuk EMBL @akreshuk
Fynn Beuttenmüller EMBL @FynnBe

Project: https://github.com/orgs/bioimage-io/projects/1

Hackathon: https://github.com/orgs/bioimage-io/projects/2

Agenda

  • Hackathon Documentation (https://bioimage.io/docs/)
    • Discuss the priority for the release
    • How to contribute models
    • Model Documentation Templates with Runner
    • Focus more on the developers
    • Roadmap

Proposals

1.- What is the bioimage model zoo? It is more than advance AI models in one click, no?

  • "About" button. It points to https://github.com/bioimage-io/bioimage-io-models.
  • I would add a simple graphical schema about how to contribute, inputs and outputs of the bioimage.io to get a general idea about all the environment (google drawings?)
  • In the contribution text, it says: "You are welcome to submit your models, datasest, applicattionionss and Jupyter notebooks to BioImage.IO." Maybe the main first description should be also oriented into this.

2.- Contributions: Can you contribute in any other way besides being a developer?

  • User: feedback as a user. Through communication channels. GitHub chat. You need to sign

  • Contribute the models to the website (main)

    • Contribute weights to existing models.

  • Contribut to the development of the zoo through GitHub.

  • Contribute to the runners.

  • Consumer software.

  • Bioengine apps.

3.- Model contribution requirements

  • The models that are upload should run at least in one community partners

  • In the begining we need to support people's contributions.

  • How does the developer decide the consumer software? Decision tree with some easy features (TF or PyTorch for example)

  • The model HAS TO BE in the 0.3.0 format.

  • Central repo where the developer can do a PR. There is a Bioimage Model repo.

  • How to contact:

    • image.sc (join the community partner? tag: bioimagemodelzoo?)
    • github issue
    • gitter

  • What are the requirements in terms of compatibility with community partners to contribute a dataset, a notebook, a model, or a workflow?

    • goto the central repo

  • What if a model cannot be loaded in any of the software? and if I provide the notebook? e.g. from https://github.com/bioimage-io/bioimage.io/blob/master/docs/resource-description-file.md#describing-ai-models, it could be assumed that almost anything can be loaded

    • This would be a special contribution. You will always need to provide a model example with the 0.3.0 format.
    • Models have attachments? The notebook could be in the attachment.

  • If I want to contribute with a model, should I create the spec and then ask a partner to include the model in its github with a PR? --> Central repo

  • Could we provide the link to the repo of each community partner to make the RDF and manifests more accessible?

  • This info is now at https://github.com/bioimage-io/bioimage.io/blob/master/docs/contribute.md

  • Description of the RDF and the manifest. Sometimes they seem to be the same and other times, not.

3.- Resource Description File.

  • The main and basic file to write is the RDF --> link to the description and examples/templates (these can be some datasets or notebooks already available). 
  • Sometimes in the text, it is used "model zoo specification" and others, Resource Description File.
  • RDF version is 0.2.0. Most probably the entire document should be reviewed: https://github.com/bioimage-io/bioimage.io/blob/master/docs/resource-description-file.md
  • The name of the RDF should be manifest.bioimage.io.yaml?
  • What is the main structure of this document and how do I start including my notebooks, datasets, and models?
  • Section "Describing AI models": how do you feel about merging it with the configuration specification? or make it very clear that there are two kinds of descriptions: for interoperable models and for just models. 

4.- Configuration specifications:

  • The example is missing, the link is broken.
  • Preprocessing and post-processing reference implementations or mathematical methods. 
  • Run modes specification is still empty. If it is free, then the spec checker should avoid this field. Also, if it is free, shouldn't it go in the config?

5.-Join as a community partner

6.- Doc for bioengine-apps

7.- Template https://deploy-preview-89--bioimage.netlify.app/docs/#/

8.- Tutorials

  • Videos
  • Embed the model runner in the page

Outcomes:

@oeway
Copy link
Collaborator Author

oeway commented Mar 5, 2021

March 5th, 2021

Attendees

Name affiliation username (github)
Tom Burke CSBD + MPI-CBG @tomburke-rse
Wei Ouyang KTH @oeway
Estibaliz Gómez UC3M @esgomezm
Arrate Muñoz-Barrutia UC3M @arratemunoz
Maksim Novikov EMBL @m-novikov
Anna Kreshuk EMBL @akreshuk
Fynn Beuttenmüller EMBL @FynnBe
Daniel Sage EPFL @dasv74
Jan Philipp Albrecht MDC @jpalbrecht
Florian Jug CSBD + HT @fjug

Agenda

  • Solving the failing tests

    • make git_repo optional
    • make cite as optional

  • Zenodo integration to simplify model upload

    • Automatic yaml config generation
    • Model upload
    • tags: bioimage-io for model discovery

  • Standardisation of the title of the pre-trained mode: some of them include U-Net, or not, some of them highlight the application "Cell segmentation". The front page seems a little weird because the name of models are not unified.

    • [TODO] Add recommendation about titles (mind set: what it does NOT what it is (e.g. U-Net))
    • [TODO] Update our core model titles
    • [TODO] Add recommendation for the covers with image before and after; If it is not image-to-image, the developer should find a way to illustrate whan the model does.

  • Smaller issues around the website:

    • Need a contact button and some e-mail to reach out to stakeholders. Can we have an e-mail at the bioimage.io domain?

      • Add an contact email to the about tab
      • Add issue page to the about tab

    • About button should be updated to contain About from docs

      • Update text in the About dialog

    • Contribute button is right now showing an old github post instead of actual guidelines

      • redirect user to the contributon in the docs

    • The bolding of the model or application word is too subtle

      • update the color

@oeway
Copy link
Collaborator Author

oeway commented Apr 17, 2024

April 10th 2024

Attendees

Name affiliation username (github)
Tomaz Vieira EMBL @Tomaz-Vieira
Mariana Ferreira IGC @mariana-gferreira
Mehdi Seifi HT @mese79
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Fynn Beuttenmüller EMBL @FynnBe
Joran Deschamps HT @jdeschamps
Arrate Muñoz-Barrutia UC3M @arratemunoz
Wei Ouyang KTH @oeway
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano

Agenda

  • The Chatbot is now working (yes, now it is working not as las time when it was "working").
  • Google form to express interest (not registration yet!) to come to Madrid in June and give a Talk/Workshop.
    • Please do it before Friday!
    • We will send a registration link for the actual hackathon/uploathon soon! ^^
    • Organization:
      • 10th to the 14th of June in Madrid
      • 2 days workshops and talks (for students)
      • 3 days parallel hackathon and uploathon
  • BMZ Typeform question: "I am looking to quantify osteons from SEM images. I tested all the models in the zoo and didn't find any that were quite the right fit (of course). I wonder if you might be able to recommend one or provide some advice."
    • If there is a question on the typeform and there is no model available to segment those images... what do we do?
  • Status of the bioimage.io > are all models displayed?
    • no, not yet
  • ok with public list of reviewers? (emails)
    • we'll use github account ids with a mapping to names in the collection repo
  • EU-OpenScreen wondering what they should do to fill their PMs in AI4Life, should I put them in touch with WP7?
    • Yes! Kathy Skopelitou seems to be replacing Katja
  • Plan for the next deliverable for BioEngine

@cfusterbarcelo
Copy link
Collaborator

April 17th 2024

Attendees

Name affiliation username (github)
Tomaz Vieira EMBL @Tomaz-Vieira
Mariana Ferreira IGC @mariana-gferreira
Mehdi Seifi HT @mese79
Fynn Beuttenmüller EMBL @FynnBe
Joran Deschamps HT @jdeschamps
Arrate Muñoz-Barrutia UC3M @arratemunoz
Wei Ouyang KTH @oeway
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Vera Galinova HT ??

Agenda

  • Model uploader, is it done? --> Urgent to plan next week's course

    • Jeremy is around finishing dev work.
    • https://github.com/bioimage-io/bioimageio-uploader-service
    • At the moment the netlify functions that served as a light backend before are being moved into the hypha service. Jeremy is actually doing this at the moment.
    • A basic uploading back up should be there again in a couple days

  • Bug in new models appearing in the website, is it solved? (Empanada models are under review)

    • MicroSAM models from Constantin are uploaded to the connection (Anna said it was smooth) but they are still not displayed in the website

  • Model retraining --> is there some toy code I could reuse for fine-tuning (out of the specs and bioimageio)

  • Paper resubmission?

    • Model upload
    • CI
      • bioimageio core is quite stable already so the notebooks can start being migrated

  • EBI course:

    • Anna gives the talk and shows the website (1h)

    • Dominik (1.15h)

    • Wei (30 mins) and Esti (1h) (after lunch)

    • Ask Dominik to switch the order (ESTI)

    • Ask craig to make more GPUs available for the test run to not collapse with Anna and the 40 participants (ESTI)

  • Bioengine HPC worker

    • Python package to install it
    • Then the user will be able to access their institutional HPC from the BioImage Model Zoo website and get new functionalities in the test run button. Basically this would support the use of local HPC from the browser.
    • It works with Singularity

@oeway oeway mentioned this issue Apr 25, 2024
Open
@cfusterbarcelo
Copy link
Collaborator

April 24th 2024

Attendees

Name affiliation username (github)
Mariana Ferreira IGC @mariana-gferreira
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Ivan Hidalgo IGC @IvanHCenalmor
Tomaz Vieira EMBL @Tomaz-Vieira

Agenda

  • Is there a meeting next week? We guess that there isn't.

  • WP4 meeting on the 2nd of April at 15:00 CEST.

@cfusterbarcelo
Copy link
Collaborator

May 8th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano

Agenda

  • Workshop & Hackathon/Uploathon in June in Madrid
    • Please, register in the form here as soon as possible.
    • Event webpage: here
    • Is the upload ready and documented?
    • Is there a list of models that we want to upload?

@cfusterbarcelo
Copy link
Collaborator

May 15th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Constantin Pape Uni Goettingen @constantinpape
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Wei Ouyang KTH @oeway
Nils Mechtel KTH @nilsmechtel
Fynn Beuttenmüller EMBL @FynnBe
Arrate Muñoz-Barrutia UC3M @arratemunoz

Agenda

  • [Caterina] Did you recieve my email as a reminder of this meeting yesterday?

    • Yes 😄

  • I2K Workshops:

    • Beginner BMZ (users): people who don't know BMZ and wants to learn about it and how to use it + chatbot for beginners - @anna
    • Advanced BMZ (developers): people who is interested on how BioEngine and core library works - @esti
    • BioImage.IO ChatBot (developers) - @caterina
    • DL4MicEverywhere - @esti

  • Workshop & Hackathon/Uploathon in June in Madrid

    • Please, register in the form here as soon as possible.
    • Event webpage: here

  • Uploathon:

    • Who is helping, the trainers
    • Is the upload ready?
    • Models to upload.
    • An idea from Anna: have a cool model such as Stardist, fine tune it in a new dataset, do some kind of competition on the best results, then the best one try to upload it to the BMZ (only uploading the best).

  • [Constantin] SAM Models in the zoo

    • Is it legal to rehost them?
    • Rough info on how to bring them into the zoo format?

  • [Constantin] Updating the zoo to new collection backend

  • [Constantin] Convenience functionality for prediction

  • [Constantin] How do I start a kaibu app (question for Wei, if there's time we can discuss, bringing it up here because its also something we should document at some point)

@cfusterbarcelo cfusterbarcelo self-assigned this May 21, 2024
@cfusterbarcelo
Copy link
Collaborator

cfusterbarcelo commented May 21, 2024 •
edited
Loading

May 15th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Constantin Pape Uni Goettingen @constantinpape
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Wei Ouyang KTH @oeway
Nils Mechtel KTH @nilsmechtel
Fynn Beuttenmüller EMBL @FynnBe
Arrate Muñoz-Barrutia UC3M @arratemunoz

Agenda

  • [Caterina] Did you recieve my email as a reminder of this meeting yesterday?

    • Yes 😄

  • I2K Workshops:

    • Beginner BMZ (users): people who don't know BMZ and wants to learn about it and how to use it + chatbot for beginners - Anna and @cfusterbarcelo
    • Advanced BMZ (developers): people who is interested on how BioEngine and core library works - @esgomezm
    • BioImage.IO ChatBot (developers) - @cfusterbarcelo
    • DL4MicEverywhere - @esgomezm

  • Workshop & Hackathon/Uploathon in June in Madrid

    • Please, register in the form here as soon as possible.
    • Event webpage: here

  • Uploathon:

    • Who is helping, the trainers
    • Is the upload ready?
    • Models to upload.
    • An idea from Anna: have a cool model such as Stardist, fine tune it in a new dataset, do some kind of competition on the best results, then the best one try to upload it to the BMZ (only uploading the best).

  • [Constantin] SAM Models in the zoo

    • Is it legal to rehost them?
    • Rough info on how to bring them into the zoo format?

  • [Constantin] Updating the zoo to new collection backend

  • [Constantin] Convenience functionality for prediction

  • [Constantin] How do I start a kaibu app (question for Wei, if there's time we can discuss, bringing it up here because its also something we should document at some point)

@cfusterbarcelo
Copy link
Collaborator

May 22th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Wei Ouyang KTH @oeway
Nils Mechtel KTH @nilsmechtel
Fynn Beuttenmüller EMBL @FynnBe
Mehdi Seifi HT @mese79
Estibaliz Gómez de Mariscal IGC @esgomezm
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano

Agenda

  • Help needed for the last consultation calls
  • Uploathon in Madrid (12th-14th June)
    • List of models - draft here
      • A CellPose model might be uploaded
      • I encourage all of you to edit and participate in this document to have a full list of models that might be interesting to upload doing the uploathon or in long-term.
    • Tomaz is doing a GUI to build the models there to get a zip and upload it in to the zoo. That's going to be very useful for the uploathon.
    • We (BMZ team) need to do a lot of testing on the upload process before the uploathon.
  • Last Call for Registering to the Hackathon in Madrid! Please, register here. We close the register this friday!
    • If someone of your group only wants to register for attending the workshop, register here.
  • Add to documentation rework spec documentation from here.

@cfusterbarcelo
Copy link
Collaborator

May 29th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Wei Ouyang KTH @oeway
Nils Mechtel KTH @nilsmechtel
Fynn Beuttenmüller EMBL @FynnBe
Mehdi Seifi HT @mese79
Estibaliz Gómez de Mariscal IGC @esgomezm
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Anna Kreshuk EMBL @akreshuk
Joran Deschamps HT @jdeschamps
Craig Russell EMBL-EBI @ctr26

Agenda

  • Internal Validation Workflow here

    • Motivation: recommendation to implement an internal way to release the small projects/components that we develop so we can all agree that the material is ready to go and report back.
    • It is important for internal testing! You can suggest reviewers if you want to point someone to test it.
    • People is encouraged to add their projects to the workflow. They can be added through issues and PR.
    • Who is a user of this board? People developing and releasing things and people interested in seeing the status of these small projects or components.

  • NEW DOCS! Visit them bioimage.io/docs!

    • Phase two: curate all the available documentation, add missing documentation and generally improve all the content.
    • Asking your contribution to update all the docs and add the missing documentation. Please give it a look!

  • Spec Docs (plan for improving the readability of the new version)

    • Readable version here - outdated
    • Also here - updated and generated automatically
    • And here - updated and generated automatically
    • Ideally: we should have an updated and automatically generated documentation human readable.

  • EBI S3 storage issue, download extremely slow: https://forum.image.sc/t/micro-sam-issues-in-napari-http/96750/9?u=oeway

  • Uploathon

    • Doc here
    • Any suggestion is encouraged!

  • Euro-BioImaging Annual report - updated factsheet

@cfusterbarcelo
Copy link
Collaborator

cfusterbarcelo commented Jun 5, 2024 •
edited
Loading

June 5th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Mariana Ferreira IGC @mariana-gferreira
Nils Mechtel KTH @nilsmechtel
Fynn Beuttenmüller EMBL @FynnBe
Mehdi Seifi HT @mese79
Vera Galinova HT ??

Agenda

  • Issues with the bioimage.io webpage @oeway
    • When clicking to a model and willing to access the source rdf file an error occours: image
    • Sometimes the webpage gets blocked while going from one card to another (don't know how to reproduce or when it happens exactly)
  • Uploader status
  • Models to upload.
    • From the OC.
      • Two Cellpose retrained models. Looking this notebook to see how they can be exported and uploaded to the zoo.
      • Some other models from students.
    • CellPose models.
      • Missing the post-processing.
    • PlantSeg.
      • It's on the way.
  • Uploathon two options:
    • Adapt ZeroCostDL4Mic notebook to the new specs - Mariana
    • Use BiaPy notebooks adapted to the new specs - Dani Franco
  • WP4 meeting tomorrow 6th June at 15:00h CET
  • Lisbon hackathon sign up

@cfusterbarcelo
Copy link
Collaborator

June 19th 2024

Attendees

Name affiliation username (github)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Mariana Ferreira IGC @mariana-gferreira
Mehdi Seifi HT @mese79
Estibaliz Gómez de Mariscal IGC @esgomezm
Florian Jug HT @fjug
Joran Deschamps HT @jdeschamps
Iván Hidalgo IGC @IvanHCenalmor
Craig Russell EBI @ctr26
Anna EMBL @akreshuk
Wei Ouyang KTH @oeway

Agenda

Madrid Workshop&Uploathon&Hackathon

  • GitHub Repo with training materials here
  • All workshop slides have been uploaded to Zenodo AI4Life Community (special thanks to Bea)
  • Hackathon Recap Slides here
    • New upload and review method for models.
      • Maybe we need some documentation on this on how to review models.
      • More reviewers are needed indeed!
      • Future work: included models which have not been reviewed with a warning or similar. A link to the model is needed before it is even reviewed and accepted.
      • Once static and automatic validation has passed, give a link without the need of a reviewer approving the model but under a warning.
  • Uploathon Recap Slides here
    • A lot of testing of BiaPy notebooks for re-training BMZ models
    • A lot of testing on their own data
    • A lot of future work on training with their own data
    • One model packaged waited for validation to be uploaded
Model Name Category Description Status Exported Uploaded Notes
µSplit: image decomposition for fluorescence microscopy Participant Models Microscopic image splitting Done Yes Yes
DeepContrast Participant Models Deep Tissue Contrast Enhancement using Synthetic Data Degradations and OOD Model Predictions Done Yes Yes
OpenCall Cellpose Model 1 Participant Models Murine Skeletal Muscle Cell Segmentation Not Started No No Waiting for Qin's cellpose wrapper code
OpenCall Cellpose Model 2 Participant Models Segmentation of Epithelial Cells Not Started No No Waiting for Qin's cellpose wrapper code
CellPose Models Community Interest Models Pre-trained Cellpose models popular in the community Not Started No No
MobileSAM Encoder Community Interest Models Only encoder part of the MobileSAM for feature extraction Done Yes Yes

BMZ Paper Updates

  • CI for all CP
    • biapy: Daniel Franco will work on the CI for biapy consumption. He's working on it and already communicating with Fynn. Shouldn't be difficult.
    • deepImageJ&deepIcy: Carlos will put some time on developping the CI for JDDL that will serve as a CI for both.
    • Pending:
      • Test the CI
      • Document how to do the CI for each CP
      • Reach to other CP that do not have a CI
  • Model cards:
    • Logos, if the logo is not there then is not cross-compatible (Fynn? or who could work on this?)
    • The CI depends on the collection, which generates the information about which model is compatible with what CP software.
    • From the CI it should be possible to generate the "links" in the specs that asign the icons in the model card. These icons link to the different apps defined in the zoo.
    • The icons should be there only when opening the model card.
  • Documentation for the specs:

OC

  • Final recommendations needed this week!

@cfusterbarcelo
Copy link
Collaborator

June 26th 2024

Attendees

Name Affiliation Username (GitHub)
Mehdi Seifi HT @mese79
Vera Galinova HT ??
Iván Hidalgo IGC @IvanHCenalmor
Mariana Ferreira IGC @mariana-gferreira
Anna EMBL @akreshuk
Wei Ouyang KTH @oeway
Nils Mechtel KTH @nilsmechtel
Craig Russell EBI @ctr26

Agenda

  • Fine-tuning notebooks by Mariana! (<3)

    • Fine-tuning models: which models can be fine-tuned and which ones are not? Can we know it and maybe even display it in the model card?
    • During Madrid's hackathon:
      • Mariana updated the pytorch notebook for BMZ models with the new core library.
      • Maraina is updating the notebook that Estibaliz created for the course.
    • Not all the models can be finetuned, depends in many aspects. For example, the format of the weights, the state of the optimizer or the pre-processing or post-processing.
    • It would be great to add a tag to indicate if a model is tunable (finetune can be applyed on a notebook or Ilastik).

  • The test run feature of all the models is tested in here

    • Models that are not working or not available for testing

      frank-water-buffalo Not available for testing
      lucky-fox Not available for testing
      pioneering-goat Not available for testing
      stupendous-sheep Not available for testing
      non-judgemental-eagle Not Working
      creative-panda Not Working
      impartial-shark Not available for testing
      placid-llama Not Working
      naked-microbe Not Working
      easy-going-sauropod Not Working
      efficient-chipmunk Not available for testing
      amiable-crocodile Not available for testing
      diplomatic-bug Not available for testing
      greedy-whale Not available for testing
      noisy-ox Not available for testing
      humorous-crab Not available for testing
      faithful-chicken Not available for testing
      idealistic-rat Not available for testing
      committed-turkey Not available for testing
      decisive-panda Not available for testing
      ambitious-sloth Not Working
      determined-hedgehog Not available for testing
      nice-peacock Not available for testing

  • KPIs

  • Distribute and promote the denoising challenge: more applications are needed! -> https://ai4life-mdc24.grand-challenge.org/

@cfusterbarcelo
Copy link
Collaborator

July 3rd 2024

Name Affiliation Username (GitHub)
Wei Ouyang KTH @oeway
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Estibaliz Gómez de Mariscal IGC @esgomezm
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Mariana Ferreira IGC @mariana-gferreira
Fynn EMBL @FynnBe

Agenda

  • Which is the state of showing which models can be fine-tuned?

    • Esti proposes that is something that should not be in the model card
    • Is more a property of the notebook or a program
    • Any model except torchscript models can be loaded in code and fine-tuned
    • It depends on the software that consumes the model, if they offer this capability or not

  • Login System of bioimage.io is offline (Hypha is down)

    • some maintainance is being done

  • Action items:

      1. Upgrade core library in BioEngine
      • Wei and Fynn having a meeting to solve this.
      1. Go through the failing models from the test run feature and fix as many as possible, remove the rest. Do it after updating the core library in BioEngine.

  • Documentation:

  • Reviewing process:

    • Review process described in bioimage-io/collection#76
      • should this be added to the documentation and made public for people (volunteers to join)?
      • Esti proposes to be closed for now.
    • How to assign reviewers to resource drafts?
    • Do we want set targets for response times along the review process?
    • Should we have a draft version of the model? How?
    • @Fynn issue related to CI status: https://bioimageio-uploader.netlify.app/#/status
    • There's a reviewers channel in Element
    • The status of each pending model could be updated to "in review" whenever someone decides to (self)assign to it. This needs to be updated
      • starting point to implement this part in the backend (bioimageio_collection_backoffice) could be here.

  • AI4EOSC technical team meeting - July 17, will people be available?

@cfusterbarcelo
Copy link
Collaborator

July 10th 2024

Attendees

Name Affiliation Username (GitHub)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Estibaliz Gómez de Mariscal IGC @esgomezm
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Fynn EMBL @FynnBe
Nils Mechtel KTH @nilsmechtel

Agenda

  • Remember that I2K deadline for Abstracts is 31 of July.

  • Reviewing process

    • How to assign reviewers to resource drafts?
    • Do we want set targets for response times along the review process?
    • Add a point to the agenda all weeks to select which models are needed to be reviewed and pending
    • Please, people join as reviewer if you haven't and you are interested

  • Missing license information for some notebooks (causing backup status)
    There are missing license field entries in published RDF versions.
    Those can be edited directly on S3, as the backup (upload to zenodo fails if the license field is missing, thus those records, eventhough published, do not have a DOI yet)
    partners that need to update some of their records (I fixed some, but in total there are 76 license entries missing):

    • DL4MicEverywhere
    • ZeroCost (notebooks in the old format 0.2, license was as optional and now is needed)
    • Biapy

    to identify which RDFs are missing their license field, search for "license": null in
    collection.json

    Either ask Wei for our S3 credentials or forward the missing license field information to someone who can edit it on S3 directly.
    Reminder: Fixing RDFs on S3 directly brings the risk of causing inconsistencies! For any other edits I highly encourage anyone to only edit resource drafts (e.g. not affable-shark/1, but affable-shark/draft), as once a published record version is backed up (and has a DOI) we do not want the version we host on S3 to deviate with the backup on zenodo.org. Thus changes should always go through a draft and be published as new versions.

    • The manifest is deprecated. DL4MicEverywhere's CI rely at this moment on the manifest to update the information on the BMZ. The license needs to be updated in the S3 server (not in the manifest). There is a programatic way to do it, DL4MicEverywhere team (Iván, Mariana and Estibaliz) will contact Fynn who knows how to implemented.

  • Quidelines for BioImage.IO team members: https://github.com/bioimage-io/bioimage.io/wiki

  • Connection with other initiatives

    • AI4EOSC (July 17, this meeting)
    • BigPicture (Sept 19, their WP4 meeting)

  • Wednesday meetings will stop on August.

  • IGC will chair wednesday's meetings on september.

@cfusterbarcelo
Copy link
Collaborator

July 17th 2024

Attendees

Name Affiliation Username (GitHub)
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Estibaliz Gómez de Mariscal IGC @esgomezm
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Fynn EMBL @FynnBe
Nils Mechtel KTH @nilsmechtel

Agenda

  • AI4EOSC presentation and discussion

    • BioImage Archive integration with AI4EOSC Platform. It is currently integrated with Zenodo. (datahugger could be an option for this integration).
      • Craig will see if this is possible and how.
    • BioImage Model Zoo for inference.
      • Allowing people to deploy the BMZ models in the AI4EOSC platform.
      • Build a template that every bioimage.io model could be runned on it.
      • Now models are runned in model containers for AI4EOSC. It could be adapted for our models. It would be a matter of creating a module with the tryton server docker image. Doing the interface with the AI4EOSC API.
      • Connect it to the notebooks for inference.
    • BioImage Model Zoo for fine-tuning.
      • This is something that we can work in the future.

  • [Caterina] There is a WP3 Meeting tomorrow at 14:00h. You can check the agenda here.

@cfusterbarcelo
Copy link
Collaborator

July 24th 2024

Attendees

Name Affiliation Username (GitHub)
Wei Ouyang KTH @oeway
Nils Mechtel KTH @nilsmechtel
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Arrate Muñoz-Barrutia UC3M @arratemunoz
Anna EMBL @akreshuk
Tomaz Vieira EMBL @Tomaz-Vieira
Fynn EMBL @FynnBe
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Vera Galinova HT @veegalinova

Agenda

  • Add the notebook that Mariana created, somewhere on the user's documentation to make it more accesible.

  • How to use the core library(add it here): https://github.com/bioimage-io/core-bioimage-io-python/blob/main/example/model_usage.ipynb

  • We need to have a the minimal example with just '3 lines of code' with the simplest usecase.

  • We need to have an image, graph or entry on the documentation that shows how the different repositories inside GitHub's bioimage community are connected in the wiki: https://github.com/bioimage-io/bioimage.io/wiki (Caterina has said that she will try to take care of this)

  • [Esti] StarDist export does not work with the new bioimageio.core and with the old one, there's no way to run the test-model export_bioimageio not working with the new biomageio.core library stardist/stardist#276

  • [Esti] Model upload not working, at least for models with format 0.4.9 model uploader failing #415

    • The problem is not with the uploader itself, but as the specs are more restrictive in the newer version, the new specs are the ones giving the issues.
    • The fact that the specs are more restrictive is not a good approach.
    • As far as understood, this problem is only concerning the source field. The source field dissapeared on version 0.3, which happened 3 years ago. Is it worth it to make it recontrabatible to that old versions? Maybe is something to take a look.

  • The BMZ exportation on ZeroCost notebooks is not working because that part of the notebook is still in the old version. So if someone is retraining a model using a ZeroCost notebook, it will not be able to be updated. This needs to be updated.

  • [Caterina] Made an updated version of the Feedback form for the BioImage Model Zoo and community parthers here. Please share it! We already have 4 answers!

  • [Carlos] Working on the CI for the JDLL almost done. He will have it very soon!

  • [Caterina] All links from bioimage.io/docs are corrected and working!

  • [Caterina] WP3 Deliverable, waiting to be commented here

  • [Caterina] Internal documentation here: https://github.com/bioimage-io/bioimage.io/wiki

  • [Fynn] onnxruntime seems to have changed such that onnx networks cannot accept input tensors with arbitrary dimensions. (affable-shark is now failing the test)

 index: 2 Got: 64 Expected: 256
 index: 3 Got: 64 Expected: 256
 Please fix either the inputs/outputs or the model.

@cfusterbarcelo
Copy link
Collaborator

July 31st 2024

Attendees

Name Affiliation Username (GitHub)
Nils Mechtel KTH @nilsmechtel
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Arrate Muñoz-Barrutia UC3M @arratemunoz
Tomaz Vieira EMBL @Tomaz-Vieira
Iván Hidalgo IGC @IvanHCenalmor
Estibaliz Gómez de Mariscal IGC @esgomezm
Mehdi Seifi HT @mese79

Agenda

  • No meetings in August! Next meeting: 4th of September
  • The uploader is not accepting models from the previous versions of the specs.
  • Reminder: WP4 regular meeting tomorrow at 15:00 CEST.

@esgomezm
Copy link
Contributor

September 4th 2024

Attendees

Name Affiliation Username (GitHub)
Anna EMBL @akreshuk
Tomaz Vieira EMBL @Tomaz-Vieira
Iván Hidalgo IGC @IvanHCenalmor
Estibaliz Gómez de Mariscal IGC @esgomezm
Mariana Ferreira IGC @mariana-gferreira
Florian Jug HT @fjug
Mehdi Seifi HT @mese79
Vera Galinova HT @veegalinova

Agenda

  • Brief comment of the hackathon at IGC
    • Slides summarising the activities in the hackathon: https://docs.google.com/presentation/d/1VKlKeP-iEUm6fnKayjyx_IYnoFXfxKIxsi8vTKrAPlw/edit#slide=id.g28c070389ad_3_766
    • Model uploader: is not in a good state but we need to have something by the general assembly.
      • Mehdi could speak with Wei and try fixing things with the newly employed software engineers.
      • There is an issue interacting with the infrastructure like the GitHub CI, servers, and supported database instead of using GitHub ci.
    • Some brainstorm: Which features would we like to have in the uploader?
      • There should be a button to publish it for the model and the reviewing should happen after the model has been "published". When published and reviewed, the model could have a "pin" with "reviewed by a human".
      • Collect a list of model uploaders' issues and then see if we can fix them.
        The spec has an issue with dependencies as it allows for arbitrary dependencies. For example, Ilastik will have its own dependencies and ZeroCostDL4Mic others. This poses a challenge when testing the models.
      • We should think about a way to get backwards compatible realising that consumers and core libraries will be unsynchronously updated. For example, a chain rule in the CI that tests the model using different versions of the core library. This is important as the software will need to run models from two years from now.
      • The uploader is not accepting models from the previous versions of the specs.
      • Have the models in containers to ensure its functionality even though old Pytorch or Keras libraries are not longer available. We need some kind of environment that we can control and that we can truly test on our computer and be sure that will pass the CI.
      • Use Hugging Face to store the models from the BMZ and have all the scientific metadata in another place.
  • I2K, anything to comment? The virtual workshops are happening the next week to I2K.
  • Hackathon in Milano before I2K:
    • Use it to see how the life demos happen before the general assembly
    • Sit all together to discuss the general assembly.

@oeway
Copy link
Collaborator Author

oeway commented Sep 11, 2024

September 11th 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Mariana Ferreira IGC @mariana-gferreira
Florian Jug HT @fjug
Mehdi Seifi HT @mese79
Vera Galinova HT @veegalinova
Wei Ouyang KTH @oeway
Marcus Andersson KTH @marcus1337

Agenda

  • Introducing Marcus (KTH)
  • Ideas about the uploader
    • integrate with the main bioimage.io page instead of on another page
  • Uploader Update & Plan
  • Model distribution via docker images
    • general docker image and conda.pak downloaded with model
    • specific docker per model - user uploaded or bMZ generated

@cfusterbarcelo
Copy link
Collaborator

cfusterbarcelo commented Sep 25, 2024 •
edited
Loading

September 18th 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Anna EMBL @akreshuk
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Mehdi Seifi HT @mese79
Vera Galinova HT @veegalinova
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano

Agenda

  • Discuss Demos for the AI4Life general assembly

    Demo 1: How can our users create and upload models today? (20’+40’)
    (Fynn Beuttenmueller, Mehdi, Tomaz)
    Note: all OC and lab specific models need to be added by the GA!!!

    Demo 2: How can users consume uploaded models? (20’+40’)
    (i) from bioimage.io (Wei Ouyang)
    (ii) from community partner tools
    (Anna Kreshuk (ilastik); Arrate Muñoz Barrutia, Ignacio Arganda-Carreras (BiaPy))
    (iii) from code, notebooks, containers
    (Estibaliz Gómez de Mariscal, Ricardo Henriques, Mariana Ferreira)

    Demo 3: FAIR is not just a buzzword! (20’+40’)
    (i) FAIR and AI-ready datasets (and formats!)
    (Matthew Hartley, Teresa Zulueta-Coarasa)
    (ii) FAIR AI models via the BioImage Model Zoo (Florian Jug (HT on premise BMZ???))
    Note: all OC and lab specific models need to be added by the GA!!!
    (iii) FAIR documentation and Chatbot (Arrate Muñoz Barrutia, Caterina 🌴)

    • We volunteered Mehdi to do model uploads at any level of manual hacking and along the way produce a documentation what steps a full model upload takes, which ones work, which ones are currently creating problems, and who created the existing solution and can help us fix issues and revamp the entire upload process. Thanks Mehdi! #hero

  • Long discussion in the dev-steering channel about compatibility with Huggingface, why (not) using GH repositories for models, run on community partner tools what it is run in the CI

  • From Mehdi to FILL please: As we discussed during the yesterday meeting, I am sharing with you a link to the document that we can share our experience / issues / ideas about the uploader and the whole uploading proccess: https://docs.google.com/document/d/1zTTlvGGjoMggYoQ5EMBM7P3fmA4sPBYwL-OurFCl39c/edit?usp=sharing

  • Wei: Test run on the website

  • ilastik (Anna): They will run some models there. They need to change to the new version of the spec.

  • Biapy: CI works. Currently, latest updates and changes related with the manuscript rebuttal. So, Daniel F. and Ignacio A. are checking that everything runs smoothly. Documentation updated. New video tutorials.

  • ZeroCost (Esti + Mariana + Iván): Work on the update of the notebooks for segmentation to use the new library and specs. This model should be compatible with Ilastik.

  • Containers (Esti + Mariana + Iván): Container standard specs + bioimageio notebook for loading a model from the zoo and retrain it with publicly available data and export back.

@cfusterbarcelo
Copy link
Collaborator

September 25th 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Anna EMBL @akreshuk
Mariana Ferreira IGC @mariana-gferreira
Iván Hidalgo IGC @IvanHCenalmor
Florian Jug HT @fjug
Mehdi Seifi HT @mese79
Vera Galinova HT @veegalinova
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Teresa Zulueta-Coarasa EBI @tzulueta

Agenda

  • Update on current state of the art of model uploading

    • Mehdi (and Tomaz (and others)) prepare a comprehensive document that describes how the current uploader + CI + model storage + model serving + model testing workflow is set up. Each part should be mentioned, currently problems and hardware on which it runs, plus know isses should be listed. Additionally it would be nict to know who created the current modules so we know who we can ask if questions or issues arise.
    • CellPose model was uploaded to zoo by Mehdi

  • The download button of the bioimage.io website is not working for some time now.

    • We need to have someone else a part from Wei being able to manage this and having the keys to the castle.

  • Demos schedule:

    • Septmber 25th - first trial runs of demos
    • October 2nd - second trial run of demos (not Jug Lab, sorry, we are all at ECCV)
    • October 9th - full demo run of all involved

  • Demos:
    Demo 1: How can our users create and upload models today? (20’+40’) (Fynn Beuttenmueller, Mehdi, Tomaz > create/GUI tool)
    Note: all OC and lab specific models need to be added by the GA!!!

    Demo 2: How can users consume uploaded models? (20’+40’)

    • (i) from bioimage.io (Wei Ouyang)
    • (ii) from community partner tools (Anna Kreshuk (ilastik); Arrate Muñoz Barrutia, Ignacio Arganda-Carreras (BiaPy))
    • (iii) from code, notebooks, containers (Estibaliz Gómez de Mariscal, Ricardo Henriques, Mariana Ferreira)

    Demo 3: FAIR is not just a buzzword! (20’+40’)

    • (i) FAIR and AI-ready datasets (and formats!) (Matthew Hartley, Teresa Zulueta-Coarasa)
    • (ii) FAIR AI models via the BioImage Model Zoo (Florian Jug (HT on premise BMZ???))
      Note: all OC and lab specific models need to be added by the GA!!!
    • (iii) FAIR documentation and Chatbot (Arrate Muñoz Barrutia, Caterina 🌴)

    PREPARE SOMETHING FOR NEXT WEEK!

  • Chatbot talk on Friday at 14pm for GloBIAS. More info here: GloBIAS Seminar Series

@cfusterbarcelo
Copy link
Collaborator

October 2nd 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Anna EMBL @akreshuk
Iván Hidalgo IGC @IvanHCenalmor
Caterina Fuster-Barceló UC3M @cfusterbarcelo

Agenda

  • GA Demos:
    Demo 1: How can our users create and upload models today? (20’+40’) (Fynn Beuttenmueller, Mehdi, Tomaz > create/GUI tool)
    Note: all OC and lab specific models need to be added by the GA!!!

    Demo 2: How can users consume uploaded models? (20’+40’)

    • (i) from bioimage.io (Wei Ouyang)
    • (ii) from community partner tools (Anna Kreshuk (ilastik); Arrate Muñoz Barrutia, Ignacio Arganda-Carreras (BiaPy))
    • (iii) from code, notebooks, containers (Estibaliz Gómez de Mariscal, Ricardo Henriques, Mariana Ferreira)

    Demo 3: FAIR is not just a buzzword! (20’+40’)

    • (i) FAIR and AI-ready datasets (and formats!) (Matthew Hartley, Teresa Zulueta-Coarasa)
    • (ii) FAIR AI models via the BioImage Model Zoo (Florian Jug (HT on premise BMZ???))
      Note: all OC and lab specific models need to be added by the GA!!!
    • (iii) FAIR documentation and Chatbot (Arrate Muñoz Barrutia, Caterina)

  • (Iván) Issues on some of the uploaded models:

    • 🐸 gleeful-bird and 🐸 poisonous-raccoon have a different animal emoji.
    • There is a model called: 🐙 10.5281/zenodo.8421755
    • 🦥 ambitious-sloth is not accesible.

  • (Iván) Questions for reviewing process:

    • Where can I find the actual model or the rdf of the uploaded models?
    • What does the feedback in the uploaded models mean? Because there are CI outputs with a cross and the CI has passed correctly.
    • Where can I now what I have to change to finally accept the model?

  • (Iván) Tomorrow, 3rd of October at 15:00 CEST, it will be WP4 regular meeting.

  • (Carlos) Cannot attend today, but will try to have deepImageJ and DeepIcy demos for next week. Integration of BiaPy in Java based software.

  • (Tomaz) Which are the differences between the Uploader and the GUI? The GUI is more like a complete solution and you can also run it locally and even test the model if you have core installed. It's another way to access the same thing.

  • (Tomaz) The new dev-docs, with flatened classes and examples, is here: Scripts for generating interactive spec docs spec-bioimage-io#630 . I see now that our CI is broken again because pyight updated is now producing other type checking errors, but you can use the PR anyway.

  • (Caterina) I2K BioImage Model Zoo workshops.

@cfusterbarcelo
Copy link
Collaborator

October 9th 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Anna EMBL @akreshuk
Iván Hidalgo IGC @IvanHCenalmor
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Arrate Muñoz-Barrutia UC3M @arratemunoz
Carlos UC3M @carlosuc3m
Vera Galinova HT @veegalinova
Florian Jug HT @fjug

  • Carlos: redoing deepImageJ and deepICY with JDLL

    • Full makeover of the interface with the models from the BioImage Model Zoo showing in the first place.
    • Working on the CI for deepIcy and deepImageJ with the BioImage Model Zoo.
    • Goal: have a demo to show on the GA

  • GA Demos:
    Demo 1: How can our users create and upload models today? (20’+40’) (Fynn Beuttenmueller, Mehdi, Tomaz > create/GUI tool)
    Note: all OC and lab specific models need to be added by the GA!!!

    Demo 2: How can users consume uploaded models? (20’+40’)

    • (i) from bioimage.io (Wei Ouyang) >> decide what model to show and who presents this, preferably, not affable-shark
    • (ii) from community partner tools (Anna Kreshuk (ilastik); Arrate Muñoz Barrutia, Ignacio Arganda-Carreras (BiaPy); Carlos and the new deep(not)ImageJ demo...)
    • (iii) from code, notebooks, containers (Estibaliz Gómez de Mariscal, Ricardo Henriques, Mariana Ferreira)

    Demo 3: FAIR is not just a buzzword! (20’+40’)

    • (i) FAIR and AI-ready datasets (and formats!) (Matthew Hartley, Teresa Zulueta-Coarasa)
    • (ii) FAIR AI models via the BioImage Model Zoo (Florian Jug (HT on premise BMZ???))
      Note: all OC and lab specific models need to be added by the GA!!!
    • (iii) FAIR documentation and Chatbot (Arrate Muñoz Barrutia, Caterina)

  • NEXT WEEK FULL DEMO! EVERYONE COME PREPARED!

@cfusterbarcelo
Copy link
Collaborator

October 16th 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Anna EMBL @akreshuk
Iván Hidalgo IGC @IvanHCenalmor
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Arrate Muñoz-Barrutia UC3M @arratemunoz
Carlos UC3M @carlosuc3m
Vera Galinova HT @veegalinova
Florian Jug HT @fjug
Mehdi Seifi HT @mese79
Marcus Andersson KTH @marcus1337
Ricardo Henriques IGC/GIMM/ITQB @HenriquesLab
Mariana Ferreira IGC @mariana-gferreira
Nils Mechtel KTH @nilsmechtel
Estibaliz Gómez de Mariscal IGC @esgomezm
Ignacio Arganda-Carreras UPV/EHU @iarganda
Teresa Zulueta-Coarasa EBI @tzulueta
Fynn EMBL @FynnBe
Daniel Franco-Barranco EHU/UPV & DIPC @danifranco
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Wei Ouyang KTH @oeway

Agenda

  • GA Demos:
    Demo 1: How can our users create and upload models today? (20’+40’) (Fynn Beuttenmueller, Mehdi, Tomaz > create/GUI tool)
    Note: all OC and lab specific models need to be added by the GA!!!

    Demo 2: How can users consume uploaded models? (20’+40’)

    • (i) from bioimage.io (Wei Ouyang) >> decide what model to show and who presents this, preferably, not affable-shark
    • (ii) from community partner tools (Anna Kreshuk (ilastik); Arrate Muñoz Barrutia, Ignacio Arganda-Carreras (BiaPy); Carlos and the new deep(not)ImageJ demo...)
    • (iii) from code, notebooks, containers (Estibaliz Gómez de Mariscal, Ricardo Henriques)

    Demo 3: FAIR is not just a buzzword! (20’+40’)

    • (i) FAIR and AI-ready datasets (and formats!) (Matthew Hartley, Teresa Zulueta-Coarasa)
    • (ii) FAIR AI models via the BioImage Model Zoo (Florian Jug (HT on premise BMZ???))
      Note: all OC and lab specific models need to be added by the GA!!!
    • (iii) FAIR documentation and Chatbot (Arrate Muñoz Barrutia, Caterina)

@cfusterbarcelo
Copy link
Collaborator

October 30th 2024

Attendees

Name Affiliation Username (GitHub)
Tomaz Vieira EMBL @Tomaz-Vieira
Anna EMBL @akreshuk
Iván Hidalgo IGC @IvanHCenalmor
Caterina Fuster-Barceló UC3M @cfusterbarcelo
Arrate Muñoz-Barrutia UC3M @arratemunoz
Vera Galinova HT @veegalinova
Marcus Andersson KTH @marcus1337
Mariana Ferreira IGC @mariana-gferreira
Nils Mechtel KTH @nilsmechtel
Fynn EMBL @FynnBe
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Wei Ouyang KTH @oeway
Joran Deschamps HT @jdeschamps

Agenda

  • (Caterina) Is any of you experiencing issues to download models from the BioImage.IO? Image.sc Forum question

    • Maybe it's related to this issue
    • There seems to be an error when downloading a model with the ilastik button, Wei will look into that

  • (Mehdi, Vera and Caterina) Problems and suggestions with the reviewing process:

    • Current reviewing process is mostly limited to mounting the S3 bucket and browse for models!
    • One credential for everyone in S3 bucket with rights to everything... a bit dangerous
    • Reading error logs on uploader to find out what went wrong can be a bit tricky
    • Do we really need sample input/output and producing exact the same result? (no other zoos has this)
      • [Wei] It can bring some issues in terms of hardware but it's the only way we can detect the errors that are in between (with the CI). It's the way to give us confidence that the model can be reproduced. "We should keep that"
    • Having the main contributor of the model stated in the model being reviewed would be helpful
    • Having a link or access to download the whole model would also be helpful and faster
    • Seeking assistance: some models have the same error ("reproduce test outputs from test inputs")... which would be the steps to follow on this error?
    • TO DO: Create an issue about this suggestions such as adding the contributor name or the link to the model, add a conda environment that can be used to test the model (uploader repo)

  • (Iván) Uploader is randomly failing: Video where same model fail and success. In case some demo plans to use it, to be aware.

    • We need the error log to see what's happening. Make an issue!

  • (Wei) Should we serve uploaded model as it is or we upgrade it? E.g. after new spec release.

    • When a new format version is released, we should update all the models or should they remain as the user upload them.
    • Think how much backward compatibility we should provide.
    • Think reproducibility for published models without dated consumer software.
    • Who updates the model? What if the people who contributed do not update it? Should only be the user or could another person update the model.
    • All the versions are published, the user just need to point that version (this could fix consumer's comatibility problems).
    • There is the parent field that can point to an older model version, but that would overpopulate the zoo with cloned models.
    • Bring this to a steering committee for a decision.

  • (Wei) Next steps for the uploader: Artifact Manager

@cfusterbarcelo cfusterbarcelo mentioned this issue Nov 4, 2024
Open
@cfusterbarcelo
Copy link
Collaborator

November 5th 2024

Attendees

Name Affiliation Username (GitHub)
Anna EMBL @akreshuk
Fynn EMBL @FynnBe
Iván Hidalgo IGC @IvanHCenalmor
Mariana Ferreira IGC @mariana-gferreira
Florian Jug HT @fjug
Vera Galinova HT @veegalinova
Mehdi Seifi HT @mese79
Wei Ouyang KTH @oeway
Arrate Muñoz UC3M @arratemunoz

Agenda

  • One of the main concerns that has arrise more than once is the difference between testing a model locally and on CI.

    • A possible proposal is using a conda environment that is the same one used on CI.
    • This could be in a checklist for the reviewing proces.
    • Using conda, issues might appear basen on the local architecture, still on those scenarios the user could go to the kubernetees option.
    • There can be a different default environments for each weight format.

  • The reviewing needs to have a checklist that facilitates the process.

    • Common checklist (upload and review) -- Caterina (@cfusterbarcelo): Please, work in the review guidelines. Give the questions below try and see if they are obvious and make or not sense.
      • Are the authors correct/updated?
      • Is the name meaningful?
      • Does the description make sense?
      • Are the citations relevant and complete?
      • Are all the checkmarks green? (aka did test_model()'s output have status=passed?)
      • Is contributed code limited to what is required to exec the model?
      • Are tags consistent with the description?
    • Checklist specific to upload
    • Checklist specific to review
    • @Fynn: Propose to make a review summary (for example, a small web page report). Automatically generated that contain the answer to the questions.

  • Any issue with the uploader: create an Issue on the bioimageio GitHub repository: https://github.com/bioimage-io/bioimage.io/issues

  • There are Licenses like the one of CellPose that are not recognised by Zenodo, but should be included. How is the way to add it to the model description? Medhi (@mese79) will create an issue of it on the bioimageio.spec repository.

  • To add a parent model, it needs to be previously added to the rdf.yaml as a field: parent: {"id": "affabel-shark/1.1"}.

  • Reduce hard requirements for ModelDescr

    • Use 'alert level' to define threshold for sharing/upload to bioimage.io

  • update recommended env to ensure compatibility to collection backoffice (e.g. py>=3.9)

@cfusterbarcelo cfusterbarcelo unpinned this issue Nov 11, 2024
@cfusterbarcelo
Copy link
Collaborator

November 13th 2024

Attendees

Name Affiliation Username (GitHub)
Anna EMBL @akreshuk
Iván Hidalgo IGC @IvanHCenalmor
Mariana Ferreira IGC @mariana-gferreira
Florian Jug HT @fjug
Vera Galinova HT @veegalinova
Mehdi Seifi HT @mese79
Wei Ouyang KTH @oeway
Estibaliz Gómez de Mariscal IGC @esgomezm
Beatriz Serrano-Solano Euro-BioImaging @beatrizserrano
Caterina Fuster-Barceló UC3M @cfusterbarcelo

Agenda

  • Draft models

    • Feeback from reviewing draft models:

  • Add CI tests for existing models in the model zoo for bioimageio.core

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cfusterbarcelo cfusterbarcelo

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@oeway @jmetz @constantinpape @FynnBe @esgomezm @cfusterbarcelo