diff --git a/.gitignore b/.gitignore index 8bbbed4b..f0d3ea62 100644 --- a/.gitignore +++ b/.gitignore @@ -133,3 +133,4 @@ lamin_sphinx docs/conf.py _docs_tmp* docs/modules.md +docs/paradisi05_laminopathic_nuclei.jpg diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 4cdc4dfa..09f85b88 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -2,8 +2,8 @@ fail_fast: false default_language_version: python: python3 default_stages: - - commit - - push + - pre-commit + - pre-push minimum_pre_commit_version: 2.16.0 repos: - repo: https://github.com/pre-commit/mirrors-prettier @@ -24,7 +24,7 @@ repos: docs/notes/ ) - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.8.6 + rev: v0.9.1 hooks: - id: ruff args: [--fix, --exit-non-zero-on-fix, --unsafe-fixes] @@ -44,7 +44,7 @@ repos: - id: trailing-whitespace - id: check-case-conflict - repo: https://github.com/pre-commit/mirrors-mypy - rev: v1.7.1 + rev: v1.14.1 hooks: - id: mypy args: [--no-strict-optional, --ignore-missing-imports] diff --git a/docs/changelog/soon/lamindb.md b/docs/changelog/soon/lamindb.md index 7bb5f3a1..028a3acc 100644 --- a/docs/changelog/soon/lamindb.md +++ b/docs/changelog/soon/lamindb.md @@ -1,7 +1,7 @@ - 🩹 migrate without setting auto-connect to False [PR](https://github.com/laminlabs/lamindb-setup/pull/942) [@Koncopd](https://github.com/Koncopd) - 🚸 Error on auto-connect [PR](https://github.com/laminlabs/lamindb-setup/pull/941) [@falexwolf](https://github.com/falexwolf) - ♻️ Add contenttypes Django plugin [PR](https://github.com/laminlabs/lamindb-setup/pull/940) [@falexwolf](https://github.com/falexwolf) -- 🚚 Rename `FeatureSet` to `Schema` and migrate to lamindb 1.0a5 [PR](https://github.com/laminlabs/lamindb/pull/2348) [@falexwolf](https://github.com/falexwolf) +- 🚚 Rename `FeatureSet` to `Schema` and migrate to lamindb 1.0a5 [PR](https://github.com/laminlabs/lamindb/pull/2348) [@falexwolf](https://github.com/falexwolf) - 🐛 Skip deleting storage when deleting outdated versions of folder-like artifacts [PR](https://github.com/laminlabs/lamindb/pull/2349) [@Koncopd](https://github.com/Koncopd) - ♻️ Refactor key and name change checks [PR](https://github.com/laminlabs/lamindb/pull/2345) [@Koncopd](https://github.com/Koncopd) - 🚚 Migrate to lamindb 1.0a4 [PR](https://github.com/laminlabs/lamindb/pull/2343) [@falexwolf](https://github.com/falexwolf) diff --git a/docs/changelog/soon/usecases.md b/docs/changelog/soon/usecases.md index 46aa5b45..e4e7f19c 100644 --- a/docs/changelog/soon/usecases.md +++ b/docs/changelog/soon/usecases.md @@ -6,7 +6,7 @@ - 🚚 Migrate to lamindb 1.0 [PR](https://github.com/laminlabs/wetlab/pull/90) [@falexwolf](https://github.com/falexwolf) - 🚚 Migrate to lamindb 1.0 [PR](https://github.com/laminlabs/ourprojects/pull/8) [@falexwolf](https://github.com/falexwolf) - 🐛 Fix default values [PR](https://github.com/laminlabs/cellxgene-lamin/pull/119) [@Zethson](https://github.com/Zethson) -- ✅ Fail CI if `cellxgene-schema` CLI validation fails [PR](https://github.com/laminlabs/cellxgene-lamin/pull/111) [@Zethson](https://github.com/Zethson) +- ✅ Fail CI if `cellxgene-schema` CLI validation fails [PR](https://github.com/laminlabs/cellxgene-lamin/pull/111) [@Zethson](https://github.com/Zethson) - 🎨 Set up sources dynamically [PR](https://github.com/laminlabs/wetlab/pull/91) [@sunnyosun](https://github.com/sunnyosun) - 🎨 Do not check embedding in .validate [PR](https://github.com/laminlabs/cellxgene-lamin/pull/114) [@sunnyosun](https://github.com/sunnyosun) - 🏗️ Integrate `lnschema-core` into `lamindb` [PR](https://github.com/laminlabs/ourprojects/pull/7) [@falexwolf](https://github.com/falexwolf) diff --git a/docs/includes/features-lamindb.md b/docs/includes/features-lamindb.md index 100caa5f..4564ee93 100644 --- a/docs/includes/features-lamindb.md +++ b/docs/includes/features-lamindb.md @@ -10,7 +10,7 @@ The following detailed specs are for the Python package `lamindb`. For the analo - Slice large array stores: {class}`~lamindb.Artifact.open` → [guide](cellxgene) - Cache & load artifacts: {class}`~lamindb.Artifact.cache`, {class}`~lamindb.Artifact.load` - Manage features & labels: {class}`~lamindb.Feature`, {class}`~lamindb.FeatureSet`, {class}`~lamindb.ULabel` -- Plug-in custom [schemas](/schemas) & manage schema migrations +- Plug-in custom [modules](/modules) & manage schema migrations - Use array formats in memory & storage: [DataFrame](/tutorial), [AnnData](/arrays), [MuData](multimodal), [SOMA](cellxgene), ... backed by [parquet](/tutorial), [zarr](/arrays), [TileDB](cellxgene), [HDF5](/arrays), [h5ad](/arrays), [DuckDB](rxrx), ... - Create iterable collections of artifacts with data loaders: {class}`~lamindb.Collection` - Version artifacts, collections & transforms: {class}`~lamindb.core.IsVersioned` diff --git a/docs/integrations.md b/docs/integrations.md index 4d8ef878..265ca8a3 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -8,14 +8,13 @@ Additional registries can be found here: - [wetlab](/wetlab): Basic wetlab entities - [clinicore](/clinicore): Basic clinical entities - [cellregistry](/cellregistry): A registry for single cells -- [ourprojects](/ourprojects): Manage projects, teams & references - [omop](/omop): OMOP Common Data Model Please [reach out](https://lamin.ai/contact) to have your schema module featured. ## Git -Sync tracked scripts in lamindb with git commits: [guide](track.ipynb#sync-scripts-with-github) +Sync tracked scripts in lamindb with git commits: [guide](track.ipynb#sync-scripts-with-git) ## MLOps diff --git a/docs/introduction.ipynb b/docs/introduction.ipynb index 67a3a0f1..8cbee47e 100644 --- a/docs/introduction.ipynb +++ b/docs/introduction.ipynb @@ -194,9 +194,8 @@ "source": [ "import lamindb as ln\n", "\n", - "# --> `ln.track()` generates a uid for your code\n", - "# --> `ln.track(uid)` initiates a tracked run\n", - "ln.track(\"FPnfDtJz8qbE0000\")" + "# track the current notebook\n", + "ln.track()" ] }, { @@ -519,10 +518,10 @@ "outputs": [], "source": [ "# get a single record by uid (here, the latest version of the current notebook)\n", - "transform = ln.Transform.get(\"FPnfDtJz8qbE\")\n", + "transform = ln.Transform.get(ln.context.transform.uid)\n", "\n", "# get a single record by matching a field\n", - "transform = ln.Transform.get(name=\"Introduction\")\n", + "transform = ln.Transform.get(type=\"notebook\")\n", "\n", "# get a set of records by filtering on description\n", "ln.Artifact.filter(description=\"my RNA-seq\").df()\n", @@ -532,7 +531,7 @@ "\n", "# query all artifacts ingested from a notebook with \"intro\" in the name and labeled \"Candidate marker study\"\n", "artifacts = ln.Artifact.filter(\n", - " transform__name__icontains=\"intro\", ulabels=candidate_marker_study\n", + " transform__description__icontains=\"intro\", ulabels=candidate_marker_study\n", ").all()" ] }, @@ -1281,7 +1280,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.7" + "version": "3.12.8" } }, "nbformat": 4, diff --git a/docs/tutorial.ipynb b/docs/tutorial.ipynb index 669f7c83..f76301b9 100644 --- a/docs/tutorial.ipynb +++ b/docs/tutorial.ipynb @@ -526,7 +526,7 @@ " ```\n", ":::\n", "\n", - "Filter the {class}`~lamindb.Transform` registry for a name:" + "Filter the {class}`~lamindb.Transform` registry for a description:" ] }, { @@ -539,7 +539,9 @@ }, "outputs": [], "source": [ - "transform = ln.Transform.get(name__icontains=\"Artifacts\") # get exactly one result\n", + "transform = ln.Transform.get(\n", + " description__icontains=\"Artifacts\"\n", + ") # get exactly one result\n", "transform" ] }, @@ -933,7 +935,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "lamindb", "language": "python", "name": "python3" }, @@ -947,12 +949,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.17" - }, - "vscode": { - "interpreter": { - "hash": "ae1fefc8646a06dd2e75004cd934adda7c5727b046986a772e3b44b0ffba9754" - } + "version": "3.12.8" } }, "nbformat": 4, diff --git a/noxfile.py b/noxfile.py index 4af58386..a81f2725 100644 --- a/noxfile.py +++ b/noxfile.py @@ -226,16 +226,27 @@ def pull_artifacts(session): @nox.session def install(session): + from pathlib import Path + + tmp_lamindb_path = Path("tmp_lamindb") + tmp_lamindb_path.mkdir(parents=True, exist_ok=True) + run( session, - "pip install --no-deps git+https://github.com/laminlabs/lnschema-core git+https://github.com/laminlabs/bionty git+https://github.com/laminlabs/lamindb-setup git+https://github.com/laminlabs/wetlab git+https://github.com/laminlabs/clinicore git+https://github.com/laminlabs/cellregistry git+https://github.com/laminlabs/ourprojects", + f"git clone -b main --depth 1 --recursive --shallow-submodules https://github.com/laminlabs/lamindb {str(tmp_lamindb_path)}", ) + run( session, - "pip install" - " lamindb[bionty,jupyter,aws]@git+https://github.com/laminlabs/lamindb@main", + f"uv pip install --system --no-deps {str(tmp_lamindb_path / 'sub/lamindb-setup')} {str(tmp_lamindb_path / 'sub/lamin-cli')} {str(tmp_lamindb_path / 'sub/bionty')} {str(tmp_lamindb_path / 'sub/wetlab')} {str(tmp_lamindb_path / 'sub/clinicore')}", ) - run(session, "pip install spatialdata") # temporarily + run( + session, + f"uv pip install --system {str(tmp_lamindb_path.resolve())}[bionty,jupyter]", + ) + run(session, "pip install --no-deps git+https://github.com/laminlabs/cellregistry") + + run(session, "uv pip install --system spatialdata") # temporarily run(session, "lamin settings set private-django-api true") @@ -250,7 +261,7 @@ def run_nbs(session): def init(session): run( session, - "lamin init --storage ./docsbuild --schema bionty,wetlab,clinicore,cellregistry,ourprojects", + "lamin init --storage ./docsbuild --schema bionty,wetlab,clinicore,cellregistry", )