fix: update plugin to include docs and fix docker build (#7)

* update plugin to include docs and fix docker build

It is really hard to install python 3.12 alongside
the latest ubuntu (jammy) and flux for it.

Signed-off-by: vsoch <[email protected]>

.github/workflows/ci_true_api.yml

@@ -0,0 +1,23 @@
+name: test-flux
+
+on:
+  workflow_dispatch:
+
+jobs:
+  testing-true-api:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.11' 
+
+      - name: Install poetry
+        run: pip install poetry
+
+      - name: Determine dependencies
+        run: poetry lock
+
+      - name: Install dependencies
+        run: poetry install

.github/workflows/main.yaml

@@ -0,0 +1,18 @@
+name: test snakemake-executor-flux
+
+on:
+  pull_request: []
+
+jobs:
+  formatting:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+
+    - name: Setup black linter
+      run: conda create --quiet --name black pytest
+
+    - name: Check Spelling
+      uses: crate-ci/typos@7ad296c72fa8265059cc03d1eda562fbdfcd6df2 # v1.9.0
+      with:
+        files: ./README.md

.github/workflows/release-please.yml

@@ -26,7 +26,7 @@ jobs:
 
       - uses: actions/setup-python@v2
         with:
-          python-version: "3.9"
+          python-version: "3.11"
 
       - name: Install poetry
         run: pipx install poetry
@@ -36,7 +36,7 @@ jobs:
 
       - uses: actions/setup-python@v4
         with:
-          python-version: "3.9"
+          python-version: "3.11"
           cache: poetry
 
       - name: Install Dependencies using Poetry

README.md

@@ -101,5 +101,11 @@ plugin for use or development (that doesn't need to be added to upstream snakema
 
 ## Developer
 
+To do the same run but bind the local plugin directory:
+
+```bash
+docker run -it -v $PWD/:/home/fluxuser/plugin flux-snake bash
+```
+
 The instructions for creating and scaffolding this plugin are [here](https://github.com/snakemake/poetry-snakemake-plugin#scaffolding-an-executor-plugin).
-Instructions for writing your plugin with examples are provided via the [snakemake-executor-plugin-interface](https://github.com/snakemake/snakemake-executor-plugin-interface).
+Instructions for writing your plugin with examples are provided via the [snakemake-executor-plugin-interface](https://github.com/snakemake/snakemake-executor-plugin-interface).

docs/further.md

@@ -0,0 +1,111 @@
+# Snakemake Executor Flux
+
+This is an example implementation for an external snakemake plugin.
+Since we already have one for Flux (and it can run in a container) the example
+is for Flux. You can use this repository as a basis to design your own executor to work
+with snakemake!
+
+## Usage
+
+### Tutorial
+
+For this tutorial you will need Docker installer.
+
+[Flux-framework](https://flux-framework.org/) is a flexible resource scheduler that can work on both high performance computing systems and cloud (e.g., Kubernetes).
+Since it is more modern (e.g., has an official Python API) we define it under a cloud resource. For this example, we will show you how to set up a "single node" local
+Flux container to interact with snakemake using the plugin here. You can use the [Dockerfile](examples/Dockerfile) that will provide a container with Flux and snakemake
+Note that we install from source and bind to `/home/fluxuser/snakemake` with the intention of being able to develop (if desired).
+
+First, build the container:
+
+```bash
+$ docker build -f example/Dockerfile -t flux-snake .
+```
+
+We will add the plugin here to `/home/fluxuser/plugin`, install it, and shell in as the fluxuser to optimally interact with flux.
+After the container builds, shell in:
+
+```bash
+$ docker run -it flux-snake bash
+```
+
+And start a flux instance:
+
+```bash
+$ flux start --test-size=4
+```
+
+Go into the examples directory (where the Snakefile is) and run snakemake, targeting your executor plugin.
+
+```bash
+$ cd ./example
+
+# This says "use the custom executor module named snakemake_executor_plugin_flux"
+$ snakemake --jobs 1 --executor flux
+```
+```console
+Building DAG of jobs...
+Using shell: /bin/bash
+Job stats:
+job                         count    min threads    max threads
+------------------------  -------  -------------  -------------
+all                             1              1              1
+multilingual_hello_world        2              1              1
+total                           3              1              1
+
+Select jobs to execute...
+
+[Fri Jun 16 19:24:22 2023]
+rule multilingual_hello_world:
+    output: hola/world.txt
+    jobid: 2
+    reason: Missing output files: hola/world.txt
+    wildcards: greeting=hola
+    resources: tmpdir=/tmp
+
+Job 2 has been submitted with flux jobid ƒcjn4t3R (log: .snakemake/flux_logs/multilingual_hello_world/greeting_hola.log).
+[Fri Jun 16 19:24:32 2023]
+Finished job 2.
+1 of 3 steps (33%) done
+Select jobs to execute...
+
+[Fri Jun 16 19:24:32 2023]
+rule multilingual_hello_world:
+    output: hello/world.txt
+    jobid: 1
+    reason: Missing output files: hello/world.txt
+    wildcards: greeting=hello
+    resources: tmpdir=/tmp
+
+Job 1 has been submitted with flux jobid ƒhAPLa79 (log: .snakemake/flux_logs/multilingual_hello_world/greeting_hello.log).
+[Fri Jun 16 19:24:42 2023]
+Finished job 1.
+2 of 3 steps (67%) done
+Select jobs to execute...
+
+[Fri Jun 16 19:24:42 2023]
+localrule all:
+    input: hello/world.txt, hola/world.txt
+    jobid: 0
+    reason: Input files updated by another job: hello/world.txt, hola/world.txt
+    resources: tmpdir=/tmp
+
+[Fri Jun 16 19:24:42 2023]
+Finished job 0.
+3 of 3 steps (100%) done
+Complete log: .snakemake/log/2023-06-16T192422.186675.snakemake.log
+```
+
+And that's it! Continue reading to learn more about plugin design, and how you can also design your own executor
+plugin for use or development (that doesn't need to be added to upstream snakemake).
+
+## Developer
+
+To do the same run but bind the local plugin directory:
+
+```bash
+docker run -it -v $PWD/:/home/fluxuser/plugin flux-snake bash
+```
+
+The instructions for creating and scaffolding this plugin are [here](https://github.com/snakemake/poetry-snakemake-plugin#scaffolding-an-executor-plugin).
+Instructions for writing your plugin with examples are provided via the [snakemake-executor-plugin-interface](https://github.com/snakemake/snakemake-executor-plugin-interface).

docs/intro.md

@@ -0,0 +1 @@
+This is the [Flux Framework](https://flux-framework.org) external executor plugin for snakemake.