Better analyze information, in all its forms
Documentation: https://icij.github.io/datashare-python
Most AI, Machine Learning, Data Engineering happens in Python. Datashare now lets you extend its backend with your own tasks implemented in Python.
Turning your own ML pipelines into Datashare tasks is very simple, learn about it inside documentation.
Turning your own ML pipelines into Datashare tasks is very simple.
Actually, it's almost as simple as cloning our template repo:
$ git clone [email protected]:ICIJ/datashare-python.git
replacing existing app tasks with your own:
from icij_worker import AsyncApp
app = AsyncApp("app")
@app.task
def hello_world() -> str:
return "Hello world"installing uv to set up dependencies and running your async Datashare worker:
$ cd datashare-python
$ curl -LsSf https://astral.sh/uv/install.sh | sh
$ uv run ./scripts/worker_entrypoint.sh
[INFO][icij_worker.backend.backend]: Loading worker configuration from env...
...
}
[INFO][icij_worker.backend.mp]: starting 1 worker for app datashare_python.app.app
...you'll then be able to execute task by starting using our HTTP client (and soon using Datashare's UI).