diff --git a/deepsearch/model/README.md b/deepsearch/model/README.md index 2d7c7580..e69de29b 100644 --- a/deepsearch/model/README.md +++ b/deepsearch/model/README.md @@ -1,83 +0,0 @@ -# Model API -> Currently in **beta**. - -The Model API is a unified and extensible inference API across different model kinds. - -Built-in model kind support includes NLP annotators and QA generators. - -## Installation -To use the Model API, install including the `api` extra, i.e.: -- with poetry: -`poetry add "deepsearch-toolkit[api]"` -- with pip: `pip install "deepsearch-toolkit[api]"` - -## Basic usage -```python -from deepsearch.model.server.config import Settings -from deepsearch.model.server.model_app import ModelApp - -# (1) create an app -app = ModelApp(settings=Settings()) - -# (2) register your model(s) -model = ... # e.g. SimpleGeoNLPAnnotator() -app.register_model(model) - -# (3) run the app -app.run(host="127.0.0.1", port=8000) -``` - -### Settings -App configuration is done in [`Settings`](server/config.py) based on -[Pydantic Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings/). - -E.g. the required API key can be injected via env var `DS_MODEL_API_KEY`. - -### OpenAPI -The OpenAPI UI is served under `/docs`, i.e. by default at http://127.0.0.1:8000/docs. - -## Developing a new model -To develop a new model class for an existing [kind](kinds/), inherit from the base model -class of that kind and implement the abstract methods and attributes. - -The framework will automatically use the correct controller for your model. - -To use a custom controller instead, pass it to `ModelApp.register_model()` via the -optional parameter `controller`. - -### Examples -- [Dummy NLP annotator](examples/dummy_nlp_annotator/) -- [Simple geo NLP annotator](examples/simple_geo_nlp_annotator/) -- [Dummy QA generator](examples/dummy_qa_generator/) - -Note: these examples configure the app with API key "example123"; when running them, use -the same key to access the protected endpoints. - -### Inference -As as example, an input payload for the `/predict` endpoint for the geography annotator -could look as follows (note that `deepsearch.res.ibm.com/x-deadline` should be a -future timestamp): -```json -{ - "apiVersion": "v1", - "kind": "NLPModel", - "metadata": { - "annotations": { - "deepsearch.res.ibm.com/x-deadline": "2024-04-20T12:26:01.479484+00:00", - "deepsearch.res.ibm.com/x-transaction-id": "abc", - "deepsearch.res.ibm.com/x-attempt-number": 5, - "deepsearch.res.ibm.com/x-max-attempts": 5 - } - }, - "spec": { - "findEntities": { - "entityNames": ["cities", "countries"], - "objectType": "text", - "texts": [ - "Bern, the capital city of Switzerland, is built around a crook in the Aare River.", - "Athens is a major coastal urban area in the Mediterranean and is both the capital and largest city of Greece." - ] - } - } -} -``` diff --git a/deepsearch/model/examples/dummy_nlp_annotator/README.md b/deepsearch/model/examples/dummy_nlp_annotator/README.md new file mode 100644 index 00000000..4b1e444c --- /dev/null +++ b/deepsearch/model/examples/dummy_nlp_annotator/README.md @@ -0,0 +1,13 @@ +# DummyNlpAnnotator +## Running the Annotator +To run this example make sure you've installed the full environment including the optional installs provided in poetry + + poetry install --all-extras + +Then simply start the server with + + python -m deepsearch.model.examples.dummy_nlp_annotator.main + +## Interaction with the Annotator + +refer to [https://ds4sd.github.io/deepsearch-toolkit/guide/](https://ds4sd.github.io/deepsearch-toolkit/guide/model/) diff --git a/deepsearch/model/examples/dummy_qa_generator/README.md b/deepsearch/model/examples/dummy_qa_generator/README.md new file mode 100644 index 00000000..98d9f2ff --- /dev/null +++ b/deepsearch/model/examples/dummy_qa_generator/README.md @@ -0,0 +1,16 @@ +# DummyNlpAnnotator +## Introduction +This is an example dummy QA kind annotator. + +## Running the Annotator +To run this example make sure you've installed the full environment including the optional installs provided in poetry + + poetry install --all-extras + +Then simply start the server with + + python -m deepsearch.model.examples.dummy_qa_generator.main + +## Interaction with the Annotator + +refer to [https://ds4sd.github.io/deepsearch-toolkit/guide/](https://ds4sd.github.io/deepsearch-toolkit/guide/model/) \ No newline at end of file diff --git a/deepsearch/model/examples/simple_geo_nlp_annotator/README.md b/deepsearch/model/examples/simple_geo_nlp_annotator/README.md new file mode 100644 index 00000000..fa5cb54b --- /dev/null +++ b/deepsearch/model/examples/simple_geo_nlp_annotator/README.md @@ -0,0 +1,16 @@ +# SimpleGeoNLPAnnotator +## Introduction +This is an example SimpleGeoNLPAnnotator NLP kind annotator it supports text data and annotates entities and relationships. + +## Running the Annotator +To run this example make sure you've installed the full environment including the optional installs provided in poetry + + poetry install --all-extras + +Then simply start the server with + + python -m deepsearch.model.examples.simple_geo_nlp_annotator.main + +## Interaction with the Annotator + +refer to [https://ds4sd.github.io/deepsearch-toolkit/guide/](https://ds4sd.github.io/deepsearch-toolkit/guide/model/) \ No newline at end of file diff --git a/docs/guide/index.md b/docs/guide/index.md index 885bf761..97be0dc8 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -16,3 +16,6 @@ - [List and manage KGs](./kgs.md) - [Operate with manual API calls](./apis.md) - [Custom CLI plugins](./cli_plugins.md) + +## Custom models +- [Model API](./model.md) \ No newline at end of file diff --git a/docs/guide/model.md b/docs/guide/model.md new file mode 100644 index 00000000..317fe6f1 --- /dev/null +++ b/docs/guide/model.md @@ -0,0 +1,402 @@ +# Model API +> Currently in **beta**. + +The Model API is a unified and extensible inference API across different model kinds. + +Built-in model kind support includes NLP annotators and QA generators. + +## Installation +To use the Model API, install including the `api` extra, i.e.: +- with poetry: +`poetry add "deepsearch-toolkit[api]"` +- with pip: `pip install "deepsearch-toolkit[api]"` + +To launch a model: + + python -m deepsearch.model.examples.. + +Illustrated by running the dummy_nlp_annotator example below + + python -m deepsearch.model.examples.dummy_nlp_annotator.main + +### Security + +By default, the API requires an API-key to be used with every request to most endpoints, this key is defined on a per model basis, as an example: + +```python + # deepsearch/model/examples/dummy_nlp_annotator/main.py + ... + + def run(): + -> settings = Settings(api_key="example123") <- + app = ModelApp(settings) + app.register_model(DummyNLPAnnotator()) + ... +``` +this API key must be provided on the authorization header for most application endpoints + +## A map of the annotator endpoints + + - / - A list of all the annotators hosted on this server with all their information. + - /model/{model_name} - You will find the annotation capabilities for the given annotator. + - /model/{model_name}/predict - You can make POST requests to have the model annotate your data, refer to the [Sample Requests](#Sample NLP kind models requests and responses) + - /health - An endpoint that will respond with a preset message letting you know that the webserver is healthy. + +### OpenAPI + +The OpenAPI UI is served under `/docs`, e.g. http://127.0.0.1:8000/docs. + +## Developing a new model +To develop a new model class for an existing [kind](kinds/), inherit from the base model +class of that kind and implement the abstract methods and attributes. + +The framework will automatically use the correct controller for your model. + +To use a custom controller instead, pass it to `ModelApp.register_model()` via the +optional parameter `controller`. + +### Examples +- [Dummy NLP annotator](examples/dummy_nlp_annotator/) +- [Simple geo NLP annotator](examples/simple_geo_nlp_annotator/) +- [Dummy QA generator](examples/dummy_qa_generator/) + +## Sample NLP kind models requests and responses + +### Entity annotation +??? note "Entity annotation request payload" + ```json + { + "apiVersion": "string", + "kind": "NLPModel", + "metadata": { + "annotations": { + "deepsearch.res.ibm.com/x-deadline": "2038-01-18T00:00:00.000Z", + "deepsearch.res.ibm.com/x-transaction-id": "string", + "deepsearch.res.ibm.com/x-attempt-number": "string", + "deepsearch.res.ibm.com/x-max-attempts": "string" + } + }, + "spec": { + "findEntities": { + "entityNames": ["entity_foo", "entity_bar"], + "objectType": "text", + "texts": [ + "A piece of text", + "Yet another piece of text" + ] + } + } + } + ``` + +??? note "Entity annotation request response" + ```json + { + "entities":[ + { + "entity_foo":[ + { + "type":"entity_foo", + "match":"a 'entity_foo' match in 'A piece of text'", + "original":"a 'entity_foo' original in 'A piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_foo", + "match":"another 'entity_foo' match in 'A piece of text'", + "original":"another 'entity_foo' original in 'A piece of text'", + "range":[ + 12, + 42 + ] + } + ], + "entity_bar":[ + { + "type":"entity_bar", + "match":"a 'entity_bar' match in 'A piece of text'", + "original":"a 'entity_bar' original in 'A piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_bar", + "match":"another 'entity_bar' match in 'A piece of text'", + "original":"another 'entity_bar' original in 'A piece of text'", + "range":[ + 12, + 42 + ] + } + ] + }, + { + "entity_foo":[ + { + "type":"entity_foo", + "match":"a 'entity_foo' match in 'Yet another piece of text'", + "original":"a 'entity_foo' original in 'Yet another piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_foo", + "match":"another 'entity_foo' match in 'Yet another piece of text'", + "original":"another 'entity_foo' original in 'Yet another piece of text'", + "range":[ + 12, + 42 + ] + } + ], + "entity_bar":[ + { + "type":"entity_bar", + "match":"a 'entity_bar' match in 'Yet another piece of text'", + "original":"a 'entity_bar' original in 'Yet another piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_bar", + "match":"another 'entity_bar' match in 'Yet another piece of text'", + "original":"another 'entity_bar' original in 'Yet another piece of text'", + "range":[ + 12, + 42 + ] + } + ] + } + ] + } + ``` + +### Relationship annotation +??? note "Relationship annotation request payload" + ```json + { + "apiVersion":"string", + "kind":"NLPModel", + "metadata":{ + "annotations":{ + "deepsearch.res.ibm.com/x-deadline":"2038-01-18T00:00:00.000Z", + "deepsearch.res.ibm.com/x-transaction-id":"string", + "deepsearch.res.ibm.com/x-attempt-number":"string", + "deepsearch.res.ibm.com/x-max-attempts":"string" + } + }, + "spec":{ + "findRelationships":{ + "relationshipNames": null, + "objectType":"text", + "texts":[ + "Lisbon, Madrid, Paris and Zurich are Capitals of european countries", + "Berlin is the capital of Germany" + ], + "entities":[ + { + "cities":[ + { + "type":"cities", + "match":"Lisbon", + "original":"Lisbon", + "range":[ + 0, + 6 + ] + }, + { + "type":"cities", + "match":"Madrid", + "original":"Madrid", + "range":[ + 8, + 14 + ] + }, + { + "type":"cities", + "match":"Paris", + "original":"Paris", + "range":[ + 16, + 21 + ] + } + ], + "countries":[ + + ] + }, + { + "cities":[ + { + "type":"cities", + "match":"Berlin", + "original":"Berlin", + "range":[ + 0, + 6 + ] + } + ], + "countries":[ + { + "type":"countries", + "match":"Germany", + "original":"Germany", + "range":[ + 25, + 32 + ] + } + ] + } + ] + } + } + } + ``` + +??? note "Relationship annotation request response" + ```json + { + "relationships": [ + { + "cities-to-countries": { + "header": [ + "cities", + "countries", + "weight", + "source" + ], + "data": [] + }, + "cities-to-provincies": { + "header": [ + "cities", + "provincies", + "weight", + "source" + ], + "data": [] + }, + "provincies-to-countries": { + "header": [ + "provincies", + "countries", + "weight", + "source" + ], + "data": [] + } + }, + { + "cities-to-countries": { + "header": [ + "cities", + "countries", + "weight", + "source" + ], + "data": [ + [ + "cities.0", + "countries.0", + 1, + "entities" + ] + ] + }, + "cities-to-provincies": { + "header": [ + "cities", + "provincies", + "weight", + "source" + ], + "data": [] + }, + "provincies-to-countries": { + "header": [ + "provincies", + "countries", + "weight", + "source" + ], + "data": [] + } + } + ] + } + ``` +### Property annotation + TBD +## Sample QAGen kind models requests and responses + +### Generate +??? note "Genarate request payload" + ```json + { + "apiVersion":"v1", + "kind":"QAGenModel", + "metadata":{ + "annotations":{ + "deepsearch.res.ibm.com/x-deadline":"2028-04-20T12:26:01.479484+00:00", + "deepsearch.res.ibm.com/x-transaction-id":"testing", + "deepsearch.res.ibm.com/x-attempt-number":5, + "deepsearch.res.ibm.com/x-max-attempts":5 + } + }, + "spec":{ + "generateAnswers":{ + "contexts":[ + [ + { + "text":"A textual transformation of a given table", + "type":"table", + "representation_type":"triplets" + }, + { + "text":"A raw paragraph as it appears on the raw text", + "type":"text", + "representation_type":"raw" + } + ] + ], + "questions":[ + "42" + ] + } + } + } + ``` + +??? note "Generate request response" + ```json + { + "answers": [ + "If you are a dummy repeat what I said!" + ] + } + ``` + +## Important considerations + +- Each annotator has a kind, for example NLPModel, as such the kind for the request must match. +- For NLP Kind annotators under the spec you must define the appropriate types to be annotated, for the dummyNLPAnnotator +[refer to this example](#marker1) you will find on the request that we would like to find *entity_foo* and *entity_bar* an empty list will lead to +no annotations being made, a null object will lead to **all** possible annotations being made. +- Each annotator declares what sort of input it supports, a list constituted of any number of (text, table and image). +- The x-deadline on each request is already implemented and must lie some time in the future. +- refer to the /docs page on any annotator instance for more specification on the request types \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 49f024f1..96e692c7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -60,6 +60,7 @@ nav: - Knowledge graphs: guide/kgs.md - APIs: guide/apis.md - Plugin system: guide/cli_plugins.md + - Model API: guide/model.md - Example gallery: gallery/index.md - API reference: - Toolkit reference: api-reference.md