Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: updating docs to support kubectl and seldon-cli #6219

Open
wants to merge 15 commits into
base: v2
Choose a base branch
from
Open

docs: updating docs to support kubectl and seldon-cli #6219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
9f6dd17
docs: updating docs to support kubectl and seldon-cli
abhimanyu003 Jan 22, 2025
94661e6
docs: updating docs to support kubectl and seldon-cli
abhimanyu003 Jan 22, 2025
a3e49db
docs: adding curl command and seldon-cli
abhimanyu003 Jan 22, 2025
59e5e53
docs: adding curl command and seldon-cli
abhimanyu003 Jan 22, 2025
4dd103c
docs: adding curl command and seldon-cli
abhimanyu003 Jan 22, 2025
7814227
docs: typo fix
abhimanyu003 Jan 22, 2025
07c2f76
docs: adding seldon-cli and curl commands
abhimanyu003 Jan 23, 2025
7438572
docs: add delete model commands
abhimanyu003 Jan 23, 2025
bad37b7
docs: updating model-zoom with cli and curl
abhimanyu003 Jan 23, 2025
af6f93f
docs: updating pandasquery.md with cli and curl
abhimanyu003 Jan 27, 2025
b31ecb6
docs: adding kubectl status
abhimanyu003 Jan 27, 2025
2e03f90
docs: adding kubectl status
abhimanyu003 Jan 27, 2025
44b2e7f
docs: updating docs for seldon cli and kubectl
abhimanyu003 Jan 27, 2025
5574d69
docs: updating docs for seldon cli and kubectl
abhimanyu003 Jan 27, 2025
576cd6c
docs: updating docs for seldon cli and kubectl
abhimanyu003 Jan 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs-gb/examples/cifar10.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,34 @@ spec:

```

{% tabs %}
{% tab title="kubectl" %}
```bash
kubectl apply -f ../../models/cifar10.yaml -n ${NAMESPACE}
kubectl apply -f ../../models/cifar10-outlier-detect.yaml -n ${NAMESPACE}
kubectl apply -f ../../models/cifar10-drift-detect.yaml -n ${NAMESPACE}
```

```
model.mlops.seldon.io/cifar10 created
model.mlops.seldon.io/cifar10-outlier created
model.mlops.seldon.io/cifar10-drift created
```

```bash
kubectl wait --for condition=ready --timeout=300s model cifar10 -n ${NAMESPACE}
kubectl wait --for condition=ready --timeout=300s model cifar10-outlier -n ${NAMESPACE}
kubectl wait --for condition=ready --timeout=300s model cifar10-drift -n ${NAMESPACE}
```

```
model.mlops.seldon.io/cifar10 condition met
model.mlops.seldon.io/cifar10-outlier met
model.mlops.seldon.io/cifar10-drift condition met
```
{% endtab %}

{% tab title="seldon-cli" %}
```bash
seldon model load -f ../../models/cifar10.yaml
seldon model load -f ../../models/cifar10-outlier-detect.yaml
Expand All @@ -186,6 +214,10 @@ seldon model status cifar10-drift -w ModelAvailable | jq .
{}

```
{% endtab %}
{% endtabs %}



```bash
cat ../../pipelines/cifar10.yaml
Expand All @@ -210,6 +242,23 @@ spec:

```

{% tabs %}
{% tab title="kubectl" %}

```bash
kubectl apply -f ../../pipelines/cifar10.yaml -n ${NAMESPACE}
```

```bash
kubectl wait --for condition=ready --timeout=300s pipelines cifar10-production -n ${NAMESPACE}
```

```
pipeline.mlops.seldon.io/cifar10-production condition met
```
{% endtab %}

{% tab title="seldon-cli" %}
```bash
seldon pipeline load -f ../../pipelines/cifar10.yaml
```
Expand Down Expand Up @@ -261,6 +310,9 @@ seldon pipeline status cifar10-production -w PipelineReady | jq -M .
}

```
{% endtab %}
{% endtabs %}


```python
infer("cifar10-production.pipeline",20, "normal")
Expand Down Expand Up @@ -457,6 +509,20 @@ seldon.default.model.cifar10.outputs cifeiq8fh5ss738i5bqg {"modelName":"cifar10_

```


{% tabs %}
{% tab title="kubectl" %}
```bash
kubectl delete -f ../../piplines/cifar10.yaml -n ${NAMESPACE}
kubectl delete -f ../../models/cifar10.yaml -n ${NAMESPACE}
kubectl delete -f ../../models/cifar10-outlier-detect.yaml -n ${NAMESPACE}
kubectl delete -f ../../models/cifar10-drift-detect.yaml -n ${NAMESPACE}
```

{% endtab %}

{% tab title="seldon-cli" %}

```bash
seldon pipeline unload cifar10-production
```
Expand All @@ -466,3 +532,5 @@ seldon model unload cifar10
seldon model unload cifar10-outlier
seldon model unload cifar10-drift
```
{% endtab %}
{% endtabs %}
162 changes: 159 additions & 3 deletions docs-gb/examples/huggingface.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,29 +20,97 @@ spec:

Load the model

{% tabs %}

{% tab title="kubectl" %}
```bash
seldon model load -f ./models/hf-text-gen.yaml
kubectl apply -f ./models/hf-text-gen.yaml
```
```bash
model.mlops.seldon.io/text-gen created
```
{% endtab %}

{% tab title="seldon-cli" %}
```bash
seldon model load -f ./models/hf-text-gen.yaml
```
```json
{}
```
{% endtab %}

{% endtabs %}


Wait for the model to be ready

{% tabs %}

{% tab title="kubectl" %}
```bash
kubectl get model text-gen -n ${NAMESPACE} -o json | jq -r '.status.conditions[] | select(.message == "ModelAvailable") | .status'
```
```bash
True
```
{% endtab %}

{% tab title="seldon-cli" %}

```bash
seldon model status text-gen -w ModelAvailable | jq -M .
```

```json
{}
```
{% endtab %}

{% endtabs %}


Do a REST inference call

{% tabs %}

{% tab title="kubectl" %}
```bash
curl --location 'http://${MESH_IP}:9000/v2/models/text-gen/infer' \
--header 'Content-Type: application/json' \
--data '{"inputs": [{"name": "args","shape": [1],"datatype": "BYTES","data": ["Once upon a time in a galaxy far away"]}]}'
```
```json
{
"model_name": "text-gen_1",
"model_version": "1",
"id": "121ff5f4-1d4a-46d0-9a5e-4cd3b11040df",
"parameters": {},
"outputs": [
{
"name": "output",
"shape": [
1,
1
],
"datatype": "BYTES",
"parameters": {
"content_type": "hg_jsonlist"
},
"data": [
"{\"generated_text\": \"Once upon a time in a galaxy far away, the planet is full of strange little creatures. A very strange combination of creatures in that universe, that is. A strange combination of creatures in that universe, that is. A kind of creature that is\"}"
]
}
]
}

```
{% endtab %}

{% tab title="seldon-cli" %}
```bash
seldon model infer text-gen \
'{"inputs": [{"name": "args","shape": [1],"datatype": "BYTES","data": ["Once upon a time in a galaxy far away"]}]}'
```

```json
{
"model_name": "text-gen_1",
Expand All @@ -68,6 +136,10 @@ seldon model infer text-gen \
}

```
{% endtab %}

{% endtabs %}


```python
res = !seldon model infer text-gen --inference-mode grpc \
Expand All @@ -88,9 +160,22 @@ b'{"generated_text": "Once upon a time in a galaxy far away\\n\\nThe Universe is

Unload the model

{% tabs %}

{% tab title="kubectl" %}
```bash
kubectl delete model text-gen
```
{% endtab %}

{% tab title="seldon-cli" %}
```bash
seldon model unload text-gen
```
{% endtab %}

{% endtabs %}


### Custom Text Generation Model

Expand All @@ -112,6 +197,61 @@ spec:

Load the model


{% tabs %}

{% tab title="kubectl" %}

```bash
kubectl apply -f ./models/hf-text-gen-custom-tiny-stories.yaml
```

```bash
model.mlops.seldon.io/custom-tiny-stories-text-gen created
```

```bash
kubectl get model custom-tiny-stories-text-gen -n ${NAMESPACE} -o json | jq -r '.status.conditions[] | select(.message == "ModelAvailable") | .status'
```
```bash
True
```

```bash
curl --location 'http://${MESH_IP}:9000/v2/models/custom-tiny-stories-text-gen/infer' \
--header 'Content-Type: application/json' \
--data '{"inputs": [{"name": "args","shape": [1],"datatype": "BYTES","data": ["Once upon a time in a galaxy far away"]}]}'
```

```json
{
"model_name": "custom-tiny-stories-text-gen_1",
"model_version": "1",
"id": "d0fce59c-76e2-4f81-9711-1c93d08bcbf9",
"parameters": {},
"outputs": [
{
"name": "output",
"shape": [
1,
1
],
"datatype": "BYTES",
"parameters": {
"content_type": "hg_jsonlist"
},
"data": [
"{\"generated_text\": \"Once upon a time in a galaxy far away. It was a very special place to live.\\n\"}"
]
}
]
}

```
{% endtab %}

{% tab title="seldon-cli" %}

```bash
seldon model load -f ./models/hf-text-gen-custom-tiny-stories.yaml
```
Expand Down Expand Up @@ -160,6 +300,10 @@ seldon model infer custom-tiny-stories-text-gen \
}

```
{% endtab %}

{% endtabs %}


```python
res = !seldon model infer custom-tiny-stories-text-gen --inference-mode grpc \
Expand All @@ -180,9 +324,21 @@ b'{"generated_text": "Once upon a time in a galaxy far away\\nOne night, a littl

Unload the model

{% tabs %}
{% tab title="kubectl" %}
```bash
kubectl delete custom-tiny-stories-text-gen
```
{% endtab %}

{% tab title="seldon-cli" %}
```bash
seldon model unload custom-tiny-stories-text-gen
```
{% endtab %}
{% endtabs %}



````
As a next step, why not try running a larger-scale model? You can find a definition for one in ./models/hf-text-gen-custom-gpt2.yaml. However, you may need to request and allocate more memory!
Expand Down
Loading
Loading