Update from https://github.com/whylabs-ai/whylogs-container-python/co…

…mmit/905b9e6a64ae02fe5be591525413a65599b81c6f

README.md

@@ -102,7 +102,7 @@ See [configure_container_python][configure_container_python] for an example that
 What goes into this Dockerfile can depend on what you're trying to do. The simplest Dockerfile would look like this.
 
 ```Dockerfile
-FROM registry.gitlab.com/whylabs/langkit-container:1.0.15
+FROM registry.gitlab.com/whylabs/langkit-container:1.0.16
 
 # Force the container to fail if the config is not present. Safeguard for messing up the
 # build in such a way that the config is not included correctly.
@@ -116,7 +116,7 @@ You're in full control of this Dockerfile and build and you can do basically any
 you might want to include some additional pip dependencies as well, which could look like this.
 
 ```Dockerfile
-FROM registry.gitlab.com/whylabs/langkit-container:1.0.15
+FROM registry.gitlab.com/whylabs/langkit-container:1.0.16
 
 # Force the container to fail if the config is not present. Safeguard for messing up the
 # build in such a way that the config is not included correctly.

RELEASE_NOTES.md

@@ -4,6 +4,34 @@
 
 - Removed the `/validate/llm` endpoint. It was deprecated for a while and the `/evaluate` endpoint can do everything it was doing and more.
 
+## Validation Levels
+
+This adds the ability to mark a validation as either block/flag. By default, validations have the block flag, which mean that the container
+will use them to determine that a request should be blocked in the `action` section of the response. Validation failures that have the flag
+`flag` won't be considered for block decisions, but will still appear in the validation report with that flag attached to them.
+
+These can be set in the policy as follows.
+
+```yaml
+id: my_id
+policy_version: 1
+schema_version: 0.0.1
+whylabs_dataset_id: default
+
+metrics:
+  - metric: prompt.topics
+    options:
+      topics:
+        - medical
+
+validators:
+  - validator: constraint
+    options:
+      target_metric: prompt.topics.medical
+      upper_threshold: 0
+      failure_level: flag # defaults to block, the previous behavior
+```
+
 ## New Refusal Metric
 
 A new pattern based refusal metric is available. This checks the response for known refusal phrases.

examples/configure_container_python/Dockerfile

@@ -1,5 +1,5 @@
 # DOCSUB_START simple_dockerfile
-FROM registry.gitlab.com/whylabs/langkit-container:1.0.15
+FROM registry.gitlab.com/whylabs/langkit-container:1.0.16
 
 # Force the container to fail if the config is not present. Safeguard for messing up the
 # build in such a way that the config is not included correctly.

examples/configure_container_python/pyproject.toml

@@ -13,7 +13,7 @@ python = "^3.10"
 # These are all dev dependencies. They're already included in the container and we don't want to
 # overwrite those versions, we just want types and auto completion in this project.
 langkit = "0.0.28.dev2"
-whylogs-container-client = "1.0.15"
+whylogs-container-client = "1.0.16"
 whylogs-container-types = "0.4.13"
 pandas = "1.3.5"
 

examples/configure_container_yaml/Dockerfile

@@ -1,4 +1,4 @@
-FROM registry.gitlab.com/whylabs/langkit-container:1.0.15
+FROM registry.gitlab.com/whylabs/langkit-container:1.0.16
 
 # Force the container to fail if the config is not present. Safeguard for messing up the
 # build in such a way that the config is not included correctly.

examples/configure_container_yaml/pyproject.toml

@@ -12,7 +12,7 @@ python = "^3.10"
 [tool.poetry.group.dev.dependencies]
 # These are all dev dependencies. They're already included in the container and we don't want to
 # overwrite those versions, we just want types and auto completion in this project.
-whylogs-container-client = "1.0.15"
+whylogs-container-client = "1.0.16"
 pandas = "1.3.5"
 
 pyright = "^1.1.347"