From 2942452434e90d638bb65a656d703b57915101c6 Mon Sep 17 00:00:00 2001 From: Adam Eri Date: Fri, 15 Nov 2024 13:21:56 +0100 Subject: [PATCH] [bitnami/mlflow] Adding missing Azure Storage configuration in the Helm Chart (#29626) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [bitnami/mlflow] Azure Storage Support in helm chart (#1) * Added Azure Storage Account as artefact store * Added environment variables for azure authentication Signed-off-by: Adam Eri * Update CHANGELOG.md Signed-off-by: Bitnami Containers * Update CHANGELOG.md Signed-off-by: Bitnami Containers * Update bitnami/mlflow/Chart.yaml Co-authored-by: Juan José Martos Signed-off-by: Adam Eri * Update bitnami/mlflow/templates/tracking/deployment.yaml Co-authored-by: Juan José Martos Signed-off-by: Adam Eri * Update bitnami/mlflow/templates/tracking/deployment.yaml Co-authored-by: Juan José Martos Signed-off-by: Adam Eri * Update bitnami/mlflow/templates/tracking/deployment.yaml Co-authored-by: Juan José Martos Signed-off-by: Adam Eri * Update CHANGELOG.md Signed-off-by: Bitnami Containers * Update CHANGELOG.md Signed-off-by: Bitnami Containers --------- Signed-off-by: Adam Eri Signed-off-by: Bitnami Containers Co-authored-by: Bitnami Containers Co-authored-by: Juan José Martos --- bitnami/mlflow/CHANGELOG.md | 6 ++- bitnami/mlflow/Chart.yaml | 2 +- bitnami/mlflow/README.md | 18 +++++++ bitnami/mlflow/templates/_helpers.tpl | 11 +++- .../mlflow/templates/tracking/deployment.yaml | 50 ++++++++++++++++++- bitnami/mlflow/values.yaml | 32 ++++++++++++ 6 files changed, 115 insertions(+), 4 deletions(-) diff --git a/bitnami/mlflow/CHANGELOG.md b/bitnami/mlflow/CHANGELOG.md index 22696e079094e4..f4cdca21866ec8 100644 --- a/bitnami/mlflow/CHANGELOG.md +++ b/bitnami/mlflow/CHANGELOG.md @@ -1,8 +1,12 @@ # Changelog +## 2.2.0 (2024-11-15) + +* [bitnami/mlflow] Adding missing Azure Storage configuration in the Helm Chart ([#29626](https://github.com/bitnami/charts/pull/29626)) + ## 2.1.0 (2024-11-04) -* [bitnami/mlflow] Make mlflow host part of values ([#29862](https://github.com/bitnami/charts/pull/29862)) +* [bitnami/mlflow] Make mlflow host part of values (#29862) ([148ff74](https://github.com/bitnami/charts/commit/148ff74ddb1e0aa6c268f37a4a8f98d08fe821fb)), closes [#29862](https://github.com/bitnami/charts/issues/29862) ## 2.0.5 (2024-10-31) diff --git a/bitnami/mlflow/Chart.yaml b/bitnami/mlflow/Chart.yaml index 825fe0611be0c0..4c8643516056a3 100644 --- a/bitnami/mlflow/Chart.yaml +++ b/bitnami/mlflow/Chart.yaml @@ -44,4 +44,4 @@ sources: - https://github.com/bitnami/charts/tree/main/bitnami/mlflow - https://github.com/bitnami/containers/tree/main/bitnami/mlflow - https://github.com/mlflow/mlflow -version: 2.1.0 +version: 2.2.0 diff --git a/bitnami/mlflow/README.md b/bitnami/mlflow/README.md index 7d4852a0b38c7c..c63f0c4d216690 100644 --- a/bitnami/mlflow/README.md +++ b/bitnami/mlflow/README.md @@ -472,6 +472,24 @@ The command deploys mlflow on the Kubernetes cluster in the default configuratio | `externalGCS.existingSecretKey` | Key in the existing secret containing the application credentials (required when useCredentialsInSecret is true) | `""` | | `externalGCS.serveArtifacts` | Whether artifact serving is enabled | `true` | +### External Azure Blob Storage parameters + +| Name | Description | Value | +| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------- | +| `externalAzureBlob.storageAccount` | Azure Blob Storage account name. Activate azure artifact storage if set, | `""` | +| `externalAzureBlob.accessKey` | Azure Blob Storage access key. Optional if connectionString is set | `""` | +| `externalAzureBlob.connectionString` | Azure Blob Storage connection string. Optional if accessKey is set. | `""` | +| `externalAzureBlob.containerName` | Azure Blob Storage container name | `mlflow` | +| `externalAzureBlob.clientId` | Azure Blob Storage client ID | `""` | +| `externalAzureBlob.tenantId` | Azure Blob Storage tenant ID | `""` | +| `externalAzureBlob.clientSecret` | Azure Blob Storage client secret | `""` | +| `externalAzureBlob.useCredentialsInSecret` | Whether to read the Azure Blob Storage credentials from a secret | `false` | +| `externalAzureBlob.existingSecret` | Name of an existing secret key containing the Azure Blob Storage credentials (required when useCredentialsInSecret is true) | `""` | +| `externalAzureBlob.existingAccessKeyKey` | Key in the existing secret containing the Azure Blob Storage access key (required when useCredentialsInSecret is true) | `""` | +| `externalAzureBlob.existingConnectionStringKey` | Key in the existing secret containing the Azure Blob Storage connection string (required when useCredentialsInSecret is true) | `""` | +| `externalAzureBlob.clientSecretKey` | Key in the existing secret containing the Azure Blob Storage client secret (required when useCredentialsInSecret is true) | `""` | +| `externalAzureBlob.serveArtifacts` | Whether artifact serving is enabled | `true` | + The MLflow chart supports three different ways to load your files in the `run` deployment. In order of priority, they are: 1. Existing config map diff --git a/bitnami/mlflow/templates/_helpers.tpl b/bitnami/mlflow/templates/_helpers.tpl index bf9a5e15823d25..b91fe3935dc8ee 100644 --- a/bitnami/mlflow/templates/_helpers.tpl +++ b/bitnami/mlflow/templates/_helpers.tpl @@ -658,12 +658,21 @@ Return the S3 secret access key inside the secret Return whether GCS is enabled */}} {{- define "mlflow.v0.gcs.enabled" -}} - {{- if and (not .Values.minio.enabled) (not .Values.externalS3.host) .Values.externalGCS.bucket -}} + {{- if and (not .Values.minio.enabled) (not .Values.externalS3.host) (not .Values.externalAzureBlob.storageAccount) .Values.externalGCS.bucket -}} {{- true }} {{- end -}} {{- end -}} +{{/* +Return whether Azure Blob is enabled +*/}} +{{- define "mlflow.v0.azureBlob.enabled" -}} + {{- if and (not .Values.minio.enabled) (not .Values.externalS3.host) (not .Values.externalGCS.bucket) .Values.externalAzureBlob.storageAccount -}} + {{- true }} + {{- end -}} +{{- end -}} + {{/* Return the proper git image name */}} diff --git a/bitnami/mlflow/templates/tracking/deployment.yaml b/bitnami/mlflow/templates/tracking/deployment.yaml index fc75031a932403..fe14af6b601650 100644 --- a/bitnami/mlflow/templates/tracking/deployment.yaml +++ b/bitnami/mlflow/templates/tracking/deployment.yaml @@ -143,6 +143,8 @@ spec: - --artifacts-destination=s3://{{ include "mlflow.v0.s3.bucket" . }} {{- else if (include "mlflow.v0.gcs.enabled" .) }} - --artifacts-destination=gs://{{ .Values.externalGCS.bucket }} + {{- else if (include "mlflow.v0.azureBlob.enabled" .) }} + - --artifacts-destination=wasbs://{{ .Values.externalAzureBlob.containerName }}@{{ .Values.externalAzureBlob.storageAccount }}.blob.core.windows.net {{- else }} - --artifacts-destination={{ .Values.tracking.persistence.mountPath }}/mlartifacts {{- end }} @@ -152,6 +154,9 @@ spec: {{- else if and (not .Values.externalGCS.serveArtifacts) (include "mlflow.v0.gcs.enabled" .) }} - --default-artifact-root=gs://{{ .Values.externalGCS.bucket }} - --no-serve-artifacts + {{- else if and (not .Values.externalAzureBlob.serveArtifacts) (include "mlflow.v0.azureBlob.enabled" .) }} + - --default-artifact-root=wasbs://{{ .Values.externalAzureBlob.containerName }}@{{ .Values.externalAzureBlob.storageAccount }}.blob.core.windows.net + - --no-serve-artifacts {{ else }} - --serve-artifacts {{- end }} @@ -200,11 +205,54 @@ spec: - name: GOOGLE_APPLICATION_CREDENTIALS value: /bitnami/gcs/key.json {{- end }} + {{- end }} + + {{- if (include "mlflow.v0.azureBlob.enabled" .) }} + {{- if .Values.externalAzureBlob.useCredentialsInSecret }} + - name: AZURE_STORAGE_CONNECTION_STRING + valueFrom: + secretKeyRef: + name: {{ .Values.externalAzureBlob.existingSecret }} + key: {{ .Values.externalAzureBlob.existingConnectionStringKey }} + - name: AZURE_STORAGE_ACCESS_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.externalAzureBlob.existingSecret }} + key: {{ .Values.externalAzureBlob.existingAccessKeyKey }} + - name: AZURE_CLIENT_SECRET + valueFrom: + secretKeyRef: + name: {{ .Values.externalAzureBlob.existingSecret }} + key: {{ .Values.externalAzureBlob.clientSecretKey }} + {{- else }} + {{- if .Values.externalAzureBlob.connectionString }} + - name: AZURE_STORAGE_CONNECTION_STRING + value: {{ .Values.externalAzureBlob.connectionString }} + {{- end }} + {{- if .Values.externalAzureBlob.accessKey }} + - name: AZURE_STORAGE_ACCESS_KEY + value: {{ .Values.externalAzureBlob.accessKey }} + {{- end }} + {{- if .Values.externalAzureBlob.clientId }} + - name: AZURE_CLIENT_ID + value: {{ .Values.externalAzureBlob.clientId }} + {{- end }} + {{- if .Values.externalAzureBlob.tenantId }} + - name: AZURE_TENANT_ID + value: {{ .Values.externalAzureBlob.tenantId }} + {{- end }} + {{- if .Values.externalAzureBlob.clientSecret }} + - name: AZURE_CLIENT_SECRET + value: {{ .Values.externalAzureBlob.clientSecret }} + {{- end }} + {{- end }} + {{- end }} + {{- if .Values.externalGCS.googleCloudProject }} - name: GOOGLE_CLOUD_PROJECT value: {{ .Values.externalGCS.googleCloudProject }} {{- end }} - {{- end }} + {{- if .Values.tracking.extraEnvVars }} {{- include "common.tplvalues.render" (dict "value" .Values.tracking.extraEnvVars "context" $) | nindent 12 }} {{- end }} diff --git a/bitnami/mlflow/values.yaml b/bitnami/mlflow/values.yaml index 5b232fab03a113..b3dc576ce13fd1 100644 --- a/bitnami/mlflow/values.yaml +++ b/bitnami/mlflow/values.yaml @@ -1441,3 +1441,35 @@ externalGCS: existingSecret: "" existingSecretKey: "" serveArtifacts: true + +## @section External Azure Blob Storage parameters +## All of these values are only used when minio.enabled is set to false and externalS3 is not configured (host is empty) +## and externalGCS is not configured (bucket is empty). +## @param externalAzureBlob.storageAccount Azure Blob Storage account name. Activate azure artifact storage if set, +## @param externalAzureBlob.accessKey Azure Blob Storage access key. Optional if connectionString is set +## @param externalAzureBlob.connectionString Azure Blob Storage connection string. Optional if accessKey is set. +## @param externalAzureBlob.containerName Azure Blob Storage container name +## @param externalAzureBlob.clientId Azure Blob Storage client ID +## @param externalAzureBlob.tenantId Azure Blob Storage tenant ID +## @param externalAzureBlob.clientSecret Azure Blob Storage client secret +## @param externalAzureBlob.useCredentialsInSecret Whether to read the Azure Blob Storage credentials from a secret +## @param externalAzureBlob.existingSecret Name of an existing secret key containing the Azure Blob Storage credentials (required when useCredentialsInSecret is true) +## @param externalAzureBlob.existingAccessKeyKey Key in the existing secret containing the Azure Blob Storage access key (required when useCredentialsInSecret is true) +## @param externalAzureBlob.existingConnectionStringKey Key in the existing secret containing the Azure Blob Storage connection string (required when useCredentialsInSecret is true) +## @param externalAzureBlob.clientSecretKey Key in the existing secret containing the Azure Blob Storage client secret (required when useCredentialsInSecret is true) +## @param externalAzureBlob.serveArtifacts Whether artifact serving is enabled +## +externalAzureBlob: + storageAccount: "" + accessKey: "" + connectionString: "" + containerName: "mlflow" + clientId: "" + tenantId: "" + clientSecret: "" + useCredentialsInSecret: false + existingSecret: "" + existingAccessKeyKey: "" + existingConnectionStringKey: "" + clientSecretKey: "" + serveArtifacts: true \ No newline at end of file