Skip to content

Commit

Permalink
GEFEST-856 авторизация в kafka по ключу
Browse files Browse the repository at this point in the history
  • Loading branch information
Donov Dmitriy committed Jul 29, 2024
1 parent 9a91285 commit d738388
Show file tree
Hide file tree
Showing 7 changed files with 439 additions and 13 deletions.
69 changes: 60 additions & 9 deletions charts/keys/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,38 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
| `tasker.affinity` | Kubernetes pod [affinity settings](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity). | `{}` |
| `tasker.tolerations` | Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |

### Counter worker settings

| Name | Description | Value |
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `counter.enabled` | Counter worker is enabled. | `false` |
| `counter.replicas` | A replica count for the pod. | `1` |
| `counter.prometheus.scrape` | If Prometheus scrape is enabled. | `true` |
| `counter.resources.requests.cpu` | A CPU request. | `20m` |
| `counter.resources.requests.memory` | A memory request. | `32Mi` |
| `counter.resources.limits.cpu` | A CPU limit. | `1000m` |
| `counter.resources.limits.memory` | A memory limit. | `512Mi` |
| `counter.preloader.refreshTick` | Refresh interval for in-memory cache with keys limitations info. The smaller the interval, the faster the worker will know about changes in limitations. | `1m` |
| `counter.updateStatusQueryTimeout` | Timeout for database queries to update key status. | `1s` |
| `counter.buffer` | **Settings for in-memory buffer for statistics data.** | |
| `counter.buffer.size` | The maximum size of the buffer. When the limit is reached, the data from the buffer is transferred to Redis. | `1000` |
| `counter.buffer.delay` | The maximum interval between data transfer operations from the buffer to Redis. | `1s` |

### Redis settings

| Name | Description | Value |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ------- |
| `counter.redis.retries` | | `5` |
| `counter.redis.minRetryBackoff` | | `100ms` |
| `counter.redis.maxRetryBackoff` | | `3s` |
| `counter.annotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `counter.labels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `counter.podAnnotations` | Kubernetes [pod annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `counter.podLabels` | Kubernetes [pod labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `counter.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
| `counter.affinity` | Kubernetes pod [affinity settings](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity). | `{}` |
| `counter.tolerations` | Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |

### Redis settings

| Name | Description | Value |
Expand Down Expand Up @@ -198,15 +230,34 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about

### Kafka settings

| Name | Description | Value |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
| `kafka.audit` | **Settings for sending audit messages.** | |
| `kafka.audit.bootstrapServers` | Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093'). | `""` |
| `kafka.audit.username` | Username for authorization (SASL/PLAINTEXT SHA-512). | `""` |
| `kafka.audit.password` | Password for authorization (SASL/PLAINTEXT SHA-512). | `""` |
| `kafka.audit.topic` | Topic to produce audit messages. | `""` |
| `kafka.audit.produce.retryCount` | Number of retries to produce a message. | `5` |
| `kafka.audit.produce.idempotentWrite` | Flag to enable/disable [idempotent write](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#enable-idempotence). | `true` |
| Name | Description | Value |
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| `kafka.main` | **Settings for read stat messages.** | |
| `kafka.main.clientPrefix` | Client prefix name. | `production` |
| `kafka.main.clientId` | Client id. If empty, then hostname will be used. | `""` |
| `kafka.main.brokers` | Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093'). | `""` |
| `kafka.main.securityProtocol` | Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. Default: PLAINTEXT. | `PLAINTEXT` |
| `kafka.main.SASLMechanism` | Authentication mechanism when security_protocol is configured for SASL_PLAINTEXT or SASL_SSL. Valid values are: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512. | `PLAIN` |
| `kafka.main.username` | Username for authorization (SASL). | `""` |
| `kafka.main.password` | Password for authorization (SASL). | `""` |
| `kafka.main.tls.skipServerCertificateVerify` | Controls whether a client verifies the server's certificate chain and host name. | `false` |
| `kafka.main.tls.rootCert` | Root certificate. | `""` |
| `kafka.main.tls.cert` | Client certificate. | `""` |
| `kafka.main.tls.key` | Client key. | `""` |
| `kafka.main.topics.stats` | Topic to consume stat messages. | `""` |
| `kafka.audit` | **Settings for sending audit messages.** | |
| `kafka.audit.bootstrapServers` | Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093'). | `""` |
| `kafka.audit.securityProtocol` | Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. Default: PLAINTEXT. | `PLAINTEXT` |
| `kafka.audit.SASLMechanism` | Authentication mechanism when security_protocol is configured for SASL_PLAINTEXT or SASL_SSL. Valid values are: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512. | `PLAIN` |
| `kafka.audit.username` | Username for authorization (SASL). | `""` |
| `kafka.audit.password` | Password for authorization (SASL). | `""` |
| `kafka.audit.topic` | Topic to produce audit messages. | `""` |
| `kafka.audit.tls.skipServerCertificateVerify` | Controls whether a client verifies the server's certificate chain and host name. | `false` |
| `kafka.audit.tls.rootCert` | Root certificate. | `""` |
| `kafka.audit.tls.cert` | Client certificate. | `""` |
| `kafka.audit.tls.key` | Client key. | `""` |
| `kafka.audit.produce.retryCount` | Number of retries to produce a message. | `5` |
| `kafka.audit.produce.idempotentWrite` | Flag to enable/disable [idempotent write](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#enable-idempotence). | `true` |

### LDAP connection settings

Expand Down
38 changes: 36 additions & 2 deletions charts/keys/templates/api/deployment.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,25 @@ spec:
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- if or (eq .Values.kafka.main.securityProtocol "SSL") (eq .Values.kafka.main.securityProtocol "SASL_SSL") }}
initContainers:
- name: copy-kafka-audit-certs
image: '{{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}'
imagePullPolicy: {{ .Values.imagePullPolicy }}
resources:
{{- toYaml .Values.api.resources | nindent 12 }}
command:
- /bin/sh
- -ec
- |
cp /tmp/certs/* /etc/2gis/secret/kafka-audit/
chmod 600 /etc/2gis/secret/kafka-audit/client.key
volumeMounts:
- name: {{ include "keys.name-kafka-audit-raw" . | quote }}
mountPath: /tmp/certs
- name: {{ include "keys.name-kafka-audit" . | quote }}
mountPath: /etc/2gis/secret/kafka-audit
{{- end }}
containers:
- name: keys-api
image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}
Expand All @@ -62,13 +81,28 @@ spec:
{{- if .Values.customCAs.bundle }}
{{- include "keys.env.custom.ca.path" . | nindent 12 }}
{{- end }}
{{- if .Values.customCAs.bundle }}
{{- if or (.Values.customCAs.bundle) (eq .Values.kafka.audit.securityProtocol "SSL") (eq .Values.kafka.audit.securityProtocol "SASL_SSL") }}
volumeMounts:
{{- if .Values.customCAs.bundle }}
{{- include "keys.custom.ca.volumeMounts" . | nindent 12 }}
{{- end }}
{{- if or (eq .Values.kafka.audit.securityProtocol "SSL") (eq .Values.kafka.audit.securityProtocol "SASL_SSL") }}
- name: {{ printf "%s-kafka-audit" (include "keys.name" .) | quote }}
mountPath: /etc/2gis/secret/kafka-audit
{{- end }}
{{- end }}
{{- if .Values.customCAs.bundle }}
{{- if or (.Values.customCAs.bundle) (eq .Values.kafka.audit.securityProtocol "SSL") (eq .Values.kafka.audit.securityProtocol "SASL_SSL") }}
volumes:
{{- if .Values.customCAs.bundle }}
{{- include "keys.custom.ca.deploys.volumes" . | nindent 8 }}
{{- end }}
{{- if or (eq .Values.kafka.audit.securityProtocol "SSL") (eq .Values.kafka.audit.securityProtocol "SASL_SSL") }}
- name: {{ include "keys.name-kafka-audit-raw" . | quote }}
secret:
secretName: {{ include "keys.name-kafka-audit" . | quote }}
- name: {{ include "keys.name-kafka-audit" . | quote }}
emptyDir: {}
{{- end }}
{{- end }}
{{- with .Values.api.nodeSelector }}
nodeSelector:
Expand Down
Loading

0 comments on commit d738388

Please sign in to comment.