Integrate BanyanDB cluster as storage solution #145
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,6 +39,6 @@ dependencies: | |
| repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami | ||
| condition: postgresql.enabled | ||
| - name: skywalking-banyandb-helm | ||
| version: 0.0.0-8bdff75 | ||
| repository: oci://ghcr.io/apache/skywalking-banyandb-helm | ||
|
Comment on lines
+43
to
+44
There was a problem hiding this comment. You should point to docker hub, rather than ghcr. Why is this changed? There was a problem hiding this comment. Could you release 0.2.0 of the banyandb helm? There was a problem hiding this comment. Meanwhile, where is There was a problem hiding this comment. We have not yet released banyandb and banyandb-helm, so I temporarily pointed the repository to ghcr. I will update these values together after the releases are completed. There was a problem hiding this comment. We have the previous(0.1.0) version, https://github.com/apache/skywalking-banyandb-helm. Are you going to release that soon? Otherwise, once skywalking-helm 4.6 is releasing, it will carry the unreleased BanyanDB version(some snapshot version). This is not correct in the ASF. There was a problem hiding this comment. As far as I know, we will release shortly after the completion of testing on banyandb, and it should be relatively fast. There was a problem hiding this comment. Could you post an issue on Helm 4.6.0 milestone to track this, and you should fix this There was a problem hiding this comment. @hanahmily told me, your deployment should not care about BanyanDB 0.5 or 0.6(dev or unreleased). So, this should not be a blocker for you. There was a problem hiding this comment.
There is a feature related to etcd authentication that has not been implemented in version 0.5. It seems that we still need to rely on the upcoming BanyanDB 0.6 version. After discussing with @hanahmily , we decide to change these tags later on. |
||
| condition: banyandb.enabled | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,208 @@ | ||
| # Licensed to the Apache Software Foundation (ASF) under one or more | ||
| # contributor license agreements. See the NOTICE file distributed with | ||
| # this work for additional information regarding copyright ownership. | ||
| # The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| # (the "License"); you may not use this file except in compliance with | ||
| # the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| # This file is used to show how to write configuration files and can be used to test. | ||
|
|
||
| setup: | ||
| env: kind | ||
| file: kind.yaml | ||
| init-system-environment: env | ||
| kind: | ||
| expose-ports: | ||
| - namespace: istio-system | ||
| resource: service/skywalking-ui | ||
| port: 80 | ||
| steps: | ||
| - name: install yq | ||
| command: bash test/e2e/setup-e2e-shell/install.sh yq | ||
| - name: install swctl | ||
| command: bash test/e2e/setup-e2e-shell/install.sh swctl | ||
| - name: install kubectl | ||
| command: bash test/e2e/setup-e2e-shell/install.sh kubectl | ||
| - name: install istio | ||
| command: | | ||
| bash test/e2e/setup-e2e-shell/install.sh istioctl | ||
| istioctl install -y --set profile=demo \ | ||
| --set meshConfig.defaultConfig.envoyAccessLogService.address=skywalking-satellite.istio-system:11800 \ | ||
| --set meshConfig.enableEnvoyAccessLogService=true | ||
| kubectl label namespace default istio-injection=enabled | ||
| - name: Install helm | ||
| command: bash test/e2e/setup-e2e-shell/install.sh helm | ||
| - name: Install SkyWalking | ||
| command: | | ||
| helm dep up chart/skywalking | ||
| helm -n istio-system install --timeout 10m skywalking chart/skywalking \ | ||
| --set fullnameOverride=skywalking \ | ||
| --set oap.env.SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh \ | ||
| --set oap.env.SW_ENVOY_METRIC_ALS_TCP_ANALYSIS=k8s-mesh \ | ||
| --set oap.env.K8S_SERVICE_NAME_RULE='e2e::${service.metadata.name}' \ | ||
| --set oap.replicas=1 \ | ||
| --set ui.image.repository=$UI_REPO \ | ||
| --set ui.image.tag=$UI_TAG \ | ||
| --set oap.image.repository=$OAP_REPO \ | ||
| --set oap.image.tag=$OAP_TAG \ | ||
| --set oap.storageType=banyandb \ | ||
| --set elasticsearch.enabled=false \ | ||
| --set banyandb.enabled=true \ | ||
| --set skywalking-banyandb-helm.standalone.enabled=false \ | ||
| --set skywalking-banyandb-helm.cluster.enabled=true \ | ||
| --set skywalking-banyandb-helm.etcd.enabled=true \ | ||
| --set satellite.enabled=true \ | ||
| --set satellite.image.repository=$SATELLITE_REPO \ | ||
| --set satellite.image.tag=$SATELLITE_TAG \ | ||
| -f test/e2e/values.yaml | ||
| wait: | ||
| - namespace: istio-system | ||
| resource: deployments/skywalking-oap | ||
| for: condition=available | ||
| - namespace: istio-system | ||
| resource: deployments/skywalking-satellite | ||
| for: condition=available | ||
| - name: Deploy demo services | ||
| command: | | ||
| kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/platform/kube/bookinfo.yaml | ||
| kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/networking/bookinfo-gateway.yaml | ||
| # Enable TCP services | ||
| kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml | ||
| kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/platform/kube/bookinfo-db.yaml | ||
| kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/networking/destination-rule-all.yaml | ||
| kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/networking/virtual-service-ratings-db.yaml | ||
| wait: | ||
| - namespace: default | ||
| resource: pod | ||
| for: condition=Ready | ||
| - name: Generate traffic | ||
| path: traffic-gen.yaml | ||
| wait: | ||
| - namespace: default | ||
| resource: pod | ||
| for: condition=Ready | ||
| timeout: 25m | ||
|
|
||
| verify: | ||
| retry: | ||
| count: 20 | ||
| interval: 10s | ||
| cases: | ||
| # service list | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql service ls | ||
| expected: expected/service.yml | ||
| # service instance list | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::productpage | ||
| expected: expected/service-instance.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::reviews | ||
| expected: expected/service-instance.yml | ||
| # service endpoint | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql endpoint list --service-name=e2e::productpage | ||
| expected: expected/service-endpoint-productpage.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql endpoint list --service-name=e2e::reviews | ||
| expected: expected/service-endpoint-reviews.yml | ||
|
|
||
| # service metrics: e2e::productpage | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_sla --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_cpm --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_resp_time --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_apdex --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| # service metrics: e2e::reviews | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_sla --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_cpm --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_resp_time --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_apdex --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| # service instance metrics: e2e::productpage | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_resp_time --service-name=e2e::productpage --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::productpage | yq e '.[0].name' - | ||
| ) |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_cpm --service-name=e2e::productpage --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::productpage | yq e '.[0].name' - | ||
| ) |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_sla --service-name=e2e::productpage --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::productpage | yq e '.[0].name' - | ||
| ) |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| # service instance metrics: e2e::reviews | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_resp_time --service-name=e2e::reviews --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::reviews | yq e '.[0].name' - | ||
| ) |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_cpm --service-name=e2e::reviews --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::reviews | yq e '.[0].name' - | ||
| ) |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_sla --service-name=e2e::reviews --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::reviews | yq e '.[0].name' - | ||
| ) |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
|
|
||
| # service endpoint metrics: e2e::productpage GET:/productpage | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=endpoint_cpm --endpoint-name=GET:/productpage --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=endpoint_resp_time --endpoint-name=GET:/productpage --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=endpoint_sla --endpoint-name=GET:/productpage --service-name=e2e::productpage |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| # service endpoint metrics: e2e::reviews GET:/reviews/0 | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=endpoint_cpm --endpoint-name=GET:/reviews/0 --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=endpoint_resp_time --endpoint-name=GET:/reviews/0 --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=endpoint_sla --endpoint-name=GET:/reviews/0 --service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
|
|
||
| # dependency service | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql dependency service --service-name=e2e::productpage | ||
| expected: expected/dependency-services-productpage.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql dependency service --service-name=e2e::reviews | ||
| expected: expected/dependency-services-reviews.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql dependency instance --service-name=e2e::productpage --dest-service-name=e2e::reviews | ||
| expected: expected/dependency-services-instance-productpage.yml | ||
|
|
||
| # service relation metrics | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_relation_client_cpm --service-name=e2e::productpage --dest-service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_relation_server_cpm --service-name=e2e::productpage --dest-service-name=e2e::reviews |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| # service instance relation metrics, e2e::productpage -> e2e::reviews | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_relation_client_cpm \ | ||
| --service-name=e2e::productpage --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::productpage | yq e '.[0].name' - ) \ | ||
| --dest-service-name=e2e::reviews --dest-instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::reviews | yq e '.[0].name' - ) \ | ||
| |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml | ||
| - query: | | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics linear --name=service_instance_relation_server_cpm \ | ||
| --service-name=e2e::productpage --instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::productpage | yq e '.[0].name' - ) \ | ||
| --dest-service-name=e2e::reviews --dest-instance-name=$( \ | ||
| swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=e2e::reviews | yq e '.[0].name' - ) \ | ||
| |yq e 'to_entries' - | ||
| expected: expected/metrics-has-value.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's rename this to skywalking-banyandb