Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resource for kibana synthetics monitors: http and tcp #699

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
2252813
feat(synthetics): add http and tcp monitor resource
biscout42 Aug 6, 2024
e88f3d3
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 6, 2024
dc8e1a9
tcp and http monitor schema
biscout42 Aug 6, 2024
a76a5a0
implement kibana api for tcp monitor
biscout42 Aug 6, 2024
3d747c1
add tfModelV0 model for synthetic monitors
biscout42 Aug 6, 2024
ed50e44
allign monitor tf schema to API
biscout42 Aug 7, 2024
4a2419c
implement read resource
biscout42 Aug 7, 2024
aa09407
first version
biscout42 Aug 7, 2024
6c31d0a
add first test
biscout42 Aug 7, 2024
218f01f
add schema transformer tests
biscout42 Aug 8, 2024
c4673b3
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 8, 2024
02c86ba
testing synthetics
biscout42 Aug 8, 2024
8a372cc
fix null pointer
biscout42 Aug 9, 2024
843890c
limit number of supported options due-to kibana api
biscout42 Aug 9, 2024
f7199f8
wip
biscout42 Aug 9, 2024
4103571
increase es docker memory due-to es error process ended by code 137
biscout42 Aug 9, 2024
43cd844
try to increase GH workflow service memory
biscout42 Aug 9, 2024
4171e75
wip
biscout42 Aug 9, 2024
797b322
test connectio to package registry
biscout42 Aug 12, 2024
6403674
test in docker
biscout42 Aug 12, 2024
f95a42b
try to run package registry in docker
biscout42 Aug 12, 2024
6a08e21
try xpack.task_manager.max_attempts
biscout42 Aug 12, 2024
f98f4a8
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 12, 2024
d0d32e5
debug kibana
biscout42 Aug 12, 2024
1990615
try xpack.fleet.isAirGapped: true
biscout42 Aug 12, 2024
7763a41
try 8.15.0
biscout42 Aug 12, 2024
ccc25f9
try package-registry distribution
biscout42 Aug 12, 2024
761003e
try to use lite version
biscout42 Aug 12, 2024
1a33422
add re-try for package registry distribution health
biscout42 Aug 12, 2024
b5fdb41
enable debug for plugins.fleet
biscout42 Aug 12, 2024
2133ca2
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 12, 2024
7c499ca
try to set kibana loggin
biscout42 Aug 12, 2024
a8e885e
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 12, 2024
c60d641
try relative path
biscout42 Aug 12, 2024
3c9019f
Merge branch 'kibana-synthetics-http-and-tcp-monitors' of github.com:…
biscout42 Aug 12, 2024
9b04aa3
explore GH workflow env
biscout42 Aug 12, 2024
06f67c3
wip
biscout42 Aug 12, 2024
792c424
wip
biscout42 Aug 12, 2024
3e21939
wip
biscout42 Aug 12, 2024
0b77e80
add status check for kibana service
biscout42 Aug 12, 2024
6bc5ae3
use dirs
biscout42 Aug 12, 2024
c1ca74e
wip
biscout42 Aug 12, 2024
b267654
try workaround due-to missing checkout
biscout42 Aug 12, 2024
a46bda7
try start
biscout42 Aug 12, 2024
a99ba14
try to run kibana manually
biscout42 Aug 12, 2024
2fee656
try run
biscout42 Aug 12, 2024
f49d151
fixing docker: invalid reference format.
biscout42 Aug 12, 2024
bdff210
wip
biscout42 Aug 12, 2024
83019cb
wip
biscout42 Aug 13, 2024
a70fb1d
try the idea with env vars and longopts from kibana
biscout42 Aug 13, 2024
f105d97
forgot to remove mounted volume
biscout42 Aug 13, 2024
cc827e8
add acc tests for tcp monitor
biscout42 Aug 13, 2024
c8e21d2
clean-up before review
biscout42 Aug 13, 2024
a1b09af
add docs
biscout42 Aug 13, 2024
55c704e
try to force-install synthetics
biscout42 Aug 13, 2024
09af708
add re-try
biscout42 Aug 13, 2024
bcde2d1
force install synthetics only for 8.14.3
biscout42 Aug 13, 2024
73b8afc
try to use status api
biscout42 Aug 13, 2024
d2a7b91
add debug
biscout42 Aug 13, 2024
daa3b4c
back to force-install (synthetics doesn't report plugin status via API)
biscout42 Aug 13, 2024
053394a
silence
biscout42 Aug 13, 2024
74445af
CR comments: make `max_redirects` as int64
biscout42 Aug 15, 2024
ef09e64
reproduce monitor id issue - need to add namespace
biscout42 Aug 15, 2024
a25b691
add context to kibana synthetic API calls
biscout42 Aug 15, 2024
a443230
use composite id for monitors to import from napespaces
biscout42 Aug 15, 2024
1cbe623
migrate private location to composite id
biscout42 Aug 15, 2024
7bda56b
implement merge attributes
biscout42 Aug 16, 2024
bc704d4
add unit tests for merge and udpate docs
biscout42 Aug 16, 2024
e7d0568
cr comments - tests
biscout42 Aug 16, 2024
e478e05
docs - add spaceId
biscout42 Aug 16, 2024
3df3e90
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 17, 2024
0888164
add 8.15.0 for synthetics install call conditions
biscout42 Aug 17, 2024
f40d508
change log
biscout42 Aug 19, 2024
dcb8391
handle private location id w/o namespace
biscout42 Aug 19, 2024
ec96f68
use lable, as before
biscout42 Aug 19, 2024
e16ba84
formatting
biscout42 Aug 19, 2024
0c8a2b0
Merge branch 'main' into kibana-synthetics-http-and-tcp-monitors
biscout42 Aug 19, 2024
254d9e1
update private location delete to support both id formats
biscout42 Aug 19, 2024
5ccf103
naming ...
biscout42 Aug 19, 2024
98038d0
split synthetics monitor resources in tests between http and tcp to p…
biscout42 Aug 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,12 @@ jobs:
image: docker.elastic.co/elasticsearch/elasticsearch:${{ matrix.version }}
env:
discovery.type: single-node
xpack.license.self_generated.type: trial
xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
xpack.security.authc.token.enabled: true
xpack.security.http.ssl.enabled: false
xpack.watcher.enabled: true
xpack.license.self_generated.type: trial
repositories.url.allowed_urls: https://example.com/*
path.repo: /tmp
ELASTIC_PASSWORD: ${{ env.ELASTIC_PASSWORD }}
Expand All @@ -78,9 +80,10 @@ jobs:
ELASTICSEARCH_USERNAME: ${{ env.KIBANA_SYSTEM_USERNAME }}
ELASTICSEARCH_PASSWORD: ${{ env.KIBANA_SYSTEM_PASSWORD }}
XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY: a7a6311933d3503b89bc2dbc36572c33a6c10925682e591bffcab6911c06786d
xpack.fleet.enabled: true
# LOGGING_ROOT_LEVEL: debug
ports:
- 5601:5601
options: --health-cmd="curl http://localhost:5601/api/status" --health-interval=10s --health-timeout=5s --health-retries=10
biscout42 marked this conversation as resolved.
Show resolved Hide resolved

timeout-minutes: 15
strategy:
Expand Down Expand Up @@ -135,6 +138,12 @@ jobs:
ELASTICSEARCH_USERNAME: "elastic"
ELASTICSEARCH_PASSWORD: ${{ env.ELASTIC_PASSWORD }}

- id: force-install-synthetics
name: Force install synthetics
if: matrix.version == '8.14.3' || matrix.version == '8.15.0'
run: |-
for i in {1..5}; do curl -s -H "Authorization: ApiKey ${{ steps.get-api-key.outputs.apikey }}" --header "Content-Type: application/json" --header "kbn-xsrf: true" --request POST --data '{ "force": true }' http://localhost:5601/api/fleet/epm/packages/synthetics/1.2.2 && break || sleep 15; done

- name: TF acceptance tests
timeout-minutes: 10
run: make testacc
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
## [Unreleased]
- Add support for Kibana synthetics http and tcp monitors ([#699](https://github.com/elastic/terraform-provider-elasticstack/pull/699))

- Add `elasticstack_kibana_spaces` data source ([#682](https://github.com/elastic/terraform-provider-elasticstack/pull/682))

Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ docker-kibana: docker-network docker-elasticsearch set-kibana-password ## Start
-e ELASTICSEARCH_USERNAME=$(KIBANA_SYSTEM_USERNAME) \
-e ELASTICSEARCH_PASSWORD=$(KIBANA_SYSTEM_PASSWORD) \
-e XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=a7a6311933d3503b89bc2dbc36572c33a6c10925682e591bffcab6911c06786d \
-e "logging.root.level=debug" \
-e LOGGING_ROOT_LEVEL=debug \
--name $(KIBANA_NAME) \
--network $(ELASTICSEARCH_NETWORK) \
docker.elastic.co/kibana/kibana:$(STACK_VERSION); \
Expand All @@ -157,7 +157,7 @@ docker-kibana-with-tls: docker-network docker-elasticsearch set-kibana-password
-e SERVER_SSL_CERTIFICATE=/certs/localhost+1.pem \
-e SERVER_SSL_KEY=/certs/localhost+1-key.pem \
-e SERVER_SSL_ENABLED=true \
-e "logging.root.level=debug" \
-e LOGGING_ROOT_LEVEL=debug \
--name $(KIBANA_NAME) \
--network $(ELASTICSEARCH_NETWORK) \
docker.elastic.co/kibana/kibana:$(STACK_VERSION); \
Expand Down
160 changes: 160 additions & 0 deletions docs/resources/kibana_synthetics_monitor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
---
subcategory: "Kibana"
layout: ""
page_title: "Elasticstack: elasticstack_kibana_synthetics_monitor Resource"
description: |-
Creates or updates a Kibana synthetics monitor.
---

# Resource: elasticstack_kibana_synthetics_monitor

Creates or updates a Kibana synthetics monitor.
See [API docs](https://www.elastic.co/guide/en/kibana/current/add-monitor-api.html)

## Supported monitor types
* `http`
* `tcp`

**NOTE:** Due-to nature of partial update API, reset values to defaults is not supported.
In case you would like to reset an optional monitor value, please set it explicitly or delete and create new monitor.


## Example Usage

```terraform
provider "elasticstack" {
kibana {}
}

resource "elasticstack_kibana_synthetics_monitor" "my_monitor" {
name = "Example http monitor"
space_id = "default"
schedule = 10
locations = ["us_west"]
enabled = false
tags = ["tag"]
alert = {
status = {
enabled = true
}
tls = {
enabled = false
}
}
service_name = "example apm service"
timeout = 30
http = {
url = "http://localhost:8080"
ssl_verification_mode = "full"
ssl_supported_protocols = ["TLSv1.2"]
max_redirects = "10"
mode = "all"
ipv4 = true
ipv6 = true
}
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `name` (String) The monitor’s name.

### Optional

- `alert` (Attributes) Alert configuration. Default: `{ status: { enabled: true }, tls: { enabled: true } }`. (see [below for nested schema](#nestedatt--alert))
- `enabled` (Boolean) Whether the monitor is enabled. Default: `true`
- `http` (Attributes) HTTP Monitor specific fields (see [below for nested schema](#nestedatt--http))
- `locations` (List of String) Where to deploy the monitor. Monitors can be deployed in multiple locations so that you can detect differences in availability and response times across those locations.
- `params` (String) Monitor parameters. Raw JSON object, use `jsonencode` function to represent JSON
- `private_locations` (List of String) These Private Locations refer to locations hosted and managed by you, whereas locations are hosted by Elastic. You can specify a Private Location using the location’s name.
- `retest_on_failure` (Boolean) Enable or disable retesting when a monitor fails. By default, monitors are automatically retested if the monitor goes from "up" to "down". If the result of the retest is also "down", an error will be created, and if configured, an alert sent. Then the monitor will resume running according to the defined schedule. Using retest_on_failure can reduce noise related to transient problems. Default: `true`.
- `schedule` (Number) The monitor’s schedule in minutes. Supported values are 1, 3, 5, 10, 15, 30, 60, 120 and 240.
- `service_name` (String) The APM service name.
- `space_id` (String) The namespace field should be lowercase and not contain spaces. The namespace must not include any of the following characters: *, \, /, ?, ", <, >, |, whitespace, ,, #, :, or -. Default: `default`
- `tags` (List of String) An array of tags.
- `tcp` (Attributes) TCP Monitor specific fields (see [below for nested schema](#nestedatt--tcp))
- `timeout` (Number) The monitor timeout in seconds, monitor will fail if it doesn’t complete within this time. Default: `16`

### Read-Only

- `id` (String) Generated identifier for the monitor

<a id="nestedatt--alert"></a>
### Nested Schema for `alert`

Optional:

- `status` (Attributes) (see [below for nested schema](#nestedatt--alert--status))
- `tls` (Attributes) (see [below for nested schema](#nestedatt--alert--tls))

<a id="nestedatt--alert--status"></a>
### Nested Schema for `alert.status`

Optional:

- `enabled` (Boolean)


<a id="nestedatt--alert--tls"></a>
### Nested Schema for `alert.tls`

Optional:

- `enabled` (Boolean)



<a id="nestedatt--http"></a>
### Nested Schema for `http`

Required:

- `url` (String) URL to monitor.

Optional:

- `check` (String) The check request settings.. Raw JSON object, use `jsonencode` function to represent JSON
- `ipv4` (Boolean) Whether to ping using the ipv4 protocol.
- `ipv6` (Boolean) Whether to ping using the ipv6 protocol.
- `max_redirects` (Number) The maximum number of redirects to follow. Default: `0`
- `mode` (String) The mode of the monitor. Can be "all" or "any". If you’re using a DNS-load balancer and want to ping every IP address for the specified hostname, you should use all.
- `password` (String) The password for authenticating with the server. The credentials are passed with the request.
- `proxy_header` (String) Additional headers to send to proxies during CONNECT requests.. Raw JSON object, use `jsonencode` function to represent JSON
- `proxy_url` (String) The URL of the proxy to use for this monitor.
- `response` (String) Controls the indexing of the HTTP response body contents to the `http.response.body.contents` field.. Raw JSON object, use `jsonencode` function to represent JSON
- `ssl_supported_protocols` (List of String) List of allowed SSL/TLS versions.
- `ssl_verification_mode` (String) Controls the verification of server certificates.
- `username` (String) The username for authenticating with the server. The credentials are passed with the request.


<a id="nestedatt--tcp"></a>
### Nested Schema for `tcp`

Required:

- `host` (String) The host to monitor; it can be an IP address or a hostname. The host can include the port using a colon (e.g., "example.com:9200").

Optional:

- `check_receive` (String) The expected answer.
- `check_send` (String) An optional payload string to send to the remote host.
- `proxy_url` (String) The URL of the SOCKS5 proxy to use when connecting to the server. The value must be a URL with a scheme of `socks5://`. If the SOCKS5 proxy server requires client authentication, then a username and password can be embedded in the URL. When using a proxy, hostnames are resolved on the proxy server instead of on the client. You can change this behavior by setting the `proxy_use_local_resolver` option.
- `proxy_use_local_resolver` (Boolean) A Boolean value that determines whether hostnames are resolved locally instead of being resolved on the proxy server. The default value is false, which means that name resolution occurs on the proxy server.
- `ssl_supported_protocols` (List of String) List of allowed SSL/TLS versions.
- `ssl_verification_mode` (String) Controls the verification of server certificates.

## Import

Import is supported using the following syntax:

```shell
terraform import elasticstack_kibana_synthetics_monitor.my_monitor <space id>/<monitor_id>
```

**NOTE:** Not all monitor fields are supported during the import due-to API limitation.
Full field support could be implemented after this [kibana issue](https://github.com/elastic/kibana/issues/189906) is resolved.

Currently not supported fields during the import: `params`, `retest_on_failure`, `http.proxy_header`, `http.username`, `http.password`, `http.check`, `http.response`, `tcp.check_send`, `tcp.check_receive`
2 changes: 1 addition & 1 deletion docs/resources/kibana_synthetics_private_location.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,5 @@ Required:
Import is supported using the following syntax:

```shell
terraform import elasticstack_kibana_synthetics_private_location.my_location <private_location_id>
terraform import elasticstack_kibana_synthetics_private_location.my_location <space id>/<private_location_id>
```
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
terraform import elasticstack_kibana_synthetics_monitor.my_monitor <space id>/<monitor_id>
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
provider "elasticstack" {
kibana {}
}

resource "elasticstack_kibana_synthetics_monitor" "my_monitor" {
name = "Example http monitor"
space_id = "default"
schedule = 10
locations = ["us_west"]
enabled = false
tags = ["tag"]
alert = {
status = {
enabled = true
}
tls = {
enabled = false
}
}
service_name = "example apm service"
timeout = 30
http = {
url = "http://localhost:8080"
ssl_verification_mode = "full"
ssl_supported_protocols = ["TLSv1.2"]
max_redirects = "10"
mode = "all"
ipv4 = true
ipv6 = true
}
}
Original file line number Diff line number Diff line change
@@ -1 +1 @@
terraform import elasticstack_kibana_synthetics_private_location.my_location <private_location_id>
terraform import elasticstack_kibana_synthetics_private_location.my_location <space id>/<private_location_id>
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ require (
github.com/hashicorp/logutils v1.0.0 // indirect
github.com/hashicorp/terraform-exec v0.21.0 // indirect
github.com/hashicorp/terraform-json v0.22.1 // indirect
github.com/hashicorp/terraform-plugin-framework-jsontypes v0.1.0 // indirect
github.com/hashicorp/terraform-registry-address v0.2.3 // indirect
github.com/hashicorp/terraform-svchost v0.1.1 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ github.com/hashicorp/terraform-json v0.22.1 h1:xft84GZR0QzjPVWs4lRUwvTcPnegqlyS7
github.com/hashicorp/terraform-json v0.22.1/go.mod h1:JbWSQCLFSXFFhg42T7l9iJwdGXBYV8fmmD6o/ML4p3A=
github.com/hashicorp/terraform-plugin-framework v1.10.0 h1:xXhICE2Fns1RYZxEQebwkB2+kXouLC932Li9qelozrc=
github.com/hashicorp/terraform-plugin-framework v1.10.0/go.mod h1:qBXLDn69kM97NNVi/MQ9qgd1uWWsVftGSnygYG1tImM=
github.com/hashicorp/terraform-plugin-framework-jsontypes v0.1.0 h1:b8vZYB/SkXJT4YPbT3trzE6oJ7dPyMy68+9dEDKsJjE=
github.com/hashicorp/terraform-plugin-framework-jsontypes v0.1.0/go.mod h1:tP9BC3icoXBz72evMS5UTFvi98CiKhPdXF6yLs1wS8A=
github.com/hashicorp/terraform-plugin-framework-validators v0.13.0 h1:bxZfGo9DIUoLLtHMElsu+zwqI4IsMZQBRRy4iLzZJ8E=
github.com/hashicorp/terraform-plugin-framework-validators v0.13.0/go.mod h1:wGeI02gEhj9nPANU62F2jCaHjXulejm/X+af4PdZaNo=
github.com/hashicorp/terraform-plugin-go v0.23.0 h1:AALVuU1gD1kPb48aPQUjug9Ir/125t+AAurhqphJ2Co=
Expand Down
Loading