[Bug] elasticstack_fleet_output produces inconsistent result after 0.11.8 upgrade

[mag-mkorn]

Describe the bug
After upgrading the provider from 0.11.7 to 0.11.8 the provider plans to perform an in-place update to a pre-existing elasticstack_fleet_output-resource and ultimately fails.

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # elasticstack_fleet_output.<redacted> will be updated in-place
  ~ resource "elasticstack_fleet_output" "cribl" {
        id                     = "<redacted>"
        name                   = "<redacted>"
        # (7 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
elasticstack_fleet_output.cribl: Modifying... [id=<redacted>]

Error: Provider produced inconsistent result after apply

When applying changes to elasticstack_fleet_output.<redacted>, provider
"provider[\"registry.terraform.io/elastic/elasticstack\"]" produced an
unexpected new value: .ca_trusted_fingerprint: was null, but now
cty.StringVal("").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.
Error: Terraform exited with code 1.
Error: Process completed with exit code 1.

To Reproduce
Resource config:

resource "elasticstack_fleet_output" "<redacted>" {
  name = "<redacted>"
  type = "elasticsearch"
  config_yaml = yamlencode({
    "ssl.verification_mode" : "full",
    "ssl.certificate_authorities" : <<EOF
-----BEGIN CERTIFICATE-----
<redacted>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<redacted>
-----END CERTIFICATE-----
EOF
  })
  default_integrations = true
  default_monitoring   = true
  hosts                = [var.<redacted>]
}

Expected behavior
Don't perform any changes without showing the reason for the change. Don't fail on comparing null values to empty strings.

Debug output
Currently not possible to provide, as TF is executed as CI Pipeline with not easily accessible secrets.

Versions (please complete the following information):

• OS: [e.g. Linux]
• Terraform Version tag "latest" on GHA hashicorp/setup-terraform@v2
• Provider version 0.11.8
• Elasticsearch Version 8.15.2

[daemitus]

Since I authored it, figured I would take a stab at this. Did you continue to have the error afterwards, or was it only transient during the upgrade?

fwiw, I can't replicate this with the following:

terraform {
  required_version = "1.8.2"
  required_providers {
    elasticstack = {
      source  = "elastic/elasticstack"
      // version = "= 0.11.7"
      version = "= 0.11.8"
    }
  }
}

provider "elasticstack" {
  elasticsearch {
    username  = "elastic"
    password  = "password"
    endpoints = ["http://localhost:9200"]
    insecure  = true
  }
  kibana {
    username  = "elastic"
    password  = "password"
    endpoints = ["http://localhost:5601"]
    insecure  = true
  }
  fleet {}
}

resource "elasticstack_fleet_output" "test" {
  name = "test"
  type = "elasticsearch"
  config_yaml = yamlencode({
    "ssl.verification_mode"       = "full",
    "ssl.certificate_authorities" = <<EOF
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
EOF
  })
  default_integrations = true
  default_monitoring   = true
  hosts                = ["http://localhost:9200"]
}

[MichalMarchewka]

I confirm this bug in "0.11.8" version.

resource "elasticstack_fleet_integration_policy" "pfsense_routers_lan" {
    name                = "pfSense- routers lan"
    namespace           = "default"
    description         = "pfSense integrations for routers in lan network"
    agent_policy_id     = elasticstack_fleet_agent_policy.routers_lan.policy_id
    integration_name    = elasticstack_fleet_integration.pfsense.name
    integration_version = elasticstack_fleet_integration.pfsense.version
    input {
        input_id = "pfsense-udp"
        enabled = true
        streams_json = jsonencode({
            "pfsense.log" : {
                "ennabled" : true,
                "vars" : {
                    "syslog_host": "localhost",
                    "syslog_port": 9001,
                    "internal_networks": [
                    "private"
                    ],
                    "tz_offset": "local",
                    "tags": [
                    "pfsense",
                    "forwarded"
                    ],
                    "preserve_original_event": false,
                    "udp_options": "#read_buffer: 100MiB\n#max_message_size: 50KiB\n#timeout: 300s\n"
                }
            }
        })
    }
    input {
        input_id = "pfsense-tcp"
        enabled = false
        streams_json = jsonencode({
            "pfsense.log": {
                "enabled": false,
                "vars": {
                    "syslog_host": "localhost",
                    "syslog_port": 9001,
                    "internal_networks": [
                    "private"
                    ],
                    "tz_offset": "local",
                    "tags": [
                    "pfsense",
                    "forwarded"
                    ],
                    "preserve_original_event": false
                }
            }
        })
    }
}

Plan: 1 to add, 0 to change, 0 to destroy.
elasticstack_fleet_integration_policy.pfsense_routers_lan: Creating...

 Error: Provider produced inconsistent result after apply

 When applying changes to elasticstack_fleet_integration_policy.pfsense_routers_lan, provider "provider[\"registry.terraform.io/elastic/elasticstack\"]" produced an unexpected new value: .input[0].streams_json:  inconsistent values for sensitive attribute.

 This is a bug in the provider, which should be reported in the provider's own issue tracker.

Successful in 0.11.7

[mag-mkorn]

Since I authored it, figured I would take a stab at this. Did you continue to have the error afterwards, or was it only transient during the upgrade?

fwiw, I can't replicate this with the following:

terraform {
  required_version = "1.8.2"
  required_providers {
    elasticstack = {
      source  = "elastic/elasticstack"
      // version = "= 0.11.7"
      version = "= 0.11.8"
    }
  }
}

provider "elasticstack" {
  elasticsearch {
    username  = "elastic"
    password  = "password"
    endpoints = ["http://localhost:9200"]
    insecure  = true
  }
  kibana {
    username  = "elastic"
    password  = "password"
    endpoints = ["http://localhost:5601"]
    insecure  = true
  }
  fleet {}
}

resource "elasticstack_fleet_output" "test" {
  name = "test"
  type = "elasticsearch"
  config_yaml = yamlencode({
    "ssl.verification_mode"       = "full",
    "ssl.certificate_authorities" = <<EOF
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
EOF
  })
  default_integrations = true
  default_monitoring   = true
  hosts                = ["http://localhost:9200"]
}

The error remained until I added ca_trusted_fingerprint = "" as a workaround.

[tobio]

@MichalMarchewka looking into your configuration you have "ennabled" : true, in the UDP input. Fixing up the typo (i.e "enabled": true) fixes the error. This is a tricky one to improve in the provider at the moment since Fleet is simply ignoring the unknown key.

[tobio]

@mag-mkorn I'm also not able to reproduce this and have tried a few permutations around the 0.11.8 upgrade path.

Are you able to reproduce this in a test setup where you're able to get debug logs?

Elasticsearch Version 8.15.2

Just checking this is accurate?

[mag-mkorn]

@mag-mkorn I'm also not able to reproduce this and have tried a few permutations around the 0.11.8 upgrade path.

Are you able to reproduce this in a test setup where you're able to get debug logs?

Elasticsearch Version 8.15.2

Just checking this is accurate?

Hi tobio,

i tried to reproduce the error on a fresh deployment. First, I didn't get the error as well. But after opening the Output Settings on the UI and clicking save, the error occurs.

So apparently saving in the UI adds the empty string variable that leads to the error.

[daemitus]

I think that's one of the nullable problem fields too, might make sense to default empty string that ca field

…

On Thu, Oct 24, 2024, 3:52 AM mag-mkorn ***@***.***> wrote: @mag-mkorn <https://github.com/mag-mkorn> I'm also not able to reproduce this and have tried a few permutations around the 0.11.8 upgrade path. Are you able to reproduce this in a test setup where you're able to get debug logs? Elasticsearch Version 8.15.2 Just checking this is accurate? Hi tobio, i tried to reproduce the error on a fresh deployment. First, I didn't get the error as well. But after opening the Output Settings on the UI and clicking save, the error occurs. So apparently saving in the UI adds the empty string variable that leads to the error. — Reply to this email directly, view it on GitHub <#836 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMTBC4LW3BL75OHTVHA33Z5CRNXAVCNFSM6AAAAABPUDIKNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUGU2TGMZUGY> . You are receiving this because you commented.Message ID: ***@***.***>