Terraform deployment does not finish.

[mosaickai]

Description

Terraform deployments seem to randomly run into API issues. Too many requests or 'unknown errors'.

Expected behavior

DCD with resources should be properly created.

Environment

Terraform version:

 1.9.8

Provider version:

6.6.1

OS:

Linux

How to Reproduce

Steps to reproduce the behavior:

1. Run any terraform deployment

Error and Debug Output

ionoscloud_nic.FIREWALLNIC1[0]: Still creating... [8m0s elapsed]
2024-11-11T09:21:53.564+0100 [WARN]  **unexpected data:**
  registry.terraform.io/ionos-cloud/ionoscloud:stderr=
  | IONOSLOG 2024/11/11 09:21:53  DumpRequestOut : GET /cloudapi/v6/requests/7939b6f1-6c83-425d-a03a-dac52de3fc8c/status HTTP/1.1\r
  | Host: api.ionos.com\r
  | User-Agent: terraform-provider/DEV_ionos-cloud-sdk-go/6.2.1_hashicorp-terraform/1.9.8_terraform-plugin-sdk/2.34.0_os/linux_arch/amd64\r
  | Authorization: Bearer REDACTED

2024-11-11T09:21:53.564+0100 [WARN]  unexpected data:
  registry.terraform.io/ionos-cloud/ionoscloud:stderr=
  | REDACTED hash
  | Accept-Encoding: gzip\r
  | \r
  |
  | IONOSLOG 2024/11/11 09:21:53
  |  try no: 1

2024-11-11T09:21:53.565+0100 [INFO]  provider.terraform-provider-ionoscloud_v6.6.1: [INFO] Checking PATH https://api.ionos.com/cloudapi/v6/requests/7939b6f1-6c83-425d-a03a-dac52de3fc8c/status
2024-11-11T09:21:53.637+0100 [DEBUG] provider.terraform-provider-ionoscloud_v6.6.1: [DEBUG] Request time : 75.290904ms for operation : GetRequestStatus
2024-11-11T09:21:53.637+0100 [DEBUG] provider.terraform-provider-ionoscloud_v6.6.1: [DEBUG] response status code : 200
2024-11-11T09:21:53.637+0100 [WARN]  unexpected data:
  registry.terraform.io/ionos-cloud/ionoscloud:stderr=
  | IONOSLOG 2024/11/11 09:21:53
  |  DumpResponse : HTTP/1.1 200 OK\r
  | Connection: close\r
  | Transfer-Encoding: chunked\r
  | Access-Control-Allow-Credentials: true\r
  | Access-Control-Allow-Headers: *\r
  | Access-Control-Allow-Methods: *\r
  | Access-Control-Allow-Origin: *\r
  | Access-Control-Expose-Headers: *\r
  | Content-Type: application/json\r
  | Date: Mon, 11 Nov 2024 08:21:53 GMT\r
  | Server: nginx\r
  | Strict-Transport-Security: max-age=31536000; includeSubDomains\r
  | Vary: Origin\r
  | X-Frame-Options: SAMEORIGIN\r
  | X-Ratelimit-Burst: 1500\r
  | X-Ratelimit-Limit: 1500\r
  | X-Ratelimit-Remaining: 1499\r
  | \r
  | 2a6\r
  | {
  |   "id" : "7939b6f1-6c83-425d-a03a-dac52de3fc8c/status",
  |   "type" : "request-status",
  |   "href" : "https://api.ionos.com/cloudapi/v6/requests/7939b6f1-6c83-425d-a03a-dac52de3fc8c/status",
  |   "metadata" : {
  |     "status" : "QUEUED",
  |     "message" : "Request has been queued",
  |     "etag" : "9df29ce7ca629ff1d47bca1149d9faae",
  |     "targets" : [ {
  |       "target" : {
  |         "id" : "0e328119-5489-461e-b357-9bbae44ade64",
  |         "type" : "nic",
  |         "href" : "https:/

2024-11-11T09:21:53.637+0100 [WARN]  unexpected data:
  registry.terraform.io/ionos-cloud/ionoscloud:stderr=
  | /api.ionos.com/cloudapi/v6/datacenters/8c0eab6e-0000-0000-0000-7e2fd4934c67/servers/6791b3d2-0000-0000-0000-029c76d2613d/nics/0e328119-0000-0000-0000-9bbae44ade64"
  |       },
  |       "status" : "QUEUED"
  |     } ]
  |   }
  | }
  | \r
  | 0

Additional Notes

Request queued but won't finish. Tried to let it run for several minutes up to 2 hours.

[cristiGuranIonos]

A few details:

1. Does this happen every time, or only from time to time?
2. Is there a plan on which this happens? On certain resources?
3. Are the resources created in DCD if you look, but not in AVAILABLE state?
   Thanks

[mosaickai]

1. It happens everytime for about 2 weeks now.
2. Right now it's LAN creation.
3. We create a new DCD with every run. It is available afterwards.

[cristiGuranIonos]

do you have a small plan where this happens? Could not reproduce requests hanging for lan creation.
Is it multiple lans?
In what location does it happen?

[mosaickai]

Region Frankfurt.
This is the entire script:

terraform {
  required_providers {
    ionoscloud = {
      source = "ionos-cloud/ionoscloud"
      version = ">= 6.4.10"
    }
  }
}

provider "ionoscloud" {
  token = var.ionos_token
}

resource "ionoscloud_datacenter" "dcd" {
  count = var.studentCount
  name = "${var.dcd_name}Student-${count.index}"
  location = var.location
}

resource "ionoscloud_lan" "public_lan_firewall" {
  depends_on = [ ionoscloud_datacenter.dcd ]
  count = var.studentCount
  datacenter_id = ionoscloud_datacenter.dcd[count.index].id
  public        = true
  name          = "PUBLIC-FIREWALL"
}

resource "ionoscloud_lan" "public_lan2_firewall" {
  count = var.studentCount
  depends_on = [ionoscloud_datacenter.dcd, ionoscloud_lan.client_lan]
  datacenter_id = ionoscloud_datacenter.dcd[count.index].id
  public        = true
  name          = "PUBLIC2-FIREWALL"
}
resource "ionoscloud_lan" "public_lan_jump" {
  count = var.studentCount
  datacenter_id = ionoscloud_datacenter.dcd[count.index].id
  public        = true
  name          = "PUBLICJUMP"
}
resource "ionoscloud_lan" "server_lan" {
  depends_on = [ ionoscloud_lan.public_lan_firewall ]
  count = var.studentCount
  datacenter_id = ionoscloud_datacenter.dcd[count.index].id
  public        = false
  name          = "SRV-LAN"
}
resource "ionoscloud_lan" "client_lan" {
  depends_on = [ ionoscloud_lan.server_lan ]
  count = var.studentCount
  datacenter_id = ionoscloud_datacenter.dcd[count.index].id
  public        = false
  name          = "CLIENT-LAN"
}

resource "ionoscloud_ipblock" "ipblock" {
  count=var.studentCount
  location = var.location
  size     = 3
  name   = "IPS_${count.index}"
}

data "ionoscloud_template" "cubemtemplate" {
  name = "Basic Cube M"
}

resource "ionoscloud_cube_server" "jumphost" {
    count                 = var.studentCount
    name                  = "Jump01"
    datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
    template_uuid         = data.ionoscloud_template.cubemtemplate.id
    availability_zone     = "AUTO"
    image_name            = "fe70bf69-0000-0000-0000-32973fa01d69"
    image_password        = "PASSWORD" 
    volume {
        name              = "Volume"
        disk_type         = "DAS"
        bus               = "IDE"
        }
    nic {
        lan               = ionoscloud_lan.public_lan_jump[count.index].id
        name              = "0-PUBLIC"
        dhcp              = true
        ips               = [ionoscloud_ipblock.ipblock[count.index].ips[0]]
        firewall_active   = true
        firewall_type     = "INGRESS"
        firewall {
            protocol          = "ANY"
            name              = "RDP"
            port_range_start  = 3389
            port_range_end    = 3389
            type              = "INGRESS"
        }
    }
    
    timeouts {
      create = "30m"
      update = "300s"
      delete = "2h"
    }
}

resource "ionoscloud_cube_server" "server" {
    count                 = var.studentCount
    name                  = "DC01"
    datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
    template_uuid         = data.ionoscloud_template.cubemtemplate.id
    availability_zone     = "AUTO"
    image_name            = var.image_name_server
    volume {
        name              = "Volume"
        disk_type         = "DAS"
        bus               = "IDE"
        }
    nic {
        lan               = ionoscloud_lan.server_lan[count.index].id
        name              = "0-PRIVATE"
        dhcp              = false
        firewall_active   = false
        firewall_type     = "BIDIRECTIONAL"
    }
    
    timeouts {
      create = "30m"
      update = "300s"
      delete = "2h"
    }
}
resource "ionoscloud_cube_server" "client" {
    count                 = var.studentCount
    name                  = "Client01"
    datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
    template_uuid         = data.ionoscloud_template.cubemtemplate.id
    availability_zone     = "AUTO"
    image_name            = var.image_name_client
    volume {
        name              = "Volume"
        disk_type         = "DAS"
        bus               = "IDE"
        }
    nic {
        lan               = ionoscloud_lan.client_lan[count.index].id
        name              = "0-PRIVATE"
        dhcp              = false
        firewall_active   = false
        firewall_type     = "BIDIRECTIONAL"
    }
    
    timeouts {
      create = "30m"
      update = "300s"
      delete = "2h"
    }
}

resource "ionoscloud_cube_server" "firewall" {
    count                 = var.studentCount
    depends_on            = [ionoscloud_lan.public_lan_firewall]
    name                  = "Firewall01"
    datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
    template_uuid         = data.ionoscloud_template.cubemtemplate.id
    availability_zone     = "AUTO"
    image_name            = var.image_name_firebox
    volume {
        name              = "Volume"
        disk_type         = "DAS"
        bus               = "IDE"
        }
    nic {
        lan               = ionoscloud_lan.public_lan_firewall[count.index].id
        name              = "0-PUBLIC"
        dhcp              = true
        ips               = [ionoscloud_ipblock.ipblock[count.index].ips[1]]
        firewall_active   = false
        firewall_type     = "BIDIRECTIONAL"
    }
    
    timeouts {
      create = "30m"
      update = "300s"
      delete = "2h"
    }
}

resource "ionoscloud_nic" "JUMPHOSTNIC" {
  count                 = var.studentCount
  datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
  server_id             = ionoscloud_cube_server.jumphost[count.index].id
  lan                   = ionoscloud_lan.server_lan[count.index].id
  name                  = "2-CLIENT2SERVER"
  dhcp                  = false
  firewall_active       = false
}

resource "ionoscloud_nic" "FIREWALLNIC1" {
  depends_on = [ ionoscloud_nic.FIREWALLNIC2 ]
  count                 = var.studentCount
  datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
  server_id             = ionoscloud_cube_server.firewall[count.index].id
  lan                   = ionoscloud_lan.client_lan[count.index].id
  name                  = "1-FW2CLIENT"
  dhcp                  = false
  firewall_active       = false
}

resource "ionoscloud_nic" "FIREWALLNIC2" {
  depends_on = [ ionoscloud_lan.public_lan_firewall ]
  count                 = var.studentCount
  datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
  server_id             = ionoscloud_cube_server.firewall[count.index].id
  lan                   = ionoscloud_lan.server_lan[count.index].id
  name                  = "2-FW2SERVER"
  dhcp                  = false
  firewall_active       = false
}
resource "ionoscloud_nic" "FIREWALLNIC3" {
  depends_on = [ ionoscloud_nic.FIREWALLNIC1]
  count                 = var.studentCount
  datacenter_id         = ionoscloud_datacenter.dcd[count.index].id
  server_id             = ionoscloud_cube_server.firewall[count.index].id
  lan                   = ionoscloud_lan.public_lan2_firewall[count.index].id
  name                  = "1-Public"
  dhcp                  = true
  ips               = [ionoscloud_ipblock.ipblock[count.index].ips[2]]
  firewall_active       = false
  firewall_type     = "BIDIRECTIONAL"
}

[cristiGuranIonos]

Tried to reproduce, but noticed only:

firewall {
  protocol          = "ANY"
  name              = "RDP"
  port_range_start  = 3389
  port_range_end    = 3389
  type              = "INGRESS"
}

Threw an error and I needed to change protocol from "ANY".
Second, LAN deletion took some time on one of the tests.

I could not get it to not finish provisioning.

Will ask another team to investigate.
Thanks!