docker_service fails on M1 Mac

[chrisbecke]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Affected Resource(s)

• docker_service

Terraform Configuration Files

resource "docker_image" "this" {
  name = "gitlab/gitlab-runner:latest"
}

resource "docker_service" "this" {
  name = "fleet-manager"
  task_spec {
    container_spec {
      image = docker_image.this.repo_digest
    }
  }
}

Expected Behaviour

Deployed on an M1 Mac the service only deploys correctly the first time. Subsequently the task_spec.placement.platforms list is updated excluding arm64.

Actual Behaviour

ID                          NAME                          IMAGE                                                                                                 NODE             DESIRED STATE   CURRENT STATE                 ERROR                                                 PORTS
ok6v0ceioukh1wv2chxxcibyb   glfleet_fleet-manager.1       gitlab/gitlab-runner:latest                                                                                            Running         Pending about a minute ago    "no suitable node (unsupported platform on 1 node)"   
r0p8xz5fl8e5but0aa11rgzxr    \_ glfleet_fleet-manager.1   gitlab/gitlab-runner:latest@sha256:d7776fd6b38f5d985832a6fde36062fb74ff38f5000d25d7843a2e3486b6d277   docker-desktop   Shutdown        Shutdown about a minute ago     

The .Spec.TaskTemplate.Placement.Platforms and .PreviousSpec.TaskTemplate.Placement.Platforms lists are rather janky in practice and do not correlate with the list of platforms supported by the target image gitlab/gitlab-runner:latest which is linux/amd64, linux/arm64, linux/ppc64le, linux/s390x.

[
  {
    "ID": "hiqog3zgxyaolwapdowrwr7vx",
    "Version": {
      "Index": 20
    },
    "CreatedAt": "2024-11-04T12:44:47.189031638Z",
    "UpdatedAt": "2024-11-04T14:30:33.955345551Z",
    "Spec": {
      "Name": "glfleet_fleet-manager",
      "Labels": {},
      "TaskTemplate": {
        "ContainerSpec": {
          "Image": "gitlab/gitlab-runner:latest",
          "StopGracePeriod": 0,
          "Healthcheck": {},
          "DNSConfig": {},
          "Configs": [
            {
              "File": {
                "Name": "/etc/gitlab-runner/config.toml",
                "UID": "0",
                "GID": "0",
                "Mode": 292
              },
              "ConfigID": "jfho1g3kw81u95uzvkk8tq4no",
              "ConfigName": "glfleet_gitlab-runner-config"
            }
          ],
          "Isolation": "default"
        },
        "Resources": {},
        "RestartPolicy": {
          "Condition": "any",
          "Delay": 0,
          "MaxAttempts": 0,
          "Window": 0
        },
        "Placement": {
          "Platforms": [
            {
              "Architecture": "amd64",
              "OS": "linux"
            }
          ]
        },
        "ForceUpdate": 0,
        "Runtime": "container"
      },
      "Mode": {
        "Replicated": {
          "Replicas": 1
        }
      },
      "UpdateConfig": {
        "Parallelism": 1,
        "FailureAction": "pause",
        "Monitor": 5000000000,
        "MaxFailureRatio": 0,
        "Order": "stop-first"
      },
      "RollbackConfig": {
        "Parallelism": 1,
        "FailureAction": "pause",
        "Monitor": 5000000000,
        "MaxFailureRatio": 0,
        "Order": "stop-first"
      },
      "EndpointSpec": {
        "Mode": "vip"
      }
    },
    "PreviousSpec": {
      "Name": "glfleet_fleet-manager",
      "Labels": {},
      "TaskTemplate": {
        "ContainerSpec": {
          "Image": "gitlab/gitlab-runner:latest@sha256:d7776fd6b38f5d985832a6fde36062fb74ff38f5000d25d7843a2e3486b6d277",
          "StopGracePeriod": 0,
          "Healthcheck": {},
          "DNSConfig": {},
          "Isolation": "default"
        },
        "Resources": {},
        "RestartPolicy": {
          "Condition": "any",
          "MaxAttempts": 0
        },
        "Placement": {
          "Platforms": [
            {
              "Architecture": "amd64",
              "OS": "linux"
            },
            {
              "Architecture": "arm64",
              "OS": "linux"
            },
            {
              "Architecture": "ppc64le",
              "OS": "linux"
            },
            {
              "Architecture": "s390x",
              "OS": "linux"
            },
            {
              "Architecture": "unknown",
              "OS": "unknown"
            },
            {
              "Architecture": "unknown",
              "OS": "unknown"
            },
            {
              "Architecture": "unknown",
              "OS": "unknown"
            },
            {
              "Architecture": "unknown",
              "OS": "unknown"
            }
          ]
        },
        "ForceUpdate": 0,
        "Runtime": "container"
      },
      "Mode": {
        "Replicated": {
          "Replicas": 1
        }
      },
      "EndpointSpec": {
        "Mode": "vip"
      }
    },
    "Endpoint": {
      "Spec": {}
    },
    "UpdateStatus": {
      "State": "updating",
      "StartedAt": "2024-11-04T14:30:33.955338468Z",
      "Message": "update in progress"
    }
  }
]

Steps to Reproduce

1. on an m1 mac
2. terraform init
3. terraform plan
4. terraform apply
5. make a trivial change so that a change will be detected
6. terraform plan
7. terraform apply

Important Factoids

Using a M1 Macbook and Docker Desktop.

[chrisbecke]

PS. I baked in the desired list of platforms (is there a way to extract these from a data.docker_image resource)?

resource "docker_service" "this" {
  name = "${local.namespace}_fleet-manager"
  task_spec {
    container_spec {
      image = docker_image.this.repo_digest
      configs {
        config_id   = docker_config.gitlab_runner_config.id
        config_name = docker_config.gitlab_runner_config.name
        file_name   = "/etc/gitlab-runner/config.toml"
      }
    }
    restart_policy {
      condition = "any"
      delay     = "5s"
      window    = "1m"
    }
    placement {
      platforms {
        os           = "linux"
        architecture = "amd64"
      }
      platforms {
        os           = "linux"
        architecture = "arm64"
      }
    }
  }
}

and docker service inspect verifies the two desired entries are present, but EVER time I run plan it detects a change:

  # module.fleet-manager.docker_service.this will be updated in-place
  ~ resource "docker_service" "this" {
        id   = "hiqog3zgxyaolwapdowrwr7vx"
        name = "glfleet_fleet-manager"

      ~ task_spec {
            # (2 unchanged attributes hidden)

          ~ placement {
                # (3 unchanged attributes hidden)

              + platforms {
                  + architecture = "arm64"
                  + os           = "linux"
                }

                # (1 unchanged block hidden)
            }

            # (3 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

docker service inspect hiqo --format '{{json .Spec.TaskTemplate.Placement}}' | jq 
{
  "Platforms": [
    {
      "Architecture": "amd64",
      "OS": "linux"
    },
    {
      "Architecture": "arm64",
      "OS": "linux"
    }
  ]
}