hashicorp/nomad docker image does not include CA certs

[Freddo3000]

Nomad version

Docker run log

docker run -it hashicorp/nomad:1.9 -v
Unable to find image 'hashicorp/nomad:1.9' locally
1.9: Pulling from hashicorp/nomad
41a4ffd8d182: Already exists 
7903fc81a5b6: Pull complete 
f15bc9e9d30a: Pull complete 
1332bb6a171a: Pull complete 
255794d141ea: Pull complete 
Digest: sha256:d9d357f44aea8c6566f1ecb87d63283fcccc63728486c452fd805383f92a12da
Status: Downloaded newer image for hashicorp/nomad:1.9
Nomad v1.9.3
BuildDate 2024-11-11T16:35:41Z
Revision d92bf1014886c0ff9f882f4a2691d5ae8ad8131c

Operating system and Environment details

• Docker version 27.3.1, build ce12230
• Ubuntu 22.04

Issue

The hashicorp/nomad image does not include ca-certificates, or any easy way to install them (being built on busybox). This causes issues in my case where Nomad is run using Let's Encrypt certs.

Reproduction steps

• Run a Nomad instance with TLS configured using a Let's Encrypt cert.
• Try to connect to it using the Nomad docker image.

Expected Result

Nomad client is able to connect to the Nomad server.

Actual Result

nomad job status
Error querying jobs: Get "https://XXXX.YYY/v1/jobs": tls: failed to verify certificate: x509: certificate signed by unknown authority

Job file (if appropriate)

n/a

Nomad Server logs (if appropriate)

n/a

Nomad Client logs (if appropriate)

See Actual Result.

[tgross]

@Freddo3000 were you to run the agent in the container, you'll note that the entrypoint script prevents you from running it as an agent without passing a specific environment variable. The container image has intentionally not been built with running the agent in mind.

You could fix that by bind-mounting the CA certs from the host into the container, but trying to use Let's Encrypt certificates for Nomad tells me you've got a very unusual configuration. Nomad uses mTLS for host-to-host communication and expect specific non-global names in those certificates, which you can't get with Let's Encrypt.

At this time we're not intending on shipping this change in the official Docker image, so I'm going to close this issue out, but I'm happy to keep chatting about what you're trying to do.

[Freddo3000]

@Freddo3000 were you to run the agent in the container, you'll note that the entrypoint script prevents you from running it as an agent without passing a specific environment variable. The container image has intentionally not been built with running the agent in mind.

You could fix that by bind-mounting the CA certs from the host into the container, but trying to use Let's Encrypt certificates for Nomad tells me you've got a very unusual configuration. Nomad uses mTLS for host-to-host communication and expect specific non-global names in those certificates, which you can't get with Let's Encrypt.

At this time we're not intending on shipping this change in the official Docker image, so I'm going to close this issue out, but I'm happy to keep chatting about what you're trying to do.

To clarify, I'm not starting Nomad as an agent in docker, I'm using it as part of a CI/CD to initiate a Nomad job using nomad job run, accessing Nomad through a Traefik proxy. This is oriented towards external clients (with some restrictions), while internally for agents and servers we use mTLS powered by Vault.

The reason for using Let's Encrypt externally is to make onboarding easier for external users. Our users are primarily university students who may deploy services and projects through simple CI/CD chains on Github or Gitlab as part of course work, so what we're aiming to do is allow access through university networks with tokens/ACLs we provide without the added complexity of mTLS PKI. For this being able to use the hashicorp/nomad image directly would be nice without having to compile our own, which may be as simple as swapping out the busybox image with alpine, adding the ca-certificates package, which I don't see as adding any security risk or significant overhead to this project.