Skip to content

Latest commit

 

History

History
134 lines (106 loc) · 5.41 KB

workflow-executors.md

File metadata and controls

134 lines (106 loc) · 5.41 KB
Raw

Workflow Executors

A workflow executor is a process that conforms to a specific interface that allows Argo to perform certain actions like monitoring pod logs, collecting artifacts, managing container lifecycles, etc..

The executor to be used in your workflows can be changed in the configmap under the containerRuntimeExecutor key.

Docker (docker)

⚠️Deprecated.

**default in <= v3.2 **

  • Reliability:
    • Most well-tested
    • Most popular
  • Least secure:
    • It requires privileged access to docker.sock of the host to be mounted which. Often rejected by Open Policy Agent (OPA) or your Pod Security Policy (PSP).
    • It can escape the privileges of the pod's service account
    • It cannot runAsNonRoot.
  • Equal most scalable:
    • It communicates directly with the local Docker daemon.
  • Artifacts:
    • Output artifacts can be located on the base layer (e.g. /tmp).
  • Configuration:
    • No additional configuration needed.

Note: when using docker as workflow executors, messages printed in both stdout and stderr are captured in the Argo variable .outputs.result.

Kubelet (kubelet)

  • Reliability:
    • Second least well-tested
    • Second least popular
  • Secure
    • No privileged access
    • Cannot escape the privileges of the pod's service account
    • runAsNonRoot - TBD, see #4186
  • Scalable:
    • Operations performed against the local Kubelet
  • Artifacts:
    • Output artifacts must be saved on volumes (e.g. emptyDir) and not the base image layer (e.g. /tmp)
  • Step/Task result:
    • Warnings that normally goes to stderr will get captured in a step or a dag task's outputs.result. May require changes if your pipeline is conditioned on steps/tasks.name.outputs.result
  • Configuration:
    • Additional Kubelet configuration maybe needed

Kubernetes API (k8sapi)

  • Reliability:
    • Well-tested
    • Popular
    • Works on GKE Autopilot
  • Most secure:
    • No privileged access
    • Cannot escape the privileges of the pod's service account
    • Can runAsNonRoot
  • Least scalable:
    • Log retrieval and container operations performed against the remote Kubernetes API
  • Artifacts:
    • Output artifacts must be saved on volumes (e.g. emptyDir) and not the base image layer (e.g. /tmp)
  • Step/Task result:
    • Warnings that normally goes to stderr will get captured in a step or a dag task's outputs.result. May require changes if your pipeline is conditioned on steps/tasks.name.outputs.result
  • Configuration:
    • No additional configuration needed.

Process Namespace Sharing (pns)

  • Reliability:
    • Well-tested
    • Popular
  • More secure:
    • No privileged access
    • cannot escape the privileges of the pod's service account
    • Can runAsNonRoot, if you use volumes (e.g. emptyDir) for your output artifacts
    • Processes are visible to other containers in the pod. This includes all information visible in /proc, such as passwords that were passed as arguments or environment variables. These are protected only by regular Unix permissions.
  • Scalable:
    • Most operations use local procfs.
    • Log retrieval uses the remote Kubernetes API
  • Artifacts:
    • Output artifacts can be located on the base layer (e.g. /tmp)
    • Cannot capture artifacts from a base layer which has a volume mounted under it
    • Cannot capture artifacts from base layer if the container is short-lived.
  • Configuration:
    • No additional configuration needed.
  • Process will no longer run with PID 1
  • Doesn't work for Windows containers.

https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/

Emissary (emissary)

alpha

v3.1 and after

**default in >= v3.3 **

This is the most fully featured executor.

  • Reliability:
    • Not yet well-tested
    • Not yet popular
    • Works on GKE Autopilot
  • More secure:
    • No privileged access
    • Cannot escape the privileges of the pod's service account
    • Can runAsNonRoot.
  • Scalable:
    • It reads and writes to and from the container's disk and typically does not use any network APIs unless resource type template is used.
  • Artifacts:
    • Output artifacts can be located on the base layer (e.g. /tmp).
  • Configuration:
    • command must be specified for containers.

You can determine the command and args as follows:

docker image inspect -f '{{.Config.Entrypoint}} {{.Config.Cmd}}' argoproj/argosay:v2

Learn more about command and args

Image Index

If the emissary cannot determine which command to run, because you did not specify it in your workflow spec, then it will look it up in the image index. This is nothing more fancy than a configuration item.

Exit Code 64

The emissary will exit with code 64 if it fails. This may indicate a bug in the emissary.