decide how to handle the breakout of the dockershim from the kubelet.

[neolit123]

this issue tracks the dockershim deprecation / removal with some key events over time.

summary of the kubeadm stance for the dockershim removal:

• if only docker is present kubeadm will try to use it but a 1.24> kubelet will fail.
  you'd have to build and deploy cri-dockerd on the host that exposes the dockershim CRI socket:
  https://github.com/Mirantis/cri-dockerd
• if you have feedback about the dockershim removal please comment on Dockershim removal feedback & issues kubernetes#106917
• if you have multiple container runtime sockets on the host kubeadm will throw an error asking you to pick one
  this is possible with init|joinconfiguration.noderegistration.crisocket
• if you only have one socket - containerd or crio, kubeadm will autodetect and use the socket
• docs / migration guides from docker to another container runtime will contain details about kubeadm
• kubeadm no longer has e2e tests for docker as the CR. in the future we can have tests if cri-dockerd becomes maintained and has docs / releases.

history:

update 19.05.2020:

• building kubelet without dockershim is now possible:
  Build Kubelet without Docker enhancements#1546
• next step is breaking out dockershim (new KEP)?

update 11.11.2020:

• dockershim has been deprecated in 1.20:
  Deprecate Dockershim kubernetes#94624
  Removing dockershim from kubelet enhancements#1985

update 1.24 / 12.2021:

• dockershim was removed. more details below:
  decide how to handle the breakout of the dockershim from the kubelet. #1412 (comment)
• kubeadm no longer tests (until further notice) with docker as the CR in e2e tests:
  kinder: switch all workflows to containerd #2620
• dockershim flags are removed:
  Clean up dockershim flags in the kubelet kubernetes#106907

[rosti]

I'll move in on this one.

/assign

[timothysc]

We're blocked on kubelet changes first.

[astrieanna]

/remove-help

This does not fit the help wanted definition because:

• It seems to involve community discussion/decisions (e.g. how to distribute the binary).
• It seems to be taken already (by @rosti).

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[neolit123]

/remove-lifecycle stale
/lifecycle frozen

[BenTheElder]

Should cri-dockerd now be distributed by mirantis etc. downstream just as docker and containerd etc. are handled by their vendor(s) / distros? (given that the kubernetes project is not going to own this binary)

[neolit123]

my vote goes for not releasing cri-dockerd as part of the k8s release packages, though this is up to sig-release to decide.
we should probably stop doing the same for crictl too. the current version is outdated but our packages depend on it:
kubernetes/release#1636
and a wild guess is that if we change this it will break apt/rpm users.

on the kubeadm side, my prior proposal was to stop managing the CR cgroup driver and sandbox image for all CRs. docker was the only one we did that for and now that it becomes external we should stop doing these actions completely.

a few documentation updates will be required for users to prepare these during node setup at:
https://kubernetes.io/docs/setup/production-environment/container-runtimes/

[xlgao-zju]

@neolit123 if the docker-shim is remove by kubelet, how will we handle the upgrade of control-plane which is using docker as runtime?

[neolit123]

The plan is for dockershim to be a separete binary downloaded from a third party location. It would become the responsibility of the cluster admin to install / configure / upgrade it.

[dims]

please see https://github.com/Mirantis/cri-dockerd

[neolit123]

status update for 1.23 around the planned dockershim removal in 1.24:

• at the k/website we already have some draft PRs to address migration guides from dockershim -> CR foo:
  Document and announce migration guide for Dockershim kubernetes#104878 (comment)
  Create a doc to explain how to change the runtime (mostly moving away from dockershim) website#25879
  Docs to change Container runtime website#30141
  (just reviewed one of these).
• despite complains from Minikube maintainers https://github.com/Mirantis/cri-dockerd seems not not be actively maintained and has no releases. this means remaining on Docker and using cri-dockerd as a replacement for dockershim after kubelet 1.24 will be unknown territory. Unless Mirantis start creating releases and write guides on how to deploy the daemon and configure it (e.g. via systemd) this will not be a recommended path.
• kubeadm has no plans to integrate with cri-dockerd and it would be up to the user to set it up if they wish to continue using Docker as the container runtime for new and old clusters after 1.24.

[neolit123]

looks like dockershim was removed from the kubelet in k/k master (PR is LGTM/approved)
kubernetes/kubernetes#97252

this means we have to switch our e2e jobs against k/k master to containerd. since our setup is rather complex it would be easier to switch the jobs against all k/k branches to containerd.

to do that we have to modify the base image in all the files here:
https://github.com/kubernetes/kubeadm/blob/ecb3d2206a00b1aab25d754a0c4c95d781ac0909/kinder/ci/tools/update-workflows/templates/workflows/
to be baseImage: kindest/base:v20191105-ee880e9b

and then run ./hack/update-workflows.sh in /kinder.

EDIT: PR for that is here:
#2620

[neolit123]

one problem in the kubeadm code base is here:
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/kubelet/flags.go#L80-L86

with our support skew we'd have to not manage this branching for kubelet N (assuming one can set it manually in cri-dockerd or if it's cni by default), while we have to manage it for kubelet N-1.

i think this issue can be closed after that.

in an ideal world the https://github.com/Mirantis/cri-dockerd project should include docs on how to deploy a service (e.g. managed by systemd) for users that never want to move away from docker(shim).also assuming the project will gain maintenance traction.

[sftim]

BTW, is the description of this issue current? (if not, would it be useful to revise it?)

[neolit123]

good point, updated with a summary

[neolit123]

cross linking to the tracking issue for dockershim removal feedback:
kubernetes/kubernetes#106917

[afbjorklund]

in an ideal world the https://github.com/Mirantis/cri-dockerd project should include docs on how to deploy a service (e.g. managed by systemd) for users that never want to move away from docker(shim).

Currently they refer to https://docs.mirantis.com, which doesn't have any information for cri-dockerd

It only documents the Mirantis Container Runtime (MCR) and Mirantis Kubernetes Engine (MKE) products.

[neolit123]

closing in favor of remain tasks for 1.24 and later TBD here:
#2626