Skip to content

Commit

Permalink
[Windows] Finish removing Docker and userspace kube-proxy support (#6255
Browse files Browse the repository at this point in the history
)

* Finish removing Docker and userspace kube-proxy support

We update the Windows documentation to remove all Docker-specific
instructions, and all mentions of (userspace) kube-proxy.

We also update Windows scripts to remove Docker support. The
Prepare-AntreaAgent.ps1 script was still defaulting to installing
kube-proxy, so we update the script to change this default behavior. For
all the scripts, "omitting" kube-proxy is now the default behavior. The
corresponding script parameters have not been removed yet, but they are
not officially deprecated, and we can remove them in a future release,
such as Antrea v2.3.

Fixes #5630

* Rename Antrea Windows YAML manifests

Now that containerd is the only supported container runtime, we rename
antrea-windows-containerd.yml to antrea-windows.yml and
antrea-windows-containerd-with-ovs.yml to antrea-windows-with-ovs.yml.

Signed-off-by: Antonin Bas <[email protected]>
  • Loading branch information
antoninbas authored Apr 24, 2024
1 parent 42e7cfa commit b11c561
Show file tree
Hide file tree
Showing 11 changed files with 90 additions and 326 deletions.
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -417,8 +417,8 @@ build-migrator:
manifest:
@echo "===> Generating dev manifest for Antrea <==="
$(CURDIR)/hack/generate-standard-manifests.sh --mode dev --out build/yamls
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev > build/yamls/antrea-windows-containerd.yml
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev --include-ovs > build/yamls/antrea-windows-containerd-with-ovs.yml
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev > build/yamls/antrea-windows.yml
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev --include-ovs > build/yamls/antrea-windows-with-ovs.yml
$(CURDIR)/hack/update-checksum-windows.sh
$(CURDIR)/hack/generate-manifest-flow-aggregator.sh --mode dev > build/yamls/flow-aggregator.yml

Expand Down
File renamed without changes.
File renamed without changes.
8 changes: 4 additions & 4 deletions ci/jenkins/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ function revert_snapshot_windows {
function build_and_deliver_antrea_windows_and_linux_containerd_images {
echo "====== Cleanup Antrea Installation Before Delivering Antrea Windows and Antrea Linux containerd Images ====="
clean_antrea
kubectl delete -f ${WORKDIR}/antrea-windows-containerd-with-ovs.yml --ignore-not-found=true || true
kubectl delete -f ${WORKDIR}/antrea-windows-with-ovs.yml --ignore-not-found=true || true
kubectl delete -f ${WORKDIR}/kube-proxy-windows-containerd.yml --ignore-not-found=true || true
kubectl delete daemonset antrea-agent -n kube-system --ignore-not-found=true || true
kubectl delete -f ${WORKDIR}/antrea.yml --ignore-not-found=true || true
Expand All @@ -392,11 +392,11 @@ function build_and_deliver_antrea_windows_and_linux_containerd_images {
${PRINT_DOCKER_STATUS}
export_govc_env_var
# Enable verbose log for troubleshooting.
sed -i "s/--v=0/--v=4/g" build/yamls/antrea.yml build/yamls/antrea-windows-containerd-with-ovs.yml
sed -i "s/--v=0/--v=4/g" build/yamls/antrea.yml build/yamls/antrea-windows-with-ovs.yml

echo "====== Updating yaml files to enable proxyAll ======"
KUBE_API_SERVER=$(kubectl --kubeconfig=$KubeConfigFile config view -o jsonpath='{.clusters[0].cluster.server}')
sed -i "s|.*kubeAPIServerOverride: \"\"| kubeAPIServerOverride: \"${KUBE_API_SERVER}\"|g" build/yamls/antrea.yml build/yamls/antrea-windows-containerd-with-ovs.yml
sed -i "s|.*kubeAPIServerOverride: \"\"| kubeAPIServerOverride: \"${KUBE_API_SERVER}\"|g" build/yamls/antrea.yml build/yamls/antrea-windows-with-ovs.yml

cp -f build/yamls/*.yml $WORKDIR
set +e
Expand Down Expand Up @@ -1004,7 +1004,7 @@ fi
trap clean_antrea EXIT
if [[ ${TESTCASE} =~ "windows" ]]; then
if [[ ${TESTCASE} =~ "containerd" ]]; then
WINDOWS_YAML_SUFFIX="windows-containerd-with-ovs"
WINDOWS_YAML_SUFFIX="windows-with-ovs"
build_and_deliver_antrea_windows_and_linux_containerd_images
if [[ ${TESTCASE} =~ "e2e" ]]; then
run_e2e_windows
Expand Down
10 changes: 5 additions & 5 deletions docs/design/windows-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,11 @@ these scenarios:

## Antrea and OVS Management on Windows

While we provide different installation methods for Windows, the recommended one starting with
Antrea v1.13 is to use the `antrea-windows-containerd-with-ovs.yml` manifest. With this method, the
antrea-agent process and the OVS daemons (ovsdb-server and ovs-vswitchd) run as a Pod on Windows
worker Nodes, and are managed by a DaemonSet. This installation method relies on
[Windows HostProcess Pod](https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/)
While we provide different installation methods for Windows, the recommended one
is to use the `antrea-windows-with-ovs.yml` manifest. With this method, the
antrea-agent process and the OVS daemons (ovsdb-server and ovs-vswitchd) run as
a Pod on Windows worker Nodes, and are managed by a DaemonSet. This installation
method relies on [Windows HostProcess Pod](https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/)
support.

## Traffic walkthrough
Expand Down
331 changes: 53 additions & 278 deletions docs/windows.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions hack/release/prepare-assets.sh
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,8 @@ export CONTROLLER_IMG_NAME=antrea/antrea-controller-ubuntu
./hack/generate-standard-manifests.sh --mode release --out "$OUTPUT_DIR"

export IMG_NAME=antrea/antrea-windows
./hack/generate-manifest-windows.sh --mode release > "$OUTPUT_DIR"/antrea-windows-containerd.yml
./hack/generate-manifest-windows.sh --mode release --include-ovs > "$OUTPUT_DIR"/antrea-windows-containerd-with-ovs.yml
./hack/generate-manifest-windows.sh --mode release > "$OUTPUT_DIR"/antrea-windows.yml
./hack/generate-manifest-windows.sh --mode release --include-ovs > "$OUTPUT_DIR"/antrea-windows-with-ovs.yml

export IMG_NAME=antrea/flow-aggregator
./hack/generate-manifest-flow-aggregator.sh --mode release > "$OUTPUT_DIR"/flow-aggregator.yml
Expand Down
4 changes: 2 additions & 2 deletions hack/update-checksum-windows.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,5 @@ for file in ${MANIFESTS[@]}; do
sed -i.bak "s/windows-config-checksum-placeholder/${checksum_windows_config}/g" ${file}
done

sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd}/g" ${YAMLS_DIR}/antrea-windows-containerd.yml
sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd_with_ovs}/g" ${YAMLS_DIR}/antrea-windows-containerd-with-ovs.yml
sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd}/g" ${YAMLS_DIR}/antrea-windows.yml
sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd_with_ovs}/g" ${YAMLS_DIR}/antrea-windows-with-ovs.yml
9 changes: 6 additions & 3 deletions hack/windows/Prepare-AntreaAgent.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ This script prepares environment needed by antrea-agent which includes:
provide the proxy for Kubernetes Services.
.PARAMETER InstallKubeProxy
Specifies whether kube-proxy interface is included in the installation. If false, this interface will not
be installed on the host.
[DEPRECATED] Specifies whether kube-proxy interface is included in the installation.
.PARAMETER RunOVSServices
Specifies whether the OVS userspace daemons should be started as Windows services.
#>
Param(
[parameter(Mandatory = $false)] [bool] $InstallKubeProxy = $true,
[parameter(Mandatory = $false)] [bool] $InstallKubeProxy = $false,
[parameter(Mandatory = $false)] [bool] $RunOVSServices= $true
)

Expand Down Expand Up @@ -48,6 +50,7 @@ if ($RunOVSServices -eq $true) {
}
# Prepare service network interface for kube-proxy.
if ($InstallKubeProxy -eq $true) {
Write-Host "Running Antrea with kube-proxy is no longer supported, this parameter will be removed soon"
Write-Host "Preparing service network interface for kube-proxy..."
& $PrepareServiceInterfaceScript
}
45 changes: 15 additions & 30 deletions hack/windows/Prepare-Node.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ script from the sig-windows-tools repo release page: https://github.com/kubernet
Kubernetes version to download and use
.PARAMETER InstallKubeProxy
Install kube-proxy
[DEPRECATED] Install kube-proxy
.PARAMETER InstallOVS
Install OVS
Expand All @@ -26,11 +26,12 @@ Install OVS
The node ip used by kubelet
.PARAMETER ContainerRuntime
Container runtime that Kubernetes will use. (docker or containerd)
[DEPRECATED] Container runtime that Kubernetes will use (docker or containerd).
Starting with Antrea v2.0, only containerd is supported.
.PARAMETER InstallOVSUserspace
Specifies whether OVS userspace processes are included in the installation. If false, these processes will not
be installed as a Windows service on the host.
Specifies whether OVS userspace processes are included in the installation. If false, these
processes will not be installed as a Windows service on the host.
.EXAMPLE
PS> .\Prepare-Node.ps1 -KubernetesVersion v1.27.0 -NodeIP 192.168.1.10 -ContainerRuntime containerd
Expand All @@ -43,8 +44,8 @@ Param(
[parameter(Mandatory = $false)] [switch] $InstallKubeProxy = $false,
[parameter(Mandatory = $false)] [switch] $InstallOVS = $false,
[parameter(Mandatory = $false, HelpMessage="Kubernetes download")] [string] $KubernetesURL="dl.k8s.io",
[parameter(HelpMessage="Container runtime that Kubernets will use")] [ValidateSet("containerd", "docker")] [string] $ContainerRuntime = "containerd",
[parameter(Mandatory = $false)] [bool] $InstallOVSUserspace = $true
[parameter(Mandatory = $false)] [ValidateSet("containerd", "docker")] [string] $ContainerRuntime = "containerd",
[parameter(Mandatory = $false)] [bool] $InstallOVSUserspace = $true
)
$ErrorActionPreference = 'Stop'

Expand All @@ -64,15 +65,13 @@ If (Get-Service kubelet -ErrorAction SilentlyContinue) {
}

if ($ContainerRuntime -eq "docker") {
if (-not(Test-Path "//./pipe/docker_engine")) {
Write-Error "Docker service was not detected - please install and start Docker before calling Prepare-Node.ps1 with -ContainerRuntime docker"
exit 1
}
} elseif ($ContainerRuntime -eq "containerd") {
if (-not(Test-Path "//./pipe/containerd-containerd")) {
Write-Error "Containerd service was not detected - please install and start Containerd before calling Prepare-Node.ps1 with -ContainerRuntime containerd"
exit 1
}
Write-Error "Docker container runtime is no longer supported"
exit 1
}

if (-not(Test-Path "//./pipe/containerd-containerd")) {
Write-Error "Containerd service was not detected - please install and start containerd before calling Prepare-Node.ps1"
exit 1
}

if (!$KubernetesVersion.StartsWith("v")) {
Expand All @@ -94,13 +93,6 @@ $env:Path += ";$global:KubernetesPath"
DownloadFile $kubeletBinPath "https:/$KubernetesURL/$KubernetesVersion/bin/windows/amd64/kubelet.exe"
DownloadFile "$global:KubernetesPath\kubeadm.exe" "https:/$KubernetesURL/$KubernetesVersion/bin/windows/amd64/kubeadm.exe"

if ($ContainerRuntime -eq "docker") {
Write-Host "Registering wins service"
DownloadFile "$global:KubernetesPath\wins.exe" https://github.com/rancher/wins/releases/download/v0.0.4/wins.exe
wins.exe srv app run --register
start-service rancher-wins
}


mkdir -force C:\var\log\kubelet
mkdir -force C:\var\lib\kubelet\etc\kubernetes
Expand All @@ -111,15 +103,8 @@ New-Item -path C:\var\lib\kubelet\etc\kubernetes\pki -type SymbolicLink -value C
$StartKubeletFileContent = '$FileContent = Get-Content -Path "/var/lib/kubelet/kubeadm-flags.env"
$global:KubeletArgs = $FileContent.Trim("KUBELET_KUBEADM_ARGS=`"")'+ [Environment]::NewLine

if ($ContainerRuntime -eq "docker") {
$StartKubeletFileContent +=[Environment]::NewLine +'$netId = docker network ls -f name=host --format "{{ .ID }}"
if ($netId.Length -lt 1) {
docker network create -d nat host
}' + [Environment]::NewLine
}

if ($InstallKubeProxy) {
Write-Host "Running Antrea with kube-proxy is no longer supported, this parameter will be removed soon"
$StartKubeletFileContent += [Environment]::NewLine + '& C:\k\Prepare-ServiceInterface.ps1 -InterfaceAlias "HNS Internal NIC"' + [Environment]::NewLine
}

Expand Down
1 change: 1 addition & 0 deletions hack/windows/Start-AntreaAgent.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ if ($LastExitCode) {
}

if ($StartKubeProxy) {
Write-Host "Running Antrea with kube-proxy is no longer supported, this parameter will be removed soon"
Write-Host "Starting kube-proxy..."
if (!(Start-KubeProxy -KubeProxy $KubernetesHome\kube-proxy.exe -KubeConfig $KubeConfig)) {
Write-Host "Failed to start kube-proxy, exit"
Expand Down

0 comments on commit b11c561

Please sign in to comment.