Skip to content

Commit

Permalink
[Windows] Optimize the containerized OVS installation (#6383)
Browse files Browse the repository at this point in the history
1. Add logic to check the installed OVSext drivers, if only the desired version
    of driver is already installed, skip the installation; otherwise, remove the
    existing drivers and re-install.
2. Add logic to check the installed VC redistributable files, if the existing
    installed vc_redist version is greater or equal than the min required version,
    skip the installation; otherwise re-install with the provided files.
3. Improve the logic updating the env paths by removing duplicated paths.
4. Improve Uninstall-OVS script by removing OVS bin paths from the system path
    after they are deleted.
5. Fix certificate import for a signed OVS driver when Node is running Windows
    Server 2022.

Note that running `.\Install-OVS.ps1` manually is no longer required when using
the containerized OVS installation method, even when using the test-signed
OVS driver that we provide.

Signed-off-by: Wenying Dong <[email protected]>
  • Loading branch information
wenyingd authored Jun 28, 2024
1 parent d029557 commit fc40157
Show file tree
Hide file tree
Showing 10 changed files with 550 additions and 259 deletions.
2 changes: 1 addition & 1 deletion build/charts/antrea-windows/conf/Run-AntreaAgent.ps1
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
41 changes: 9 additions & 32 deletions build/charts/antrea-windows/conf/ovs/Install-OVSDriver.ps1
Original file line number Diff line number Diff line change
@@ -1,37 +1,14 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
$result = netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
if ($result -like '*failed*') {
Write-Host "Failed to install OVSExt driver: $result"
exit 1
}
Write-Host "OVSExt driver has been installed"
$OVSInstallScript = "$mountPath\k\antrea\Install-OVS.ps1"
if (-not (Test-Path $OVSInstallScript)) {
Write-Host "Installation script not found: $OVSInstallScript, you may be using an invalid antrea-windows container image"
exit 1
}

# Check if the VC redistributable is already installed.
$OVSRedistDir="$mountPath\openvswitch\redist"
if (Test-Path $OVSRedistDir) {
$dllFound = $false
$paths = $env:PATH -split ';'
foreach ($path in $paths) {
$dllFiles = Get-ChildItem -Path $path -Filter "vcruntime*.dll" -File -ErrorAction SilentlyContinue
if ($dllFiles.Count -gt 0) {
$dllFound = $true
break
}
}

# vcruntime dlls are not installed on the host, then install the binaries.
if (-not $dllFound) {
Get-ChildItem $OVSRedistDir -Filter *.exe | ForEach-Object {
Start-Process -FilePath $_.FullName -Args '/install /passive /norestart' -Verb RunAs -Wait
}
}
& $OVSInstallScript -LocalFile "$mountPath/openvswitch" -InstallUserspace $false
If (!$?) {
Write-Host "Failed to install OVS driver"
exit 1
}
Write-Host "Completed OVS driver installation"
2 changes: 1 addition & 1 deletion build/charts/antrea-windows/conf/ovs/Run-AntreaOVS.ps1
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Configure OVS processes
Expand Down
3 changes: 2 additions & 1 deletion build/images/Dockerfile.build.windows
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ RUN --mount=type=cache,target=/go/pkg/mod/ \
RUN mkdir -p /go/k/antrea/bin && \
cp /antrea/bin/antrea-agent.exe /go/k/antrea/bin/ && \
cp /antrea/bin/antctl.exe /go/k/antrea/bin/ && \
cp /antrea/bin/antrea-cni.exe /go/k/antrea/cni/antrea.exe
cp /antrea/bin/antrea-cni.exe /go/k/antrea/cni/antrea.exe && \
cp /antrea/hack/windows/Install-OVS.ps1 /go/k/antrea/

FROM antrea/windows-ovs:${OVS_VERSION} AS antrea-ovs

Expand Down
47 changes: 12 additions & 35 deletions build/yamls/antrea-windows-with-ovs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,51 +40,28 @@ data:
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
Install-OVSDriver.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$OVSDriverDir = "$mountPath\openvswitch\driver"
# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
$result = netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
if ($result -like '*failed*') {
Write-Host "Failed to install OVSExt driver: $result"
exit 1
}
Write-Host "OVSExt driver has been installed"
$OVSInstallScript = "$mountPath\k\antrea\Install-OVS.ps1"
if (-not (Test-Path $OVSInstallScript)) {
Write-Host "Installation script not found: $OVSInstallScript, you may be using an invalid antrea-windows container image"
exit 1
}
# Check if the VC redistributable is already installed.
$OVSRedistDir="$mountPath\openvswitch\redist"
if (Test-Path $OVSRedistDir) {
$dllFound = $false
$paths = $env:PATH -split ';'
foreach ($path in $paths) {
$dllFiles = Get-ChildItem -Path $path -Filter "vcruntime*.dll" -File -ErrorAction SilentlyContinue
if ($dllFiles.Count -gt 0) {
$dllFound = $true
break
}
}
# vcruntime dlls are not installed on the host, then install the binaries.
if (-not $dllFound) {
Get-ChildItem $OVSRedistDir -Filter *.exe | ForEach-Object {
Start-Process -FilePath $_.FullName -Args '/install /passive /norestart' -Verb RunAs -Wait
}
}
& $OVSInstallScript -LocalFile "$mountPath/openvswitch" -InstallUserspace $false
If (!$?) {
Write-Host "Failed to install OVS driver"
exit 1
}
Write-Host "Completed OVS driver installation"
Run-AntreaOVS.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$OVSDriverDir = "$mountPath\openvswitch\driver"
# Configure OVS processes
Expand Down Expand Up @@ -328,7 +305,7 @@ spec:
template:
metadata:
annotations:
checksum/agent-windows: 5efe6525007ef87c58914b37d190f84bc93b8cf081d204979dffce0859ee2da3
checksum/agent-windows: 86f999cb18501659a52d982f20b3df5cdf666ffd849f50ed183c366e75d01ac5
checksum/windows-config: 10ad2be0a04b1752abc224fed0124f7b1da36efc5e7323e193eb38e11b25e798
microsoft.com/hostprocess-inherit-user: "true"
labels:
Expand Down
4 changes: 2 additions & 2 deletions build/yamls/antrea-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ data:
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
kind: ConfigMap
metadata:
Expand Down Expand Up @@ -233,7 +233,7 @@ spec:
template:
metadata:
annotations:
checksum/agent-windows: adb135c962fe85e0a2bc86a45f4b8c72d89b09a1da35bb16775e547813295679
checksum/agent-windows: 63f16e1fadb6b1354efda21c73702b4290400181136d4d47d4b1cd6a5f82d037
checksum/windows-config: 10ad2be0a04b1752abc224fed0124f7b1da36efc5e7323e193eb38e11b25e798
microsoft.com/hostprocess-inherit-user: "true"
labels:
Expand Down
7 changes: 4 additions & 3 deletions ci/jenkins/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -778,15 +778,16 @@ function run_install_windows_ovs {
govc vm.power -on $OVS_VM_NAME || true
echo "===== Testing VM has been reverted and powered on ====="
IP=$(govc vm.ip $OVS_VM_NAME)
scp -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -T hack/windows/Install-OVS.ps1 Administrator@${IP}:
ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} '/bin/bash -lc "cp Install-OVS.ps1 C:/k && powershell.exe -File C:/k/Install-OVS.ps1"'
scp -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -T hack/windows/Install-OVS.ps1 Administrator@${IP}:/cygdrive/c/k/
ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} '/bin/bash -lc "powershell.exe -File c:/k/Install-OVS.ps1 -InstallUserspace:$true"'

set +e
RC_SERVER=$(ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} 'powershell.exe -Command "(get-service ovsdb-server).Status -eq \"Running\""')
RC_VSWITCHD=$(ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} 'powershell.exe -Command "(get-service ovs-vswitchd).Status -eq \"Running\""')
OVSDriverStatus=$(ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} 'netcfg.exe -q ovsext')
set -e

if [[ $RC_SERVER != *True* || $RC_VSWITCHD != *True* ]]; then
if [[ $RC_SERVER != *True* || $RC_VSWITCHD != *True* || $OVSDriverStatus != *"is installed"* ]]; then
echo "=== TEST FAILURE !!! ==="
TEST_FAILURE=true
ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} "tar zcf openvswitch.tar.gz -C /cygdrive/c/openvswitch/var/log openvswitch"
Expand Down
26 changes: 8 additions & 18 deletions docs/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,12 @@ depending on whether you are using your own [signed](https://docs.microsoft.com/
OVS kernel driver or you want to use the test-signed driver provided by Antrea,
you will need to invoke the `Install-OVS.ps1` script differently (or not at all).

| Containerized OVS daemons? | Test-signed OVS driver? | Run this command |
| -------------------------- | ----------------------- | ---------------- |
| Yes | Yes | `.\Install-OVS.ps1 -InstallUserspace $false` |
| Yes | No | N/A |
| No | Yes | `.\Install-OVS.ps1` |
| No | No | `.\Install-OVS.ps1 -ImportCertificate $false -Local -LocalFile <PathToOVSPackage>` |
| Containerized OVS daemons? | Test-signed OVS driver? | Run this command |
| -------------------------- | ----------------------- |---------------------------------------------------------------------------|
| Yes | Yes | Not required |
| Yes | No | Not required |
| No | Yes | `.\Install-OVS.ps1 -InstallUserspace $true` |
| No | No | `.\Install-OVS.ps1 -InstallUserspace $true -LocalFile <PathToOVSPackage>` |

If you used `antrea-windows-with-ovs.yml` to create the antrea-agent
Windows DaemonSet, then you are using "Containerized OVS daemons". For all other
Expand All @@ -193,22 +193,12 @@ Bcdedit.exe -set TESTSIGNING ON
Restart-Computer
```

As an example, if you are using containerized OVS
(`antrea-windows-with-ovs.yml`), and you want to use the test-signed
OVS kernel driver provided by Antrea (not recommended for production), you would
run the following commands:

```powershell
curl.exe -LO https://raw.githubusercontent.com/antrea-io/antrea/main/hack/windows/Install-OVS.ps1
.\Install-OVS.ps1 -InstallUserspace $false
```

And, if you want to run OVS as Windows native services, and you are bringing
If you want to run OVS as Windows native services, and you are bringing
your own OVS package with a signed OVS kernel driver, you would run:

```powershell
curl.exe -LO https://raw.githubusercontent.com/antrea-io/antrea/main/hack/windows/Install-OVS.ps1
.\Install-OVS.ps1 -ImportCertificate $false -Local -LocalFile <PathToOVSPackage>
.\Install-OVS.ps1 -InstallUserspace $true -LocalFile <PathToOVSPackage>
# verify that the OVS services are installed
get-service ovsdb-server
Expand Down
Loading

0 comments on commit fc40157

Please sign in to comment.