[BUG] modified cni binaries do not support Pod IP address kept as expected

[joez]

What happened:

As mentioned by the official doc: yurtadm join

The process of yurtadm join will pull specially modified cni binaries, the modifications can be found here.

The modified host-local should support IPAM: Pod IP address kept

But the result is not as expected, everytime I rebooted the edge node, the deployed Pod got a new IP address

And the IP address records are not as expected:

$ sudo ls /var/lib/cni/networks/cbr0
10.244.0.2  10.244.0.3  last_reserved_ip.0  lock

After build the host-local with patch 0002-ipam-keep-pod-ip and install it, things work as expected.

So I think the latest modified cni binaries provided by OpenYurt did not include the correct patches

What you expected to happen:

The Pod should get the same IP address after edge node rebooted, the IP address records should be like:

$ sudo ls /var/lib/cni/networks/cbr0/
10.244.1.12  10.244.1.13  10.244.1.13_default_nginx-6799fc88d8-j7fwc  last_reserved_ip.0  lock

How to reproduce it (as minimally and precisely as possible):

Deploy OpenYurt in a Kubernetes cluster with two nodes, details in #1400

Install the modified cni binaries which is used by yurtadm:

Tried yurtadm join, got the same result

tar czf cni-plugin-backup.tar.gz -C /opt/cni/bin/ .
curl -LO https://aliacs-edge-k8s-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/public/pkg/openyurt/cni/v0.8.0/cni-plugins-linux-amd64-v0.8.0.tgz
sudo tar xzf cni-plugins-linux-amd64-v0.8.0.tgz -C /opt/cni/bin

Now, deploy nginx server and check the result

kubectl create deployment nginx --image=nginx --dry-run=client -o=yaml > nginx.yaml
echo '---' >> nginx.yaml
kubectl create service nodeport nginx --dry-run=client -o yaml --tcp=80:80 --node-port=30080 >> nginx.yaml

kubectl apply -f nginx.yaml
# wait until ready
kubectl get po -w
# the po should be scheduled on the worker node
kubectl get all -o wide

# check web server
curl localhost:30080

On the edge node, check the assigned IP:

docker exec $(docker ps -f 'ancestor=nginx' --format '{{.ID}}') cat /proc/net/fib_trie | awk '/32 host/ { print i } {i=$2}' | grep -v 127.0 | uniq
# the output IP should be in
sudo ls /var/lib/cni/networks/cbr0/

Reboot the edge node and check it again after booted, the result IP is different

Anything else we need to know?:

Environment:

• OpenYurt version: 1.2.0
• Kubernetes version (use kubectl version): 1.22.0
• OS (e.g: cat /etc/os-release): Ubuntu 20.04.5 LTS
• Kernel (e.g. uname -a): Linux xxx 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Others:

others

/kind bug

[rambohe-ch]

@joez Thanks for raising issue, it looks like that cni binaries has been changed unintentionally, and i will update cni binaries as soon as possible.

[rambohe-ch]

/assign @rambohe-ch

[luc99hen]

/assign @luc99hen

[luc99hen]

The CNI binaries have been updated. @rambohe-ch

[rambohe-ch]

/close

[openyurt-bot]

@rambohe-ch: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.