[Release-1.27] - Agent config retrieval can cause excessive CPU load on servers during bulk restart of nodes

[brandond]

Backport fix for Agent config retrieval can cause excessive CPU load on servers during bulk restart of nodes

• Agent config retrieval can cause excessive CPU load on servers during bulk restart of nodes #8862

[endawkins]

Validated on branch 1.27 with commit 0abb8a5 / version 1.27

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-4-130 5.14.21-150400.22-default #1 SMP PREEMPT_DYNAMIC Wed May 11 06:57:18 UTC 2022 (49db222) x86_64 x86_64 x86_64 GNU/Linux
NAME="SLES"
VERSION="15-SP4"
VERSION_ID="15.4"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP4"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp4"
DOCUMENTATION_URL="https://documentation.suse.com/"

Cluster Configuration:

3 servers
3 agents

Additional files

N/A

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s
3. Reboot AWS worker instances
4. Verify the worker nodes come back up

Validation Results:

• k3s version used for validation:

k3s -v
k3s version v1.27.8-rc1+k3s1 (0abb8a5a)
go version go1.20.11

NAME               STATUS     ROLES                       AGE    VERSION            INTERNAL-IP     EXTERNAL-IP     OS-IMAGE                              KERNEL-VERSION              CONTAINER-RUNTIME
ip-172-31-10-238   Ready      control-plane,etcd,master   113m   v1.27.8-rc1+k3s1   172.31.10.238   13.58.97.126    SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-15-133   Ready      control-plane,etcd,master   111m   v1.27.8-rc1+k3s1   172.31.15.133   18.224.33.42    SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-3-252    NotReady   <none>                      109m   v1.27.8-rc1+k3s1   172.31.3.252    3.144.101.172   SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-7-155    NotReady   <none>                      110m   v1.27.8-rc1+k3s1   172.31.7.155    3.137.180.185   SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-8-201    Ready      control-plane,etcd,master   111m   v1.27.8-rc1+k3s1   172.31.8.201    3.145.69.30     SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-9-159    NotReady   <none>                      110m   v1.27.8-rc1+k3s1   172.31.9.159    52.14.233.202   SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27

NAME               STATUS   ROLES                       AGE    VERSION            INTERNAL-IP     EXTERNAL-IP     OS-IMAGE                              KERNEL-VERSION              CONTAINER-RUNTIME
ip-172-31-10-238   Ready    control-plane,etcd,master   114m   v1.27.8-rc1+k3s1   172.31.10.238   13.58.97.126    SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-15-133   Ready    control-plane,etcd,master   112m   v1.27.8-rc1+k3s1   172.31.15.133   18.224.33.42    SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-3-252    Ready    <none>                      110m   v1.27.8-rc1+k3s1   172.31.3.252    3.144.101.172   SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-7-155    Ready    <none>                      111m   v1.27.8-rc1+k3s1   172.31.7.155    3.137.180.185   SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-8-201    Ready    control-plane,etcd,master   112m   v1.27.8-rc1+k3s1   172.31.8.201    3.145.69.30     SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27
ip-172-31-9-159    Ready    <none>                      111m   v1.27.8-rc1+k3s1   172.31.9.159    52.14.233.202   SUSE Linux Enterprise Server 15 SP4   5.14.21-150400.22-default   containerd://1.7.7-k3s1.27

Additional context / logs:

N/A