[release-1.31] Backport RBAC changes for v1.32 compat

[brandond]

Backport changes made for compat with AuthorizeNodeWithSelectors feature-gate in v1.32.

From the release notes:

Kubernetes 1.32 moves the AuthorizeNodeWithSelectors feature gate to Beta and on by default. See KEP-4601 for more information.

This feature-gate breaks some of the RBAC that previous releases of K3s relied upon. The January releases of K3s v1.29, v1.30, and v1.31 will contain backported fixes. Until then, you must set --kube-apiserver-arg=feature-gates=AuthorizeNodeWithSelectors=false on server nodes, if you want to mix K3s v1.32 nodes with nodes of other versions (within the limits of what is supported by the Kubernetes Version Skew Policy).

• V1.32.0+k3s1 #11478 - b474770

[VestigeJ]

This is predominantly an aspect only for the 1.32 minor where this feature gate is present and modified. Testing involves backwards compatibility with prior agent versions.

$ kubectl get --raw /metrics | grep -i kubernetes_feature_enabled{name="AuthorizeNodeWithSelectors",

DEBU[0000] Asset dir /var/lib/rancher/k3s/data/d24c784b7efbd0fd681bc8b3de245cd1bc2ca768ea0af38826b9875932cadbba
DEBU[0000] Running /var/lib/rancher/k3s/data/d24c784b7efbd0fd681bc8b3de245cd1bc2ca768ea0af38826b9875932cadbba/bin/kubectl [kubectl get --raw /metrics]
kubernetes_feature_enabled{name="AuthorizeNodeWithSelectors",stage="BETA"} 1

$ k3s -v

k3s version v1.32.1-rc1+k3s1 (57f4a479)
go version go1.23.4

kg no,po,svc -A
DEBU[0000] Asset dir /var/lib/rancher/k3s/data/d24c784b7efbd0fd681bc8b3de245cd1bc2ca768ea0af38826b9875932cadbba
DEBU[0000] Running /var/lib/rancher/k3s/data/d24c784b7efbd0fd681bc8b3de245cd1bc2ca768ea0af38826b9875932cadbba/bin/kubectl [kubectl get no,po,svc -A]
NAME STATUS ROLES AGE VERSION
node/ip-1-1-1-226 Ready 2m17s v1.32.0+k3s1
node/ip-1-1-1-13 Ready control-plane,etcd,master 2m38s v1.32.1-rc1+k3s1

NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-ff8999cc5-4sfpc 1/1 Running 0 2m31s
kube-system pod/helm-install-traefik-crd-zxw86 0/1 Completed 0 2m31s
kube-system pod/helm-install-traefik-nddsb 0/1 Completed 2 2m31s
kube-system pod/local-path-provisioner-698b58967b-8dlsj 1/1 Running 0 2m31s
kube-system pod/metrics-server-8584b5786c-zpsv2 1/1 Running 0 2m31s
kube-system pod/svclb-traefik-02a91249-8nst8 2/2 Running 0 118s
kube-system pod/svclb-traefik-02a91249-fnk7h 2/2 Running 0 118s
kube-system pod/traefik-5c57d75bc9-xbzfx 1/1 Running 0 118s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.43.0.1 443/TCP 2m38s
kube-system service/kube-dns ClusterIP 10.43.0.10 53/UDP,53/TCP,9153/TCP 2m35s
kube-system service/metrics-server ClusterIP 10.43.139.140 443/TCP 2m35s
kube-system service/traefik LoadBalancer 10.43.132.107 1.1.1.226,1.1.1.13 80:31307/TCP,443:31756/TCP 118s

NAME STATUS ROLES AGE VERSION
node/ip-1-1-1-2 Ready 3m42s v1.31.4+k3s1
node/ip-1-1-1-13 Ready control-plane,etcd,master 4m3s v1.32.1-rc1+k3s1

NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-ff8999cc5-4sfpc 1/1 Running 0 3m56s
kube-system pod/helm-install-traefik-crd-zxw86 0/1 Completed 0 3m56s
kube-system pod/helm-install-traefik-nddsb 0/1 Completed 2 3m56s
kube-system pod/local-path-provisioner-698b58967b-8dlsj 1/1 Running 0 3m56s
kube-system pod/metrics-server-8584b5786c-zpsv2 1/1 Running 0 3m56s
kube-system pod/svclb-traefik-02a91249-8nst8 2/2 Running 2 3m23s
kube-system pod/svclb-traefik-02a91249-fnk7h 2/2 Running 0 3m23s
kube-system pod/traefik-5c57d75bc9-xbzfx 1/1 Running 1 3m23s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.43.0.1 443/TCP 4m3s
kube-system service/kube-dns ClusterIP 10.43.0.10 53/UDP,53/TCP,9153/TCP 4m
kube-system service/metrics-server ClusterIP 10.43.139.140 443/TCP 4m
kube-system service/traefik LoadBalancer 10.43.132.107 1.1.1.2,1.1.1.13 80:31307/TCP,443:31756/TCP 3m23s

NAME STATUS ROLES AGE VERSION
node/ip-1-1-1-19 Ready 5m45s v1.30.8+k3s1
node/ip-1-1-1-13 Ready control-plane,etcd,master 6m6s v1.32.1-rc1+k3s1

NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-ff8999cc5-4sfpc 1/1 Running 0 5m59s
kube-system pod/helm-install-traefik-crd-zxw86 0/1 Completed 0 5m59s
kube-system pod/helm-install-traefik-nddsb 0/1 Completed 2 5m59s
kube-system pod/local-path-provisioner-698b58967b-8dlsj 1/1 Running 0 5m59s
kube-system pod/metrics-server-8584b5786c-zpsv2 1/1 Running 0 5m59s
kube-system pod/svclb-traefik-02a91249-8nst8 2/2 Running 4 5m26s
kube-system pod/svclb-traefik-02a91249-fnk7h 2/2 Running 0 5m26s
kube-system pod/traefik-5c57d75bc9-xbzfx 1/1 Running 2 5m26s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.43.0.1 443/TCP 6m6s
kube-system service/kube-dns ClusterIP 10.43.0.10 53/UDP,53/TCP,9153/TCP 6m3s
kube-system service/metrics-server ClusterIP 10.43.139.140 443/TCP 6m3s
kube-system service/traefik LoadBalancer 10.43.132.107 1.1.1.19,1.1.1.13 80:31307/TCP,443:31756/TCP 5m26s

NAME                    STATUS   ROLES                       AGE     VERSION
node/ip-1-1-1-73        Ready    <none>                      7m57s   v1.29.12+k3s1
node/ip-1-1-1-13        Ready    control-plane,etcd,master   8m18s   v1.32.1-rc1+k3s1

NAMESPACE     NAME                                          READY   STATUS      RESTARTS   AGE
kube-system   pod/coredns-ff8999cc5-4sfpc                   1/1     Running     0          8m11s
kube-system   pod/helm-install-traefik-crd-zxw86            0/1     Completed   0          8m11s
kube-system   pod/helm-install-traefik-nddsb                0/1     Completed   2          8m11s
kube-system   pod/local-path-provisioner-698b58967b-8dlsj   1/1     Running     0          8m11s
kube-system   pod/metrics-server-8584b5786c-zpsv2           1/1     Running     0          8m11s
kube-system   pod/svclb-traefik-02a91249-8nst8              2/2     Running     6          7m38s
kube-system   pod/svclb-traefik-02a91249-fnk7h              2/2     Running     0          7m38s
kube-system   pod/traefik-5c57d75bc9-xbzfx                  1/1     Running     3          7m38s

NAMESPACE     NAME                     TYPE           CLUSTER-IP      EXTERNAL-IP                  PORT(S)                      AGE
default       service/kubernetes       ClusterIP      10.43.0.1       <none>                       443/TCP                      8m18s
kube-system   service/kube-dns         ClusterIP      10.43.0.10      <none>                       53/UDP,53/TCP,9153/TCP       8m15s
kube-system   service/metrics-server   ClusterIP      10.43.139.140   <none>                       443/TCP                      8m15s
kube-system   service/traefik          LoadBalancer   10.43.132.107   1.1.1.73,1.1.1.13   80:31307/TCP,443:31756/TCP   7m38s