K8s fix flannel after upgrading cni plugins
Saturday Aug 28, 2021 | Series KubernetesA couple days ago my kubernetes cluster broke after a system upgrade. Resulting in the cluster generating the errors below.
Error syncing pod a2a53290-07ce-4c3b-b848-469f1916f2ab ("nginx-deployment-58ff77f57f-4w7mn_default(a2a53290-07ce-4c3b-b848-469f1916f2ab)"), skipping: failed to "CreatePodSandbox" for "nginx-deployment-58ff77f57f-4w7mn_default(a2a53290-07ce-4c3b-b848-469f1916f2ab)" with CreatePodSandboxError: "CreatePodSandbox for pod "nginx-deployment-58ff77f57f-4w7mn_default(a2a53290-07ce-4c3b-b848-469f1916f2ab)" failed: rpc error: code = Unknown desc = failed to setup network for sandbox "1cf0dc5013868b30f8adfac1f7e9dca2ba444d2678e4a7efe442607e148dbfa9": failed to find plugin "flannel" in path [/usr/libexec/cni]"
AND
killPodWithSyncResult failed: failed to "KillPodSandbox" for "f33e71c4-9f4a-41c3-b930-409b4093dc5d" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox "9d6db1d0b017cdb493bb8951e9da20aed48ecf1cd607c15e3d71951b5051b8d8": failed to find plugin "flannel" in path [/usr/libexec/cni]"
The key here was that it was suddenly missing the flannel plugin. This was due to a breaking change, detailed here https://github.com/containernetworking/plugins/releases. As the changelog states, the decision was made to remove the flannel from the containernetworking-plugins
package.
For now I have chosen to downgrade the plugin sudo dnf downgrade containernetworking-plugins
. Until the repository https://github.com/flannel-io/cni-plugin is more mature and picked up by package managers.
If that takes to long I’ll have to manually build the repo and put the plugin into the cni directory at /usr/libexec/cni
manually.
Again for now I’m ok with a downgrade, I do hope I can upgrade in the future without issues.