Before you start, understand this part. Rancher is a deployment running on Kubernetes and they are upgraded separately. Since Rancher is depending on Kubernetes you need to first figure out if the version of Rancher you want to upgrade to can run on the current Kubernetes version.
There are also multiple ways of initially setting up a cluster. This article will cover RKE. If you have used RKE2 or K3s you can stop reading now.
Login to the node you used to create the local cluster. It will have the cluster.yaml file in the rancher user’s home directory. In this directory you’ll also have the Kubconfig file for the local cluster. Start by setting the KUBECONFIG environment variable and check out the Kubernetes version of the nodes and my installed Docker version.
# The file name of the Kubeconfig could differ export KUBECONFIG=~/kube_config_rancher-cluster.yml kubectl get nodes # NAME STATUS ROLES AGE VERSION # 192.168.50.232 Ready controlplane,etcd,worker 480d v1.21.13 # 192.168.50.233 Ready controlplane,etcd,worker 480d v1.21.13 # 192.168.50.234 Ready controlplane,etcd,worker 480d v1.21.13 docker --version # Docker version 20.10.17, build 100c701
The Kubernetes version is v1.21.13. Save this for later in case you need to roll back! Now it’s time to check out if the Rancher version you want to upgrade to can support this version. You can get this information from the Rancher Support Matrix.
Now, their table was confusing to me too and it took embarrassingly long to decipher it.
- I want to upgrade to Rancher v2.6.5.
- I am running my local cluster on Ubuntu 20.04
- My Kubernetes version is v1.21.13 (as determined above)
The table then recommends RKE version v1.3.8:
Scrolling down to the Ubuntu section we can find that 20.04 running Kubernetes v1.21.x and Docker version 20.10.x is supported.
Important, do not upgrade Kubernetes to a version that your current Rancher version does not support! If that is the case you need to upgrade Rancher first!
Before going ahead with this, make sure that you have done and read the previous section, especially the last paragraph.
I don’t really need to do this, but for the sake of this article I’ll cover how I did it anyway. First, download the recommended version of RKE from Github to the same folder as your cluster.yaml. I can highly recommend to name it after the version to keep them apart just in case.
curl https://github.com/rancher/rke/releases/download/v1.3.8/rke_linux-amd64 -o rke-v1.3.8 chmod +x ./rke-v1.3.8
Once you have the file you can check out the supported Kubernetes versions by running this command:
./rke-v1.3.8 config --list-version --all # v1.23.4-rancher1-1 # v1.18.20-rancher1-3 # v1.20.15-rancher1-2 # v1.19.16-rancher1-4 # v1.21.10-rancher1-1 # v1.22.7-rancher1-1
Pick one that is supported by your intended Rancher version and maybe not the latest unless you have a practical reason to do so. Then edit your cluster.yml file and add a key called kubernetes_version with a value of the Kubernetes version you picked above. In my case the cluster.yaml file looks like this:
kubernetes_version: v1.21.10-rancher1-1 nodes: - address: 192.168.50.232 user: rancher role: [controlplane, worker, etcd] ssh_key_path: ~/.ssh/id_rsa - address: 192.168.50.233 user: rancher role: [controlplane, worker, etcd] ssh_key_path: ~/.ssh/id_rsa - address: 192.168.50.234 user: rancher ssh_key_path: ~/.ssh/id_rsa role: [controlplane, worker, etcd] services: etcd: snapshot: true creation: 6h retention: 24h # Required for external TLS termination with # ingress-nginx v0.22+ ingress: provider: nginx options: use-forwarded-headers: "true"
Make a backup of your cluster state using the following command:
./rke-v1.3.8 etcd snapshot-save --config rancher-cluster.yaml --name $(date '+%Y%m%d%H%m')
Then you can upgrade your Kubernetes version using the following command:
./rke up -config cluster.yml
You can now follow the progress with:
kubectl get nodes --watch NAME STATUS ROLES AGE VERSION 192.168.50.232 Ready controlplane,etcd,worker 480d v1.21.13 192.168.50.233 Ready controlplane,etcd,worker 480d v1.21.13 192.168.50.234 Ready controlplane,etcd,worker 480d v1.21.13
Once all of the nodes has the intended version it’s time to update Rancher.
This is pretty easy. Start with updating helm and look at which values that was used to provision the cluster to begin with:
helm repo update helm get values rancher -n cattle-system # Location: /home/rancher/kube_config_rancher-cluster.yml # USER-SUPPLIED VALUES: # hostname: rancher.xip.se # Ingress: # tls: # source: secret
Then use the values to generate the upgrade command while supplying the intended Rancher version:
helm upgrade rancher rancher-latest/rancher --version=2.6.6 --set hostname=rancher.xip.se --set ingress.tls.source=secret --namespace cattle-system
Once this is done you should have an upgraded cluster!