Lab notes: Upgrading vSphere 6.0 and NSX 6.2 to vSphere 6.5a and NSX 6.3
Sam McGeown
I’ve been holding off upgrading my lab to vSphere 6.5 because NSX 6.2.x doesn’t support it. With the release of NSX 6.3 and vSphere 6.5a, I can now upgrade.
The sequence of the upgrade is slightly different to the generic one published by VMware because vSphere 6.5 isn’t supported with NSX 6.2. If follows that I need to upgrade NSX to 6.3 (which supports vSphere 6.0u2) before I can upgrade vSphere to 6.5a.
- Upgrade NSX Manager
- Upgrade NSX Controllers
- Upgrade NSX Host Clusters
- Update VXLAN port
- Upgrade NSX Edges
- Run Migration Assistant on VUM server
- Upgrade Platform Services Controller
- Upgrade vCenter Server Appliance
- Upgrade ESXi Hosts
- Upgrade VM Hardware and Tools
Upgrade NSX Manager
Upgrading NSX manager is simply a case of logging onto the admin web page, ensuring you have a recent backup, and then uploading the upgrade bundle. I had issues with my uploads taking a very long time, and failing to “take”. The solution was to move the NSX manager to an SSD (my NFS shares were taking a bit of a hammering at the time).
Upgrade NSX Controllers
Again, a simple process, clicking the upgrade link in the Networking and Security console. This was hampered slightly by the web client being cached by Chrome, which meant that the controllers didn’t show in the older NSX plugin - I switched to Firefox and ran the upgrade fine.
Upgrade NSX Host Clusters
My lab seems to be dogged by host preparation errors with NSX - I’ve never really got to the bottom of it (I think it’s to do with VSAN maintenance mode and only having 3 hosts), but in my experience it’s pretty easy in most cases. A host reboot is required, so make sure you’re not impacting running workloads.
I found that I needed to manually put the hosts in maintenance mode, remove the old VIBs (esxcli software vib remove -n esx-vsip -n esx-vxlan) and then re-run the host prep.
Update VXLAN port
In NSX 6.2.3 and later, the default VXLAN port is 4789, the standard port assigned by IANA. Before NSX 6.2.3, the default VXLAN UDP port number was 8472.
If you're upgrading from an older version it's probably good practice to change your port to the IANA assigned one.
Upgrade NSX Edges
Thankfully, my NSX Edges upgraded easily, first time, without any issue!
Run Migration Assistant on VUM server
My vCenter setup is based on an external Platform Services Controller (PSC), a vCenter Server Appliance (VCSA), and vSphere Update Manager (VUM) installed on a Windows server. With 6.5 the VUM service moves onto the VCSA, so you need to use the Migration Assistant to help with the migration.
This is just a case of mounting the vCenter Server Appliance ISO image on the VUM server and running the vmware-migration-assistant.exe. Once the assistant has run and done it’s thing, you leave it running while you upgrade the rest of the platform.
Upgrade Platform Services Controller
Upgrading the PSC was very straightforward - I use the GUI installer and ran the update pointed at my PSC. The update first deploys a new PSC on a temporary network address, before copying the configuration over from the old PSC and shutting it down. The new PSC becomes live with the old PSCs IP address.
Upgrade vCenter Server Appliance
Upgrading my VCSA was slightly trickier, but only just - because my VCSA was managing the hosts I was deploying to, I needed to disable DRS on the cluster and point the destination at a specific host instead of a cluster on the VCSA. It makes sense, but it won’t complain until you validate the config. Other than that, it’s the same two-step process as the PSC - deploy a new one and migrate the config.
Upgrade ESXi Hosts
I had originally planned to upgrade the hosts using VUM, but decided against it because the hosts I have in my lab have all required some little tweaks to get them working, for various reasons - mostly that it’s old hardware that isn’t supported! Instead I went with the esxcli option to upgrade the hosts so that I could see exactly what was happening as I did it.
esxcli software profile update -p ESXi-6.5.0-4564106-standard -d http://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
Once that had run, a quick reboot and the host was at 6.5. Because of the fun and games I had installing NSX VIBs on my hosts, I had to re-prepare the hosts once again at this point.
Upgrade VM Hardware and Tools
The final step was to upgrade the relevant VM Hardware and Tools levels. I typically set my VMs to upgrade Tools on a reboot, so that pretty much takes care of itself over time. The VM hardware I do manually, simply because I want to make sure that I can control what’s upgraded and ensure compatibility.