Reading Time: 5 minutes

VMware vSAN 6.7U3 it’s out and maybe you would like to upgrade your environment.

But note that there are some possible performance issue with vSphere 6.7U3 on Dell servers.

In my opinion, one reason to upgrade is to finally have the automatical proactive rebalancing feature.

You can automate all rebalancing activities with cluster-wide configuration and threshold settings. Prior to this release, proactive rebalancing was manually initiated after being alerted by vSAN health checks.

Automatic rebalance

You don’t have to periodically check your environment and manually rebalance it… Just enable this feature at cluster level!

But if you have only updated vCenter and not the ESXi, this will be a minor issue: you cannot perform automatically rebalance (seems that works only in some cases), and you cannot perform manually rebalance anymore form the GUI.

Also, until you don’t upgrade all the ESXi hosts and you enable the automatic rebalance, you will have some health issues because the hosts are not configured in the same way.

Another minor possible issue is the “vSAN health alarms are suppressed” message, that has a quick fix (and seems not only related to the Update 3).

Also remember that in vSphere 6.7 Update 3, almost all of the vSphere Web Client functionality is implemented in the vSphere Client. For an up-to-date list of any remaining unsupported functionality, see Functionality Updates for the vSphere Client.

In vSAN 6.7 Update 3, Configuration Assist and Updates are available only in the Flex-based vSphere Web Client. 

Before upgrade

Always check the hardware and software compatibility and also check the vSAN upgrade guide. It could be difficoult rolling back!

Plan and design your upgrade to be fail-safe. Before you attempt to upgrade vSAN, verify that your environment meets the vSphere hardware and software requirements. 

For the software, for example, Veeam Backup 9.5 Update 4b and VMware Horizon 7.9 are already compatible.

For hardware compatibility check also the I/O card compatibility. Don’t assume that your hardware will be compatible because it was with vSphere 6.5 Update 2!

Same also with vSAN Ready node: they are not necessary compatible with the Update 3.

VMware is probably too slow in the process of hardware re-certification after each update, and you can have big surprise.

For example, on this Lenovo System the hardware compatible with 6.7U2 it’s not compatible with 6.7U3 if you are using the latest firmware.

Really boring and probably will be fixed in the next week, but you have to take care.

By the way, I’ve got some big performance issues with some Broadcom NICs that are supported in the HCL, but they really slow down with the Update 3 (also without vSAN).

So another importan note is to make some tests, before upgrade a production environment.

Upgrade steps

VMware vSAN 6.7 Update 3 is a new release that requires a full upgrade to vSphere 6.7 Update 3.

At a high level, you have to perform the following tasks to complete the upgrade:

  1. First upgrade to vCenter Server 6.7 Update 3. For more information, see the VMware vSphere 6.7 Release Notes
  2. Then upgrade hosts to ESXi 6.7 Update 3. For more information, see the VMware vSphere 6.7 Release Notes
  3. Then upgrade the vSAN on-disk format to version 10.0. If upgrading from on-disk format version 5.0 or later, no data evacuation is required (metadata update only).
  4. Keep the cluster updated. VMware vSAN generates system baselines and baseline groups for use with vSphere Update Manager. You can use these recommended baselines to update software, patches, and extensions for hosts in your vSAN cluster.

On-disk Format 10

When all the ESXi nodes are updated you may upgrade also the disk format (it’s not mandatory, but highly recommened).

During an upgrade of the vSAN on-disk format, a disk group evacuation is performed. The disk group is removed and upgraded to on-disk format version 10.0, and the disk group is added back to the cluster.

For two-node or three-node clusters, or clusters without enough capacity to evacuate each disk group, select Allow Reduced Redundancy from the vSphere Client. You also can use the following RVC command to upgrade the on-disk format: vsan.ondisk_upgrade –allow-reduced-redundancy

When you allow reduced redundancy, your VMs are unprotected for the duration of the upgrade, because this method does not evacuate data to the other hosts in the cluster. It removes each disk group, upgrades the on-disk format, and adds the disk group back to the cluster. All objects remain available, but with reduced redundancy.

If you enable deduplication and compression during the upgrade to vSAN 6.7, you can select Allow Reduced Redundancy from the vSphere Client.

Verifying Health Check

During upgrades of the vSAN on-disk format, the Physical Disk Health – Metadata Health check can fail intermittently.

These failures can occur if the destaging process is slow, most likely because vSAN must allocate physical blocks on the storage devices.

Before you take action, verify the status of this health check after the period of high activity, such as multiple virtual machine deployments, is complete. If the health check is still red, the warning is valid. If the health check is green, you can ignore the previous warning. For more information, see Knowledge Base article 2108690.