As you probably know VMware vSphere 6.0 had a critical issue con its Change Block Tracking (CBT) implementation that can impact all incremental backup with “VMware native” backup program (all agent-less implementation using the VMware VDAP API).
This issue occurs due to an issue with CBT in the disklib area, this causes the change tracking information of I/Os that occur during snapshot consolidation to be lost. The main backup payload data is never lost and it is always written to the backend device. However, the corresponding change tracking information entries which occur during the consolidation task are missed. Subsequent calls to CBT APIs do not include these missed blocks, hence a backup based on this CBT data is inconsistent.
VMware has released on Nov 25 a patch (patch ESXi600-201511401-BG) that should fix this issue as described on KB 2137546.
But formally it’s not enough. Veeam team has analyse the problem and Anton Gostev explain well the complete resolution.
Installaing the patch to bring ESXi 6.0 to Update 1a build 3073146 is only a part of the solution, but if you have already done some backup you need also a complete CBT reset for all of your VMs.
This is because existing CBT maps may already contain issues, which may result in inconsistent full backups in future. Having CBT reset will also make the following job run perform “full scan” incremental, thus fixing any existing inconsistencies in backups and replicas.
When CBT reset has been performed, for Veeam the Active Full backup is not required. However, performing Active Full backups by itself cannot be considered as a substitute to CBT reset with this particular issue.
Reset the CBT file is a quite long procedure with the official way:
1. Power off the VM.
2. Right-click the VM, click [Edit settings], click the Options tab, select the General section, then click Configuration Parameters
3. Set the “ctkEnabled” value to false
4. Set the all of the “scsi0:x.ctkEnabled” values to false
5. Open the source folder via Datastore Browser and delete any -CTK.VMDK files
6. Power the VM on
7. Rerun Backup or Replication job to re-enable CBT
Veeam KB1113 (How to reset VMware Virtual Machine CBT) provide also another method with a PowerCLI based script.