This post is also available in: Italian

As you probably know VMware vSphere 6.0 had a critical issue con its Change Block Tracking (CBT) implementation that can impact all incremental backup with “VMware native” backup program (all agent-less implementation using the VMware VDAP API).

This issue occurs due to an issue with CBT in the disklib area, this causes the change tracking information of I/Os that occur during snapshot consolidation to be lost. The main backup payload data is never lost and it is always written to the backend device. However, the corresponding change tracking information entries which occur during the consolidation task are missed. Subsequent calls to CBT APIs do not include these missed blocks, hence a backup based on this CBT data is inconsistent.

VMware has released on Nov 25 a patch (patch ESXi600-201511401-BG) that should fix this issue  as described on KB 2137546.

But formally it’s not enough. Veeam team has analyse the problem and Anton Gostev explain well the complete resolution.

Installaing the patch to bring ESXi 6.0 to Update 1a build 3073146 is only a part of the solution, but if you have already done some backup you need also a complete CBT reset for all of your VMs.

This is because existing CBT maps may already contain issues, which may result in inconsistent full backups in future. Having CBT reset will also make the following job run perform “full scan” incremental, thus fixing any existing inconsistencies in backups and replicas.

When CBT reset has been performed, for Veeam the Active Full backup is not required. However, performing Active Full backups by itself cannot be considered as a substitute to CBT reset with this particular issue.

Reset the CBT file is a quite long procedure with the official way:

1. Power off the VM.
2. Right-click the VM, click [Edit settings], click the Options tab, select the General section, then click Configuration Parameters

User-added image

3. Set the “ctkEnabled” value to false

User-added image

4.  Set the all of the “scsi0:x.ctkEnabled” values to false

User-added image

5.  Open the source folder via Datastore Browser and delete any -CTK.VMDK files

User-added image

6.  Power the VM on
7.  Rerun Backup or Replication job to re-enable CBT

Veeam KB1113 (How to reset VMware Virtual Machine CBT) provide also another method with a PowerCLI based script.

Andrea MauroAbout Andrea Mauro (2489 Posts)

Virtualization & Cloud Architect. VMUG IT Co-Founder and board member. VMware VMTN Moderator and vExpert (2010, 2011, 2012, 2013, 2014, 2015). PernixPro 2014. Dell TechCenter Rockstar 2014. MVP 2014. Several certifications including: VCDX-DCV, VCP-DCV/DT/Cloud, VCAP-DCA/DCD/CIA/CID/DTA/DTD, MCSA, MCSE, MCITP, CCA, NPP.


Share