Reading Time: 3 minutes

VMware vSAN has become a reliable solution with a good and usable HTML5 GUI.

But sometimes you have have some issues in object health with the result of some inaccessible vSAN objects.

vSAN inaccessible object error

Usually you have a good integrity of your vSAN objects, but in some rare cases you may have this type of issues.

If you check the details of the oject you will probably see something like this:

Unknown object type

What is an “Unknown object type”? Usually are corrupted object, that are no more reference by a configuration file or a directory index.

It can happen, for example during a disk format upgrade.

It happen with old swap object format, but in this case it’s easy because there is a specific link in the new GUI (Purge Inaccessible VM Swap Object).

But there isn’t nothing for other type of inaccessible objects.

You need the command line, and you have to start from your vCenter and the RVC console.

First you have to login:

root@vcenter [ ~ ]# rvc administrator@SSO-DOMAIN@localhost
Install the "ffi" gem for better tab completion.
password:

Then you need to navigate into your vSAN cluster by selecting the datacenter:

ls
0 /
1 localhost/
cd 1
/localhost> ls
0 DC (datacenter)
/localhost> cd 0
/localhost/DC> ls
0 storage/
1 computers [host]/
2 networks [network]/
3 datastores [datastore]/
4 vms [vm]/
/localhost/DC> cd 1
/localhost/DC/computers> ls
0 Stretched-Cluster (cluster): cpu 203 GHz, memory 1522 GB
/localhost/DC/computers> cd 0

At this point you can use the vsan.check_state command to verify if there are some stale objects:

/localhost/DC/computers/Stretched-Cluster> vsan.check_state -r .
 2019-11-20 07:48:35 +0000: Step 1: Check for inaccessible vSAN objects
 Detected f92ea65d-33c3-357d-f88e-1866daf3f196 to be inaccessible, refreshing state
2019-11-20 07:48:40 +0000: Step 1b: Check for inaccessible vSAN objects, again
 Detected f92ea65d-33c3-357d-f88e-1866daf3f196 is still inaccessible2019-11-20 07:48:40 +0000: Step 2: Check for invalid/inaccessible VMs2019-11-20 07:48:40 +0000: Step 2b: Check for invalid/inaccessible VMs again

Now you can check the details of the inaccessible object (in this example the object f92ea65d-33c3-357d-f88e-1866daf3f196):

/localhost/DC/computers/Stretched-Cluster> vsan.cmmds_find -u f92ea65d-33c3-357d-f88e-1866daf3f196 

You will get details on where is located this object (on which ESXi host).

Now you need to log into the ESXi (using, for example, SSH) and you can delete the object with this command:

/usr/lib/vmware/osfs/bin/objtool delete -u f92ea65d-33c3-357d-f88e-1866daf3f196 -f -v 10

On latest version of vSAN seems that you can delete a “corrupted” object from one ESXi node, not necessary the owner of the resource.

That’s much more faster because it does not require to use the RVC console to locate the owner of each object.

Another option, but much more longer it’s to remove and reformat again the physical disk group where the objects are stored. The issue with this approach (apart the required time and space) is that the object may be re-created on other disks.

Share

Virtualization, Cloud and Storage Architect. Tech Field delegate. VMUG IT Co-Founder and board member. VMware VMTN Moderator and vExpert 2010-24. Dell TechCenter Rockstar 2014-15. Microsoft MVP 2014-16. Veeam Vanguard 2015-23. Nutanix NTC 2014-20. Several certifications including: VCDX-DCV, VCP-DCV/DT/Cloud, VCAP-DCA/DCD/CIA/CID/DTA/DTD, MCSA, MCSE, MCITP, CCA, NPP.