In the VMware ESXi 6.x partitions layout, usually, there is a partition called “scratch” that hosts the log, the updates, and other temporal files. Scratch space is configured automatically during installation or first boot of an ESXi host and is not required to be manually configured.
If you install ESXi on a local hardware disk (or also if you are using a remote LUN in “boot from SAN” mode), this partition is built during the installation phase (it’s 4 GB Fat16 partition created on the target device during installation, if there is sufficient space). If you install ESXi on a USB connected mass storage (like the SD cards), then this partition is usually skipped and the ramdisk will be used instead.
But with ESXi 6.5 there are some issues on how scratch will be managed.
The first issue is described in KB 2151270 (“ScratchConfig.ConfiguredScratchLocation” for persistent scratch location is reverted to the default value after ESXi 6.x host reboot) and it’s related both to ESXi 6.0 and 6.5 and actually it hasn’t yet a resolution.
In this case the advanced setting ScratchConfig.ConfiguredScratchLocation for persistent scratch location is reverted to the default value after ESXi 6.x host reboot. The default value of ScratchConfig.CurrentScratchLocation is /tmp/scratch.
To work around this issue, follow the steps provided in KB 2003322 (Configuring syslog on ESXi) and configure the advanced setting Syslog.global.logDir with a datastore in the format: [DatastoreName] DirectoryName.
The second issue is specific only for ESXi 6.5 and only with Dell customized ISOs. VMware KB 2151209 (ScratchConfig.CurrentScratchLocation is set to /scratch after upgrading to ESXi 6.x) describe this issue, with a tricky and not always easy solution.
If you are using the Dell EMC Customized image ESXi 6.5 A04 (but I’ve got the same issues with other Dell’s customized images, still with 6.5), after upgrading or also installing the ESXi host, you experience these symptoms:
- The ScratchConfig.CurrentScratchLocation is set to /scratch.
- You see these warnings:System logs on host [] are stored on non-persistent storage.AndNo scratch partition has been configured. Recommended scratch partition size is {} MiB.” (Syslog.global.logdir = /scratch/log).
- Changing the ScratchConfig.CurrentScratchLocation does not persist between reboots.
To resolve this issue, see VMware vSphere 6.5.x on Dell EMC PowerEdge Servers Release Notes. The root cause is a specific version of the elxiscsi Emulex driver, but the doc says only when you enable the hardware or software iSCSI. Not true, considering that I’ve got exactly the same issue with a new clean installation of a vSAN cluster without ANY iSCSI initiator or target (vSAN can export blocks with iSCSI, but was not enabled at all).
This issue prevents the redirection of logging to a persistent data store causing loss of log data across reboots, and this can be easily solved with an external syslog server. But also is going to fill the root partition (that works on a ramdisk) to 100%… and this cannot easily solve with an external syslog server!
The solution is to uninstall the elxiscsi driver and the associated shared library elx-esx-libelxima, if Emulex is not part of the configuration. To uninstall the elxiscsi driver, run the following commands:
- /etc/init.d/hostd stop
- localcli software vib remove -n elxiscsi -n elx-esx-libelxima.so
- reboot
Be sure that the host is in maintenance mode BEFORE start this procedure!
Related Posts
Andrea Mauro
Virtualization, Cloud and Storage Architect. Tech Field delegate. VMUG IT Co-Founder and board member. VMware VMTN Moderator and vExpert 2010-24. Dell TechCenter Rockstar 2014-15. Microsoft MVP 2014-16. Veeam Vanguard 2015-23. Nutanix NTC 2014-20. Several certifications including: VCDX-DCV, VCP-DCV/DT/Cloud, VCAP-DCA/DCD/CIA/CID/DTA/DTD, MCSA, MCSE, MCITP, CCA, NPP.
