Update to ESXi 6.7 U3 using esxcli fails with error „Failed to setup upgrade using esx-update VIB“

After the gerneral availability of ESXi 6.7 U3, I decided it is time to update my homelab. I used the integrated vSphere Update Manager (VUM) for my nested vSAN Cluster without any issues. During my VCAP-DVC Deploy preparation earlier this year I used an old Intel NUC system to deploy an additional vCenter Server Appliance (VCSA) in order to play around with Enhanced Linked Mode (ELM). I wanted to have the VCSAs in different Layer 3 networks, so I copied the design from the nested cluster and installed a nested router alongside with a nested ESXi host. The nested ESXi is used to host the second VCSA. Due to this design I can not use the integrated VUM of the second VCSA to update this standalone ESXi host.

The virtual machines (VMs) on the standalone ESXi host are not very important, so I decided to shut them down together with the VCSA and update the nested ESXi host via CLI. For details on how to update ESXi 6.x using the CLI, take a look at one of my previous blog posts here. I used this method quite some time so I was certain to have no issues with the update. But this time it was different, the update failed.

esxcli software profile update failes

Not sure on what went wrong this time, I digged into the logs. Starting with the vmkernel.log I found the first hints.

VMware-bootbank-esx-update-6.7.0-3.73.14320388 is busy

I jumped over to the esxupdate.log and looked for some matching entries. Using the timestamps I quickly found the entries I was looking for.

Failed to setup upgrade using esx-update VIB

Based on the logs, I assume that the corresponding VIB could be successfully downloaded from the VMware depot, but an error occurred during mounting. I did not have a real idea where the issue came from so I did some testing. One of the things I tried was to update all the installed vSphere Installation Bundles (VIBs) using the same online depot without referencing to a certain profile. The following command accomplishes this task.

esxcli software vib update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

To my own surprise, this worked without any problems.

Updating ESXi VIBs

At the end of the update I restarted the ESXi host as requested. After rebooting I checked the version and compared it with my nested vSAN cluster. To my delight, the versions of the standalone host and the hosts within the cluster matched.

ESXi version after the VIB update

Wrap-Up

Even though I can not say exactly where the problem was with the update via the profile, the uncoupled update of the VIBs seems to be a workaround for the problem.