As every year, some of my customers use the weeks after Christmas to update their environments. Nearly all of them run their ESXi hosts with vendor-specific Custom Images that provide additional drivers or agents over the standard VMware image. Unfortunately there are almost always problems with conflicting VIBs when they are updated. Of course it was the same this time. In my case, this time it was about a custom image from Fujitsu.
To perform the update successfully the problematic VIB must be removed. The necessary steps for this I would like to point out below.
After the gerneral availability of ESXi 6.7 U3, I decided it is time to update my homelab. I used the integrated vSphere Update Manager (VUM) for my nested vSAN Cluster without any issues. During my VCAP-DVC Deploy preparation earlier this year I used an old Intel NUC system to deploy an additional vCenter Server Appliance (VCSA) in order to play around with Enhanced Linked Mode (ELM). I wanted to have the VCSAs in different Layer 3 networks, so I copied the design from the nested cluster and installed a nested router alongside with a nested ESXi host. The nested ESXi is used to host the second VCSA. Due to this design I can not use the integrated VUM of the second VCSA to update this standalone ESXi host.
The virtual machines (VMs) on the standalone ESXi host are not very important, so I decided to shut them down together with the VCSA and update the nested ESXi host via CLI. For details on how to update ESXi 6.x using the CLI, take a look at one of my previous blog posts here. I used this method quite some time so I was certain to have no issues with the update. But this time it was different, the update failed.
Recently I was asked to update some ESXi 6.7 U1 hosts to the latest patch release. A quick look at the list of build and version numbers showed me that there is an actual express patch (EP6) which could be installed. Before performing this update in production I decided to update my test ESXi host first. If the update completes successfully and there are no issues after a few days I will continue to update the production hosts to EP6.
The following list outlines the steps necessary to perform the update using CLI (this is just a test ESXi host so there is no vSphere Update Manager (VUM) available). This procedure should be the same for other versions of ESXi 6.x even if a restart of the ESXi host may be necessary at the end.
Step 1: Make sure SSH is enabled on the host
Because I perform the update using CLI I need to login to the host via SSH. Therefore I need to check if SSH is enabled or not. There are at least three ways to check:
Try and error: Use your favorite SSH client and simply try to connect to the management IP of your ESXi host.
VMware Host Client: Use your webbrowser of choice and connect to the management IP of your ESXi host. Navigate to Host > Manage > Services. Check if the TSM-SSH service is in Running state, else start up the service using the corresponding button.
Direct Console User Interface (DCUI): Use the IPMI of your server vendor, a KVM system or direct connected peripherie to access the DCUI. Navigate to Troubleshooting Mode Options > Enable/Disable SSH and check if SSH is enabled, else enable it.
Step 2: Enable firewall rule for web traffic
Connect to the ESXi host via SSH and enable the firewall rule to allow web traffic. This is necessary to use the VMware Online Depot in the next steps.
esxcli network firewall ruleset set -e true -r httpClient
Optional Step 3: Check the current ESXi build and version
To confirm the current version of your ESXi host use the following command.
esxcli system version get
Step 4: List all available profiles in the VMware Online Depot
To get al list of all available profiles in the VMware Online Depot use the following command.
esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
This list can be very extensive so it is recommended to work with filters. Appending a | grep -i ESXi-6.7.*standard lists only the profiles for Version 6.7 including VMware Tools.
If the update finishes with Reboot Required: true use the following command to reboot the host. Make sure there are no running VMs and optional the host is in maintenance mode.
Optional Step 7: Check ESXi version again to ensure the update completed successfully
I finished the update with a final check of the running ESXi build and version using esxcli again.
esxcli system version get
With just a few commands it is possible to update an existing host to the current available EP6 using only the VMware Online Depot. Just make sure that host is configured with correct DNS settings and is able to access the internet. If this is not possible, a corresponding offline bundle can still be used for the update.