Although ESXi does allow for limited hardware reporting from via IPMI, you may find yourself in a situation where you need to interact with your BMC. A tool that I’ve been using for some time for this purpose is ipmitool, which can be obtained for just about any Linux distribution. The ipmitool utility can be used to pull fan and sensor data, change the BMC network configuration and can even be executed against an IPMI compliant BMC remotely over the network.
Normally, you can just use the web-based IPMI user interface to do a lot of what ipmitool can do, but there are situations where this won’t cut it. A couple of examples would be locking yourself out of the BMC by setting an incorrect VLAN tag or having to modify a setting that isn’t exposed in the UI. My motivation for getting ipmitool running within the ESXi shell was the latter. I needed to be able to modify fan thresholds to keep my slow-spinning fans from triggering critical alarms on my hosts. These fan thresholds aren’t exposed in the web UI and I have to modify them using ipmitool. Normally, to do this I’d have to shut down the host, and boot it up using an install of Debian on a USB stick – a bit of a pain. Why not just run ipmitool from directly within ESXi instead?
A Bit of Background
I first came across this post, which pointed me in the right direction. I couldn’t find a copy of the correct binary, so I just went ahead and compiled a build of ipmitool from source to get a 32-bit Linux executable. Unfortunately, the newest version of ipmitool that ESXi seems to satisfy the library dependencies for is a relatively old 1.8.11 from almost ten years ago. I tried compiling versions 1.8.18 and 1.8.12, but neither would run in ESXi 6.7 U2. Thankfully, 1.8.11 still seems to work fine for most systems and provides the basic functionality required.
Once I had a functional executable, I was able to package it using the really handy ESXi Community Packaging Tools at v-front.de.
File Download
You can download the files here:
ipmitool-esxi-vib-1.8.11-2.zip
Within the zip file, you’ll find the following:
ipmitool-1.8.11-2-offline_bundle.zip - offline bundle for Update Manager or ISO customization. ipmitool-1.8.11-2.x86_64.vib - standalone VIB for CLI installation readme.txt - installation instructions and examples
MD5 checksums:
ipmitool-esxi-vib-1.8.11-2.zip = 444af4ce1dc68418583dba2926093980 ipmitool-1.8.11-2-offline_bundle.zip = 44cb77cc8bd7f969b8e74eb33aec8d71 ipmitool-1.8.11-2.x86_64.vib = 173db5b9f205d9c10d9458ff76d8954b
Installing the VIB
To make installation and removal easier, I packaged up the ipmitool executable as a VIB as well as an offline bundle. To install the VIB, use the following steps:
- If you haven’t already done so, you’ll need to set your software acceptance level to ‘CommunitySupported’. This can be accomplished by running the following command:
[root@esx-e1:~] esxcli software acceptance set --level=CommunitySupported Host acceptance level changed to 'CommunitySupported'.
- Next, copy ipmitool-1.8.11-2.x86_64.vib to the ESXi host. To do this you can SCP file file over using WinSCP or the scp command in Linux. The /tmp location is recommended for this purpose. Don’t forget to enable SSH on the ESXi host or you won’t be able to scp the file over. Alternatively, you can copy the file to a shared datastore that the host has access to using the datastore browser.
- Install ipmitool using the esxcli software vib install -v /tmp/ipmitool-1.8.11-2.x86_64.vib –no-sig-check command:
[root@esx-e1:~] esxcli software vib install -v /tmp/ipmitool-1.8.11-2.x86_64.vib Installation Result Message: Operation finished successfully. Reboot Required: false VIBs Installed: ipmitool_bootbank_ipmitool_1.8.11-2 VIBs Removed: VIBs Skipped:
Note: If the install fails, try to add the –no-sig-check option to bypass the file signature check. It’s not necessary to reboot the host after installation.
- If ipmitool installed successfully, you can run the ipmitool command from the /opt/ipmitool/ directory. If you get the help output returned, it was successful:
[root@esx-e1:~] /opt/ipmitool/ipmitool No command provided! Commands: raw Send a RAW IPMI request and print response i2c Send an I2C Master Write-Read command and print response spd Print SPD info from remote I2C device lan Configure LAN Channels chassis Get chassis status and set power state power Shortcut to chassis power commands event Send pre-defined events to MC mc Management Controller status and global enables sdr Print Sensor Data Repository entries and readings sensor Print detailed sensor information fru Print built-in FRU and scan SDR for FRU locators <snip>
- You can validate the VIB was installed by running the following command:
[root@esx-e1:~] esxcli software vib get -n ipmitool ipmitool_bootbank_ipmitool_1.8.11-2 Name: ipmitool Version: 1.8.11-2 Type: bootbank Vendor: ipmitool Acceptance Level: CommunitySupported Summary: ipmitool 1.8.11 CLI utility Description: Used for managing IPMI baseboard controllers on the host. ReferenceURLs: kb|https://vswitchzero.com/ipmitool-vib Creation Date: 2019-08-20 Depends: Conflicts: Replaces: Provides: Maintenance Mode Required: False Hardware Platforms Required: Live Install Allowed: True Live Remove Allowed: True Stateless Ready: True Overlay: False Tags: ipmi, tool, utility Payloads: ipmitool
The executable will be installed to the /opt/ipmitool/ location on the ESXi host:
[root@esx-e1:~] ls -lha /opt/ipmitool/ipmitool -r-xr-xr-x 1 root root 787.6K Aug 20 16:00 /opt/ipmitool/ipmitool
Also included in the download is an offline bundle .zip file. This file can be used for including the VIB in custom ISOs or for uploading multiple hosts via Update Manager. It is not necessary for installing on an individual host.
Removing ipmitool
Removing ipmitool is simple. Use the esxcli software vib remove command as listed below:
[root@esx-e1:~] esxcli software vib remove -n ipmitool Removal Result Message: Operation finished successfully. Reboot Required: false VIBs Installed: VIBs Removed: ipmitool_bootbank_ipmitool_1.8.11-1 VIBs Skipped:
A host reboot is not required. You can validate that it was removed successfully if the executable is gone:
[root@esx-e1:/tmp] ls -lha /opt/ipmitool/ipmitool ls: /opt/ipmitool/ipmitool: No such file or directory
Using ipmitool
I won’t get too much into using ipmitool as it’s well documented, but a few examples you include the following:
Get an output of all sensor readings:
[root@esx-e1:~] /opt/ipmitool/ipmitool sensor CPU Temp | 34.000 | degrees C | ok | 0.000 | 0.000 | 0.000 | 84.000 | 87.000 | 89.000 System Temp | 36.000 | degrees C | ok | -9.000 | -7.000 | -5.000 | 80.000 | 85.000 | 90.000 Peripheral Temp | 38.000 | degrees C | ok | -9.000 | -7.000 | -5.000 | 80.000 | 85.000 | 90.000 PCH Temp | 52.000 | degrees C | ok | -11.000 | -8.000 | -5.000 | 90.000 | 95.000 | 100.000 P1-DIMMA1 TEMP | 44.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMA2 TEMP | 44.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMB1 TEMP | 43.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMB2 TEMP | 41.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMC1 TEMP | 42.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMC2 TEMP | 41.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMD1 TEMP | 44.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 P1-DIMMD2 TEMP | 42.000 | degrees C | ok | 1.000 | 2.000 | 4.000 | 80.000 | 85.000 | 90.000 FAN 1 | 300.000 | RPM | ok | 75.000 | 75.000 | 75.000 | 18975.000 | 19050.000 | 19125.000 FAN 2 | 300.000 | RPM | ok | 75.000 | 75.000 | 75.000 | 18975.000 | 19050.000 | 19125.000 FAN 3 | na | RPM | na | na | na | na | na | na | na FAN 4 | 1050.000 | RPM | ok | 75.000 | 75.000 | 75.000 | 18975.000 | 19050.000 | 19125.000 FAN A | 300.000 | RPM | ok | 75.000 | 75.000 | 75.000 | 18975.000 | 19050.000 | 19125.000 Vcore | 0.880 | Volts | ok | 0.480 | 0.512 | 0.544 | 1.488 | 1.520 | 1.552 3.3VCC | 3.376 | Volts | ok | 2.816 | 2.880 | 2.944 | 3.584 | 3.648 | 3.712 12V | 12.349 | Volts | ok | 10.494 | 10.600 | 10.706 | 13.091 | 13.197 | 13.303 VDIMM | 1.504 | Volts | ok | 1.152 | 1.216 | 1.280 | 1.760 | 1.776 | 1.792 5VCC | 5.088 | Volts | ok | 4.096 | 4.320 | 4.576 | 5.344 | 5.600 | 5.632 CPU VTT | 1.000 | Volts | ok | 0.872 | 0.896 | 0.920 | 1.344 | 1.368 | 1.392 VBAT | 3.440 | Volts | ok | 2.816 | 2.880 | 2.944 | 3.584 | 3.648 | 3.712 VSB | 3.568 | Volts | ok | 3.008 | 3.072 | 3.136 | 3.856 | 3.920 | 3.984 AVCC | 3.376 | Volts | ok | 2.816 | 2.880 | 2.944 | 3.584 | 3.648 | 3.712 Chassis Intru | 0x0 | discrete | 0x0000| na | na | na | na | na | na
Get detailed information for a specific sensor:
[root@esx-e1:~] /opt/ipmitool/ipmitool sensor get "FAN 1" Locating sensor record... Sensor ID : FAN 1 (0x41) Entity ID : 29.1 Sensor Type (Analog) : Fan Sensor Reading : 300 (+/- 0) RPM Status : ok Lower Non-Recoverable : 75.000 Lower Critical : 75.000 Lower Non-Critical : 75.000 Upper Non-Critical : 18975.000 Upper Critical : 19050.000 Upper Non-Recoverable : 19125.000 Assertion Events : Assertions Enabled : lcr- lnr- unc+ ucr+ unr+ Deassertions Enabled : lcr- lnr- unc+ ucr+ unr+
Set all “FAN 1” alarm thresholds to 750RPM:
[root@esx-e1:~] /opt/ipmitool/ipmitool sensor thresh "FAN 1" lower 750 750 750 Locating sensor record 'FAN 1'... Setting sensor "FAN 1" Lower Non-Recoverable threshold to 750.000 Setting sensor "FAN 1" Lower Critical threshold to 750.000 Setting sensor "FAN 1" Lower Non-Critical threshold to 750.000
Get IPMI LAN configuration:
[root@esx-e1:/tmp] /opt/ipmitool/ipmitool lan print Set in Progress : Set Complete Auth Type Support : NONE MD2 MD5 PASSWORD Auth Type Enable : Callback : MD2 MD5 PASSWORD : User : MD2 MD5 PASSWORD : Operator : MD2 MD5 PASSWORD : Admin : MD2 MD5 PASSWORD : OEM : MD2 MD5 PASSWORD IP Address Source : Static Address IP Address : 172.16.1.61 Subnet Mask : 255.255.255.0 MAC Address : 00:25:90:7d:ea:1e SNMP Community String : public IP Header : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00 BMC ARP Control : ARP Responses Enabled, Gratuitous ARP Disabled Default Gateway IP : 0.0.0.0 Default Gateway MAC : 00:00:00:00:00:00 Backup Gateway IP : 0.0.0.0 Backup Gateway MAC : 00:00:00:00:00:00 802.1q VLAN ID : 1 802.1q VLAN Priority : 0 RMCP+ Cipher Suites : 1,2,3,6,7,8,11,12 Cipher Suite Priv Max : aaaaXXaaaXXaaXX : X=Cipher Suite Unused : c=CALLBACK : u=USER : o=OPERATOR : a=ADMIN : O=OEM
Change the BMC’s VLAN ID:
[root@esx-e1:~] /opt/ipmitool/ipmitool lan set 1 vlan id 1
Updates
1.8.11-2 – August 20, 2019: If you downloaded the older version that I had originally posted (1.8.11-1) please remove it and install the 1.8.11-2 version posted above instead. I had used /bin/ to store the executable in the older version. Although it worked fine, this causes a lot of unnecessary log messages from VisorFS in vmkernel.log similar to the following:
2019-08-19T18:29:56.215Z cpu14:2150082)WARNING: VisorFS: 1093: Attempt to remove non sticky dir/file from tar mount
The executable is now stored in /opt/ipmitool/ instead, which means this won’t happen. As an added bonus, the –no-sig-check option is no longer required while installing.
1.8.11-1 – August 19, 2019: Original release version.
