Hayden

The “bnx2x_dynamic_alloc_rx_queue_single” error that suddenly appears in VMware ESXi environments can be a real headache for system administrators. When this error occurs, network connectivity becomes unstable or completely drops out. Let me walk you through the root causes and step-by-step solutions to resolve this issue once and for all.

 

The bnx2x_dynamic_alloc_rx_queue_single error is a memory leak issue that occurs in the bnx2x driver for Broadcom NetXtreme II network cards. This error typically happens during network queue allocation when memory becomes insufficient, and in severe cases, it can trigger an ESXi host PSOD (Purple Screen of Death).

 

 

1. ESXi “bnx2x_dynamic_alloc_rx_queue_single” Error : Root Cause and Symptoms

Primary Cause

This issue stems from a memory leak identified in bnx2x driver version 2.712.70.x.3. The core problem occurs when NetPacketPool becomes depleted, causing buffer allocation failures in network queues with Large Receive Offload (LRO) enabled.

Key Symptoms

You may encounter the following log messages:

2017-03-15T17:17:14.810Z cpu1:32868)<3>[bnx2x_dynamic_alloc_rx_queue_single:789(vmnic2)]Could not start queue:1
2017-03-15T17:17:14.788Z cpu1:32868)<3>[bnx2x_esx_init_rx_ring:1944(vmnic2)]was only able to allocate 0 rx sges
2017-03-15T17:17:19.786Z cpu0:32868)<3>[bnx2x_alloc_rx_sge:734(vmnic2)]Can't alloc sge

 

 

2. Pre-Diagnosis Steps

Check Current Driver Version

SSH into your ESXi host and run the following commands:

# List network interfaces
esxcli network nic list

# Check specific network card driver information
esxcli network nic get -n vmnic0

Verify Hardware Information

# Check PCI information
vmkchdev -l | grep vmnic

The following table shows problematic hardware identifiers:

VID DID SVID SSID Product
14e4 16a2 103c 22fa HPE Broadcom BCM5709C
14e4 163d – – Broadcom NetXtreme II
14e4 163e – – Broadcom NetXtreme II

 

 

3. Driver Upgrade (Recommended Solution)

Download Latest Driver

Upgrading to bnx2x driver version 2.713.70.v60.6 or higher is the most reliable solution.

  1. Visit VMware Customer Connect
  2. Search for the compatible driver for your ESXi version
  3. Download the Offline Bundle (.zip) file

Installation Process

Step 1: Put host in maintenance mode

# Via vSphere Client or command line
esxcli system maintenanceMode set --enable true

Step 2: Install driver

# Install VIB
esxcli software vib install -d /vmfs/volumes/datastore1/driver-offline-bundle.zip

# Verify installation
esxcli software vib list | grep bnx2x

Step 3: Reboot host

reboot

 

 

4. Network Driver Migration

For ESXi 6.7 and later, you can use the native qfle3 driver instead of bnx2x.

Switch from bnx2x to qfle3

# Enable maintenance mode
esxcli system maintenanceMode set --enable true

# Enable qfle3 driver
esxcli system module set --enabled=true --module=qfle3

# Disable bnx2x driver
esxcli system module set --enabled=false --module=bnx2x

# Reboot
reboot

Revert from qfle3 to bnx2x

# Enable bnx2x driver
esxcli system module set --enabled=true --module=bnx2x

# Disable qfle3 driver
esxcli system module set --enabled=false --module=qfle3

# Reboot
reboot

 

 

5. Monitoring and Verification

Check Driver Status

# Verify current active driver
esxcli network nic list

# Check memory usage
esxcli system stats get

Monitor Error Logs

# Real-time log monitoring
tail -f /var/log/vmkernel.log | grep bnx2x

 

 

6. Prevention Measures

Regular Maintenance Items

  • Driver Version Management: Check for latest drivers quarterly
  • Memory Usage Monitoring: Regular NetPacketPool utilization checks
  • Hardware Compatibility Verification: Review VMware Compatibility Guide for updates

Recommended Settings

Configuration Item Recommended Value Description
bnx2x driver version 2.713.70.v60.8 or higher Resolves memory leak issue
NetPacketPool size 150% of default Prevents memory shortage
Check frequency Monthly Driver and firmware update verification

 

The bnx2x_dynamic_alloc_rx_queue_single error may look complex, but it’s usually resolved with a simple driver upgrade. However, in production environments, always ensure you have maintenance windows scheduled and backup plans in place before proceeding. 🙂

 

Leave a Reply