Changing the HBA queue depths on multiple dual-port adapters

Following on from optimising the storage for a customer, I decided to change the queue depths for the Emulex HBAs. The ESX hosts, each have two dual-port Emulex HBAs, the diagram below shows the setup..









Only two ports are in use at the moment, vmhba2 and vmhba5. To determine the instance numbers that are in use by the Emulex ESX driver - lpfc (use qla2300 or similar for QLogic), the output of the ls command includes a number for each active HBA in the system. We can then use the instance numbers to find the active adapters.

Emulex example

# ls /proc/scsi/lpfc

You should get an output similar to




Because of the way that the host is connected and from the picture above, I already know that 2 and 5 are the active adapters. Running the following command will confirm

# cat /proc/scsi/lpfc/2

this shows that vmhba2 is currently active and has 4-paths to the SAN








Running the same command on vmhba3 gives the following as expected







Running the command on vmhba5 is also as expected.

Now that we've found out which vmhbas are active, we can use the output to find out which lpfc# options we should add to the lpfc_740.o module to configure the queue length.

The outputs of # cat /proc/scsi/lpfc/2 and # cat /proc/scsi/lpfc/5, give us lpfc numbers of 0 and 3 respectively. So to configure a queue depth of 64 for lpfc2 and lpfc5 we run the following command

# esxcfg-module -s "lpfc0_lun_queue_depth=64 lpfc3_lun_queue_depth=64" lpfc_740

and

# esxcfg-boot -b

The -q option shows configured options for a module.

Now we reboot for the changes to take effect.

In this case, both HBAs lpfc0 (vmhba2) and lpfc3 (vmhba5) will have their queue depths set to 64.

With this post and the previous one, we have set manual load balancing for the LUNs over eight different paths and also changed the queue depth to 64, this should keep the ESX optimised for now, maybe I'll change the VMFS3.MaxHeapSizeMB to 64 for good measure!