Table of ContentsView in Frames

Calculating queue depth

You might need to tune your FC queue depth on the host to achieve the maximum values for hosts per controller and FC port fan-in. The maximum number of LUNs and the number of HBAs that can connect to an FC port are limited by the available queue depth on the FC target ports.

About this task

Queue depth is the number of I/O requests (SCSI commands) that can be queued at one time on a storage controller. Each I/O request from the host's initiator HBA to the storage controller's target adapter consumes a queue entry. Typically, a higher queue depth equates to better performance. However, if the storage controller's maximum queue depth is reached, that storage controller rejects incoming commands by returning a QFULL response to them. If a large number of hosts are accessing a storage controller, plan carefully to avoid QFULL conditions, which significantly degrade system performance and can lead to errors on some systems.

In a configuration with multiple initiators (hosts), all hosts should have similar queue depths. This prevents hosts with small queue depths from being starved by hosts with large queue depths.

The following general recommendations can be made about "tuning" queue depths.
  • For small to mid-size systems, use a HBA queue depth of 32.
  • For large systems, use a HBA queue depth of 128.
  • For exception cases or performance testing, use a queue depth of 256 to avoid possible queuing problems.
  • All hosts should have the queue depths set to similar values to give equal access to all hosts.
  • Ensure that the storage controller target FC port queue depth is not exceeded to avoid performance penalties or errors.

Steps

  1. Count the total number of FC initiators in all the hosts that connect to one FC target port.
  2. Multiply by 128.
    • If the result is less than 2,048, set the queue depth for all initiators to 128.

    Example

    You have 15 hosts with one initiator connected to each of two target ports on the storage controller. 15 x 128 = 1,920. 1,920 is less than the total queue depth limit of 2,048, so you can set the queue depth for all your initiators to 128.

    • If the result is greater than 2,048, go to step 3.

    Example

    You have 30 hosts with one initiator connected to each of two target ports on the storage controller. 30 x 128 = 3,840. 3,840 is greater than the total queue depth limit of 2,048, so you should choose one of the options under step 3 for remediation.

  3. Choose one of the following options.
    • Option 1:
      1. Add more FC target ports.
      2. Redistribute your FC initiators.
      3. Repeat steps 1 and 2.

    Example

    The desired queue depth of 3,840 exceeds the available queue depth per port. To remedy this, you can add a two-port FC target adapter to each controller, then rezone your FC switches so that 15 of your 30 hosts connect to one set of ports, and the remaining 15 hosts connect to a second set of ports. The queue depth per port is then reduced to 15 x 128 = 1,920.

    • Option 2:
      1. Designate each host as "large" or "small" based on its expected I/O need.
      2. Multiply the number of large initiators by 128.
      3. Multiply the number of small initiators by 32.
      4. Add the two results together.
      5. If the result is less than 2,048, set the queue depth for "large" host to 128 and the queue depth for "small" host to 32.
      6. If the result is still greater than 2,048 per port, reduce the queue depth per initiator until the total queue depth is less than or equal to 2,048.
        Note: To estimate the queue depth needed to achieve a certain I/O per second throughput, use this formula.

        Needed queue depth = (Number of I/O per second) x (Response time)

        For example, if you need 40,000 I/O per second with a response time of 3 milliseconds, the needed queue depth = 40,000 x (.003) = 120.

    Example

    The desired queue depth of 3,840 exceeds the available queue depth per port. You have 10 "large" hosts that have high storage I/O needs, and 20 "small" hosts that have low I/O needs. Set the initiator queue depth on the "large" hosts to 128 and the initiator queue depth on the "small" hosts to 32.

    Your resulting total queue depth is (10 x 128) + (20 x 32) = 1,920.

    Example

    You can spread the available queue depth equally across each initiator.

    Your resulting queue depth per initiator is 2,048/30 = 68