Hot-removing disk shelves in systems running Data ONTAP 8.2.1 or later

If your system is running Data ONTAP 8.2.1 or later, you can hot-remove disk shelves—physically remove disk shelves that have had the aggregates removed from the disk drives—in a 7-Mode multipath HA configuration with DS14 disk shelves that is up and serving data. You can hot-remove one or more disk shelves from anywhere within a loop of disk shelves or remove a loop of disk shelves.

Note: If you have a MetroCluster configuration with SAS disk shelves, see the appropriate MetroCluster document for hot-removing SAS disk shelves.

For stretch MetroCluster systems using SAS optical cables, see the Configuring a stretch MetroCluster system with SAS disk shelves and SAS optical cables document. For fabric and stretch MetroCluster systems using FibreBridge 6500N bridges and SAS copper cables, see the Configuring a MetroCluster system with SAS disk shelves and FibreBridge 6500N bridges document.

These documents can be found on the NetApp Support Site at support.netapp.com.

  • Your storage system must be running Data ONTAP 8.2.1 or later.
  • Your storage system must be a multipath HA system.
  • For FAS2240 configurations, the external storage must be cabled as multipath HA.
  • You must have already removed all aggregates from the disk drives in the disk shelves you are removing.
    Attention: If you attempt this procedure on non MetroCluster configurations with aggregates on the disk shelf you are removing, you could fail the system with a multi-disk panic.

    For MetroCluster configurations, removing aggregates involves splitting the mirrored aggregates on the disk shelves you are removing and then re-creating the mirrored aggregates with another set of disk drives. For procedures, see Splitting a mirrored aggregate and Rejoining split aggregates in the Data ONTAP Data Protection Online Backup and Recovery Guide for 7-Mode.

    For non MetroCluster configurations, see Taking an aggregate offline in the Storage Management Guide for 7-Mode for information. This document is available on the NetApp Support Site at support.netapp.com.

  • As a best practice, you should remove disk drive ownership after you remove the aggregates from the disk drives in the disk shelves you are removing.
    Note: This procedure follows the best practice of removing disk drive ownership; therefore, steps are written with the assumption that you have removed disk drive ownership.
    For information about removing disk drive ownership, see the Removing ownership from a disk procedure in the Storage Management Guide for 7-Mode. This document is available on the NetApp Support Site at support.netapp.com.
    Note: The procedure for removing ownership from disk drives requires you to disable disk autoassignment. You reenable disk autoassignment when prompted at the end of this shelf hot-remove procedure.
  • Multipath HA configurations cannot be in a takeover state.
  • If you are removing one or more disk shelves from within a loop, you must have factored the distance to bypass the disk shelves you are removing; therefore, if the current cables are not long enough, you need to have longer cables available.

    The Hardware Universe at hwu.netapp.com contains information about supported cables.

  • This procedure follows cabling best practices; therefore, references to modules and module input and output ports align with the best practices. If your storage system is cabled differently from what is prescribed as best practice, the modules and/or module input and output ports might be different.
  • Path A refers to the A-side disk shelf module (module A) located in the top of the disk shelf.
  • Path B refers to the B-side disk shelf module or module B located in the bottom location in the disk shelf.
  • The first disk shelf in the loop is the disk shelf with the input ports directly connected to the controllers or switches.
  • The interim disk shelf in the loop is the disk shelf directly connected to other disk shelves in the loop.
  • The last disk shelf in the loop is the disk shelf with output ports directly connected to the controllers, except for fabric MetroCluster systems.

    For fabric MetroCluster systems, the output ports of the last disk shelf are not connected; they are not used.

  • The next disk shelf is the disk shelf downstream of the disk shelf being removed, in depth order.
  • The previous disk shelf is the disk shelf upstream of the disk shelf being removed, in depth order.
  • For MetroCluster configurations, after completing this procedure for the local site, you are instructed to repeat this procedure for the remote site if you are also removing disk shelves for the remote site.
  1. Verify that your system configuration is Multi-Path HA by entering the following command from the console of either controller: sysconfig It might take up to a minute for the system to complete discovery. The configuration is listed in the System Storage Configuration field.
    Note: For FAS2240 systems with external storage, the output is displayed as Mixed-Path HA because the internal storage is cabled as single-path HA and the external storage is cabled as multipath HA.
    Attention: If your non FAS2240 system configuration is shown as something other than Multi-Path HA, you cannot continue with this procedure. Your system must meet the prerequisites stated in the Before you begin section of this procedure.
  2. Verify that no aggregates are on the disk drives in the disk shelves you are removing, by completing the following substeps:
    1. Enter the following command at the console of either controller: aggr status -rIf you have traditional volumes, you enter the vol status -r command.
    2. Check the output SHELF column to verify that the disk shelves from which you have removed aggregates do not appear.
    Attention: If any disk drives in the disk shelves you are removing are anything other than spares or failed, you cannot continue with this procedure. Your system must meet the prerequisites stated in the Before you begin section of this procedure.
    The following aggr status -r output is for a 7-Mode system with disk shelf 3 being removed. Because aggregates have been removed from disk shelf 3, only disk shelves in the loop with aggregates (disk shelves 2 and 4) appear in the SHELF column:
    fas6200> aggr status -r
    
    RAID Disk   Device    HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
    ---------   ------    ------------- ---- ---- ---- ----- --------------    --------------
    ...
    dparity       0c.33     0c    2   1   FC:A   0  FCAL 15000 136000/278528000  137104/280790184
    parity        0c.38     0c    2   6   FC:A   0  FCAL 15000 136000/278528000  137104/280790184  
    data          0c.64     0c    4   0   FC:A   0  FCAL 15000 136000/278528000  137104/280790185 
    ...
    
  3. Verify that ownership is removed from the disk drives on the disk shelves you are removing:
    1. Enter the following command at the console of either controller: disk show -v
    2. Check the output and take the appropriate action:
      If the output shows... Then...

      Not Owned or FAILED

      Go to the next step.

      The disk drives in the disk shelves you are removing are in the correct state.

      Any disk drives in the disk shelves you are removing have ownership You can use the Removing ownership from a disk procedure referenced in the Before you begin section of this procedure.
      The following output for the disk show -v command shows that disk drives on the disk shelf being removed (disk shelf 3) are in a correct state for removal. Two disk drives have the ownership removed; therefore, Not Owned appears in the OWNER column. Two disk drives are failed; therefore, FAILED appears in the POOL column:
      fas6200> disk show -v
      
      DISK      OWNER          POOL     SERIAL NUMBER   DR HOME         CHKSUM
      --------  -------------  ----     -------------   ------------   -------
      ...
      4b.5      Not Owned      NONE     YHGLB7PB        Block
      4b.11     Not Owned      NONE     YHGL33ZA        Block
      4b.15                    FAILED                   Block
      4b.13                    FAILED                   Block
      ...
  4. Turn on the LEDs for each disk drive in the disk shelves you are removing so that the disk shelves are physically identifiable by completing the following substeps:You need to be certain of which disk shelves you are removing so that you can correctly recable path A and path B later in this procedure.

    You can enter the commands from the console of either controller.

    1. Identify the disk drives in each disk shelf you are removing: fcadmin device_map In this output, the shelf mapping shows three disk shelves in a loop and their respective 14 disk drives. If disk shelf 3 is being removed, disk drives 45 44 43 42 41 40 39 38 37 36 35 34 33 32 are applicable.
      fas6200> fcadmin device_map
      
      Loop Map for channel 0c:
      ...
      Shelf mapping:
      Shelf 3:  45  44  43  42  41  40  39  38  37  36  35  34  33  32
      Shelf 4:  77  76  75  74  73  72  71  70  69  68  67  66  65  64
      Shelf 5:  93  92  91  90  89  88  87  86  85  84  83  82  81  80
      ...
    2. Turn on the LEDs for the disk drives you identified in Substep a: led_on disk_nameYou must be in advanced privilege level to enter this command.

      The fault LED on the front of the disk drive illuminates—solid. Additionally, if you have any failed disk drives in the disk shelves you are removing, the activity LED on the front of those disk drives blinks.

      It is recommended that you turn on the LED for a minimum of four disk drives so that the disk shelves you are removing can be visually identified. You must repeat the command for each disk drive.

      To turn on the fault LED for disk drive 0c.45 in disk shelf 3 identified in Substep a, you enter led_on 0c.45
  5. If you are removing an entire loop of disk shelves, complete the following substeps; otherwise, go to the next step:
    1. Remove all cables on path A and path B.This includes controller-to-shelf cables, switch-to-shelf cables, and shelf-to-shelf cables for all disk shelves in the loop you are removing
    2. Go to Step 9.
  6. If you are removing one or more disk shelves from a loop (but keeping the loop), recable the applicable path A loop connections to bypass the disk shelves you are removing by completing the applicable set of substeps:If you are removing more than one disk shelf, complete the applicable set of substeps one disk shelf at a time.

    If you need a graphical system cabling reference, use the platform specific Installation and Setup Instructions document that ships with each platform, or access these documents on the NetApp Support Site at support.netapp.com by searching on your specific platform. For example, to find the Installation and Setup Instructions document for FAS3200 systems, search on FAS3200 series.

    If you are removing... Then...
    The first disk shelf in a loop
    1. Remove the cable connecting the module A output port of the first disk shelf and the module A input port of the second disk shelf in the loop and set it aside.
    2. Move the cable connecting the controller or switch to the module A input port of the first disk shelf to the module A input port of the second disk shelf in the loop
    An interim disk shelf in a loop
    1. Remove the cable connecting the module A output port of the disk shelf being removed and the module A input port of the next disk shelf in the loop and set it aside.
    2. Move the cable connecting the module A input port of the disk shelf being removed to the module A input port of the next disk shelf in the loop
    The last disk shelf in a loop
    1. Remove the cable connecting the module A input port of the last disk shelf and the module A output port of the previous disk shelf in the loop and set it aside.
    2. If you have a non fabric MetroCluster system, move the cable connecting the controller to the module A output port of the last disk shelf to the module A output port of the previous disk shelf in the loop
    Note: For fabric MetroCluster systems, the last disk shelf in a loop is not connected to the switch; therefore, removing the last disk shelf involves removing shelf-to-shelf cables only.
  7. Verify that the cabling on path A has successfully bypassed the disk shelves you are removing and all disk drives on the disk shelves you are removing are still connected through path B, by entering the following command from the console of either controller: storage show disk -pIt might take up to a minute for the system to complete discovery. In this example of how the output should appear for a stretch MetroCluster system, the disk shelf being removed is disk shelf 3. One line item appears for each disk drive connected through path B (now the primary path); therefore, the disk drives are listed in the PRIMARY column and B appears in the first PORT column. There is no connectivity through path A for any of the disk drives in the disk shelf being removed; therefore, no information is shown in the SECONDARY or second PORT columns:
    fas6200> storage show disk -p
    
    PRIMARY    PORT  SECONDARY  PORT SHELF BAY
    ---------- ----  ---------- ---- ---------
    ...  
    0d.64       B                      3    0  
    0d.65       B                      3    1  
    0d.66       B                      3    2  
    0d.67       B                      3    3  
    0d.68       B                      3    4  
    0d.69       B                      3    5  
    0d.70       B                      3    6  
    0d.71       B                      3    7  
    ...
    Attention: If the output shows anything other than all the disk drives connected only through path B, you must correct the cabling by repeating Step 6.
  8. Complete the following substeps:
    1. Repeat Step 6 and Step 7 for path B.
    2. Repeat Step 1 to confirm that your system configuration is the same as before you began this procedure.
    3. Go to the next step.
  9. If, when you removed ownership from the disk drives as part of the preparation for this procedure, you disabled disk autoassignment, then reenable disk autoassignment by entering the following command; otherwise, go to the next step: options disk.auto_assign onEnter the applicable command from the console of each controller.
  10. Power off the disk shelves you disconnected and unplug the power cords from the disk shelves.
  11. Remove the disk shelves from the rack or cabinet.To make disk shelves lighter and easier to maneuver, remove the power supplies and modules. Avoid removing the disk drives if possible, because excessive handling can lead to internal damage.
  12. If you are removing disk shelves at the remote site, repeat this procedure.