Hot-removing disk shelves or loops in systems running Data ONTAP 8.2.1 or later

If your system is running Data ONTAP 8.2.1 or later, you can hot-remove disk shelves—physically remove disk shelves that have had the aggregates removed from the disk drives—in a clustered Data ONTAP multipath HA configuration with DS14 disk shelves that is up and serving data. You can hot-remove one or more disk shelves from anywhere within a loop of disk shelves or remove a loop of disk shelves.

  • Your system must not have SAS disk shelves.

    See the appropriate guide for clustered Data ONTAP systems with SAS disk shelves.

    SAS Disk Shelves Installation and Service Guide for DS4243, DS2246, DS4486, and DS4246

  • Your storage system must be running Data ONTAP 8.2.1 or later.
  • Your storage system must be a multipath HA system.
  • For FAS2240 configurations, the external storage must be cabled as multipath HA.
  • You must have already removed all aggregates from the disk drives in the disk shelves you are removing.
    Attention: If you attempt this procedure with aggregates on the disk shelf you are removing, you could fail the system with a multi-disk panic.

    For information about taking an aggregate offline for clustered systems, see Commands for managing aggregates in the Clustered Data ONTAP Physical Storage Management Guide. This document is available on the NetApp Support Site at mysupport.netapp.com.

  • As a best practice, you should remove disk drive ownership after you remove the aggregates from the disk drives in the disk shelves you are removing.
    Note: This procedure follows the best practice of removing disk drive ownership; therefore, steps are written with the assumption that you have removed disk drive ownership.
    The Clustered Data ONTAP Physical Storage Management Guide includes the Removing ownership from a disk procedure for removing disk drive ownership. This document is available on the NetApp Support Site at mysupport.netapp.com.
    Note: The procedure for removing ownership from disk drives requires you to disable disk autoassignment. You reenable disk autoassignment when prompted at the end of this shelf hot-remove procedure.
  • Multipath HA configurations cannot be in a takeover state.
  • If you are removing one or more disk shelves from within a loop, you must have factored the distance to bypass the disk shelves you are removing; therefore, if the current cables are not long enough, you need to have longer cables available.

    The Hardware Universe at hwu.netapp.com contains information about supported cables.

  • This procedure follows cabling best practices; therefore, references to modules and module input and output ports align with the best practices. If your storage system is cabled differently from what is prescribed as best practice, the modules and/or module input and output ports might be different.
  • Path A refers to the A-side disk shelf module (module A) located in the top of the disk shelf.
  • Path B refers to the B-side disk shelf module (module B) located in the bottom of the disk shelf.
  • The first disk shelf in the loop is the disk shelf with the input ports directly connected to the controllers.
  • The interim disk shelf in the loop is the disk shelf directly connected to other disk shelves in the loop.
  • The last disk shelf in the loop is the disk shelf with output ports directly connected to the controllers.
  • The next disk shelf is the disk shelf downstream of the disk shelf being removed, in depth order.
  • The previous disk shelf is the disk shelf upstream of the disk shelf being removed, in depth order.
  • Clustered Data ONTAP commands and 7-Mode commands are used; therefore, you will be entering commands from the clustershell and from the nodeshell.
  1. Verify that your system configuration is Multi-Path HA by entering the following command from the nodeshell of either controller: sysconfig It might take up to a minute for the system to complete discovery. The configuration is listed in the System Storage Configuration field.
    Note: For FAS2240 systems with external storage, the output is displayed as Mixed-Path HA because the internal storage is cabled as single-path HA and the external storage is cabled as multipath HA.
    Attention: If your non FAS2240 system is shown as something other than Multi-Path HA, you cannot continue with this procedure. Your system must meet the prerequisites stated in the Before you begin section of this procedure.
  2. Verify that the disk drives in the disk shelves you are removing have no aggregates (are spares) and ownership is removed, by completing the following substeps:
    1. Enter the following command from the clustershell of either controller: storage disk show -shelf shelf_number
    2. Check the output to verify there are no aggregates on the disk drives in the disk shelves you are removing.Disk drives with no aggregates have a dash in the Aggregate column.
      Attention: If disk drives in the disk shelves you are removing have aggregates, you cannot continue with this procedure. Your system must meet the prerequisites stated in the Before you begin section of this procedure.
    3. Check the output to verify that ownership is removed from the disk drives on the disk shelves you are removing or that the disk drives are failed.
      If the output shows... Then...
      unassigned or broken for all disk drives Go to the next step.

      The disk drives in the disk shelves you are removing are in the correct state.

      Any disk drives in the disk shelves you are removing have ownership You can use the Removing ownership from a disk procedure referenced in the Before you begin section of this procedure.
    The following output for the storage disk show -shelf 3 command shows disk drives on the disk shelf being removed (disk shelf 3). All of the disk drives in disk shelf 3 have a dash in the Aggregate column. Two disk drives have the ownership removed; therefore, unassigned appears in the Container Type column. And two disk drives are failed; therefore, broken appears in the Container Type column:
    cluster::> storage disk show -shelf 3
    
               Usable           Disk   Container   Container
    Disk         Size Shelf Bay Type   Type        Name       Owner
    -------- -------- ----- --- ------ ----------- ---------- ---------
    ...
    1.3.4           -     3   4 SAS    unassigned           -         -
    1.3.5           -     3   5 SAS    unassigned           -         -
    1.3.6           -     3   6 SAS    broken               -         -
    1.3.7           -     3   7 SAS    broken               -         -
    ...
    
  3. Turn on the LEDs for each disk drive in the disk shelves you are removing so that the disk shelves are physically identifiable by completing the following substeps:You need to be certain of which disk shelves you are removing so that you can correctly recable path A and path B later in this procedure.

    You enter the commands from the nodeshell of either controller.

    1. Identify the disk drives in each disk shelf you are removing: fcadmin device_map In this output, the shelf mapping shows three disk shelves in a loop and their respective 14 disk drives. If disk shelf 3 is being removed, disk drives 45 44 43 42 41 40 39 38 37 36 35 34 33 32 are applicable.
      fas6200> fcadmin device_map
      
      Loop Map for channel 0c:
      ...
      Shelf mapping:
      Shelf 3:  45  44  43  42  41  40  39  38  37  36  35  34  33  32
      Shelf 4:  77  76  75  74  73  72  71  70  69  68  67  66  65  64
      Shelf 5:  93  92  91  90  89  88  87  86  85  84  83  82  81  80
      ...
    2. Turn on the LEDs for the disk drives you identified in Substep a: led_on disk_nameYou must be in advanced privilege level to enter this command.

      The fault LED on the front of the disk drive illuminates—solid. Additionally, if you have any failed disk drives in the disk shelves you are removing, the activity LED on the front of those disk drives blinks.

      It is recommended that you turn on the LED for a minimum of four disk drives so that the disk shelves you are removing can be visually identified. You must repeat the command for each disk drive.

      To turn on the fault LED for disk drive 0c.45 in disk shelf 3 identified in Substep a, you enter led_on 0c.45
  4. If you are removing an entire loop of disk shelves, complete the following substeps; otherwise, go to the next step:
    1. Remove all cables on path A and path B.This includes controller-to-shelf cables and shelf-to-shelf cables for all disk shelves in the loop you are removing.
    2. Go to Step 8.
  5. If you are removing one or more disk shelves from a loop (but keeping the loop), recable the applicable path A loop connections to bypass the disk shelves you are removing by completing the applicable set of substeps:If you are removing more than one disk shelf, complete the applicable set of substeps one disk shelf at a time.

    If you need a graphical system cabling reference, use the platform specific Installation and Setup Instructions document that ships with each platform, or access these documents on the NetApp Support Site at mysupport.netapp.com by searching on your specific platform. For example, to find the Installation and Setup Instructions document for FAS3200 systems, search on FAS3200 series.

    If you are removing... Then...
    The first disk shelf in a loop
    1. Remove the cable connecting the module A output port of the first disk shelf and the module A input port of the second disk shelf in the loop and set it aside.
    2. Move the cable connecting the controller to the module A input port of the first disk shelf to the module A input port of the second disk shelf in the loop
    An interim disk shelf in a loop
    1. Remove the cable connecting the module A output port of the disk shelf being removed and the module A input port of the next disk shelf in the loop and set it aside.
    2. Move the cable connecting the module A input port of the disk shelf being removed to the module A input port of the next disk shelf in the loop
    The last disk shelf in a loop
    1. Remove the cable connecting the module A input port of the last disk shelf and the module A output port of the previous disk shelf in the loop and set it aside.
    2. Move the cable connecting the controller to the module A output port of the last disk shelf to the module A output port of the previous disk shelf in the loop
  6. Verify that the cabling on path A has successfully bypassed the disk shelves you are removing and all disk drives on the disk shelves you are removing are still connected through path B, by entering the following command from the nodeshell of either controller: storage show disk -p It might take up to a minute for the system to complete discovery. In this example of how the output should appear, the disk shelf being removed is disk shelf 3. One line item appears for each disk drive connected through path B (now the primary path); therefore, the disk drives are listed in the PRIMARY column and B appears in the first PORT column. There is no connectivity through path A for any of the disk drives in the disk shelf being removed; therefore, no information is shown in the SECONDARY or second PORT columns:
    fas6200> storage show disk -p
    
    PRIMARY    PORT  SECONDARY  PORT SHELF BAY
    ---------- ----  ---------- ---- ---------
    ...  
    0d.64       B                      3    0  
    0d.65       B                      3    1  
    0d.66       B                      3    2  
    0d.67       B                      3    3  
    0d.68       B                      3    4  
    0d.69       B                      3    5  
    0d.70       B                      3    6  
    0d.71       B                      3    7   
    ...
    Attention: If the output shows anything other than all the disk drives connected only through path B, you must correct the cabling by repeating Step 5.
  7. Complete the following substeps:
    1. Repeat Step 5 and Step 6 for path B.
    2. Repeat Step 1 to confirm that your system configuration is the same as before you began this procedure.
    3. Go to the next step.
  8. If, when you removed ownership from the disk drives as part of the preparation for this procedure, you disabled disk autoassignment, then reenable disk autoassignment by entering the following command; otherwise, go to the next step: storage disk option modify -autoassign onEnter the applicable command from the clustershell of each controller.
  9. Power off the disk shelves you disconnected and unplug the power cords from the disk shelves.
  10. Remove the disk shelves from the rack or cabinet.To make disk shelves lighter and easier to maneuver, remove the power supplies and modules. Avoid removing the disk drivesor carriers if possible, because excessive handling can lead to internal damage.