[OmniOS-discuss] Pool degraded

[Alexander Lesle]

Hello All,

I have a pool with mirrors and one spare.
Now my pool is degraded and I though that Omnios/ZFS activate the
spare itself and make a resilvering.

# zpool status -x
  pool: pool_ripley
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q
  scan: resilvered 84K in 0h0m with 0 errors on Sun Mar 23 15:09:08 2014
config:

        NAME                       STATE     READ WRITE CKSUM
        pool_ripley                DEGRADED     0     0     0
          mirror-0                 DEGRADED     0     0     0
            c1t5000CCA22BC16BC5d0  ONLINE       0     0     0
            c1t5000CCA22BEEF6A3d0  UNAVAIL      0     0     0  cannot open
          mirror-1                 ONLINE       0     0     0
            c1t5000CCA22BC8D31Ad0  ONLINE       0     0     0
            c1t5000CCA22BF612C4d0  ONLINE       0     0     0
          .
          .
          .

        spares
          c1t5000CCA22BF5B9DEd0    AVAIL

But nothing done.
OK, then I do it myself.
# zpool replace -f pool_ripley c1t5000CCA22BEEF6A3d0 c1t5000CCA22BF5B9DEd0
Resilvering is starting immediately.

# zpool status -x
  pool: pool_ripley
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q
  scan: resilvered 1.53T in 3h12m with 0 errors on Sun Apr  6 17:48:51 2014
config:

        NAME                         STATE     READ WRITE CKSUM
        pool_ripley                  DEGRADED     0     0     0
          mirror-0                   DEGRADED     0     0     0
            c1t5000CCA22BC16BC5d0    ONLINE       0     0     0
            spare-1                  DEGRADED     0     0     0
              c1t5000CCA22BEEF6A3d0  UNAVAIL      0     0     0  cannot open
              c1t5000CCA22BF5B9DEd0  ONLINE       0     0     0
          mirror-1                   ONLINE       0     0     0
            c1t5000CCA22BC8D31Ad0    ONLINE       0     0     0
            c1t5000CCA22BF612C4d0    ONLINE       0     0     0
         .
         .
         .
        spares
          c1t5000CCA22BF5B9DEd0      INUSE     currently in use

After resilvering I made power-off, unplugged the broken HDD from
Case-slot 1 and switched the Spare from Slot 21 to Slot 1.
The pool is still degraded. The broken HDD I cant remove it.

# zpool remove pool_ripley c1t5000CCA22BEEF6A3d0
cannot remove c1t5000CCA22BEEF6A3d0: only inactive hot spares,
cache, top-level, or log devices can be removed

What can I do to through out the broken HDD and tell ZFS that the
spare is now member of mirror-0 and remove it from the spare list?
Why does not automatically jump in the Spare device and resilver the
pool?

Thanks.

-- 
Best Regards
Alexander
April, 08 2014