[OmniOS-discuss] 4kn or 512e with ashift=12

[Richard Jahnel]

It should be noted that using a 512e disk as a 512n disk subjects you to a significant risk of silent corruption in the event of power loss. Because 512e disks does a read>modify>write operation to modify 512byte chunk of a 4k sector, zfs won't know about the other 7 corrupted 512e sectors in the event of a power loss during a write operation. So when discards the incomplete txg on reboot, it won't do anything about the other 7 512e sectors it doesn't know were affected.

Richard Jahnel
Network Engineer
On-Site.com | Ellipse Design
866-266-7483 ext. 4408
Direct: 669-800-6270


-----Original Message-----
From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Chris Siebenmann
Sent: Wednesday, March 23, 2016 9:36 AM
To: Fred Liu <Fred_Liu at issi.com>
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12

> > 	The sd.conf whitelist also requires a reboot to activate if you need
> > 	to add a new entry, as far as I know.
> > 
> >     (Nor do I know what happens if you have some 512n disks and
> >     some 512e disks, both correctly recognized and in different
> >     pools, and now you need to replace a 512n disk with a spare 512e
> >     disk so you change sd.conf to claim that all of the 512e disks
> >     are 512n. I'd like to think that ZFS will carry on as normal,
> >     but I'm not sure.  This makes it somewhat dangerous to change
> >     sd.conf on a live system.)
> 
> There are two cases if we don't use the remedy (whitelist in illumos 
> or -o ashift in ZoL) here:
> a): 512n <---> 512e. This replacement should work *in theory* if the 
> lie works *correctly*.

 This will not work without the sd.conf workaround in Illumos.

 All 512e disks that I know of correctly report their actual physical disk size to Illumos (and to Linux/ZoL). When a disk reports a 4K physical sector size, ZFS will refuse to allow it into an ashift=9 vdev *regardless* of the fact that it is 512e and will accept reads and writes in 512-byte sectors.

 In Illumos, you can use sd.conf to lie to the system and claim that this is not a 512e but a 512n disk (ie, it has a 512 byte physical sector size). I don't believe there's an equivalent on ZoL, but I haven't looked.

 This absolute insistence on ZFS's part is what makes ashift=9 vdevs so dangerous today, because you cannot replace existing 512n disks in them with 512e disks without (significant) hackery.

(Perhaps I'm misunderstanding what people mean by '512e' here; I've been assuming it means a disk which reports 512 byte logical sectors and 4k physical sectors. Such disks are what you commonly get today.)

	- cks
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss