[OmniOS-discuss] Illumos and Infiniband

[Dan Swartzendruber]

> Well, this is all very encouraging! I guess there is interest and actual
> usage out there.
>
> Unfortunately, I need FDR-ish speeds (at least 4GB/s), the entire network
> is setup for FDR, and I only have ConnectX-3 adapters, so it's Linux for
> me
> until some new developments emerge.
>
> Dan, I can somewhat vouch for the stability of ZFS on Debian and CentOS
> using the ZoL kernel implementation. My Debian system (the one using IB)
> is
> at 65 days uptime with a 24/7 heavy load, managing 100TB+ of datasets in a
> quasi-production (replicated) environment. First disk replacement went
> smoothly last week. Time will tell... The CentOS install has been up for
> close to a year. However, in terms of raw disk access speeds, neither of
> these systems performs anywhere near as well as OmniOS or OI.

Chris, let me clarify.  I had this same converasation on a forum somewhere
where some guy got all defensive and borderline-accused me of being an
anti-ZoL troll or something.  Anyway, for basic I/O and serving up NFS
(andmaybe even iSCSI) clients, sure, I agree!  The places it still falls
down (not directly related to ZoL itself) are things where if your HBA is
a little slow off the mark presenting the disks to the ZFS driver, you can
end up with zvols that are not visible, datasets that have to be manually
exported, iscsi target daemon needing to be manually restarted, etc... 
All having to do with bad timing between udev and ZFS.  Oh, also, no zfs
root pool unless you want to slide down a very sharp razor blade...