[OmniOS-discuss] Big Data

[Michael Talbott]

We've been using OmniOS happily for years now for our storage server needs. But we're rapidly increasing our data footprint and growing so much (multiple PBs per year) that ideally I'd like to move to a cluster based object store based system ontop of OmniOS. I successfully use BeeGFS inside lxzones in OmniOS which seems to work nicely for our HPC scratch volume, but, it doesn't sound like it scales to hundreds of millions of files very well.

I am hoping that someone has some ideas for me. Ideally I'd like something that's cluster capable and has erasure coding like Ceph and have cluster aware snapshots (not local zfs snaps) and an s3 compatibility/access layer.

Any thoughts on the topic are greatly appreciated.

Thanks,

Michael
Sent from my iPhone