<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Enabling IB support for our ZFS-based filers is important enough to us that we have made the decision to migrate away from OmniOS/Illumos over to FreeBSD. FreeBSD 9 has a partial OFED stack, FreeBSD 10 is further along, and development continues for IB support in FreeBSD. I’ve successfully tested a Mellanox ConnectX-3 card on our FDR fabric on a FreeBSD 10.0-RELEASE system. This was with the subnet manager running on a Mellanox switch (which we regularly upgrade).<div class=""><div class=""><br class=""></div><div class="">If you’d like more details, please contact me off-list, I’m happy to share :)<div class=""><br class=""></div><div class="">-Greg</div><div class=""><br class=""><div apple-content-edited="true" class=""> <div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Greg Mason<br class="">Michigan State University<br class="">Institute for Cyber-Enabled Research<br class="">High Performance Computing Center<br class=""><br class="">web: <a href="http://www.hpcc.msu.edu" class="">www.hpcc.msu.edu</a><br class="">email: <a href="mailto:gmason@msu.edu" class="">gmason@msu.edu</a><br class=""></div></div> </div> <br class=""><div><blockquote type="cite" class=""><div class="">On Oct 24, 2014, at 11:35 AM, Ian Kaufman <<a href="mailto:ikaufman@eng.ucsd.edu" class="">ikaufman@eng.ucsd.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">I wonder if Nexenta has added more/better support for IB? I believe<br class="">that Illumian 1.0 is finally out (after having run the Alpha for<br class="">years), and I know that they did have some interest in supporting IB<br class="">originally. If they have, then I would hope some of that support would<br class="">make it back upstream.<br class=""><br class="">Mayhaps I have an email to send out ...<br class=""><br class="">Ian<br class=""><br class="">On Fri, Oct 24, 2014 at 7:58 AM, Ian Kaufman <<a href="mailto:ikaufman@eng.ucsd.edu" class="">ikaufman@eng.ucsd.edu</a>> wrote:<br class=""><blockquote type="cite" class="">I use a QLogic 12300 with SM built in. I haven't had any issues,<br class="">saving my cluster frontend's cycles for other things.<br class=""><br class="">Ian<br class=""><br class="">On Fri, Oct 24, 2014 at 7:55 AM, Michael Rasmussen <<a href="mailto:mir@miras.org" class="">mir@miras.org</a>> wrote:<br class=""><blockquote type="cite" class="">On Fri, 24 Oct 2014 13:17:39 +0200<br class="">Johan Kragsterman <<a href="mailto:johan.kragsterman@capvert.se" class="">johan.kragsterman@capvert.se</a>> wrote:<br class=""><br class=""><blockquote type="cite" class=""><br class="">Well, I can only find one person in that thread that BELIEVES that running SM on a switch is preferable to run OpenSM software. He hardly comes with any documentation of this, and I can't see any other that agrees...<br class=""><br class="">But of coarse, if you provide too little resources for OpenSM it will cause problems, or if you have any problems with the S/W or the configuration...<br class=""><br class=""></blockquote>I think the issue here is that OpenSM is a single threaded application<br class="">so when the CPU which is assigned to it maxes out performance will<br class="">drop. So I guess it comes down to a trade-off between the<br class="">performance of the host CPU and the performance of the switch hardware.<br class=""><br class="">--<br class="">Hilsen/Regards<br class="">Michael Rasmussen<br class=""><br class="">Get my public GnuPG keys:<br class="">michael <at> rasmussen <dot> cc<br class=""><a href="http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E" class="">http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E</a><br class="">mir <at> datanom <dot> net<br class="">http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C<br class="">mir <at> miras <dot> org<br class="">http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917<br class="">--------------------------------------------------------------<br class="">/usr/games/fortune -es says:<br class="">The default Magic Word, "Abracadabra", actually is a corruption of the<br class="">Hebrew phrase "ha-Bracha dab'ra" which means "pronounce the blessing".<br class=""><br class="">_______________________________________________<br class="">OmniOS-discuss mailing list<br class="">OmniOS-discuss@lists.omniti.com<br class="">http://lists.omniti.com/mailman/listinfo/omnios-discuss<br class=""><br class=""></blockquote><br class=""><br class=""><br class="">--<br class="">Ian Kaufman<br class="">Research Systems Administrator<br class="">UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu<br class=""></blockquote><br class=""><br class=""><br class="">-- <br class="">Ian Kaufman<br class="">Research Systems Administrator<br class="">UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu<br class="">_______________________________________________<br class="">OmniOS-discuss mailing list<br class=""><a href="mailto:OmniOS-discuss@lists.omniti.com" class="">OmniOS-discuss@lists.omniti.com</a><br class="">http://lists.omniti.com/mailman/listinfo/omnios-discuss<br class=""></div></blockquote></div><br class=""></div></div></div></body></html>