<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>On Aug 21, 2013, at 3:53 PM, <a href="mailto:steve@linuxsuite.org">steve@linuxsuite.org</a> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><blockquote type="cite">SATA drives behind SAS expanders have a pathological error case. If a<br>drive encounters errors and it needs to be reset then the entire set of<br>drives will take the reset. If any io is in flight this will error as a<br>result often causing another reset. The result can best be described as a<br>cascade failure.<br><br>There may be things software can do to mitigate this but experience is<br>that it doesn't.<br></blockquote><br> Hmmm..... I have put.<br><br> allow-bus-device-reset=0;<br><br> In sd.conf<br><br> as it seemed to mitigate that problem?<br></blockquote><div><br></div><div>IIRC, this is a nop for mpt and mptsas drivers.</div><div> -- richard</div><br><blockquote type="cite"><br><br><blockquote type="cite"><br>So just because the array is working fine now does not mean that it won't<br>fail tragically when the first problems occur.<br><br>I have personal experience with this failure mode.<br><br>As a result I strongly discourage the use of SATA drives unless they are<br>directly connected to the hba without any expanders or port multipliers.<br><br>You ignore this advice at your own risk. Don't penny pinch on the drives<br> It ALWAYS costs you in the long run.<br><br></blockquote><br> The advice in this thread, wrt SATA vs SAS, has been my thoughts<br>and position<br>for a while based on my initial research. I was required to explore and<br>verify this option<br>simply because of the significant economic benefit if it was possible. <br>Thanks for your<br>input. I will use this to make my case<br><br> So now I just need some cheap SAS drives ;-)<br><br> thanx - steve<br><br><blockquote type="cite">Sent from my iPhone<br><br>On Aug 19, 2013, at 4:25 PM, <a href="mailto:steve@linuxsuite.org">steve@linuxsuite.org</a> wrote:<br><br><blockquote type="cite"><blockquote type="cite">Can't agree more here. Desktop firmwares are designed to try harder to<br>never return an error. While this is what home users want, it's an<br>anathema to large configurations with redundancy where you would prefer<br>to<br>just get the error so you can handle it - usually by doing the op on<br>another drive.<br><br>Use enterprise drives if you love your data and your ability to access<br>it.<br></blockquote><br> Yes I know, I know.... Desktop hardware sucks. Didn't know there<br>was<br>Enterprise (ie none Desktop) SATA. Perhaps the solution is to partition<br>the data and do "deep" storage that never gets read on large bulk SATA<br>and<br>then have a smaller SAS pool for the often read/written stuff.<br><br> Anyway, large storage rollouts will always be looking for<br>the cheapest solution. Even if it isn't perfect. I do backups<br>to a SATA pool (behind SAS expanders) with zfs send,.. no problems<br>errors<br>whatever YET!!<br>(cross fingers)<br><br> -steve<br><br><blockquote type="cite"><br>Sent from my iPhone<br><br>On Aug 19, 2013, at 2:47 PM, Eric Sproul <<a href="mailto:esproul@omniti.com">esproul@omniti.com</a>> wrote:<br><br><blockquote type="cite">On Mon, Aug 19, 2013 at 5:35 PM, <<a href="mailto:steve@linuxsuite.org">steve@linuxsuite.org</a>> wrote:<br><blockquote type="cite"> 4T SATA here<br><br><a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16822178338">http://www.newegg.com/Product/Product.aspx?Item=N82E16822178338</a><br><br> are $179<br></blockquote><br>That's a desktop drive, not nearline enterprise, so it's<br>apples-oranges. You get what you pay for. Desktop drives can take<br>much, much longer to respond to commands, leading the HBA/expander to<br>declare them dead and reset them, and the rest is history.<br><br>Avoid desktop parts. :)<br><br>Eric<br>_______________________________________________<br>OmniOS-discuss mailing list<br><a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>http://lists.omniti.com/mailman/listinfo/omnios-discuss<br></blockquote></blockquote><br>_______________________________________________<br>OmniOS-discuss mailing list<br><a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>http://lists.omniti.com/mailman/listinfo/omnios-discuss<br></blockquote><br></blockquote><br><br>_______________________________________________<br>OmniOS-discuss mailing list<br><a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>http://lists.omniti.com/mailman/listinfo/omnios-discuss<br></blockquote></div><br><div apple-content-edited="true"> <div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">--</div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br></div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><a href="mailto:Richard.Elling@RichardElling.com">Richard.Elling@RichardElling.com</a><br>+1-760-896-4422<br><br><br></div></span></div></span></div></span></div></span></div> </div> <br></body></html>