<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 08/10/13 13:24, Narayan Desai wrote:<br>
</div>
<blockquote
cite="mid:CABweQmKo7s6J1BNp0B2ZW1=sFqVBoNMW+sq9Qg3G1FvXdbNMbw@mail.gmail.com"
type="cite">
<div dir="ltr">I actually hadn't posted a full description of our
systems. Here is prtconf output.
<div><br>
</div>
<div>The system is a:</div>
<div>
<div>System Configuration: Supermicro X8DAH</div>
<div>BIOS Configuration: American Megatrends Inc. 2.1
12/30/2011</div>
</div>
<div>a pile of LSI 92xx controllers (mix of internal and
external)</div>
<div>Mellanox ConnectX 2 nic</div>
<div>Intel gigabit (igb)</div>
<div><br>
</div>
<div>Someone suggested that this might be some sort of ACPI
problem at one point. Does that seem reasonable?</div>
<div> -nld</div>
<div><br>
</div>
</div>
</blockquote>
<br>
I had some 'luck' when I disabled the following in the BIOS:<br>
Advanced->ACPI Settings->ACPI Sleep State = Suspend Disabled<br>
<br>
By 'luck' I mean it rebooted without hanging once. Subsequent
reboots kept hanging on 'rebooting...'.<br>
<br>
I then made the following BIOS config changes:<br>
<br>
Advanced->CPU Power Management Configuration-> C1E Support =
Disabled<br>
CPU C3 Report = Disabled<br>
CPU C6 Report = Disabled<br>
CPU C7 Report = Disabled
<br>
<br>
But that didn't stop the reboot hang.<br>
<br>
Would it have anything to do with NUMA?<br>
<br>
<br>
t.<br>
<br>
<blockquote
cite="mid:CABweQmKo7s6J1BNp0B2ZW1=sFqVBoNMW+sq9Qg3G1FvXdbNMbw@mail.gmail.com"
type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Oct 7, 2013 at 9:13 PM, Garrett
D'Amore <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:garrett.damore@dey-sys.com" target="_blank">garrett.damore@dey-sys.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><br>
<div>
<div class="im">
<div>On Oct 7, 2013, at 6:08 PM, Narayan Desai <<a
moz-do-not-send="true"
href="mailto:narayan.desai@gmail.com"
target="_blank">narayan.desai@gmail.com</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">So, specifically wrt the mellanox
cards, we have four identical systems with
mellanox cards running with the hermon driver. One
is running OI 151a3, and reboots properly. The
ones running a recent omnios don't with fastboot.
Just one data point.</div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
</div>
Oh, that's interesting. Would be interesting to see
what's different. I doubt its the hermon driver. More
suspicious of the Intel ethernet now.</div>
<span class="HOEnZb"><font color="#888888">
<div>
<br>
</div>
<div><span style="white-space:pre-wrap"> </span>-
Garrett</div>
</font></span>
<div>
<div class="h5">
<div><br>
<blockquote type="cite">
<div dir="ltr">
<div>
-nld</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Oct 7, 2013 at
7:10 PM, Garrett D'Amore <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:garrett.damore@dey-sys.com"
target="_blank">garrett.damore@dey-sys.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><br>
<div>
<div>
<div>On Oct 7, 2013, at 4:29 PM, Tom
Robinson <<a
moz-do-not-send="true"
href="mailto:tom.robinson@motec.com.au"
target="_blank">tom.robinson@motec.com.au</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div bgcolor="#FFFFFF"
text="#000000">
<div><tt>Hi Garret,<br>
<br>
Thanks for your message.<br>
<br>
The host configuration is as
follows:<br>
<br>
Supermicro X9DRi-F<br>
</tt><tt><tt>256GB RAM (16x
Hynix 16GB ECC Reg. DDR3
1600MHz)<br>
</tt>2 x Intel Xeon E5-2620<br>
2 x Intel SSD 320 80GB (rpool)<br>
4 x STEC Enterprise S842 200GB
(ARC)<br>
1 x STEC ZeusRAM 8GB 3.5" SAS
SSD (ZIL)<br>
1 x LSI SAS 9207-8i (internal
drives)<br>
1 x Intel Ethernet Server
Adapter X520-DA2, Dual Port
10Gbps SFP+ Direct Attach
Copper, PCI-e 2.0 5GT/s 1<br>
2 x Mellanox ConnectX®-2 VPI<br>
2 x LSI SAS 9207-8e (JBODS)<br>
</tt></div>
</div>
</blockquote>
<div><br>
</div>
</div>
I'd be suspicious of the Mellanox cards.
Are these the hermon driver? It looks
like there is an attempt to do the right
thing for those drivers, but… I don't
know if I believe it all works properly.
The other cards should be fine.</div>
<div><br>
</div>
<div><span style="white-space:pre-wrap"> </span>-
Garrett</div>
<div><br>
<blockquote type="cite">
<div>
<div>
<div bgcolor="#FFFFFF"
text="#000000">
<div><tt> <br>
That is connected via
external SAS to two JBODS
containing 28 x 1TB disks
each for mirrored zfs vdevs.<br>
<br>
The output from prtconf -vp
is attached as it's very
long (2042 lines).<br>
<br>
Kind regards,<br>
Tom<br>
<br>
</tt>On 08/10/13 02:06,
Garrett D'Amore wrote:<br>
</div>
<blockquote type="cite"> If
you're seeing hangs like this,
I would appreciate knowing the
hardware configuration.
Prtconf -vp might be helpful.
Presumably this is the result
of one or more devices not
doing the right thing for
quiesce().
<div><br>
</div>
<div><span
style="white-space:pre-wrap">
</span>- Garrett</div>
<div><br>
<div>
<div>On Oct 7, 2013, at
4:34 AM, Narayan Desai
<<a
moz-do-not-send="true"
href="mailto:narayan.desai@gmail.com" target="_blank">narayan.desai@gmail.com</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">This is
caused by the system
attempting to use
fastboot on default
reboot. We've disabled
that and things seem
to work properly; the
following commands do
the trick. (the first
changes reboot not to
use fastboot, the
second causes the
system to do a full
reboot upon panic)
<div> -nld</div>
<div><br>
</div>
<div>
<pre style="margin-top:10px;margin-bottom:10px;color:rgb(85,85,85);font-size:12px;line-height:18px"># svccfg -s "system/boot-config:default" setprop config/fastreboot_default=false
# svcadm refresh svc:/system/boot-config:default
</pre>
<pre style="margin-top:10px;margin-bottom:10px;color:rgb(85,85,85);font-size:12px;line-height:18px"><pre style="margin-top:10px;margin-bottom:10px"># svccfg -s "system/boot-config:default" setprop config/fastreboot_onpanic=false
# svcadm refresh svc:/system/boot-config:default</pre></pre>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Sun, Oct 6, 2013 at
11:26 PM, Tom
Robinson <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:tom.robinson@motec.com.au" target="_blank">tom.robinson@motec.com.au</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0
0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">Hi,<br>
<br>
I'm running OmniOS
r151006 on a
SuperMicro X9
motherboard.<br>
<br>
when I do:<br>
<br>
reboot -- -r<br>
<br>
The system hangs
on the
'rebooting...'
message.<br>
<br>
I've disabled
Suspend and
C-states in the
BIOS. I had one
successful
'reboot' but now
it's hanging<br>
again.<br>
<br>
Anyone have any
clues on how to
fix this?<br>
<br>
Kind regards,<br>
<br>
--<br>
<br>
Tom Robinson<br>
IT Manager/System
Administrator<br>
<br>
MoTeC Pty Ltd<br>
<br>
121 Merrindale
Drive<br>
Croydon South<br>
3136 Victoria<br>
Australia<br>
<br>
T: <a
moz-do-not-send="true"
href="tel:%2B61%203%209761%205050" value="+61397615050" target="_blank">+61
3 9761 5050</a><br>
F: <a
moz-do-not-send="true"
href="tel:%2B61%203%209761%205051" value="+61397615051" target="_blank">+61
3 9761 5051</a><br>
E: <a
moz-do-not-send="true"
href="mailto:tom.robinson@motec.com.au" target="_blank">tom.robinson@motec.com.au</a><br>
<br>
<br>
<br>
_______________________________________________<br>
OmniOS-discuss
mailing list<br>
<a
moz-do-not-send="true"
href="mailto:OmniOS-discuss@lists.omniti.com" target="_blank">OmniOS-discuss@lists.omniti.com</a><br>
<a
moz-do-not-send="true"
href="http://lists.omniti.com/mailman/listinfo/omnios-discuss"
target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
<br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
OmniOS-discuss mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:OmniOS-discuss@lists.omniti.com" target="_blank">OmniOS-discuss@lists.omniti.com</a><br>
<a
moz-do-not-send="true"
href="http://lists.omniti.com/mailman/listinfo/omnios-discuss"
target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<span><prtconf-vp.out></span></blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>