<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Am 17.01.17 um 17:37 schrieb Dale
Ghent:<br>
</div>
<blockquote
cite="mid:A5DE7D1A-2629-48FB-9BB8-FA1EFF8F9E17@omniti.com"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">On Jan 17, 2017, at 11:31 AM, Stephan Budach <a class="moz-txt-link-rfc2396E" href="mailto:stephan.budach@JVM.DE"><stephan.budach@JVM.DE></a> wrote:
Hi Dale,
Am 17.01.17 um 17:22 schrieb Dale Ghent:
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">On Jan 17, 2017, at 11:12 AM, Stephan Budach <a class="moz-txt-link-rfc2396E" href="mailto:stephan.budach@JVM.DE"><stephan.budach@JVM.DE></a>
wrote:
Hi guys,
I am sorry, but I do have to undig this old topic, since I do now have three hosts running omniOS 018/020, which show these pesky issues with flapping their ixgbeN links on my Nexus FEXes…
Does anyone know, if there has any change been made to the ixgbe drivers since 06/2016?
</pre>
</blockquote>
<pre wrap="">Since June 2016? Yes! A large update to the ixgbe driver happened in August. This added X550 support, and also brought the Intel Shared Code it uses from its 2012 vintage up to current. The updated driver is available in 014 and later.
/dale
</pre>
</blockquote>
<pre wrap="">
do you know of any option to get to know, why three of my boxes are flapping their 10GbE ports? It's actually not only when in aggr mode, but on single use as well. Last week I presumeably had one of my RSF-1 nodes panic, since it couldn't get to it's iSCSI LUNs anymore. The thing ist, that somewhere doen the line, the ixgbe driver seems to be fine, to configure one port to 1GbE instead of 10GbE, which will stop the flapping, but wich will break the VPC on my Nexus nevertheless.
In syslog, this looks like this:
...
Jan 17 14:46:07 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe1 link up, 1000 Mbps, full duplex
Jan 17 14:46:21 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Jan 17 14:46:22 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe3 link up, 10000 Mbps, full duplex
Jan 17 14:46:22 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Jan 17 14:46:26 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe3 link up, 10000 Mbps, full duplex
Jan 17 14:52:22 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Jan 17 14:52:22 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe3 link up, 10000 Mbps, full duplex
Jan 17 14:52:22 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Jan 17 14:52:32 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe3 link up, 10000 Mbps, full duplex
Jan 17 14:54:50 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Jan 17 14:54:55 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe3 link up, 10000 Mbps, full duplex
Jan 17 14:58:12 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Jan 17 14:58:16 zfsha02gh79 mac: [ID 435574 kern.info] NOTICE: ixgbe3 link up, 10000 Mbps, full duplex
Jan 17 14:59:46 zfsha02gh79 mac: [ID 486395 kern.info] NOTICE: ixgbe3 link down
Note on 14:46:07, where the system settles on a 1GbE connection…
</pre>
</blockquote>
<pre wrap="">
Sounds like a cabling issue? Are the runs too long or are you not using CAT6a? It flapping at 10Gb and then settling at 1Gb would indicate a cabling issue to me. The driver will always try to link at the fastest speed that the local controller and the remote peer will negotiate at... it will not proactively downgrade the link speed. If that happens, it is because that is what the controller managed to negotiate with the remote peer at.
Are you using jumbo frames or anything outside of a normal 1500mtu link?
/dale</pre>
</blockquote>
The cables are actually specifically purchased cat6 cables. They run
about 2m, not more. It could be tna cables, but I am running a
couple of those and afaik, I only get these issues on these three
nodes. I can try some other cables, but I hoped to be able to get
maybe some kind of debug messages from the driver.<br>
<br>
Thanks,<br>
Stephan<br>
</body>
</html>