<div dir="ltr"><div><br></div>I was at about 6 months of uptime, then added some new SSD's for cache to the motherboard SATA ports. They weren't hot-plug recognized, so I rebooted over the weekend. Added the caches, all seemed good.<div>
<br></div><div>Five days later, the system was locked. No kernel panic, just a frozen console and no network access. Not ping-able.</div><div><br></div><div>Looking through the logs, I saw mostly just the typical (and benign?) netatalk messages:</div>
<div><br></div><div>------</div><div>
<p class="">mDNSResponder: [ID 702911 daemon.error] ERROR: getOptRdata - unknown opt 4</p><p class="">
</p><p class="">mDNSResponder: [ID 702911 daemon.error] Correcting TTL from 4500 to 3600 for 312 nexus </p><p class="">-------</p><p class=""><br></p><p class="">Etc.</p><p class="">But also, something new right before the crash:</p>
<p class=""><br></p><p class="">------</p><p class="">Mar 7 11:18:28 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb3 link down</p><p class="">Mar 7 11:18:28 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb4 link down</p>
<p class="">Mar 7 11:18:28 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb2 link down</p><p class="">Mar 7 11:18:28 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb5 link down</p>
<p class="">Mar 7 11:18:28 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: aggr1000 link down</p><p class="">Mar 7 11:18:30 colossus mac: [ID 435574 <a href="http://kern.info">kern.info</a>] NOTICE: igb3 link up, 1000 Mbps, full duplex</p>
<p class="">Mar 7 11:18:30 colossus mac: [ID 435574 <a href="http://kern.info">kern.info</a>] NOTICE: aggr1000 link up, 1000 Mbps, full duplex</p><p class="">Mar 7 11:18:30 colossus mac: [ID 435574 <a href="http://kern.info">kern.info</a>] NOTICE: igb2 link up, 1000 Mbps, full duplex</p>
<p class="">Mar 7 11:18:30 colossus mac: [ID 435574 <a href="http://kern.info">kern.info</a>] NOTICE: igb4 link up, 1000 Mbps, full duplex</p><p class="">Mar 7 11:18:30 colossus mac: [ID 435574 <a href="http://kern.info">kern.info</a>] NOTICE: igb5 link up, 1000 Mbps, full duplex</p>
<p class="">Mar 7 11:18:35 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb3 link down</p><p class="">Mar 7 11:18:35 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb4 link down</p>
<p class="">Mar 7 11:18:35 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb2 link down</p><p class="">Mar 7 11:18:36 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: igb5 link down</p>
<p class="">
</p><p class="">Mar 7 11:18:36 colossus mac: [ID 486395 <a href="http://kern.info">kern.info</a>] NOTICE: aggr1000 link down</p><p class="">------</p><p class=""><br></p><p class="">This goes on indefinitely, interfaces going down, coming up, over and over. All of the igb interfaces listed here are part of an aggregate group (although it's actually called aggr1, not aggr1000?). The other interfaces (2 additional igb's and 4 ixgbe's) did not log error messages, but by this point the server is unresponsive via ssh over the network and at the console. Interesting however, is that established file-sharing connections over the unaffected interfaces continue to function for quite a whole after the lockup, all night in one case. This includes AFP, SMB, and iSCSI (giving me enough time to shut down my virtual machines and log off some key clients). In other words, the zpools are functional, and so are enough services to keep that particular type of access alive. Establishing new connections over those protocols after the incident doesn't appear to be possible. A hard reboot is necessary to regain access to the console and permit new connections.</p>
<p class="">My initial thought was that it could be an issue with the switch, but that seems unlikely because I have other LACP groups that are unaffected. I'm also thinking that it can't be a coincidence that this only started happening right after that initial reboot?</p>
<p class="">Since that reboot, this crash has happened three times. The first, as I noted, was five days after the reconfiguration, but now they seem to be happening slightly more frequently, although they're always several days apart.</p>
<p class="">I'm considering reverting to a base install and rebuilding the system config this weekend, as it's very basic.. but still curious if anyone has seen this type of behavior before.<br></p><p class="">Regards,</p>
<p class="">Chris</p></div></div>