From danmcd at omniti.com Mon Jan 4 13:02:43 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 4 Jan 2016 08:02:43 -0500 Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: References: Message-ID: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> > On Dec 28, 2015, at 5:50 PM, Bob Friesenhahn wrote: > > Due to the existing problem, I don't think that there are existing dependencies to worry about. It is clear that for existing release branches, gcc-5-runtime can not add libraries without the risk of an application not running because the gcc-5-runtime vintage is too old. This is perhaps not so much of a problem since OmniOS systems should be updating regularly. > > The gomp library is a bit large so some users might be happier if it was optional via its own package and not part of the OmniOS baseline. A theoretical gomp package might be a variant of gcc-5-runtime: bloody(~)[0]% pkg contents gcc-5-runtime PATH usr/lib/amd64 usr/lib/amd64/libgcc_s.so usr/lib/amd64/libgcc_s.so.1 usr/lib/libgcc_s.so usr/lib/libgcc_s.so.1 bloody(~)[0]% pkg contents gcc51 | grep libgcc opt/gcc-5.1.0/lib/amd64/libgcc_s.so opt/gcc-5.1.0/lib/amd64/libgcc_s.so.1 opt/gcc-5.1.0/lib/gcc/i386-pc-solaris2.11/5.1.0/amd64/libgcc.a opt/gcc-5.1.0/lib/gcc/i386-pc-solaris2.11/5.1.0/amd64/libgcc_eh.a opt/gcc-5.1.0/lib/gcc/i386-pc-solaris2.11/5.1.0/libgcc.a opt/gcc-5.1.0/lib/gcc/i386-pc-solaris2.11/5.1.0/libgcc_eh.a opt/gcc-5.1.0/lib/libgcc-unwind.map opt/gcc-5.1.0/lib/libgcc_s.so opt/gcc-5.1.0/lib/libgcc_s.so.1 bloody(~)[0]% digest -a md5 /opt/gcc-5.1.0/lib/amd64/libgcc_s.so 341b9d1d3c5bb99196a57712725fd8a2 bloody(~)[0]% digest -a md5 /usr/lib/amd64/libgcc_s.so 341b9d1d3c5bb99196a57712725fd8a2 bloody(~)[0]% pkg contents gcc51 | grep gomp opt/gcc-5.1.0/lib/amd64/libgomp-plugin-host_nonshm.so opt/gcc-5.1.0/lib/amd64/libgomp-plugin-host_nonshm.so.1 opt/gcc-5.1.0/lib/amd64/libgomp-plugin-host_nonshm.so.1.0.0 opt/gcc-5.1.0/lib/amd64/libgomp.a opt/gcc-5.1.0/lib/amd64/libgomp.so opt/gcc-5.1.0/lib/amd64/libgomp.so.1 opt/gcc-5.1.0/lib/amd64/libgomp.so.1.0.0 opt/gcc-5.1.0/lib/amd64/libgomp.spec opt/gcc-5.1.0/lib/libgomp-plugin-host_nonshm.so opt/gcc-5.1.0/lib/libgomp-plugin-host_nonshm.so.1 opt/gcc-5.1.0/lib/libgomp-plugin-host_nonshm.so.1.0.0 opt/gcc-5.1.0/lib/libgomp.a opt/gcc-5.1.0/lib/libgomp.so opt/gcc-5.1.0/lib/libgomp.so.1 opt/gcc-5.1.0/lib/libgomp.so.1.0.0 opt/gcc-5.1.0/lib/libgomp.spec opt/gcc-5.1.0/share/info/libgomp.info bloody(~)[0]% Note how gcc-5-runtime is just a copy of what's in /opt/gcc-5.1.0 ? One could do the same for libgomp. 2016's already shaping up to be swamped, but maybe that'll get in at some point. Dan From danmcd at omniti.com Mon Jan 4 13:48:11 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 4 Jan 2016 08:48:11 -0500 Subject: [OmniOS-discuss] OmniOS r151016 zone has difficulties shutting down In-Reply-To: References: <536501D2-EA96-4F6B-8CB2-39A0F9698267@omniti.com> Message-ID: <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> > On Dec 31, 2015, at 10:45 AM, Bob Friesenhahn wrote: > > I am accessing the system via ssh which uses '~.' to terminate the ssh session. This is also the default shutdown sequence for 'zlogin -C'. The idea is that after doing some initial administration using 'zlogin -C', the "~." sequence was used to quit it. This terminates the ssh session rather than the zone console login. While subsequent 'zlogin -C' works, it may be that the unclean/violent termination of the zone console login has left behind residue which prevents the zone from shutting down. Dumb questions: which ssh client, and which ssh server? It's highly possible the process you have simply backgrounds once ssh quits. Knowing which server + client combo may help... especially given that you now have OpenSSH doing a lot more in recent OmniOS updates. Dan From bfriesen at simple.dallas.tx.us Mon Jan 4 14:23:16 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Mon, 4 Jan 2016 08:23:16 -0600 (CST) Subject: [OmniOS-discuss] OmniOS r151016 zone has difficulties shutting down In-Reply-To: <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> References: <536501D2-EA96-4F6B-8CB2-39A0F9698267@omniti.com> <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> Message-ID: On Mon, 4 Jan 2016, Dan McDonald wrote: > >> On Dec 31, 2015, at 10:45 AM, Bob Friesenhahn wrote: >> >> I am accessing the system via ssh which uses '~.' to terminate the ssh session. This is also the default shutdown sequence for 'zlogin -C'. The idea is that after doing some initial administration using 'zlogin -C', the "~." sequence was used to quit it. This terminates the ssh session rather than the zone console login. While subsequent 'zlogin -C' works, it may be that the unclean/violent termination of the zone console login has left behind residue which prevents the zone from shutting down. > > Dumb questions: which ssh client, and which ssh server? > > It's highly possible the process you have simply backgrounds once ssh quits. Knowing which server + client combo may help... especially given that you now have OpenSSH doing a lot more in recent OmniOS updates. I no longer believe that the problem is due to hung state associated with a ssh session since I have since seen it several times using 'zlogin' without -C and using 'exit' to quit the session. Regardless, this is the answer to your question: On Solaris 10 client system: Sun_SSH_1.1.5, SSH protocols 1.5/2.0, OpenSSL 0x0090704f On OmniOS system Sun_SSH_1.5, SSH protocols 1.5/2.0, OpenSSL 0x1000205f As far as I am aware, the problem has not happened without first having done a 'zlogin' to the zone. Also, as far as I am aware, the problem afflicts the most recently created zone (until at least system reboot) but it is also not consistent in that sometimes the shutdown succeeds and sometimes it fails. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From bfriesen at simple.dallas.tx.us Mon Jan 4 14:43:15 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Mon, 4 Jan 2016 08:43:15 -0600 (CST) Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> Message-ID: On Mon, 4 Jan 2016, Dan McDonald wrote: > bloody(~)[0]% pkg contents gcc51 | grep gomp > opt/gcc-5.1.0/lib/amd64/libgomp-plugin-host_nonshm.so > opt/gcc-5.1.0/lib/amd64/libgomp-plugin-host_nonshm.so.1 > opt/gcc-5.1.0/lib/amd64/libgomp-plugin-host_nonshm.so.1.0.0 > opt/gcc-5.1.0/lib/amd64/libgomp.a > opt/gcc-5.1.0/lib/amd64/libgomp.so > opt/gcc-5.1.0/lib/amd64/libgomp.so.1 > opt/gcc-5.1.0/lib/amd64/libgomp.so.1.0.0 > opt/gcc-5.1.0/lib/amd64/libgomp.spec > opt/gcc-5.1.0/lib/libgomp-plugin-host_nonshm.so > opt/gcc-5.1.0/lib/libgomp-plugin-host_nonshm.so.1 > opt/gcc-5.1.0/lib/libgomp-plugin-host_nonshm.so.1.0.0 > opt/gcc-5.1.0/lib/libgomp.a > opt/gcc-5.1.0/lib/libgomp.so > opt/gcc-5.1.0/lib/libgomp.so.1 > opt/gcc-5.1.0/lib/libgomp.so.1.0.0 > opt/gcc-5.1.0/lib/libgomp.spec > opt/gcc-5.1.0/share/info/libgomp.info > bloody(~)[0]% > > Note how gcc-5-runtime is just a copy of what's in /opt/gcc-5.1.0 ? > One could do the same for libgomp. 2016's already shaping up to be > swamped, but maybe that'll get in at some point. Yes, but skip the .a files. :-) Due to the libgomp availability issue, packages targeting OmniOS which normally support OpenMP have OpenMP specifically disabled. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From jimklimov at cos.ru Mon Jan 4 15:30:35 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Mon, 04 Jan 2016 16:30:35 +0100 Subject: [OmniOS-discuss] OmniOS r151016 zone has difficulties shutting down In-Reply-To: <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> References: <536501D2-EA96-4F6B-8CB2-39A0F9698267@omniti.com> <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> Message-ID: <0E8786F4-44FD-44A5-8F69-B5A8EB02812C@cos.ru> 4 ?????? 2016??. 14:48:11 CET, Dan McDonald ?????: > >> On Dec 31, 2015, at 10:45 AM, Bob Friesenhahn > wrote: >> >> I am accessing the system via ssh which uses '~.' to terminate the >ssh session. This is also the default shutdown sequence for 'zlogin >-C'. The idea is that after doing some initial administration using >'zlogin -C', the "~." sequence was used to quit it. This terminates >the ssh session rather than the zone console login. While subsequent >'zlogin -C' works, it may be that the unclean/violent termination of >the zone console login has left behind residue which prevents the zone >from shutting down. > >Dumb questions: which ssh client, and which ssh server? > >It's highly possible the process you have simply backgrounds once ssh >quits. Knowing which server + client combo may help... especially >given that you now have OpenSSH doing a lot more in recent OmniOS >updates. > >Dan > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Just in case, also note that you can pass the sequence through compliant clients by stacking e.g. ~~. (after a newline generally so everyone picks it up). So you might want to end ssh+zlogin sessions with ENTER ~~. ~. if only to see if this behaves differently. Other things that might block a zone are processes with CWD under zone root (e.g. managing its files from GZ with a Midnight Commander session); maybe the abruptly dead zlogin leaves some lock of this sort... Jim -- Typos courtesy of K-9 Mail on my Samsung Android From danmcd at omniti.com Mon Jan 4 15:57:55 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 4 Jan 2016 10:57:55 -0500 Subject: [OmniOS-discuss] Happy New Year & Happy New libxml2 Message-ID: <8831C2B7-3A08-4CD1-8CF2-25F0949BBD81@omniti.com> There's a security-related fix for libxml2. It's not a security hole for running OmniOS services per se as much as it is for other consumers of libxml2. Please utter "pkg update" on your LTS (r151014) or Stable (r151016) systems. Thanks, Dan From janus at volny.cz Mon Jan 4 20:29:57 2016 From: janus at volny.cz (Jan Vlach) Date: Mon, 4 Jan 2016 21:29:57 +0100 Subject: [OmniOS-discuss] OmniOS r151016 zone has difficulties shutting down In-Reply-To: <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> References: <536501D2-EA96-4F6B-8CB2-39A0F9698267@omniti.com> <9B83EFAC-D5C9-4000-8647-0FAF7004033C@omniti.com> Message-ID: <20160104202957.GB8382@volny.cz> Hi, you can setup ad-hoc override for ssh, so the escape would be sent to zlogin or you can send ~~. to send it to the "inner session". If you overdo it with the tildes, just press enter and try again with one less. override the escape sequence for ssh (man ssh): -e escape_char Sets the escape character for sessions with a pty (default: ?~?). The escape character is only recognized at the beginning of a line. The escape character followed by a dot (?.?) closes the connection; followed by control-Z suspends the connection; and followed by itself sends the escape character once. Setting the character to ?none? disables any escapes and makes the session fully transparent. Jan On Mon, Jan 04, 2016 at 08:48:11AM -0500, Dan McDonald wrote: > > > On Dec 31, 2015, at 10:45 AM, Bob Friesenhahn wrote: > > > > I am accessing the system via ssh which uses '~.' to terminate the ssh session. This is also the default shutdown sequence for 'zlogin -C'. The idea is that after doing some initial administration using 'zlogin -C', the "~." sequence was used to quit it. This terminates the ssh session rather than the zone console login. While subsequent 'zlogin -C' works, it may be that the unclean/violent termination of the zone console login has left behind residue which prevents the zone from shutting down. > > Dumb questions: which ssh client, and which ssh server? > > It's highly possible the process you have simply backgrounds once ssh quits. Knowing which server + client combo may help... especially given that you now have OpenSSH doing a lot more in recent OmniOS updates. > > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Be the change you want to see in the world. From danmcd at omniti.com Mon Jan 4 21:40:46 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 4 Jan 2016 16:40:46 -0500 Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> Message-ID: <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> > On Jan 4, 2016, at 9:43 AM, Bob Friesenhahn wrote: > > Due to the libgomp availability issue, packages targeting OmniOS which normally support OpenMP have OpenMP specifically disabled. If this were a bigger thing, I'd worry about design-on-the-fly, but would it be perhaps reasonable to place libgomp in with the existing gcc-5-runtime? Or should it be its own? We could go either way on this, I think. If it becomes its own, the name would be likely named "system/library/gomp" or ".../libgomp". Opinions welcome! Dan From bfriesen at simple.dallas.tx.us Mon Jan 4 21:53:26 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Mon, 4 Jan 2016 15:53:26 -0600 (CST) Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> Message-ID: On Mon, 4 Jan 2016, Dan McDonald wrote: > >> On Jan 4, 2016, at 9:43 AM, Bob Friesenhahn wrote: >> >> Due to the libgomp availability issue, packages targeting OmniOS which normally support OpenMP have OpenMP specifically disabled. > > If this were a bigger thing, I'd worry about design-on-the-fly, but > would it be perhaps reasonable to place libgomp in with the existing > gcc-5-runtime? Or should it be its own? We could go either way on > this, I think. If it becomes its own, the name would be likely > named "system/library/gomp" or ".../libgomp". While OmniOS observes a 'KYSTY' philosophy, the default created OmniOS zone size still seems rather large. Avoiding expanding this size should be part of the decision criteria. Other than that, including it in gcc-5-runtime would make things easier for packagers and avoid additional moving parts. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Mon Jan 4 21:55:16 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 4 Jan 2016 16:55:16 -0500 Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> Message-ID: > On Jan 4, 2016, at 4:53 PM, Bob Friesenhahn wrote: > > While OmniOS observes a 'KYSTY' philosophy, the default created OmniOS zone size still seems rather large. That can get fixed once we reform "entire". I was thinking that if it didn't get included in gcc-5-runtime it'd NOT be in "entire". > Avoiding expanding this size should be part of the decision criteria. Other than that, including it in gcc-5-runtime would make things easier for packagers and avoid additional moving parts. And you've illuminated why it's not an obvious decision. Hence my putting the question out here. Dan From henson at acm.org Mon Jan 4 22:11:36 2016 From: henson at acm.org (Paul B. Henson) Date: Mon, 04 Jan 2016 14:11:36 -0800 Subject: [OmniOS-discuss] usb ups support for nut Message-ID: <077601d1473c$e159bea0$a40d3be0$@acm.org> Is there any updated status on getting nut working with usb-based UPSes under omnios? It is so easy under linux, you just plug them in, and you're basically done :). The last I looked for solaris/illumos, it involved arcane wizardry that scared me :). I just replaced a UPS that had a serial interface with a newer one that only has USB, it would be nice to be able to monitor it. Thanks. From gary at genashor.com Mon Jan 4 22:16:07 2016 From: gary at genashor.com (Gary Gendel) Date: Mon, 04 Jan 2016 17:16:07 -0500 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <077601d1473c$e159bea0$a40d3be0$@acm.org> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> Message-ID: <20160104221607.5816398.54360.2443@genashor.com> Try apcups. It works fine for me. ? Original Message ? From: Paul B. Henson Sent: Monday, January 4, 2016 5:13 PM To: 'omnios-discuss' Subject: [OmniOS-discuss] usb ups support for nut Is there any updated status on getting nut working with usb-based UPSes under omnios? It is so easy under linux, you just plug them in, and you're basically done :). The last I looked for solaris/illumos, it involved arcane wizardry that scared me :). I just replaced a UPS that had a serial interface with a newer one that only has USB, it would be nice to be able to monitor it. Thanks. _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From henson at acm.org Mon Jan 4 22:38:58 2016 From: henson at acm.org (Paul B. Henson) Date: Mon, 04 Jan 2016 14:38:58 -0800 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <20160104221607.5816398.54360.2443@genashor.com> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> Message-ID: <077a01d14740$b3c55c70$1b501550$@acm.org> Hmm, it's not an APC UPS :)? Unless it has changed since the last time I looked at it, apcups was pretty much dedicated to APC hardware Also, I thought the underlying issue was with the illumos usb drivers, not the UPS management software layer? > -----Original Message----- > From: Gary Gendel [mailto:gary at genashor.com] > Sent: Monday, January 04, 2016 2:16 PM > To: Paul B. Henson ; 'omnios-discuss' discuss at lists.omniti.com> > Subject: Re: [OmniOS-discuss] usb ups support for nut > > Try apcups. It works fine for me. > > ? Original Message > From: Paul B. Henson > Sent: Monday, January 4, 2016 5:13 PM > To: 'omnios-discuss' > Subject: [OmniOS-discuss] usb ups support for nut > > Is there any updated status on getting nut working with usb-based UPSes > under omnios? It is so easy under linux, you just plug them in, and you're > basically done :). The last I looked for solaris/illumos, it involved arcane > wizardry that scared me :). I just replaced a UPS that had a serial > interface with a newer one that only has USB, it would be nice to be able to > monitor it. > > Thanks. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From gary at genashor.com Mon Jan 4 22:58:59 2016 From: gary at genashor.com (Gary Gendel) Date: Mon, 04 Jan 2016 17:58:59 -0500 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <077a01d14740$b3c55c70$1b501550$@acm.org> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> <077a01d14740$b3c55c70$1b501550$@acm.org> Message-ID: <20160104225859.5816398.10140.2451@genashor.com> Apcups has been generalized for other UPS manufacturers but YMMV. You do need the libusb and libugen drivers installed. They are in the hipster repository. ? Original Message ? From: Paul B. Henson Sent: Monday, January 4, 2016 5:39 PM To: 'Gary Gendel'; 'omnios-discuss' Subject: RE: [OmniOS-discuss] usb ups support for nut Hmm, it's not an APC UPS :)? Unless it has changed since the last time I looked at it, apcups was pretty much dedicated to APC hardware Also, I thought the underlying issue was with the illumos usb drivers, not the UPS management software layer? > -----Original Message----- > From: Gary Gendel [mailto:gary at genashor.com] > Sent: Monday, January 04, 2016 2:16 PM > To: Paul B. Henson ; 'omnios-discuss' discuss at lists.omniti.com> > Subject: Re: [OmniOS-discuss] usb ups support for nut > > Try apcups. It works fine for me. > > ? Original Message > From: Paul B. Henson > Sent: Monday, January 4, 2016 5:13 PM > To: 'omnios-discuss' > Subject: [OmniOS-discuss] usb ups support for nut > > Is there any updated status on getting nut working with usb-based UPSes > under omnios? It is so easy under linux, you just plug them in, and you're > basically done :). The last I looked for solaris/illumos, it involved arcane > wizardry that scared me :). I just replaced a UPS that had a serial > interface with a newer one that only has USB, it would be nice to be able to > monitor it. > > Thanks. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From henson at acm.org Tue Jan 5 01:00:55 2016 From: henson at acm.org (Paul B. Henson) Date: Mon, 4 Jan 2016 17:00:55 -0800 Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> Message-ID: <871A6C36-A202-4996-96B2-9B3170B4383D@acm.org> Ah, bloating the total package count vs bloating the default installed file system, decisions, decisions :). I think on this one I'd vote for a separate package. > On Jan 4, 2016, at 1:55 PM, Dan McDonald wrote: > > >> On Jan 4, 2016, at 4:53 PM, Bob Friesenhahn wrote: >> >> While OmniOS observes a 'KYSTY' philosophy, the default created OmniOS zone size still seems rather large. > > That can get fixed once we reform "entire". I was thinking that if it didn't get included in gcc-5-runtime it'd NOT be in "entire". > >> Avoiding expanding this size should be part of the decision criteria. Other than that, including it in gcc-5-runtime would make things easier for packagers and avoid additional moving parts. > > And you've illuminated why it's not an obvious decision. Hence my putting the question out here. > > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From henson at acm.org Tue Jan 5 03:20:13 2016 From: henson at acm.org (Paul B. Henson) Date: Mon, 04 Jan 2016 19:20:13 -0800 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <20160104225859.5816398.10140.2451@genashor.com> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> <077a01d14740$b3c55c70$1b501550$@acm.org> <20160104225859.5816398.10140.2451@genashor.com> Message-ID: <07bb01d14767$fe32ca00$fa985e00$@acm.org> > From: Gary Gendel > Sent: Monday, January 04, 2016 2:59 PM > > Apcups has been generalized for other UPS manufacturers but YMMV. Hmm, I wasn't aware of that; I switched to nut a while ago. > You do need the libusb and libugen drivers installed. They are in the hipster > repository. hipster is basically the OI beta or alpha repository? I'm not sure I really want to install device driver packages from another distributions dev repository on a production server 8-/. Upstream still doesn't even officially support Solaris/illumos, so these packages are basically just the stuff pulled along from the old Sun opensolaris port? Dan, any chance you might chime in on this with any thoughts? Are these packages something that would be suitable for pulling into omnios itself? I'd feel a lot more confident installing them from the omnios stable repo :). Thanks. From richard at netbsd.org Tue Jan 5 15:21:07 2016 From: richard at netbsd.org (Richard PALO) Date: Tue, 05 Jan 2016 16:21:07 +0100 Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: <871A6C36-A202-4996-96B2-9B3170B4383D@acm.org> References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> <871A6C36-A202-4996-96B2-9B3170B4383D@acm.org> Message-ID: <568BDF63.8040204@netbsd.org> Le 05/01/16 02:00, Paul B. Henson a ?crit : > Ah, bloating the total package count vs bloating the default installed file system, decisions, decisions :). > > I think on this one I'd vote for a separate package. > I might suggest parity with the other packagers of it.. taking a look at oi-userland and solaris-userland both install it with the default c runtime libraries. It *is* easier dealing with fewer packages for standard gcc runtime libraries... as to size, even the .a is way under a meg (uncompressed). Who doesn't compress their BE's? -- Richard PALO From danmcd at omniti.com Tue Jan 5 16:10:49 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 5 Jan 2016 11:10:49 -0500 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <07bb01d14767$fe32ca00$fa985e00$@acm.org> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> <077a01d14740$b3c55c70$1b501550$@acm.org> <20160104225859.5816398.10140.2451@genashor.com> <07bb01d14767$fe32ca00$fa985e00$@acm.org> Message-ID: <683E3820-833F-4622-AEAD-07D1B2450D55@omniti.com> > On Jan 4, 2016, at 10:20 PM, Paul B. Henson wrote: > > Dan, any chance you might chime in on this with any thoughts? Are these > packages something that would be suitable for pulling into omnios itself? > I'd feel a lot more confident installing them from the omnios stable repo > :). If I merely invoked the spirit of OmniOS's KTSTY, my initial answer is "no". This seems like something, however, that may be useful in a built-but-not-actually-supported repo like ms.omniti.com (i.e. where we share tools OmniTI itself uses internally). However, with this month being what it is (another customer-biting ZFS bug, FOSDEM at the end of the month, a trip to HQ, and setting plans to kick off work on LX), I'm not sure if I'd be able to bring it up right now anyway. Sorry, Dan From bhildebrandt at exegy.com Tue Jan 5 19:15:46 2016 From: bhildebrandt at exegy.com (Hildebrandt, Bill) Date: Tue, 5 Jan 2016 19:15:46 +0000 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <683E3820-833F-4622-AEAD-07D1B2450D55@omniti.com> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> <077a01d14740$b3c55c70$1b501550$@acm.org> <20160104225859.5816398.10140.2451@genashor.com> <07bb01d14767$fe32ca00$fa985e00$@acm.org> <683E3820-833F-4622-AEAD-07D1B2450D55@omniti.com> Message-ID: ". . . customer-biting ZFS bug". Can you elaborate? -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Dan McDonald Sent: Tuesday, January 05, 2016 10:11 AM To: Paul B. Henson; Dan McDonald Cc: omnios-discuss Subject: Re: [OmniOS-discuss] usb ups support for nut > On Jan 4, 2016, at 10:20 PM, Paul B. Henson wrote: > > Dan, any chance you might chime in on this with any thoughts? Are > these packages something that would be suitable for pulling into omnios itself? > I'd feel a lot more confident installing them from the omnios stable > repo :). If I merely invoked the spirit of OmniOS's KTSTY, my initial answer is "no". This seems like something, however, that may be useful in a built-but-not-actually-supported repo like ms.omniti.com (i.e. where we share tools OmniTI itself uses internally). However, with this month being what it is (another customer-biting ZFS bug, FOSDEM at the end of the month, a trip to HQ, and setting plans to kick off work on LX), I'm not sure if I'd be able to bring it up right now anyway. Sorry, Dan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ________________________________ This e-mail and any documents accompanying it may contain legally privileged and/or confidential information belonging to Exegy, Inc. Such information may be protected from disclosure by law. The information is intended for use by only the addressee. If you are not the intended recipient, you are hereby notified that any disclosure or use of the information is strictly prohibited. If you have received this e-mail in error, please immediately contact the sender by e-mail or phone regarding instructions for return or destruction and do not use or disclose the content to others. From danmcd at omniti.com Tue Jan 5 19:21:40 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 5 Jan 2016 14:21:40 -0500 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> <077a01d14740$b3c55c70$1b501550$@acm.org> <20160104225859.5816398.10140.2451@genashor.com> <07bb01d14767$fe32ca00$fa985e00$@acm.org> <683E3820-833F-4622-AEAD-07D1B2450D55@omniti.com> Message-ID: <7B5FF3B9-CF14-4940-A06E-EC80C693D0C9@omniti.com> > On Jan 5, 2016, at 2:15 PM, Hildebrandt, Bill wrote: > > ". . . customer-biting ZFS bug". Can you elaborate? illumos 4986 doesn't solve all of the cases of refquota + incremental sends/receives. The new one looks more like a space accounting problem. I'd recommend looking at the illumos ZFS or the OpenZFS developer list for that ongoing conversation. Dan From henson at acm.org Tue Jan 5 21:39:07 2016 From: henson at acm.org (Paul B. Henson) Date: Tue, 05 Jan 2016 13:39:07 -0800 Subject: [OmniOS-discuss] usb ups support for nut In-Reply-To: <683E3820-833F-4622-AEAD-07D1B2450D55@omniti.com> References: <077601d1473c$e159bea0$a40d3be0$@acm.org> <20160104221607.5816398.54360.2443@genashor.com> <077a01d14740$b3c55c70$1b501550$@acm.org> <20160104225859.5816398.10140.2451@genashor.com> <07bb01d14767$fe32ca00$fa985e00$@acm.org> <683E3820-833F-4622-AEAD-07D1B2450D55@omniti.com> Message-ID: <084901d14801$81eee7e0$85ccb7a0$@acm.org> > From: Dan McDonald > Sent: Tuesday, January 05, 2016 8:11 AM > > If I merely invoked the spirit of OmniOS's KTSTY, my initial answer is "no". Do you mean nut itself, or the supporting usb/ugen pieces? I would agree about nut/acupsd, but libusb/libugen at least to me seem like part of an os distribution. They are basic libraries, no? I know you have to draw the line somewhere, but there are already a number of general-purpose libraries part of the core distribution. And given that these two are illumos specific and you can't even compile them yourself from upstream sources, they seem like pretty good candidates for just being included :). > However, with this month being what it is (another customer-biting ZFS bug, Yes, definitely focus on that customer biting bug :), as when I wear my $DAYJOB hat it is very annoying ;). Thanks. From wonko at 4amlunch.net Wed Jan 6 02:59:38 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Tue, 5 Jan 2016 21:59:38 -0500 Subject: [OmniOS-discuss] COMSTAR hanging Message-ID: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> So this is the second time this has happened to me. The COMSTAR layer appears to be getting hung. At first I thought it was just the IB/SRP target stuff, but the iSCSI target also stops working. So far the only solution I?ve found is a reboot. This is very concerning and I?d like to try to get it figured out. The next time it happens, what is the best course of action in order to get the information you all need to debug this? I?m assuming force a crashdump, but is there anything else I could be doing? Thanks! -brian PS: Latest OmniOS-stable From garrett at damore.org Wed Jan 6 04:45:27 2016 From: garrett at damore.org (Garrett D'Amore) Date: Tue, 5 Jan 2016 20:45:27 -0800 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> Message-ID: Actually, what is probably the most useful is this command: # echo ?$ wrote: > So this is the second time this has happened to me. The COMSTAR layer > appears to be getting hung. At first I thought it was just the IB/SRP > target stuff, but the iSCSI target also stops working. So far the only > solution I?ve found is a reboot. > > This is very concerning and I?d like to try to get it figured out. > > The next time it happens, what is the best course of action in order to > get the information you all need to debug this? I?m assuming force a > crashdump, but is there anything else I could be doing? > > Thanks! > > -brian > > PS: Latest OmniOS-stable > > ------------------------------------------- > illumos-discuss > Archives: https://www.listbox.com/member/archive/182180/=now > RSS Feed: > https://www.listbox.com/member/archive/rss/182180/22003744-9012f59c > Modify Your Subscription: > https://www.listbox.com/member/?member_id=22003744&id_secret=22003744-e9cd8436 > Powered by Listbox: http://www.listbox.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Wed Jan 6 14:51:41 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 6 Jan 2016 08:51:41 -0600 (CST) Subject: [OmniOS-discuss] OmniOS and OpenMP In-Reply-To: <568BDF63.8040204@netbsd.org> References: <6DA5A2E8-78BB-4A2C-AEAB-14E6FB9840A0@omniti.com> <4C18D604-E4F2-4993-817C-ACE5952606E3@omniti.com> <871A6C36-A202-4996-96B2-9B3170B4383D@acm.org> <568BDF63.8040204@netbsd.org> Message-ID: On Tue, 5 Jan 2016, Richard PALO wrote: > Le 05/01/16 02:00, Paul B. Henson a ?crit : >> Ah, bloating the total package count vs bloating the default installed file system, decisions, decisions :). >> >> I think on this one I'd vote for a separate package. >> > > I might suggest parity with the other packagers of it.. taking a look at oi-userland and solaris-userland > both install it with the default c runtime libraries. It *is* easier dealing with fewer packages for > standard gcc runtime libraries... as to size, even the .a is way under a meg (uncompressed). > Who doesn't compress their BE's? This is a very good point. I notice that OI and Solaris seem to try to use the same package names. SFE packages (and other packages) need to list their dependencies, which may include gcc-5-runtime. If OmniOS provides different components in gcc-5-runtime from other distributions then this serves as an additional hurdle to being able to re-use existing package build configurations (which work for OpenIndiana and Solaris) on OmniOS. OmniOS currently suffers from a "package gap" caused by its reduced server package offerings as compared with OpenIndiana, Solaris 10, Solaris 11, and Linux distributions. There are a profusion of "shovel ready" packages ready to be built once this gap is addressed. OmniOS does not need to (and should not) fill the gap, but it should not make it more difficult for others to do so. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From wonko at 4amlunch.net Wed Jan 6 20:29:49 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Wed, 6 Jan 2016 15:29:49 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> Message-ID: <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> Great, look for one in the future when this happens again. :) -brian > On Jan 5, 2016, at 11:45 PM, Garrett D'Amore wrote: > > Actually, what is probably the most useful is this command: > > # echo ?$ > A full crashdump will have that inside it, as well, but that first list of threads (and therefore will include the comstar threads) and backtraces will probably yield the most fruit for least effort on your part. > > On Tue, Jan 5, 2016 at 6:59 PM, Brian Hechinger > wrote: > So this is the second time this has happened to me. The COMSTAR layer appears to be getting hung. At first I thought it was just the IB/SRP target stuff, but the iSCSI target also stops working. So far the only solution I?ve found is a reboot. > > This is very concerning and I?d like to try to get it figured out. > > The next time it happens, what is the best course of action in order to get the information you all need to debug this? I?m assuming force a crashdump, but is there anything else I could be doing? > > Thanks! > > -brian > > PS: Latest OmniOS-stable > > ------------------------------------------- > illumos-discuss > Archives: https://www.listbox.com/member/archive/182180/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182180/22003744-9012f59c > Modify Your Subscription: https://www.listbox.com/member/?member_id=22003744&id_secret=22003744-e9cd8436 > Powered by Listbox: http://www.listbox.com > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug at will.to Wed Jan 6 21:54:54 2016 From: doug at will.to (Doug Hughes) Date: Wed, 6 Jan 2016 16:54:54 -0500 Subject: [OmniOS-discuss] release 16 package dist flummoxed? Message-ID: <568D8D2E.80202@will.to> (this is a new thing. I've updated a bunch of other machines last week) root at xyr-3-3-1:/root# /usr/bin/pkg unset-publisher omnios icy=require-signatures -g http://pkg.omniti.com/omnios/r151016/ omniosature-pol 1-0.151016-3-1:/rot# /usr/bin/pkg update --be-name=omnios-r151016 entire at 11,5.1 Creating Plan /Traceback (most recent call last): File "/usr/bin/pkg", line 6002, in handle_errors __ret = func(*args, **kwargs) File "/usr/bin/pkg", line 5985, in main_func pargs=pargs, **opts) File "/usr/bin/pkg", line 2344, in update reject_list=reject_pats, update_index=update_index) File "/usr/bin/pkg", line 1575, in __api_op rv = __api_plan_exception(_op, _noexecute, _verbose, _api_inst) File "/usr/bin/pkg", line 1557, in __api_op for pd in api_plan_func(**kwargs): File "/usr/lib/python2.6/vendor-packages/pkg/client/api.py", line 1118, in __plan_op log_op_end_all=True) File "/usr/lib/python2.6/vendor-packages/pkg/client/api.py", line 1072, in __plan_op self._img.make_update_plan(**kwargs) File "/usr/lib/python2.6/vendor-packages/pkg/client/image.py", line 4244, in make_update_plan reject_list=reject_list) File "/usr/lib/python2.6/vendor-packages/pkg/client/image.py", line 4123, in __make_plan_common self.__call_imageplan_evaluate(ip) File "/usr/lib/python2.6/vendor-packages/pkg/client/image.py", line 4054, in __call_imageplan_evaluate ip.evaluate() File "/usr/lib/python2.6/vendor-packages/pkg/client/imageplan.py", line 2151, in evaluate self.evaluate_pkg_plans() File "/usr/lib/python2.6/vendor-packages/pkg/client/imageplan.py", line 2272, in evaluate_pkg_plans self.__get_manifest(oldfmri, old_in), File "/usr/lib/python2.6/vendor-packages/pkg/client/imageplan.py", line 2056, in __get_manifest intent=intent) File "/usr/lib/python2.6/vendor-packages/pkg/client/image.py", line 2466, in get_manifest intent=intent, alt_pub=alt_pub) File "/usr/lib/python2.6/vendor-packages/pkg/client/image.py", line 2423, in __get_manifest pathname=self.get_manifest_path(fmri)) File "/usr/lib/python2.6/vendor-packages/pkg/manifest.py", line 981, in __init__ self.__load() File "/usr/lib/python2.6/vendor-packages/pkg/manifest.py", line 995, in __load self.set_content(excludes=self.excludes, pathname=self.pathname) File "/usr/lib/python2.6/vendor-packages/pkg/manifest.py", line 563, in set_content raise apx._convert_error(e) IOError: [Errno 5] I/O error pkg: This is an internal error in pkg(5) version 1408388823. Please log a Service Request about this issue including the information above and this message. repeatable. From danmcd at omniti.com Wed Jan 6 23:29:14 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 6 Jan 2016 18:29:14 -0500 Subject: [OmniOS-discuss] release 16 package dist flummoxed? In-Reply-To: <568D8D2E.80202@will.to> References: <568D8D2E.80202@will.to> Message-ID: <50F91B93-06A9-4216-B8E0-93829731523B@omniti.com> > On Jan 6, 2016, at 4:54 PM, Doug Hughes wrote: > > File "/usr/lib/python2.6/vendor-packages/pkg/manifest.py", line 563, in set_content > raise apx._convert_error(e) > IOError: [Errno 5] I/O error This stack is weird. Line 563 is a hash function. Why it would return EIO is interesting. Also, the line numbers in your python stack don't seem to match up with anything in the source. Finally, your command seems to be garbled: > root at xyr-3-3-1:/root# /usr/bin/pkg unset-publisher omnios > icy=require-signatures -g http://pkg.omniti.com/omnios/r151016/ omniosature-pol > 1-0.151016-3-1:/rot# /usr/bin/pkg update --be-name=omnios-r151016 entire at 11,5.1 Can you resend the output without what appears to be copy/paste-in-middle problems? Thanks, Dan From doug at will.to Thu Jan 7 00:50:37 2016 From: doug at will.to (Doug Hughes) Date: Wed, 6 Jan 2016 19:50:37 -0500 Subject: [OmniOS-discuss] release 16 package dist flummoxed? In-Reply-To: <50F91B93-06A9-4216-B8E0-93829731523B@omniti.com> References: <568D8D2E.80202@will.to> <50F91B93-06A9-4216-B8E0-93829731523B@omniti.com> Message-ID: <568DB65D.2070803@will.to> DOH.. I tried this multiple times and didn't notice the clear symptoms of paste error... (pasting from 'more' -- bbbbbaaaaaaddd) thanks for the clue-by-4. all good. On 1/6/2016 6:29 PM, Dan McDonald wrote: >> On Jan 6, 2016, at 4:54 PM, Doug Hughes wrote: >> >> File "/usr/lib/python2.6/vendor-packages/pkg/manifest.py", line 563, in set_content >> raise apx._convert_error(e) >> IOError: [Errno 5] I/O error > This stack is weird. Line 563 is a hash function. Why it would return EIO is interesting. > > Also, the line numbers in your python stack don't seem to match up with anything in the source. > > Finally, your command seems to be garbled: > >> root at xyr-3-3-1:/root# /usr/bin/pkg unset-publisher omnios >> icy=require-signatures -g http://pkg.omniti.com/omnios/r151016/ omniosature-pol >> 1-0.151016-3-1:/rot# /usr/bin/pkg update --be-name=omnios-r151016 entire at 11,5.1 > Can you resend the output without what appears to be copy/paste-in-middle problems? > > Thanks, > Dan > From gordon at hafnia.dk Fri Jan 8 08:27:38 2016 From: gordon at hafnia.dk (Gordon Selisek) Date: Fri, 08 Jan 2016 09:27:38 +0100 Subject: [OmniOS-discuss] Setting LUN properties Message-ID: <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> Hello, and happy new year to everyone. Can anybody enlighten me on how to set any unique LUN property, that I'm able to retrieve on the Initiator using Win32_DiskDrive ? You know like on Solaris 11.2 create-lu [?p, ?-lu-prop logical-unit-property=val ?-s, ?-size size] lu-file Btw, I'm on r151016 Cordial regards Gordon -------------- next part -------------- An HTML attachment was scrubbed... URL: From matej at zunaj.si Fri Jan 8 14:36:05 2016 From: matej at zunaj.si (Matej Zerovnik) Date: Fri, 8 Jan 2016 15:36:05 +0100 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> Message-ID: I had the same problems? In my case, comstart hanging went away with downgrade to early 014 version, but then ZFS started to lock. What is your hardware? Any JBODs? SAS or SATA drives? Expanders? Currently, I didn?t had Comstar lock for about a month, and I?m running latest 014 (but I did reduce the number of users for about 25%, so maybe I removed some some of the problematic users). Matej > On 06 Jan 2016, at 21:29, Brian Hechinger wrote: > > Great, look for one in the future when this happens again. :) > > -brian > >> On Jan 5, 2016, at 11:45 PM, Garrett D'Amore > wrote: >> >> Actually, what is probably the most useful is this command: >> >> # echo ?$> >> A full crashdump will have that inside it, as well, but that first list of threads (and therefore will include the comstar threads) and backtraces will probably yield the most fruit for least effort on your part. >> >> On Tue, Jan 5, 2016 at 6:59 PM, Brian Hechinger > wrote: >> So this is the second time this has happened to me. The COMSTAR layer appears to be getting hung. At first I thought it was just the IB/SRP target stuff, but the iSCSI target also stops working. So far the only solution I?ve found is a reboot. >> >> This is very concerning and I?d like to try to get it figured out. >> >> The next time it happens, what is the best course of action in order to get the information you all need to debug this? I?m assuming force a crashdump, but is there anything else I could be doing? >> >> Thanks! >> >> -brian >> >> PS: Latest OmniOS-stable >> >> ------------------------------------------- >> illumos-discuss >> Archives: https://www.listbox.com/member/archive/182180/=now >> RSS Feed: https://www.listbox.com/member/archive/rss/182180/22003744-9012f59c >> Modify Your Subscription: https://www.listbox.com/member/?member_id=22003744&id_secret=22003744-e9cd8436 >> Powered by Listbox: http://www.listbox.com >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From wonko at 4amlunch.net Fri Jan 8 15:23:35 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Fri, 8 Jan 2016 10:23:35 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> Message-ID: <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> ZFS lockups don?t seem much better an option. :) Hardware is a Xeon box with 6x SATA disks in a raid10 wired straight to the controller. No expanders. This is not a heavily loaded system (yet) so I don?t think it?s a load issue, sadly. I *might* be able to downgrade to 014? I?m not sure I want to. I?d rather help get things fixed going forward. This is for my home VMware stack. I have important services on disks local to the ESXi hosts (not idea, but makes this less painful when it happens) so COMSTAR locking up is mostly an inconvenience yet at this point. I?d rather it didn?t though because I?d like to be using this more heavily than it is now. Thanks for the data points! -brian > On Jan 8, 2016, at 9:36 AM, Matej Zerovnik wrote: > > I had the same problems? > > In my case, comstart hanging went away with downgrade to early 014 version, but then ZFS started to lock. > > What is your hardware? Any JBODs? SAS or SATA drives? Expanders? > > Currently, I didn?t had Comstar lock for about a month, and I?m running latest 014 (but I did reduce the number of users for about 25%, so maybe I removed some some of the problematic users). > > Matej > > >> On 06 Jan 2016, at 21:29, Brian Hechinger > wrote: >> >> Great, look for one in the future when this happens again. :) >> >> -brian >> >>> On Jan 5, 2016, at 11:45 PM, Garrett D'Amore > wrote: >>> >>> Actually, what is probably the most useful is this command: >>> >>> # echo ?$>> >>> A full crashdump will have that inside it, as well, but that first list of threads (and therefore will include the comstar threads) and backtraces will probably yield the most fruit for least effort on your part. >>> >>> On Tue, Jan 5, 2016 at 6:59 PM, Brian Hechinger > wrote: >>> So this is the second time this has happened to me. The COMSTAR layer appears to be getting hung. At first I thought it was just the IB/SRP target stuff, but the iSCSI target also stops working. So far the only solution I?ve found is a reboot. >>> >>> This is very concerning and I?d like to try to get it figured out. >>> >>> The next time it happens, what is the best course of action in order to get the information you all need to debug this? I?m assuming force a crashdump, but is there anything else I could be doing? >>> >>> Thanks! >>> >>> -brian >>> >>> PS: Latest OmniOS-stable >>> >>> ------------------------------------------- >>> illumos-discuss >>> Archives: https://www.listbox.com/member/archive/182180/=now >>> RSS Feed: https://www.listbox.com/member/archive/rss/182180/22003744-9012f59c >>> Modify Your Subscription: https://www.listbox.com/member/?member_id=22003744&id_secret=22003744-e9cd8436 >>> Powered by Listbox: http://www.listbox.com >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Fri Jan 8 15:34:21 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 8 Jan 2016 10:34:21 -0500 Subject: [OmniOS-discuss] Setting LUN properties In-Reply-To: <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> References: <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> Message-ID: > On Jan 8, 2016, at 3:27 AM, Gordon Selisek wrote: > > > Hello, and happy new year to everyone. > > Can anybody enlighten me on how to set any unique LUN property, that I'm able to retrieve on the Initiator using Win32_DiskDrive ? > > You know like on Solaris 11.2 > > create-lu [?p, ?-lu-prop logical-unit-property=val ?-s, ?-size size] lu-file > > Btw, I'm on r151016 bloody(~)[1]% stmfadm create-lu -\? Usage: stmfadm create-lu [OPTIONS] OPTIONS: -p, --lu-prop -s, --size . . . Looks like you'd do it that way. Dan From danmcd at omniti.com Fri Jan 8 16:13:38 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 8 Jan 2016 11:13:38 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> Message-ID: > On Jan 8, 2016, at 10:23 AM, Brian Hechinger wrote: > > Hardware is a Xeon box with 6x SATA disks in a raid10 wired straight to the controller. No expanders. > You're using hardware raid?!? That could, to be honest, be part of the problem. Dan From wonko at 4amlunch.net Fri Jan 8 16:22:42 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Fri, 8 Jan 2016 11:22:42 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> Message-ID: <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> No, ZFS raid10 -brian > On Jan 8, 2016, at 11:13 AM, Dan McDonald wrote: > > >> On Jan 8, 2016, at 10:23 AM, Brian Hechinger wrote: >> >> Hardware is a Xeon box with 6x SATA disks in a raid10 wired straight to the controller. No expanders. >> > > You're using hardware raid?!? That could, to be honest, be part of the problem. > > Dan > > > > ------------------------------------------- > illumos-discuss > Archives: https://www.listbox.com/member/archive/182180/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182180/27886204-900409ed > Modify Your Subscription: https://www.listbox.com/member/?member_id=27886204&id_secret=27886204-941ea7c7 > Powered by Listbox: http://www.listbox.com From danmcd at omniti.com Fri Jan 8 16:26:26 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 8 Jan 2016 11:26:26 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> Message-ID: <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> > On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: > > No, ZFS raid10 Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? As far as COMSTAR goes, there's been a nontrivial amount of work in Nexenta's public repo (illumos-nexenta), but nobody there or elsewhere has made the effort to upstream. Dan From johan.kragsterman at capvert.se Fri Jan 8 16:32:35 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Fri, 8 Jan 2016 17:32:35 +0100 Subject: [OmniOS-discuss] Ang: Re: Setting LUN properties In-Reply-To: References: , <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> Message-ID: Hi! -----"OmniOS-discuss" skrev: ----- Till: Gordon Selisek Fr?n: Dan McDonald S?nt av: "OmniOS-discuss" Datum: 2016-01-08 16:35 Kopia: "omnios-discuss at lists.omniti.com" ?rende: Re: [OmniOS-discuss] Setting LUN properties > On Jan 8, 2016, at 3:27 AM, Gordon Selisek wrote: > > > Hello, and happy new year to everyone. > > Can anybody enlighten me on how to set any unique LUN property, that I'm able to retrieve on the Initiator using Win32_DiskDrive ? > > You know like on Solaris 11.2 > > create-lu [–p, –-lu-prop logical-unit-property=val –-s, –-size size] lu-file > > Btw, I'm on r151016 bloody(~)[1]% stmfadm create-lu -\? Usage: ?stmfadm create-lu [OPTIONS] ?? ? ? ?OPTIONS: ?? ? ? ? ? ? ? ?-p, --lu-prop ? ?? ? ? ? ? ? ? ?-s, --size ? . ?. ?. Looks like you'd do it that way. Dan You just check out stmfadm admin commands on this site: http://docs.oracle.com/cd/E26502_01/html/E29031/stmfadm-1m.html#scrolltoc There are 13 different property possibilities listed there. /Johan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From dave-oo at pooserville.com Fri Jan 8 16:50:16 2016 From: dave-oo at pooserville.com (Dave Pooser) Date: Fri, 08 Jan 2016 10:50:16 -0600 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> Message-ID: >>On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >> >> No, ZFS raid10 > >Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? It's a zpool with multiple mirror vdevs. -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com From gordon at hafnia.dk Fri Jan 8 16:59:39 2016 From: gordon at hafnia.dk (Gordon Selisek) Date: Fri, 08 Jan 2016 17:59:39 +0100 Subject: [OmniOS-discuss] Ang: Re: Setting LUN properties In-Reply-To: References: , <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> Message-ID: <152222de6f2.11d812a1c19174.4693355061681301428@hafnia.dk> Dan + Johan, thank you, but the omnios man page says: sbdadm create-lu [-s, --size size] filename there are no -p option. and even if i use it, i get the error message create-lu: illegal option -- p i know my question might come over a bit amateurish, but it's legit ;-) so are there any other way to do it? ---- On Fri, 08 Jan 2016 17:32:35 +0100 Johan Kragsterman <johan.kragsterman at capvert.se>wrote ---- Hi! -----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: ----- Till: Gordon Selisek <gordon at hafnia.dk> Fr?n: Dan McDonald S?nt av: "OmniOS-discuss" Datum: 2016-01-08 16:35 Kopia: "omnios-discuss at lists.omniti.com" <omnios-discuss at lists.omniti.com> ?rende: Re: [OmniOS-discuss] Setting LUN properties > On Jan 8, 2016, at 3:27 AM, Gordon Selisek <gordon at hafnia.dk> wrote: > > > Hello, and happy new year to everyone. > > Can anybody enlighten me on how to set any unique LUN property, that I'm able to retrieve on the Initiator using Win32_DiskDrive ? > > You know like on Solaris 11.2 > > create-lu [&#8211;p, &#8211;-lu-prop logical-unit-property=val &#8211;-s, &#8211;-size size] lu-file > > Btw, I'm on r151016 bloody(~)[1]% stmfadm create-lu -\? Usage: stmfadm create-lu [OPTIONS] <lu file> OPTIONS: -p, --lu-prop <logical-unit-property=value> -s, --size <size K/M/G/T/P> . . . Looks like you'd do it that way. Dan You just check out stmfadm admin commands on this site: http://docs.oracle.com/cd/E26502_01/html/E29031/stmfadm-1m.html#scrolltoc There are 13 different property possibilities listed there. /Johan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From wonko at 4amlunch.net Fri Jan 8 17:10:52 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Fri, 8 Jan 2016 12:10:52 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> Message-ID: <5C0AFC96-07A4-4DEB-8514-E90398A1DDB8@4amlunch.net> Yeah, it doesn?t have a great name, really. raidz10? :) -brian > On Jan 8, 2016, at 11:50 AM, Dave Pooser wrote: > >>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>> >>> No, ZFS raid10 >> >> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? > > It's a zpool with multiple mirror vdevs. > > -- > Dave Pooser > Cat-Herder-in-Chief, Pooserville.com > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Fri Jan 8 17:12:48 2016 From: mir at miras.org (Michael Rasmussen) Date: Fri, 8 Jan 2016 18:12:48 +0100 Subject: [OmniOS-discuss] Ang: Re: Setting LUN properties In-Reply-To: <152222de6f2.11d812a1c19174.4693355061681301428@hafnia.dk> References: <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> <152222de6f2.11d812a1c19174.4693355061681301428@hafnia.dk> Message-ID: <20160108181248.04477bd5@sleipner.datanom.net> On Fri, 08 Jan 2016 17:59:39 +0100 Gordon Selisek wrote: > > sbdadm create-lu [-s, --size size] filename > > It is stmfadm. sbdadm is the old one. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Debian Hint #7: You can use the cron-apt package to do automatic nightly downloads of updates for packages installed on your system. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From matej at zunaj.si Fri Jan 8 17:24:55 2016 From: matej at zunaj.si (Matej Zerovnik) Date: Fri, 8 Jan 2016 18:24:55 +0100 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> Message-ID: <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> Which controller exactly do you have? Do you know firmware version? Which hard drives? It might not tell much, but it?s good to have as much information as possible. When comstar hangs, can you telnet to the iSCSI port? What does svcs says, is the service running? What happens in you try to restart it? How do you restart it? In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). Matej > On 08 Jan 2016, at 17:50, Dave Pooser wrote: > >>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>> >>> No, ZFS raid10 >> >> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? > > It's a zpool with multiple mirror vdevs. > > -- > Dave Pooser > Cat-Herder-in-Chief, Pooserville.com > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From gordon at hafnia.dk Fri Jan 8 17:37:35 2016 From: gordon at hafnia.dk (Gordon Selisek) Date: Fri, 08 Jan 2016 18:37:35 +0100 Subject: [OmniOS-discuss] Ang: Re: Setting LUN properties In-Reply-To: <20160108181248.04477bd5@sleipner.datanom.net> References: <1522059223e.d4ceb07113804.8502809651213649833@hafnia.dk> <152222de6f2.11d812a1c19174.4693355061681301428@hafnia.dk> <20160108181248.04477bd5@sleipner.datanom.net> Message-ID: <1522250a001.10857983824145.503236806192168161@hafnia.dk> thanks michael, allow me to externalize; i blame my poor dioptre for the mistake ;-) ---- On Fri, 08 Jan 2016 18:12:48 +0100 Michael Rasmussen <mir at miras.org>wrote ---- On Fri, 08 Jan 2016 17:59:39 +0100 Gordon Selisek <gordon at hafnia.dk> wrote: > > sbdadm create-lu [-s, --size size] filename > > It is stmfadm. sbdadm is the old one. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael <at> rasmussen <dot> cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir <at> datanom <dot> net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir <at> miras <dot> org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Debian Hint #7: You can use the cron-apt package to do automatic nightly downloads of updates for packages installed on your system. _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From wonko at 4amlunch.net Fri Jan 8 18:16:10 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Fri, 8 Jan 2016 13:16:10 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> Message-ID: <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> > Which controller exactly do you have? Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. > Do you know firmware version? I?m assuming this is linked to the BIOS version? > Which hard drives? Hitachi-HUA723030ALA640-MKAOAA50-2.73TB > It might not tell much, but it?s good to have as much information as possible. > > When comstar hangs, can you telnet to the iSCSI port? > What does svcs says, is the service running? > What happens in you try to restart it? > How do you restart it? I?ll try all these things next time. > In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). All good info. Thanks! -brian > > Matej > >> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >> >>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>> >>>> No, ZFS raid10 >>> >>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >> >> It's a zpool with multiple mirror vdevs. >> >> -- >> Dave Pooser >> Cat-Herder-in-Chief, Pooserville.com >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From rjahnel at ellipseinc.com Fri Jan 8 18:31:54 2016 From: rjahnel at ellipseinc.com (Richard Jahnel) Date: Fri, 8 Jan 2016 18:31:54 +0000 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> Message-ID: <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> First off, love SuperMicro good choice IMHO. This board has two on board controllers. LSI SAS1068E (not 100% sure there are working illumos drivers for this one) And Intel ICH10R SATA (So I'm guessing your using this one.) -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger Sent: Friday, January 08, 2016 12:16 PM To: Matej Zerovnik Cc: omnios-discuss Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging > Which controller exactly do you have? Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. > Do you know firmware version? I?m assuming this is linked to the BIOS version? > Which hard drives? Hitachi-HUA723030ALA640-MKAOAA50-2.73TB > It might not tell much, but it?s good to have as much information as possible. > > When comstar hangs, can you telnet to the iSCSI port? > What does svcs says, is the service running? > What happens in you try to restart it? > How do you restart it? I?ll try all these things next time. > In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). All good info. Thanks! -brian > > Matej > >> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >> >>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>> >>>> No, ZFS raid10 >>> >>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >> >> It's a zpool with multiple mirror vdevs. >> >> -- >> Dave Pooser >> Cat-Herder-in-Chief, Pooserville.com >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From wonko at 4amlunch.net Fri Jan 8 19:11:50 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Fri, 8 Jan 2016 14:11:50 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> Message-ID: Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. -brian > On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: > > First off, love SuperMicro good choice IMHO. > > This board has two on board controllers. > > LSI SAS1068E (not 100% sure there are working illumos drivers for this one) > > And > > Intel ICH10R SATA (So I'm guessing your using this one.) > > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger > Sent: Friday, January 08, 2016 12:16 PM > To: Matej Zerovnik > Cc: omnios-discuss > Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging > > >> Which controller exactly do you have? > > Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. > >> Do you know firmware version? > > I?m assuming this is linked to the BIOS version? > >> Which hard drives? > > Hitachi-HUA723030ALA640-MKAOAA50-2.73TB > >> It might not tell much, but it?s good to have as much information as possible. >> >> When comstar hangs, can you telnet to the iSCSI port? >> What does svcs says, is the service running? >> What happens in you try to restart it? >> How do you restart it? > > I?ll try all these things next time. > >> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). > > All good info. Thanks! > > -brian > >> >> Matej >> >>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>> >>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>> >>>>> No, ZFS raid10 >>>> >>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>> >>> It's a zpool with multiple mirror vdevs. >>> >>> -- >>> Dave Pooser >>> Cat-Herder-in-Chief, Pooserville.com >>> >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From matej at zunaj.si Fri Jan 8 20:11:45 2016 From: matej at zunaj.si (Matej Zerovnik) Date: Fri, 8 Jan 2016 21:11:45 +0100 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> Message-ID: Is the pool usable during comstar hang? Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. Matej > On 08 Jan 2016, at 20:11, Brian Hechinger wrote: > > Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. > > It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. > > -brian > >> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >> >> First off, love SuperMicro good choice IMHO. >> >> This board has two on board controllers. >> >> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >> >> And >> >> Intel ICH10R SATA (So I'm guessing your using this one.) >> >> -----Original Message----- >> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >> Sent: Friday, January 08, 2016 12:16 PM >> To: Matej Zerovnik >> Cc: omnios-discuss >> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >> >> >>> Which controller exactly do you have? >> >> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >> >>> Do you know firmware version? >> >> I?m assuming this is linked to the BIOS version? >> >>> Which hard drives? >> >> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >> >>> It might not tell much, but it?s good to have as much information as possible. >>> >>> When comstar hangs, can you telnet to the iSCSI port? >>> What does svcs says, is the service running? >>> What happens in you try to restart it? >>> How do you restart it? >> >> I?ll try all these things next time. >> >>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >> >> All good info. Thanks! >> >> -brian >> >>> >>> Matej >>> >>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>> >>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>> >>>>>> No, ZFS raid10 >>>>> >>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>> >>>> It's a zpool with multiple mirror vdevs. >>>> >>>> -- >>>> Dave Pooser >>>> Cat-Herder-in-Chief, Pooserville.com >>>> >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From wonko at 4amlunch.net Fri Jan 8 20:24:21 2016 From: wonko at 4amlunch.net (wonko at 4amlunch.net) Date: Fri, 8 Jan 2016 15:24:21 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> Message-ID: I don't remember if I checked that or not. I'll add it to the list. -brian > On Jan 8, 2016, at 15:11, Matej Zerovnik wrote: > > Is the pool usable during comstar hang? > Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). > > Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. > > Matej > >> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: >> >> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >> >> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >> >> -brian >> >>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >>> >>> First off, love SuperMicro good choice IMHO. >>> >>> This board has two on board controllers. >>> >>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>> >>> And >>> >>> Intel ICH10R SATA (So I'm guessing your using this one.) >>> >>> -----Original Message----- >>> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>> Sent: Friday, January 08, 2016 12:16 PM >>> To: Matej Zerovnik >>> Cc: omnios-discuss >>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>> >>> >>>> Which controller exactly do you have? >>> >>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>> >>>> Do you know firmware version? >>> >>> I?m assuming this is linked to the BIOS version? >>> >>>> Which hard drives? >>> >>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>> >>>> It might not tell much, but it?s good to have as much information as possible. >>>> >>>> When comstar hangs, can you telnet to the iSCSI port? >>>> What does svcs says, is the service running? >>>> What happens in you try to restart it? >>>> How do you restart it? >>> >>> I?ll try all these things next time. >>> >>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>> >>> All good info. Thanks! >>> >>> -brian >>> >>>> >>>> Matej >>>> >>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>>> >>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>>> >>>>>>> No, ZFS raid10 >>>>>> >>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>> >>>>> It's a zpool with multiple mirror vdevs. >>>>> >>>>> -- >>>>> Dave Pooser >>>>> Cat-Herder-in-Chief, Pooserville.com >>>>> >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > From rafibeyli at gmail.com Fri Jan 8 20:50:50 2016 From: rafibeyli at gmail.com (Hafiz Rafiyev) Date: Fri, 8 Jan 2016 22:50:50 +0200 (EET) Subject: [OmniOS-discuss] comstar illumos-nexenta upstream Message-ID: <1564189811.226153.1452286250826.JavaMail.zimbra@cantekstil.com.tr> Hello Dan, Anyway to help illumos-nexenta upstream?any voting or survey? It's really very important and useful part of omnios to sharing block zfs volumes. Hafiz. ------------------------------------------------------------------------- Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? As far as COMSTAR goes, there's been a nontrivial amount of work in Nexenta's public repo (illumos-nexenta), but nobody there or elsewhere has made the effort to upstream. Dan From danmcd at omniti.com Fri Jan 8 20:59:37 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 8 Jan 2016 15:59:37 -0500 Subject: [OmniOS-discuss] comstar illumos-nexenta upstream In-Reply-To: <1564189811.226153.1452286250826.JavaMail.zimbra@cantekstil.com.tr> References: <1564189811.226153.1452286250826.JavaMail.zimbra@cantekstil.com.tr> Message-ID: <7DCF9AB3-7F9D-4A39-8D67-73711E5BDECE@omniti.com> > On Jan 8, 2016, at 3:50 PM, Hafiz Rafiyev wrote: > > Hello Dan, > > Anyway to help illumos-nexenta upstream?any voting or survey? > It's really very important and useful part of omnios to sharing block zfs volumes. Good question. You're asking the wrong list about this, however. It's more of an illumos community question. Dan From doug at will.to Fri Jan 8 21:19:12 2016 From: doug at will.to (Doug Hughes) Date: Fri, 8 Jan 2016 16:19:12 -0500 Subject: [OmniOS-discuss] flow control with Intel x520da Message-ID: Interesting issue with flow control that I can't prove but suspect is ixgbe driver related. I wonder if anybody has seen this. The switch is Dell Force10 S4810. I have enabled bidirectional flow control on the port. (standard broadcomm chipset). If I turn on rx flow control in the driver by editing /kernel/drv/ixgbe.conf, everything works just fine. If I turn on bidir flow control in the driver, the port doesn't work at all. Instead, what I see on the switch is approximately 1000 multicast packets inbound per second (or something that registers as multicast that is probably some attempt at flow control). Running snoop on the ixgbe interface shows nothing at all in this mode. Not even standard multicast traffic from the switch. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Fri Jan 8 21:26:56 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 8 Jan 2016 16:26:56 -0500 Subject: [OmniOS-discuss] flow control with Intel x520da In-Reply-To: References: Message-ID: > On Jan 8, 2016, at 4:19 PM, Doug Hughes wrote: > > Interesting issue with flow control that I can't prove but suspect is ixgbe driver related. I wonder if anybody has seen this. The switch is Dell Force10 S4810. I have enabled bidirectional flow control on the port. (standard broadcomm chipset). > > If I turn on rx flow control in the driver by editing /kernel/drv/ixgbe.conf, everything works just fine. > > If I turn on bidir flow control in the driver, the port doesn't work at all. Instead, what I see on the switch is approximately 1000 multicast packets inbound per second (or something that registers as multicast that is probably some attempt at flow control). Running snoop on the ixgbe interface shows nothing at all in this mode. Not even standard multicast traffic from the switch. I've seen issues with flow-control before in ixgbe. Can you take this to the illumos developer's list, as it's better suited for the wider audience? Thanks, Dan From geoffn at gnaa.net Tue Jan 12 20:55:47 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Tue, 12 Jan 2016 12:55:47 -0800 Subject: [OmniOS-discuss] disable kvm to run Virtualbox Message-ID: <56956853.9040404@gnaa.net> Hi. What is the best way to remove/disable kvm from Omnos. It prevents Virtualbox from grabbing the VT-x. #pkg uninstall system/kvm Creating Plan (Solver setup): /pkg uninstall: Cannot remove 'pkg://omnios/system/kvm at 1.0.5.11,5.11-0.151016:20151112T214129Z' due to the following packages that depend on it: pkg://omnios/entire at 11,5.11-0.151016:20151202T161203Z I ended up adding "exclude: kvm" in the /etc/system file. Is there an more elegant solution? thanks, Geoff From danmcd at omniti.com Tue Jan 12 23:02:55 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 12 Jan 2016 18:02:55 -0500 Subject: [OmniOS-discuss] ISC DHCP now at 4.3.3-P1 - security update! Message-ID: If you run the ISC DHCP server on your machine, please utter "pkg update". The act of updating should restart your service (I just rechecked the manifest). This applies to r151014 (LTS) and r151016 (Stable). Thanks, Dan From wonko at 4amlunch.net Wed Jan 13 03:55:00 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Tue, 12 Jan 2016 22:55:00 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> Message-ID: Ok, it has happened. Checking this here, the pool seems to be fine. I can read and write files. except ?zpool status? is now currently hanging. I can still read/write from the pool, however. I can telnet to port 3260, but restarting target services has hung. root at basket1:/tank/Share# svcs -a | grep stmf online Jan_05 svc:/system/stmf:default root at basket1:/tank/Share# svcs -a | grep target disabled Jan_05 svc:/system/fcoe_target:default online Jan_05 svc:/network/iscsi/target:default online Jan_05 svc:/system/ibsrp/target:default root at basket1:/tank/Share# svcadm restart /system/ibsrp/target root at basket1:/tank/Share# svcadm restart /network/iscsi/target root at basket1:/tank/Share# svcadm restart /system/stmf root at basket1:/tank/Share# svcs -a | grep target disabled Jan_05 svc:/system/fcoe_target:default online* 22:43:03 svc:/system/ibsrp/target:default online* 22:43:13 svc:/network/iscsi/target:default root at basket1:/tank/Share# svcs -a | grep stmf online* 22:43:18 svc:/system/stmf:default root at basket1:/tank/Share# I?m doing a crash dump reboot. I?ll post the output somewhere. The output of echo '$ -------------- next part -------------- > On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: > > Is the pool usable during comstar hang? > Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). > > Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. > > Matej > >> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: >> >> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >> >> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >> >> -brian >> >>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >>> >>> First off, love SuperMicro good choice IMHO. >>> >>> This board has two on board controllers. >>> >>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>> >>> And >>> >>> Intel ICH10R SATA (So I'm guessing your using this one.) >>> >>> -----Original Message----- >>> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>> Sent: Friday, January 08, 2016 12:16 PM >>> To: Matej Zerovnik >>> Cc: omnios-discuss >>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>> >>> >>>> Which controller exactly do you have? >>> >>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>> >>>> Do you know firmware version? >>> >>> I?m assuming this is linked to the BIOS version? >>> >>>> Which hard drives? >>> >>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>> >>>> It might not tell much, but it?s good to have as much information as possible. >>>> >>>> When comstar hangs, can you telnet to the iSCSI port? >>>> What does svcs says, is the service running? >>>> What happens in you try to restart it? >>>> How do you restart it? >>> >>> I?ll try all these things next time. >>> >>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>> >>> All good info. Thanks! >>> >>> -brian >>> >>>> >>>> Matej >>>> >>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>>> >>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>>> >>>>>>> No, ZFS raid10 >>>>>> >>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>> >>>>> It's a zpool with multiple mirror vdevs. >>>>> >>>>> -- >>>>> Dave Pooser >>>>> Cat-Herder-in-Chief, Pooserville.com >>>>> >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > From wonko at 4amlunch.net Wed Jan 13 04:01:17 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Tue, 12 Jan 2016 23:01:17 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> Message-ID: <81E2F528-3E41-47D0-BC45-73B248CDFE81@4amlunch.net> Or there won?t be a crash dump. Unless I get one after resetting the box. ?reboot -d? has hung as well. :( -brian > On Jan 12, 2016, at 10:55 PM, Brian Hechinger wrote: > > Ok, it has happened. > > Checking this here, the pool seems to be fine. I can read and write files. > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > I can telnet to port 3260, but restarting target services has hung. > > root at basket1:/tank/Share# svcs -a | grep stmf > online Jan_05 svc:/system/stmf:default > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online Jan_05 svc:/network/iscsi/target:default > online Jan_05 svc:/system/ibsrp/target:default > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > root at basket1:/tank/Share# svcadm restart /system/stmf > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online* 22:43:03 svc:/system/ibsrp/target:default > online* 22:43:13 svc:/network/iscsi/target:default > root at basket1:/tank/Share# svcs -a | grep stmf > online* 22:43:18 svc:/system/stmf:default > root at basket1:/tank/Share# > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > The output of echo '$ > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: >> >> Is the pool usable during comstar hang? >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). >> >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. >> >> Matej >> >>> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: >>> >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >>> >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >>> >>> -brian >>> >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >>>> >>>> First off, love SuperMicro good choice IMHO. >>>> >>>> This board has two on board controllers. >>>> >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>>> >>>> And >>>> >>>> Intel ICH10R SATA (So I'm guessing your using this one.) >>>> >>>> -----Original Message----- >>>> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>>> Sent: Friday, January 08, 2016 12:16 PM >>>> To: Matej Zerovnik >>>> Cc: omnios-discuss >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>>> >>>> >>>>> Which controller exactly do you have? >>>> >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>>> >>>>> Do you know firmware version? >>>> >>>> I?m assuming this is linked to the BIOS version? >>>> >>>>> Which hard drives? >>>> >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>>> >>>>> It might not tell much, but it?s good to have as much information as possible. >>>>> >>>>> When comstar hangs, can you telnet to the iSCSI port? >>>>> What does svcs says, is the service running? >>>>> What happens in you try to restart it? >>>>> How do you restart it? >>>> >>>> I?ll try all these things next time. >>>> >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>>> >>>> All good info. Thanks! >>>> >>>> -brian >>>> >>>>> >>>>> Matej >>>>> >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>>>> >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>>>> >>>>>>>> No, ZFS raid10 >>>>>>> >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>>> >>>>>> It's a zpool with multiple mirror vdevs. >>>>>> >>>>>> -- >>>>>> Dave Pooser >>>>>> Cat-Herder-in-Chief, Pooserville.com >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> OmniOS-discuss mailing list >>>>>> OmniOS-discuss at lists.omniti.com >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >> > From wonko at 4amlunch.net Wed Jan 13 04:07:30 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Tue, 12 Jan 2016 23:07:30 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> Message-ID: In the meantime I?ve removed the SLOG and L2ARC just in case. I don?t think that?s it though. At least will have some sort of data point to work with here. :) -brian > On Jan 12, 2016, at 10:55 PM, Brian Hechinger wrote: > > Ok, it has happened. > > Checking this here, the pool seems to be fine. I can read and write files. > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > I can telnet to port 3260, but restarting target services has hung. > > root at basket1:/tank/Share# svcs -a | grep stmf > online Jan_05 svc:/system/stmf:default > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online Jan_05 svc:/network/iscsi/target:default > online Jan_05 svc:/system/ibsrp/target:default > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > root at basket1:/tank/Share# svcadm restart /system/stmf > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online* 22:43:03 svc:/system/ibsrp/target:default > online* 22:43:13 svc:/network/iscsi/target:default > root at basket1:/tank/Share# svcs -a | grep stmf > online* 22:43:18 svc:/system/stmf:default > root at basket1:/tank/Share# > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > The output of echo '$ > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: >> >> Is the pool usable during comstar hang? >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). >> >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. >> >> Matej >> >>> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: >>> >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >>> >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >>> >>> -brian >>> >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >>>> >>>> First off, love SuperMicro good choice IMHO. >>>> >>>> This board has two on board controllers. >>>> >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>>> >>>> And >>>> >>>> Intel ICH10R SATA (So I'm guessing your using this one.) >>>> >>>> -----Original Message----- >>>> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>>> Sent: Friday, January 08, 2016 12:16 PM >>>> To: Matej Zerovnik >>>> Cc: omnios-discuss >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>>> >>>> >>>>> Which controller exactly do you have? >>>> >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>>> >>>>> Do you know firmware version? >>>> >>>> I?m assuming this is linked to the BIOS version? >>>> >>>>> Which hard drives? >>>> >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>>> >>>>> It might not tell much, but it?s good to have as much information as possible. >>>>> >>>>> When comstar hangs, can you telnet to the iSCSI port? >>>>> What does svcs says, is the service running? >>>>> What happens in you try to restart it? >>>>> How do you restart it? >>>> >>>> I?ll try all these things next time. >>>> >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>>> >>>> All good info. Thanks! >>>> >>>> -brian >>>> >>>>> >>>>> Matej >>>>> >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>>>> >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>>>> >>>>>>>> No, ZFS raid10 >>>>>>> >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>>> >>>>>> It's a zpool with multiple mirror vdevs. >>>>>> >>>>>> -- >>>>>> Dave Pooser >>>>>> Cat-Herder-in-Chief, Pooserville.com >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> OmniOS-discuss mailing list >>>>>> OmniOS-discuss at lists.omniti.com >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >> > From john.barfield at bissinc.com Wed Jan 13 04:16:43 2016 From: john.barfield at bissinc.com (John Barfield) Date: Wed, 13 Jan 2016 04:16:43 +0000 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> , Message-ID: <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> My input may or may not be valid but Im going to throw it out there anyway :) do you have any Mpt disconnect errors in /var/adm/messages? Also do you have smartmontools installed? I ran into similiar issues just booting a sunfire x4540 recently off of OmniOS live, i/o would just hang while probing device nodes. I found the drive that was acting up and pulled it. All of a sudden everything miraculously worked amazing. I compiled smartmontools after I got it to boot and found 10 drives out of 48 with bad sectors in prefail state. I dont know if this happens with SAS drives or not but Im using SATA and saw this was a common issue in old opensolaris threads. -barfield Sent from Outlook Mobile On Tue, Jan 12, 2016 at 8:08 PM -0800, "Brian Hechinger" > wrote: In the meantime I?ve removed the SLOG and L2ARC just in case. I don?t think that?s it though. At least will have some sort of data point to work with here. :) -brian > On Jan 12, 2016, at 10:55 PM, Brian Hechinger wrote: > > Ok, it has happened. > > Checking this here, the pool seems to be fine. I can read and write files. > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > I can telnet to port 3260, but restarting target services has hung. > > root at basket1:/tank/Share# svcs -a | grep stmf > online Jan_05 svc:/system/stmf:default > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online Jan_05 svc:/network/iscsi/target:default > online Jan_05 svc:/system/ibsrp/target:default > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > root at basket1:/tank/Share# svcadm restart /system/stmf > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online* 22:43:03 svc:/system/ibsrp/target:default > online* 22:43:13 svc:/network/iscsi/target:default > root at basket1:/tank/Share# svcs -a | grep stmf > online* 22:43:18 svc:/system/stmf:default > root at basket1:/tank/Share# > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > The output of echo '$ > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: >> >> Is the pool usable during comstar hang? >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). >> >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. >> >> Matej >> >>> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: >>> >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >>> >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >>> >>> -brian >>> >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >>>> >>>> First off, love SuperMicro good choice IMHO. >>>> >>>> This board has two on board controllers. >>>> >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>>> >>>> And >>>> >>>> Intel ICH10R SATA (So I'm guessing your using this one.) >>>> >>>> -----Original Message----- >>>> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>>> Sent: Friday, January 08, 2016 12:16 PM >>>> To: Matej Zerovnik >>>> Cc: omnios-discuss >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>>> >>>> >>>>> Which controller exactly do you have? >>>> >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>>> >>>>> Do you know firmware version? >>>> >>>> I?m assuming this is linked to the BIOS version? >>>> >>>>> Which hard drives? >>>> >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>>> >>>>> It might not tell much, but it?s good to have as much information as possible. >>>>> >>>>> When comstar hangs, can you telnet to the iSCSI port? >>>>> What does svcs says, is the service running? >>>>> What happens in you try to restart it? >>>>> How do you restart it? >>>> >>>> I?ll try all these things next time. >>>> >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>>> >>>> All good info. Thanks! >>>> >>>> -brian >>>> >>>>> >>>>> Matej >>>>> >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>>>> >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>>>> >>>>>>>> No, ZFS raid10 >>>>>>> >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>>> >>>>>> It's a zpool with multiple mirror vdevs. >>>>>> >>>>>> -- >>>>>> Dave Pooser >>>>>> Cat-Herder-in-Chief, Pooserville.com >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> OmniOS-discuss mailing list >>>>>> OmniOS-discuss at lists.omniti.com >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >> > ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/26677440-40b316d8 Modify Your Subscription: https://www.listbox.com/member/?member_id=26677440&id_secret=26677440-8fd7f4fe Powered by Listbox: http://www.listbox.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.barfield at bissinc.com Wed Jan 13 04:19:53 2016 From: john.barfield at bissinc.com (John Barfield) Date: Wed, 13 Jan 2016 04:19:53 +0000 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> , , <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> Message-ID: <867309FA36CA349E.A61AF59B-1738-47A9-8918-7281B3A691CA@mail.outlook.com> BTW I left off that it has the same LSI controller chipset Sent from Outlook Mobile _____________________________ From: John Barfield Sent: Tuesday, January 12, 2016 10:17 PM Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging To: , omnios-discuss My input may or may not be valid but Im going to throw it out there anyway :) do you have any Mpt disconnect errors in /var/adm/messages? Also do you have smartmontools installed? I ran into similiar issues just booting a sunfire x4540 recently off of OmniOS live, i/o would just hang while probing device nodes. I found the drive that was acting up and pulled it. All of a sudden everything miraculously worked amazing. I compiled smartmontools after I got it to boot and found 10 drives out of 48 with bad sectors in prefail state. I dont know if this happens with SAS drives or not but Im using SATA and saw this was a common issue in old opensolaris threads. -barfield Sent from Outlook Mobile On Tue, Jan 12, 2016 at 8:08 PM -0800, "Brian Hechinger" > wrote: In the meantime I?ve removed the SLOG and L2ARC just in case. I don?t think that?s it though. At least will have some sort of data point to work with here. :) -brian > On Jan 12, 2016, at 10:55 PM, Brian Hechinger wrote: > > Ok, it has happened. > > Checking this here, the pool seems to be fine. I can read and write files. > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > I can telnet to port 3260, but restarting target services has hung. > > root at basket1:/tank/Share# svcs -a | grep stmf > online Jan_05 svc:/system/stmf:default > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online Jan_05 svc:/network/iscsi/target:default > online Jan_05 svc:/system/ibsrp/target:default > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > root at basket1:/tank/Share# svcadm restart /system/stmf > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online* 22:43:03 svc:/system/ibsrp/target:default > online* 22:43:13 svc:/network/iscsi/target:default > root at basket1:/tank/Share# svcs -a | grep stmf > online* 22:43:18 svc:/system/stmf:default > root at basket1:/tank/Share# > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > The output of echo '$ > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: >> >> Is the pool usable during comstar hang? >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). >> >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. >> >> Matej >> >>> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: >>> >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >>> >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >>> >>> -brian >>> >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: >>>> >>>> First off, love SuperMicro good choice IMHO. >>>> >>>> This board has two on board controllers. >>>> >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>>> >>>> And >>>> >>>> Intel ICH10R SATA (So I'm guessing your using this one.) >>>> >>>> -----Original Message----- >>>> From: OmniOS-discuss [ mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>>> Sent: Friday, January 08, 2016 12:16 PM >>>> To: Matej Zerovnik >>>> Cc: omnios-discuss >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>>> >>>> >>>>> Which controller exactly do you have? >>>> >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>>> >>>>> Do you know firmware version? >>>> >>>> I?m assuming this is linked to the BIOS version? >>>> >>>>> Which hard drives? >>>> >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>>> >>>>> It might not tell much, but it?s good to have as much information as possible. >>>>> >>>>> When comstar hangs, can you telnet to the iSCSI port? >>>>> What does svcs says, is the service running? >>>>> What happens in you try to restart it? >>>>> How do you restart it? >>>> >>>> I?ll try all these things next time. >>>> >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>>> >>>> All good info. Thanks! >>>> >>>> -brian >>>> >>>>> >>>>> Matej >>>>> >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: >>>>>> >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: >>>>>>>> >>>>>>>> No, ZFS raid10 >>>>>>> >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>>> >>>>>> It's a zpool with multiple mirror vdevs. >>>>>> >>>>>> -- >>>>>> Dave Pooser >>>>>> Cat-Herder-in-Chief, Pooserville.com >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> OmniOS-discuss mailing list >>>>>> OmniOS-discuss at lists.omniti.com >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >> > http://www.listbox.com illumos-discuss | Archives [http://postlink.www.listbox.com/2033704/833487e62783d55fe81f119fb93ef644/26677440/3044d385.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc] | Modify Your Subscription [http://postlink.www.listbox.com/2033705/3379085af0f1cf7fc3708f04b4471ae2/26677440/3044d385.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc] -------------- next part -------------- An HTML attachment was scrubbed... URL: From wonko at 4amlunch.net Wed Jan 13 04:20:34 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Tue, 12 Jan 2016 23:20:34 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> Message-ID: I will look to do this. The shared storage is on SATA disks, so maybe? Although they are new. I hope they are fine. :) I don?t see anything about mpt in /var/adm/messages, no. -brian > On Jan 12, 2016, at 11:16 PM, John Barfield wrote: > > My input may or may not be valid but Im going to throw it out there anyway :) > > do you have any Mpt disconnect errors in /var/adm/messages? > > Also do you have smartmontools installed? > > I ran into similiar issues just booting a sunfire x4540 recently off of OmniOS live, i/o would just hang while probing device nodes. > > I found the drive that was acting up and pulled it. > > All of a sudden everything miraculously worked amazing. > > I compiled smartmontools after I got it to boot and found 10 drives out of 48 with bad sectors in prefail state. > > I dont know if this happens with SAS drives or not but Im using SATA and saw this was a common issue in old opensolaris threads. > > -barfield > > Sent from Outlook Mobile > > > > On Tue, Jan 12, 2016 at 8:08 PM -0800, "Brian Hechinger" > wrote: > > In the meantime I?ve removed the SLOG and L2ARC just in case. I don?t think that?s it though. At least will have some sort of data point to work with here. :) > > -brian > > > On Jan 12, 2016, at 10:55 PM, Brian Hechinger wrote: > > > > Ok, it has happened. > > > > Checking this here, the pool seems to be fine. I can read and write files. > > > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > > > I can telnet to port 3260, but restarting target services has hung. > > > > root at basket1:/tank/Share# svcs -a | grep stmf > > online Jan_05 svc:/system/stmf:default > > root at basket1:/tank/Share# svcs -a | grep target > > disabled Jan_05 svc:/system/fcoe_target:default > > online Jan_05 svc:/network/iscsi/target:default > > online Jan_05 svc:/system/ibsrp/target:default > > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > > root at basket1:/tank/Share# svcadm restart /system/stmf > > root at basket1:/tank/Share# svcs -a | grep target > > disabled Jan_05 svc:/system/fcoe_target:default > > online* 22:43:03 svc:/system/ibsrp/target:default > > online* 22:43:13 svc:/network/iscsi/target:default > > root at basket1:/tank/Share# svcs -a | grep stmf > > online* 22:43:18 svc:/system/stmf:default > > root at basket1:/tank/Share# > > > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > > > The output of echo '$ > > > > > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: > >> > >> Is the pool usable during comstar hang? > >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). > >> > >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. > >> > >> Matej > >> > >>> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: > >>> > >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. > >>> > >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. > >>> > >>> -brian > >>> > >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: > >>>> > >>>> First off, love SuperMicro good choice IMHO. > >>>> > >>>> This board has two on board controllers. > >>>> > >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) > >>>> > >>>> And > >>>> > >>>> Intel ICH10R SATA (So I'm guessing your using this one.) > >>>> > >>>> -----Original Message----- > >>>> From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com ] On Behalf Of Brian Hechinger > >>>> Sent: Friday, January 08, 2016 12:16 PM > >>>> To: Matej Zerovnik > >>>> Cc: omnios-discuss > >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging > >>>> > >>>> > >>>>> Which controller exactly do you have? > >>>> > >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. > >>>> > >>>>> Do you know firmware version? > >>>> > >>>> I?m assuming this is linked to the BIOS version? > >>>> > >>>>> Which hard drives? > >>>> > >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB > >>>> > >>>>> It might not tell much, but it?s good to have as much information as possible. > >>>>> > >>>>> When comstar hangs, can you telnet to the iSCSI port? > >>>>> What does svcs says, is the service running? > >>>>> What happens in you try to restart it? > >>>>> How do you restart it? > >>>> > >>>> I?ll try all these things next time. > >>>> > >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). > >>>> > >>>> All good info. Thanks! > >>>> > >>>> -brian > >>>> > >>>>> > >>>>> Matej > >>>>> > >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: > >>>>>> > >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: > >>>>>>>> > >>>>>>>> No, ZFS raid10 > >>>>>>> > >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? > >>>>>> > >>>>>> It's a zpool with multiple mirror vdevs. > >>>>>> > >>>>>> -- > >>>>>> Dave Pooser > >>>>>> Cat-Herder-in-Chief, Pooserville.com > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> OmniOS-discuss mailing list > >>>>>> OmniOS-discuss at lists.omniti.com > >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >>>>> > >>>>> _______________________________________________ > >>>>> OmniOS-discuss mailing list > >>>>> OmniOS-discuss at lists.omniti.com > >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >>>> > >>>> _______________________________________________ > >>>> OmniOS-discuss mailing list > >>>> OmniOS-discuss at lists.omniti.com > >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >>> > >> > > > > > > ------------------------------------------- > illumos-discuss > Archives: https://www.listbox.com/member/archive/182180/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182180/26677440-40b316d8 > Modify Your Subscription: https://www.listbox.com/member/?member_id=26677440&id_secret=26677440-8fd7f4fe > Powered by Listbox: http://www.listbox.com > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From wonko at 4amlunch.net Wed Jan 13 04:21:03 2016 From: wonko at 4amlunch.net (Brian Hechinger) Date: Tue, 12 Jan 2016 23:21:03 -0500 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: <867309FA36CA349E.A61AF59B-1738-47A9-8918-7281B3A691CA@mail.outlook.com> References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> <867309FA36CA349E.A61AF59B-1738-47A9-8918-7281B3A691CA@mail.outlook.com> Message-ID: In my case the SATA disks aren?t on the 1068E. -brian > On Jan 12, 2016, at 11:19 PM, John Barfield wrote: > > BTW I left off that it has the same LSI controller chipset > > Sent from Outlook Mobile > _____________________________ > From: John Barfield > Sent: Tuesday, January 12, 2016 10:17 PM > Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging > To: , omnios-discuss > > > My input may or may not be valid but Im going to throw it out there anyway :) > > do you have any Mpt disconnect errors in /var/adm/messages? > > Also do you have smartmontools installed? > > I ran into similiar issues just booting a sunfire x4540 recently off of OmniOS live, i/o would just hang while probing device nodes. > > I found the drive that was acting up and pulled it. > > All of a sudden everything miraculously worked amazing. > > I compiled smartmontools after I got it to boot and found 10 drives out of 48 with bad sectors in prefail state. > > I dont know if this happens with SAS drives or not but Im using SATA and saw this was a common issue in old opensolaris threads. > > -barfield > > Sent from Outlook Mobile > > > > On Tue, Jan 12, 2016 at 8:08 PM -0800, "Brian Hechinger" > wrote: > > In the meantime I?ve removed the SLOG and L2ARC just in case. I don?t think that?s it though. At least will have some sort of data point to work with here. :) > > -brian > > > On Jan 12, 2016, at 10:55 PM, Brian Hechinger wrote: > > > > Ok, it has happened. > > > > Checking this here, the pool seems to be fine. I can read and write files. > > > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > > > I can telnet to port 3260, but restarting target services has hung. > > > > root at basket1:/tank/Share# svcs -a | grep stmf > > online Jan_05 svc:/system/stmf:default > > root at basket1:/tank/Share# svcs -a | grep target > > disabled Jan_05 svc:/system/fcoe_target:default > > online Jan_05 svc:/network/iscsi/target:default > > online Jan_05 svc:/system/ibsrp/target:default > > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > > root at basket1:/tank/Share# svcadm restart /system/stmf > > root at basket1:/tank/Share# svcs -a | grep target > > disabled Jan_05 svc:/system/fcoe_target:default > > online* 22:43:03 svc:/system/ibsrp/target:default > > online* 22:43:13 svc:/network/iscsi/target:default > > root at basket1:/tank/Share# svcs -a | grep stmf > > online* 22:43:18 svc:/system/stmf:default > > root at basket1:/tank/Share# > > > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > > > The output of echo '$ > > > > > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik wrote: > >> > >> Is the pool usable during comstar hang? > >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). > >> > >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. > >> > >> Matej > >> > >>> On 08 Jan 2016, at 20:11, Brian Hechinger wrote: > >>> > >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. > >>> > >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. > >>> > >>> -brian > >>> > >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel wrote: > >>>> > >>>> First off, love SuperMicro good choice IMHO. > >>>> > >>>> This board has two on board controllers. > >>>> > >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) > >>>> > >>>> And > >>>> > >>>> Intel ICH10R SATA (So I'm guessing your using this one.) > >>>> > >>>> -----Original Message----- > >>>> From: OmniOS-discuss [ mailto:omnios-discuss-bounces at lists.omniti.com ] On Behalf Of Brian Hechinger > >>>> Sent: Friday, January 08, 2016 12:16 PM > >>>> To: Matej Zerovnik > >>>> Cc: omnios-discuss > >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging > >>>> > >>>> > >>>>> Which controller exactly do you have? > >>>> > >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. > >>>> > >>>>> Do you know firmware version? > >>>> > >>>> I?m assuming this is linked to the BIOS version? > >>>> > >>>>> Which hard drives? > >>>> > >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB > >>>> > >>>>> It might not tell much, but it?s good to have as much information as possible. > >>>>> > >>>>> When comstar hangs, can you telnet to the iSCSI port? > >>>>> What does svcs says, is the service running? > >>>>> What happens in you try to restart it? > >>>>> How do you restart it? > >>>> > >>>> I?ll try all these things next time. > >>>> > >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). > >>>> > >>>> All good info. Thanks! > >>>> > >>>> -brian > >>>> > >>>>> > >>>>> Matej > >>>>> > >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser wrote: > >>>>>> > >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger wrote: > >>>>>>>> > >>>>>>>> No, ZFS raid10 > >>>>>>> > >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? > >>>>>> > >>>>>> It's a zpool with multiple mirror vdevs. > >>>>>> > >>>>>> -- > >>>>>> Dave Pooser > >>>>>> Cat-Herder-in-Chief, Pooserville.com > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> OmniOS-discuss mailing list > >>>>>> OmniOS-discuss at lists.omniti.com > >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >>>>> > >>>>> _______________________________________________ > >>>>> OmniOS-discuss mailing list > >>>>> OmniOS-discuss at lists.omniti.com > >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >>>> > >>>> _______________________________________________ > >>>> OmniOS-discuss mailing list > >>>> OmniOS-discuss at lists.omniti.com > >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >>> > >> > > > > > > http://www.listbox.com > illumos-discuss | Archives | Modify Your Subscription > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.barfield at bissinc.com Wed Jan 13 04:52:14 2016 From: john.barfield at bissinc.com (John Barfield) Date: Wed, 13 Jan 2016 04:52:14 +0000 Subject: [OmniOS-discuss] [discuss] COMSTAR hanging In-Reply-To: References: <0C7A62FA-5D1C-42A4-B171-FAA3AC34AC3D@4amlunch.net> <97F9575A-C086-4862-B5CA-BA2893E48C08@4amlunch.net> <6D6F42AE-611E-4C7C-88AE-23543CA66DA4@4amlunch.net> <2F515DC7-7803-4814-99D1-06AE3E3A196C@4amlunch.net> <52C14C1A-80F1-4B5D-AE7D-9999AB1B758F@omniti.com> <90EFFEAE-2967-49C7-80F1-AB82E95CB52E@zunaj.si> <11A7C643-506B-441D-B320-79C75B8EAFCE@4amlunch.net> <65DC5816D4BEE043885A89FD54E273FC6CF6BC33@MAIL101.Ellipseinc.com> <867309FA36CA349E.2808DEE6-8CB2-4717-B123-26F8E9BF10C5@mail.outlook.com> <867309FA36CA349E.A61AF59B-1738-47A9-8918-7281B3A691CA@mail.outlook.com>, Message-ID: <867309FA36CA349E.0C1F93E3-1045-4C65-9084-870490D3BF34@mail.outlook.com> Oh I didnt catch that detail. Okay well nevermind :) Sent from Outlook Mobile On Tue, Jan 12, 2016 at 8:21 PM -0800, "Brian Hechinger" > wrote: In my case the SATA disks aren?t on the 1068E. -brian On Jan 12, 2016, at 11:19 PM, John Barfield > wrote: BTW I left off that it has the same LSI controller chipset Sent from Outlook Mobile _____________________________ From: John Barfield > Sent: Tuesday, January 12, 2016 10:17 PM Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging To: >, omnios-discuss > My input may or may not be valid but Im going to throw it out there anyway :) do you have any Mpt disconnect errors in /var/adm/messages? Also do you have smartmontools installed? I ran into similiar issues just booting a sunfire x4540 recently off of OmniOS live, i/o would just hang while probing device nodes. I found the drive that was acting up and pulled it. All of a sudden everything miraculously worked amazing. I compiled smartmontools after I got it to boot and found 10 drives out of 48 with bad sectors in prefail state. I dont know if this happens with SAS drives or not but Im using SATA and saw this was a common issue in old opensolaris threads. -barfield Sent from Outlook Mobile On Tue, Jan 12, 2016 at 8:08 PM -0800, "Brian Hechinger" > wrote: In the meantime I?ve removed the SLOG and L2ARC just in case. I don?t think that?s it though. At least will have some sort of data point to work with here. :) -brian > On Jan 12, 2016, at 10:55 PM, Brian Hechinger > wrote: > > Ok, it has happened. > > Checking this here, the pool seems to be fine. I can read and write files. > > except ?zpool status? is now currently hanging. I can still read/write from the pool, however. > > I can telnet to port 3260, but restarting target services has hung. > > root at basket1:/tank/Share# svcs -a | grep stmf > online Jan_05 svc:/system/stmf:default > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online Jan_05 svc:/network/iscsi/target:default > online Jan_05 svc:/system/ibsrp/target:default > root at basket1:/tank/Share# svcadm restart /system/ibsrp/target > root at basket1:/tank/Share# svcadm restart /network/iscsi/target > root at basket1:/tank/Share# svcadm restart /system/stmf > root at basket1:/tank/Share# svcs -a | grep target > disabled Jan_05 svc:/system/fcoe_target:default > online* 22:43:03 svc:/system/ibsrp/target:default > online* 22:43:13 svc:/network/iscsi/target:default > root at basket1:/tank/Share# svcs -a | grep stmf > online* 22:43:18 svc:/system/stmf:default > root at basket1:/tank/Share# > > I?m doing a crash dump reboot. I?ll post the output somewhere. > > The output of echo '$ > > >> On Jan 8, 2016, at 3:11 PM, Matej Zerovnik > wrote: >> >> Is the pool usable during comstar hang? >> Can you write and read from the pool (test both, in my case, when pool froze, I wasn?t able to write to the pool, but I could read). >> >> Again, this might not be connected with Comstar, but in my case, Comstar and pool hang were exchanging. >> >> Matej >> >>> On 08 Jan 2016, at 20:11, Brian Hechinger > wrote: >>> >>> Yeah, I?m using the 1068E to boot from (this has been supported since before Illumos) but that doesn?t have anything accessed by COMSTAR. >>> >>> It?s the ICH10R SATA that hosts the disks that COMSTAR shares out space from. >>> >>> -brian >>> >>>> On Jan 8, 2016, at 1:31 PM, Richard Jahnel > wrote: >>>> >>>> First off, love SuperMicro good choice IMHO. >>>> >>>> This board has two on board controllers. >>>> >>>> LSI SAS1068E (not 100% sure there are working illumos drivers for this one) >>>> >>>> And >>>> >>>> Intel ICH10R SATA (So I'm guessing your using this one.) >>>> >>>> -----Original Message----- >>>> From: OmniOS-discuss [ mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Brian Hechinger >>>> Sent: Friday, January 08, 2016 12:16 PM >>>> To: Matej Zerovnik > >>>> Cc: omnios-discuss > >>>> Subject: Re: [OmniOS-discuss] [discuss] COMSTAR hanging >>>> >>>> >>>>> Which controller exactly do you have? >>>> >>>> Whatever ACHI stuff is built into the motherboard. Motherboard is X8DTL-3F. >>>> >>>>> Do you know firmware version? >>>> >>>> I?m assuming this is linked to the BIOS version? >>>> >>>>> Which hard drives? >>>> >>>> Hitachi-HUA723030ALA640-MKAOAA50-2.73TB >>>> >>>>> It might not tell much, but it?s good to have as much information as possible. >>>>> >>>>> When comstar hangs, can you telnet to the iSCSI port? >>>>> What does svcs says, is the service running? >>>>> What happens in you try to restart it? >>>>> How do you restart it? >>>> >>>> I?ll try all these things next time. >>>> >>>>> In my case, svcs reported service running, but when I tried to telnet, there was no connection as well as there was no listening port opened when checking with 'netstat -an'. If I tried to restart target and stmf service, but stmf service got stucked in online* state and would not start. Reboot was the only solution in my case, but as I said, latest 014 release is working OK (but then again, load got reduced). >>>> >>>> All good info. Thanks! >>>> >>>> -brian >>>> >>>>> >>>>> Matej >>>>> >>>>>> On 08 Jan 2016, at 17:50, Dave Pooser > wrote: >>>>>> >>>>>>>> On Jan 8, 2016, at 11:22 AM, Brian Hechinger > wrote: >>>>>>>> >>>>>>>> No, ZFS raid10 >>>>>>> >>>>>>> Saw the HW-RAID term, and got concerned. That's what, raidz2 in ZFS-ese? >>>>>> >>>>>> It's a zpool with multiple mirror vdevs. >>>>>> >>>>>> -- >>>>>> Dave Pooser >>>>>> Cat-Herder-in-Chief, Pooserville.com >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> OmniOS-discuss mailing list >>>>>> OmniOS-discuss at lists.omniti.com >>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>>> >>>>> _______________________________________________ >>>>> OmniOS-discuss mailing list >>>>> OmniOS-discuss at lists.omniti.com >>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >> > http://www.listbox.com illumos-discuss | Archives [http://postlink.www.listbox.com/2033704/833487e62783d55fe81f119fb93ef644/26677440/3044d385.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc] | Modify Your Subscription [http://postlink.www.listbox.com/2033705/3379085af0f1cf7fc3708f04b4471ae2/26677440/3044d385.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc] _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at marzocchi.net Wed Jan 13 14:16:35 2016 From: lists at marzocchi.net (Olaf Marzocchi) Date: Wed, 13 Jan 2016 15:16:35 +0100 Subject: [OmniOS-discuss] .$EXTEND/$QUOTA Message-ID: Hello, I was doing remote rsync backups and I got this error on the source computer: rsync: get_xattr_data: lgetxattr(""/tank/eBooks/.$EXTEND/$QUOTA"","SUNWsmb:$Q:$INDEX_ALLOCATION",0) failed: No data available (61) I'm using kernel CIFS on the latest LTS and Windows 10 (previously 8.1) and OS X 10.10 as client. I found some info online about the folders .$EXTEND/$QUOTA but I understand they are created on NTFS disks, not on network shares. I don't have quotas set, could someone give me hints about possible solutions to the issue? Since the folder(s) generating the error are empty, what about removing them? In the past I was using netatalk but not anymore. Thanks Olaf -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Jan 13 14:28:53 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 13 Jan 2016 09:28:53 -0500 Subject: [OmniOS-discuss] .$EXTEND/$QUOTA In-Reply-To: References: Message-ID: <58F4F49F-0E05-44DC-90B7-912C6A05C8C9@omniti.com> I'd HIGHLY recommend upgrading your LTS to r151016 (current Stable) if you're using kernel CIFS. MANY new features, including SMB 2.0, are in that release from upstream illumos. Dan From lists at marzocchi.net Wed Jan 13 16:36:33 2016 From: lists at marzocchi.net (Olaf Marzocchi) Date: Wed, 13 Jan 2016 17:36:33 +0100 Subject: [OmniOS-discuss] .$EXTEND/$QUOTA In-Reply-To: <58F4F49F-0E05-44DC-90B7-912C6A05C8C9@omniti.com> References: <58F4F49F-0E05-44DC-90B7-912C6A05C8C9@omniti.com> Message-ID: Ok, if it's that much recommended I will. In the meanwhile (or even afterwards), should I keep or delete those folders? Put it another way: what would I lose if I delete them? I still don't have this clear. Thanks Olaf Il 13 gennaio 2016 15:28:53 CET, Dan McDonald ha scritto: >I'd HIGHLY recommend upgrading your LTS to r151016 (current Stable) if >you're using kernel CIFS. MANY new features, including SMB 2.0, are in >that release from upstream illumos. > >Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From natxo.asenjo at gmail.com Wed Jan 13 17:12:08 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Wed, 13 Jan 2016 18:12:08 +0100 Subject: [OmniOS-discuss] smb 2 omnios OmniOS r151016 Message-ID: hi, in another thread (.$EXTEND/$QUOTA) I read that smb v2 is enabled (yay!). Do we need to do something to use that or it will "just work"? Thanks! -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From natxo.asenjo at gmail.com Wed Jan 13 17:13:16 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Wed, 13 Jan 2016 18:13:16 +0100 Subject: [OmniOS-discuss] smb 2 omnios OmniOS r151016 In-Reply-To: References: Message-ID: by the way, I cannot see anything regarding smb/cifs 2 in http://omnios.omniti.com/wiki.php/ReleaseNotes/r151016 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Wed Jan 13 17:13:28 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 13 Jan 2016 11:13:28 -0600 (CST) Subject: [OmniOS-discuss] .$EXTEND/$QUOTA In-Reply-To: References: Message-ID: On Wed, 13 Jan 2016, Olaf Marzocchi wrote: > Hello, > I was doing remote rsync backups and I got this error on the source computer: > > rsync: get_xattr_data: lgetxattr(""/tank/eBooks/.$EXTEND/$QUOTA"","SUNWsmb:$Q:$INDEX_ALLOCATION",0) failed: No data available (61) > > I'm using kernel CIFS on the latest LTS and Windows 10 (previously 8.1) and OS X 10.10 as client. > I found some info online about the folders .$EXTEND/$QUOTA but I understand they are created on NTFS disks, not on network shares. > I don't have quotas set, could someone give me hints about possible solutions to the issue? Since the folder(s) generating the > error are empty, what about removing them? > In the past I was using netatalk but not anymore. If the problem is just getting rsync working, then I suggest using the --exclude or --exclude-from rsync option such that rsync simply ignores and file/directory names which cause problems. Deleting the directories without full understanding might result in harm. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Wed Jan 13 17:48:45 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 13 Jan 2016 12:48:45 -0500 Subject: [OmniOS-discuss] smb 2 omnios OmniOS r151016 In-Reply-To: References: Message-ID: <7A6F1CD2-FB59-4A64-B584-7366A1BB8AC3@omniti.com> > On Jan 13, 2016, at 12:13 PM, Natxo Asenjo wrote: > > by the way, I cannot see anything regarding smb/cifs 2 in http://omnios.omniti.com/wiki.php/ReleaseNotes/r151016 SHOOT! I have to apologize. I got my order of arrival wrong. SMB2 hasn't shown up in a stable yet. It arrived JUST AFTER I closed r151016 for upstream merges. Sorry about that misinformation, folks! Dan From martin at c-home.cz Wed Jan 13 20:13:30 2016 From: martin at c-home.cz (Martin Cerveny) Date: Wed, 13 Jan 2016 21:13:30 +0100 (CET) Subject: [OmniOS-discuss] Ang: Re: Ang: Re: Ang: Re: How to configure FCoE target in OmniOS? Message-ID: Hello. I try to setup FCoE target before year, but unsuccessful. This is very old code (~2009) before finalization of FCoE protocol (LLDP+DCBX missing (not mandatory) and FCoE Inicialization Protocol (FIP) (mandatory) (ethertype 0x8914) missing too). I am unable to register target to FC fabric without FIP. Oracle Solaris driver v20140219-1.10 works. Verified with HP/H3C 5900AF switch. see: https://www.illumos.org/issues/5544 M.C> From lists at marzocchi.net Wed Jan 13 21:46:00 2016 From: lists at marzocchi.net (Olaf Marzocchi) Date: Wed, 13 Jan 2016 22:46:00 +0100 Subject: [OmniOS-discuss] .$EXTEND/$QUOTA In-Reply-To: References: Message-ID: <5696C598.7010300@marzocchi.net> On 13/01/2016 18:13, Bob Friesenhahn wrote: > On Wed, 13 Jan 2016, Olaf Marzocchi wrote: > >> Hello, >> I was doing remote rsync backups and I got this error on the source >> computer: >> >> rsync: get_xattr_data: >> lgetxattr(""/tank/eBooks/.$EXTEND/$QUOTA"","SUNWsmb:$Q:$INDEX_ALLOCATION",0) >> failed: No data available (61) >> >> I'm using kernel CIFS on the latest LTS and Windows 10 (previously >> 8.1) and OS X 10.10 as client. >> I found some info online about the folders .$EXTEND/$QUOTA but I >> understand they are created on NTFS disks, not on network shares. >> I don't have quotas set, could someone give me hints about possible >> solutions to the issue? Since the folder(s) generating the >> error are empty, what about removing them? >> In the past I was using netatalk but not anymore. > > If the problem is just getting rsync working, then I suggest using the > --exclude or --exclude-from rsync option such that rsync simply ignores > and file/directory names which cause problems. > > Deleting the directories without full understanding might result in harm. And that's why I would like to understand why they appeared there and what is the role they play in kernel CIFS under OmniOS/illumos :) So that I can take an informed decision whether to keep them or not: this server went through a long history and maybe they are leftovers no more required. Olaf From danmcd at omniti.com Thu Jan 14 15:59:51 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 14 Jan 2016 10:59:51 -0500 Subject: [OmniOS-discuss] OpenSSH now at 7.1p2 Message-ID: <0B2192D0-B191-404D-B190-4E004FFF7B08@omniti.com> Users of r151014 & r151016 can now "pkg update" to get OpenSSH 7.1p2. I'm sure some of you have seen that OpenSSH 7.1p2 patches a vulnerability. There is a workaround if you can't update your systems, see here: http://undeadly.org/cgi?action=article&sid=20160114142733 You don't have to worry if you're a SunSSH user, the "roaming" functionality just isn't there. Also, the next bloody release update will contain this fix as well. Dan From danmcd at omniti.com Thu Jan 14 17:01:10 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 14 Jan 2016 12:01:10 -0500 Subject: [OmniOS-discuss] Also updated: NTP 4.2.8p5 Message-ID: <359D6D0A-F689-4C01-ADE4-5D00DB455CEF@omniti.com> Thanks to Bob Friesenhahn for reminding me that NTP is updated now as well. As with OpenSSH, it's a non-reboot, self-restarting update. 014 and 016 are now updated, and bloody will be as well soon enough. Dan From richard at netbsd.org Thu Jan 14 18:39:36 2016 From: richard at netbsd.org (Richard PALO) Date: Thu, 14 Jan 2016 19:39:36 +0100 Subject: [OmniOS-discuss] updating to openssh on onu'd bloody Message-ID: Sorry if this has already been discussed, my search didn't turn up anything... I'm seeing this: > richard at omnis:/home/richard# /usr/bin/pkg install --no-backup-be --reject pkg:/network/ssh --reject pkg:/network/ssh/ssh-key --reject pkg:/service/network/ssh pkg:/network/openssh pkg:/network/openssh-server > Creating Plan (Checking for conflicting actions): / > pkg install: The following packages deliver conflicting action types to etc/ssh/moduli: > > link: > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > file: > pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages all deliver file actions to etc/ssh/ssh_config: > > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > pkg://omnios/network/openssh at 7.1.1,5.11-0.151017:20151211T022728Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages deliver conflicting action types to usr/share/man/man4/sshd_config.4: > > link: > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > file: > pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages deliver conflicting action types to usr/share/man/man4/ssh_config.4: > > link: > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > file: > pkg://omnios/network/openssh at 7.1.1,5.11-0.151017:20151211T022728Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages all deliver file actions to lib/svc/method/sshd: > > pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages all deliver file actions to etc/ssh/sshd_config: > > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages all deliver file actions to lib/svc/manifest/network/ssh.xml: > > pkg://on-nightly/service/network/ssh-common at 0.5.11,5.11-0.151017:20160111T193446Z > pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. okay, so I try: > richard at omnis:/home/richard# /usr/bin/pkg install --no-backup-be --reject pkg:/network/ssh --reject pkg:/network/ssh/ssh-key --reject pkg:/service/network/ssh --reject pkg:/service/network/ssh-common pkg:/network/openssh pkg:/network/openssh-server > WARNING: The boot environment being modified is not the active one. Changes > made will not be reflected on the next boot. > > Packages to remove: 4 > Packages to install: 2 > Mediators to change: 1 > Services to change: 2 > Create boot environment: No > Create backup boot environment: No > > DOWNLOAD PKGS FILES XFER (MB) SPEED > Completed 6/6 32/32 1.7/1.7 0B/s > > PHASE ITEMS > Removing old actions 131/131 > Installing new actions 1/68Action install failed for 'sshd' (pkg://omnios/network/openssh-server): > ActionExecutionError: Requested operation failed for package pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z: > sshd is an unknown or invalid group > The Boot Environment b151017-20160111 failed to be updated. A snapshot was taken before the failed attempt and is mounted here /tmp/tmpPDKodN. Use 'beadm unmount b151017-20160112' and then 'beadm activate b151017-20160112' if you wish to boot to this BE. > > pkg: Requested operation failed for package pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z: > sshd is an unknown or invalid group Hopefully there a way to avoid going back to vanilla omnios? -- Richard PALO From richard at netbsd.org Thu Jan 14 21:48:38 2016 From: richard at netbsd.org (Richard PALO) Date: Thu, 14 Jan 2016 22:48:38 +0100 Subject: [OmniOS-discuss] updating to openssh on onu'd bloody In-Reply-To: References: Message-ID: <569817B6.3080706@netbsd.org> Le 14/01/16 19:39, Richard PALO a ?crit : > okay, so I try: >> richard at omnis:/home/richard# /usr/bin/pkg install --no-backup-be --reject pkg:/network/ssh --reject pkg:/network/ssh/ssh-key --reject pkg:/service/network/ssh --reject pkg:/service/network/ssh-common pkg:/network/openssh pkg:/network/openssh-server >> WARNING: The boot environment being modified is not the active one. Changes >> made will not be reflected on the next boot. >> >> Packages to remove: 4 >> Packages to install: 2 >> Mediators to change: 1 >> Services to change: 2 >> Create boot environment: No >> Create backup boot environment: No >> >> DOWNLOAD PKGS FILES XFER (MB) SPEED >> Completed 6/6 32/32 1.7/1.7 0B/s >> >> PHASE ITEMS >> Removing old actions 131/131 >> Installing new actions 1/68Action install failed for 'sshd' (pkg://omnios/network/openssh-server): >> ActionExecutionError: Requested operation failed for package pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z: >> sshd is an unknown or invalid group >> The Boot Environment b151017-20160111 failed to be updated. A snapshot was taken before the failed attempt and is mounted here /tmp/tmpPDKodN. Use 'beadm unmount b151017-20160112' and then 'beadm activate b151017-20160112' if you wish to boot to this BE. >> >> pkg: Requested operation failed for package pkg://omnios/network/openssh-server at 7.1.1,5.11-0.151017:20151211T022742Z: >> sshd is an unknown or invalid group > > Hopefully there a way to avoid going back to vanilla omnios? > and: > richard at omnis:/home/richard# getent group sshd > sshd::92: ? -- Richard PALO From stephan.budach at JVM.DE Mon Jan 18 13:54:59 2016 From: stephan.budach at JVM.DE (Stephan Budach) Date: Mon, 18 Jan 2016 14:54:59 +0100 Subject: [OmniOS-discuss] How to best build a HA ZFS/NFS storage? Message-ID: <569CEEB3.9010106@jvm.de> Hi all, I am currently thinking of setting up a HA ZFS/NFS filer using RSF-1 for my Oracle VM infrastructure and I would like to collect some opinions on how to do that? Especially, I'd like to know, if there're any objections against setting up ZFS on an iSCSI target which is also setup from a ZFS volume. So, the basic setup would be either - 2 ZFS boxes, exporting either raw disks via COMSTAR iSCSI and the ZFS/NFS host would create the ZPOOL/ZFS - 2 ZFS boxes, exporting less iSCSI targets from a local ZPOOL/ZFS and the ZFS/NFS host would then create it's own ZPOOL/ZFS on top Each of these setups has it's obvious advantanges, where on the HA host, you will have to deal with maybe quite a lot of iSCSI targets, which may increase the fail-over time, as there will be a lot of iSCSI logging-on going on. While having less iSCSI targets will be easier to administer, it's somewhat against ZFS' best practises, I guess. Also, performance may be a point, where the 2nd setup has an advatange over the 1st one. So, how would anyone else here do that? Cheers, Stephan From martijn at fennis.tk Tue Jan 19 20:20:35 2016 From: martijn at fennis.tk (Martijn Fennis) Date: Tue, 19 Jan 2016 21:20:35 +0100 Subject: [OmniOS-discuss] OmniOS lockups Message-ID: Hi all, For personal storage i?m using omnios with napp-it. It?s running on top of a cheap mobo. But every now and then it just locks up (1 to 3 weeks) and does not respond to reboot commands. I tried another mobo with the same specs but it also locks up. Specs are: AM1M-A / AM1M-S2H Kabini 5350 cpu 16GB ecc mem Booting from onboard; ssd 16 port lsi 9201-16i with version 19 of IT-firmware. FC-card qlogic Already turned off all the power saving options from Bios, changed the FC-card. Someone could tell me where/how to investigate? Thanks, Martijn -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Wed Jan 20 14:27:06 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 20 Jan 2016 08:27:06 -0600 (CST) Subject: [OmniOS-discuss] Update BIND to 9.10.3-P3? Message-ID: There is a BIND update to address security issues, including some which impact client software and libraries. Last time there was a BIND release, OmniOS was totally on top of it, releasing at about the time the announcements came out to the general public. Will network/dns/bind be updated to be based on BIND 9.10.3-P3? Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Wed Jan 20 16:12:04 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 20 Jan 2016 11:12:04 -0500 Subject: [OmniOS-discuss] Update BIND to 9.10.3-P3? In-Reply-To: References: Message-ID: > On Jan 20, 2016, at 9:27 AM, Bob Friesenhahn wrote: > > There is a BIND update to address security issues, including some which impact client software and libraries. Last time there was a BIND release, OmniOS was totally on top of it, releasing at about the time the announcements came out to the general public. > > Will network/dns/bind be updated to be based on BIND 9.10.3-P3? This afternoon, US/Eastern. I just found out about this via you. Usually I find out other ways, but I've been deep in something, and I'm trying not to break my concentration. Dan From danmcd at omniti.com Wed Jan 20 16:17:52 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 20 Jan 2016 11:17:52 -0500 Subject: [OmniOS-discuss] Update BIND to 9.10.3-P3? In-Reply-To: References: Message-ID: > On Jan 20, 2016, at 11:12 AM, Dan McDonald wrote: > > This afternoon, US/Eastern. I just found out about this via you. Usually I find out other ways, but I've been deep in something, and I'm trying not to break my concentration. BTW, the bind we ship via the omnios publisher doesn't have named in it, just the client-side. Since I said I'd update, however, I will do so, but in the future, remember that the omnios publisher (i.e. what we officially support) doesn't have named in it. Dan From chip at innovates.com Wed Jan 20 18:40:30 2016 From: chip at innovates.com (Schweiss, Chip) Date: Wed, 20 Jan 2016 12:40:30 -0600 Subject: [OmniOS-discuss] zlib/zlib-devel packages Message-ID: Is anyone aware of zlib and zlib-devel packages available anywhere for OmniOS? These are needed for building any Samba version 4.2.0 or greater. Thanks! -Chip -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Jan 20 19:00:18 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 20 Jan 2016 14:00:18 -0500 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: > On Jan 20, 2016, at 1:40 PM, Schweiss, Chip wrote: > > Is anyone aware of zlib and zlib-devel packages available anywhere for OmniOS? r151014(~)[0]% pkg list zlib NAME (PUBLISHER) VERSION IFO library/zlib 1.2.8-0.151014 i-- r151014(~)[0]% pkg contents zlib PATH usr/include usr/include/zconf.h usr/include/zlib.h usr/lib/amd64 usr/lib/amd64/libz.a usr/lib/amd64/libz.so usr/lib/amd64/libz.so.1 usr/lib/amd64/libz.so.1.2.8 usr/lib/amd64/llib-lz.ln usr/lib/amd64/pkgconfig usr/lib/amd64/pkgconfig/zlib.pc usr/lib/libz.a usr/lib/libz.so usr/lib/libz.so.1 usr/lib/libz.so.1.2.8 usr/lib/llib-lz.ln usr/lib/pkgconfig usr/lib/pkgconfig/zlib.pc usr/share/man/man3 usr/share/man/man3/zlib.3 r151014(~)[0]% Is this sufficient? You probably want the headers for maximum fun, though. Dan From peter.tribble at gmail.com Wed Jan 20 19:02:15 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Wed, 20 Jan 2016 19:02:15 +0000 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: On Wed, Jan 20, 2016 at 6:40 PM, Schweiss, Chip wrote: > Is anyone aware of zlib and zlib-devel packages available anywhere for > OmniOS? > Installed in OmniOS by default, and cannot be uninstalled. > These are needed for building any Samba version 4.2.0 or greater. > We build samba (4.3.x) on OmniOS without any issues. We haven't done anything beyond install the basic build tools. What sort of error are you getting? -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From chip at innovates.com Wed Jan 20 19:48:02 2016 From: chip at innovates.com (Schweiss, Chip) Date: Wed, 20 Jan 2016 13:48:02 -0600 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: I ended up downloading and building zlib separately and got it to build. The problem only occurs when selecting --with-ads during configure. It fails on checking gnutls, which needs zlib-devel. My build will not join the domain, but that's out of the scope of this list.. Thanks! -Chip On Wed, Jan 20, 2016 at 1:02 PM, Peter Tribble wrote: > On Wed, Jan 20, 2016 at 6:40 PM, Schweiss, Chip > wrote: > >> Is anyone aware of zlib and zlib-devel packages available anywhere for >> OmniOS? >> > > Installed in OmniOS by default, and cannot be uninstalled. > > >> These are needed for building any Samba version 4.2.0 or greater. >> > > We build samba (4.3.x) on OmniOS without any issues. We haven't done > anything > beyond install the basic build tools. What sort of error are you getting? > > -- > -Peter Tribble > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Jan 20 20:03:35 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 20 Jan 2016 15:03:35 -0500 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: Probably not useful now for you on LTS, but Nexenta's SMB2 is available for r151016 and later. Dan From mtalbott at lji.org Wed Jan 20 20:29:40 2016 From: mtalbott at lji.org (Michael Talbott) Date: Wed, 20 Jan 2016 12:29:40 -0800 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: I have samba 4.2.3 working with all the bells and whistles (including winbind). Also I have netatalk working along side it ;) Hope this helps: Here's what I installed as prereqs before compiling them: pkg install \ library/security/openssl \ naming/ldap \ system/library/iconv/unicode \ system/library/dbus \ system/library/libdbus \ system/library/libdbus-glib \ developer/gnu-binutils \ developer/build/libtool \ developer/build/autoconf \ system/library/math/header-math \ /system/library/dbus \ /system/library/libdbus-glib \ /omniti/database/bdb \ /text/gnu-gettext \ /service/network/dns/mdns \ /developer/build/gnu-make \ /developer/build/automake \ /developer/build/libtool \ /developer/macro/gnu-m4 \ /developer/build/gnu-make \ /developer/gnu-binutils \ developer/build/autoconf \ developer/build/automake \ developer/lexer/flex \ developer/parser/bison \ developer/object-file \ developer/linker \ developer/library/lint \ developer/build/gnu-make \ library/idnkit \ library/idnkit/header-idnkit \ system/header \ system/library/math/header-math \ gcc44 \ gcc48 pkg install /omniti/perl/dbd-mysql \ /omniti/database/mysql-55/library pkg install libgcrypt And, this is what I use for building samba (I force 32 bit so winbind plays nicely with other 32 bit only tools like the "id" command): export ISALIST=i386 CFLAGS=-m32 CXXFLAGS=-m32 CPPFLAGS=-m32 LDFLAGS=-m32 \ ./configure \ --prefix=/usr/local \ --bindir=/usr/local/bin \ --sbindir=/usr/local/sbin \ --libdir=/usr/local/lib/ \ --mandir=/usr/local/man \ --infodir=/usr/local/info \ --sysconfdir=/etc/samba \ --with-configdir=/etc/samba \ --with-privatedir=/etc/samba/private \ --localstatedir=/var \ --sharedstatedir=/var \ --bundled-libraries=ALL \ --with-winbind \ --with-ads \ --with-ldap \ --with-pam \ --with-iconv \ --with-acl-support \ --with-syslog \ --with-aio-support \ --enable-fhs \ --without-ad-dc \ --with-shared-modules=idmap_ad,vfs_zfsacl,vfs_audit,vfs_catia,vfs_full_audit,vfs_readahead,vfs_streams_xattr,time_audit,vfs_fruit \ --enable-gnutls gmake gmake install Good luck! Michael > On Jan 20, 2016, at 11:48 AM, Schweiss, Chip wrote: > > I ended up downloading and building zlib separately and got it to build. > > The problem only occurs when selecting --with-ads during configure. It fails on checking gnutls, which needs zlib-devel. > > My build will not join the domain, but that's out of the scope of this list.. > > Thanks! > -Chip > > > > On Wed, Jan 20, 2016 at 1:02 PM, Peter Tribble > wrote: > On Wed, Jan 20, 2016 at 6:40 PM, Schweiss, Chip > wrote: > Is anyone aware of zlib and zlib-devel packages available anywhere for OmniOS? > > Installed in OmniOS by default, and cannot be uninstalled. > > These are needed for building any Samba version 4.2.0 or greater. > > We build samba (4.3.x) on OmniOS without any issues. We haven't done anything > beyond install the basic build tools. What sort of error are you getting? > > -- > -Peter Tribble > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From chip at innovates.com Wed Jan 20 21:00:27 2016 From: chip at innovates.com (Schweiss, Chip) Date: Wed, 20 Jan 2016 15:00:27 -0600 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: On Wed, Jan 20, 2016 at 2:03 PM, Dan McDonald wrote: > Probably not useful now for you on LTS, but Nexenta's SMB2 is available > for r151016 and later. > My biggest challenge is I have to support multiple domains on one server. That's forcing me to build from source because several paths get compiled in and break things. -Chip > > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chip at innovates.com Wed Jan 20 21:04:39 2016 From: chip at innovates.com (Schweiss, Chip) Date: Wed, 20 Jan 2016 15:04:39 -0600 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: I'll definitely be trying your build config. Is that joined to an Active Directory domain? If so I'm confused on how that works with the '--without-ad-dc' flag. Thanks! -Chip On Wed, Jan 20, 2016 at 2:29 PM, Michael Talbott wrote: > I have samba 4.2.3 working with all the bells and whistles (including > winbind). Also I have netatalk working along side it ;) > > Hope this helps: > > Here's what I installed as prereqs before compiling them: > > pkg install \ > library/security/openssl \ > naming/ldap \ > system/library/iconv/unicode \ > system/library/dbus \ > system/library/libdbus \ > system/library/libdbus-glib \ > developer/gnu-binutils \ > developer/build/libtool \ > developer/build/autoconf \ > system/library/math/header-math \ > /system/library/dbus \ > /system/library/libdbus-glib \ > /omniti/database/bdb \ > /text/gnu-gettext \ > /service/network/dns/mdns \ > /developer/build/gnu-make \ > /developer/build/automake \ > /developer/build/libtool \ > /developer/macro/gnu-m4 \ > /developer/build/gnu-make \ > /developer/gnu-binutils \ > developer/build/autoconf \ > developer/build/automake \ > developer/lexer/flex \ > developer/parser/bison \ > developer/object-file \ > developer/linker \ > developer/library/lint \ > developer/build/gnu-make \ > library/idnkit \ > library/idnkit/header-idnkit \ > system/header \ > system/library/math/header-math \ > gcc44 \ > gcc48 > > pkg install /omniti/perl/dbd-mysql \ > /omniti/database/mysql-55/library > > pkg install libgcrypt > > And, this is what I use for building samba (I force 32 bit so winbind > plays nicely with other 32 bit only tools like the "id" command): > > export ISALIST=i386 > CFLAGS=-m32 CXXFLAGS=-m32 CPPFLAGS=-m32 LDFLAGS=-m32 \ > ./configure \ > --prefix=/usr/local \ > --bindir=/usr/local/bin \ > --sbindir=/usr/local/sbin \ > --libdir=/usr/local/lib/ \ > --mandir=/usr/local/man \ > --infodir=/usr/local/info \ > --sysconfdir=/etc/samba \ > --with-configdir=/etc/samba \ > --with-privatedir=/etc/samba/private \ > --localstatedir=/var \ > --sharedstatedir=/var \ > --bundled-libraries=ALL \ > --with-winbind \ > --with-ads \ > --with-ldap \ > --with-pam \ > --with-iconv \ > --with-acl-support \ > --with-syslog \ > --with-aio-support \ > --enable-fhs \ > --without-ad-dc \ > > --with-shared-modules=idmap_ad,vfs_zfsacl,vfs_audit,vfs_catia,vfs_full_audit,vfs_readahead,vfs_streams_xattr,time_audit,vfs_fruit > \ > --enable-gnutls > > gmake > gmake install > > Good luck! > > Michael > > > On Jan 20, 2016, at 11:48 AM, Schweiss, Chip wrote: > > I ended up downloading and building zlib separately and got it to build. > > The problem only occurs when selecting --with-ads during configure. It > fails on checking gnutls, which needs zlib-devel. > > My build will not join the domain, but that's out of the scope of this > list.. > > Thanks! > -Chip > > > > On Wed, Jan 20, 2016 at 1:02 PM, Peter Tribble > wrote: > >> On Wed, Jan 20, 2016 at 6:40 PM, Schweiss, Chip >> wrote: >> >>> Is anyone aware of zlib and zlib-devel packages available anywhere for >>> OmniOS? >>> >> >> Installed in OmniOS by default, and cannot be uninstalled. >> >> >>> These are needed for building any Samba version 4.2.0 or greater. >>> >> >> We build samba (4.3.x) on OmniOS without any issues. We haven't done >> anything >> beyond install the basic build tools. What sort of error are you getting? >> >> -- >> -Peter Tribble >> http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ >> > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mtalbott at lji.org Wed Jan 20 21:12:35 2016 From: mtalbott at lji.org (Michael Talbott) Date: Wed, 20 Jan 2016 13:12:35 -0800 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: It is joined to an AD domain. The without-ad-dc flag just specifies that samba will be built without the ability for it to be an AD domain controller itself. But it doesn't prevent it from joining an existing domain. ________________________ Michael Talbott Systems Administrator La Jolla Institute > On Jan 20, 2016, at 1:00 PM, Schweiss, Chip wrote: > > > > On Wed, Jan 20, 2016 at 2:03 PM, Dan McDonald > wrote: > Probably not useful now for you on LTS, but Nexenta's SMB2 is available for r151016 and later. > > My biggest challenge is I have to support multiple domains on one server. That's forcing me to build from source because several paths get compiled in and break things. > > -Chip > > Dan > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Jan 20 21:34:32 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 20 Jan 2016 16:34:32 -0500 Subject: [OmniOS-discuss] Update BIND to 9.10.3-P3? In-Reply-To: References: Message-ID: <7A8CC3D8-7E1A-41C7-8352-237CFA9CC47C@omniti.com> > On Jan 20, 2016, at 11:17 AM, Dan McDonald wrote: > > Since I said I'd update, however, I will do so, but in the future, remember that the omnios publisher (i.e. what we officially support) doesn't have named in it. 014 & 016 now have bind 9.10.3-p3 Dan From contact at jacobvosmaer.nl Wed Jan 20 21:40:42 2016 From: contact at jacobvosmaer.nl (Jacob Vosmaer) Date: Wed, 20 Jan 2016 22:40:42 +0100 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: Dan, you mean to say that SMB2 is in bloody, right? Cheers, Jacob 2016-01-20 21:03 GMT+01:00 Dan McDonald : > Probably not useful now for you on LTS, but Nexenta's SMB2 is available > for r151016 and later. > > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Jan 20 21:42:53 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 20 Jan 2016 16:42:53 -0500 Subject: [OmniOS-discuss] zlib/zlib-devel packages In-Reply-To: References: Message-ID: <50983962-1F0C-4064-937A-111E59FE2C90@omniti.com> > On Jan 20, 2016, at 4:40 PM, Jacob Vosmaer wrote: > > Dan, you mean to say that SMB2 is in bloody, right? Dammit! That's TWICE I've flubbed that. Yes, SMB2 is in bloody. SMB-server-in-a-zone is in 016. SMB2 was too late upstream for the 016 cutoff. Sorry again, folks, Dan From youzhong at gmail.com Mon Jan 25 20:50:51 2016 From: youzhong at gmail.com (Youzhong Yang) Date: Mon, 25 Jan 2016 15:50:51 -0500 Subject: [OmniOS-discuss] Is it possible to roll back a ZPOOL(which cannot be imported) to its last known good state? Message-ID: Hi all, Just wondering if anyone has done similar recovery using txg stuff. We have a zpool attached to two hosts physically, ideally at any time only one host imports this zpool. Due to some operational mistake this zpool was corrupted when the two hosts tried to have access to it. Here is the crash stack: Jan 25 10:07:17 batfs0346 genunix: [ID 403854 kern.notice] assertion failed: 0 == dmu_bonus_hold(spa->spa_meta_objset, obj, FTAG, &db), file: ../../common/fs/zfs/spa.c, line: 1549 Jan 25 10:07:17 batfs0346 unix: [ID 100000 kern.notice] Jan 25 10:07:17 batfs0346 genunix: [ID 802836 kern.notice] ffffff017495c920 fffffffffba6b1f8 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495c9a0 zfs:load_nvlist+e8 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495ca90 zfs:spa_load_impl+10bb () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cb30 zfs:spa_load+14e () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cb80 zfs:spa_tryimport+aa () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cbd0 zfs:zfs_ioc_pool_tryimport+51 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cc80 zfs:zfsdev_ioctl+4a7 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495ccc0 genunix:cdev_ioctl+39 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cd10 specfs:spec_ioctl+60 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cda0 genunix:fop_ioctl+55 () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cec0 genunix:ioctl+9b () Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] ffffff017495cf10 unix:brand_sys_sysenter+1c9 () Is it possible to roll back the zpool to its last known good txg? We know when the zpool should be in good state. Any suggestion would be very much appreciated. We can build a kernel if needed. Thanks, - Youzhong -------------- next part -------------- An HTML attachment was scrubbed... URL: From jstockett at molalla.com Tue Jan 26 17:30:54 2016 From: jstockett at molalla.com (Jeff Stockett) Date: Tue, 26 Jan 2016 17:30:54 +0000 Subject: [OmniOS-discuss] 9207-8i P19 poor write performance Message-ID: <136C13E89D22BB468B2A7025993639733EB096FC@EXMCCMB.molalla.com> I'm trying to repurpose a couple older Dell R720xds we have that have 26x1TB SAS 10K spinners to host Windows Server backups via iSCSI since Windows Server backup does not work well with SMB shares and I'd like to get the backup I/O off our main storage servers. I've replaced the hopelessly broken onboard Dell H710 controller with a new 9207-8i that I downgraded from P20 to P19 per various posts I've seen. Installed latest version of R151016, everything went smoothly, and seems to work fine, but when I do basic performance tests on the pool using dd, I only get around 800MB/s for writes when I would expect > 2GB/s - read speeds are fine - about 2.5GB/s. The pool is 5x5raidz1 vdevs+1 boot drive (I also tried 4x6raidz2+2 mirrored boot with similar but even slightly slower results as expected). To rule out hardware issues, I re-installed Ubuntu 14.04 with ZFS, imported the pool, and write speeds were right where they should be around the 2.5GB/s mark on a 512GB dd (machine has 256GB of RAM at the moment but will probably put a lot of that somewhere else soon). Anyone aware of any issues or special tuning with the 2308 chipset and R151016? All our other storage servers are supermicro and have the 3008 chipset which seems to work great at least in the older build of OmniOS that they are all running. Thanks, Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: From youzhong at gmail.com Tue Jan 26 20:11:47 2016 From: youzhong at gmail.com (Youzhong Yang) Date: Tue, 26 Jan 2016 15:11:47 -0500 Subject: [OmniOS-discuss] Is it possible to roll back a ZPOOL(which cannot be imported) to its last known good state? In-Reply-To: References: Message-ID: Thanks Richard and Matthew. After applying fix for https://www.illumos.org/issues/5770, I was able to run zdb and zpool import with -X, -F, -T and etc. But unfortunately I am no luck to successfully import the zpool. -F or -T returned "cannot import 'zp13': one or more devices is currently unavailable" (which seems to fail at the following code block), -X seems to run forever so I just killed it. /* * Find the best uberblock. */ vdev_uberblock_load(rvd, ub, &label); /* * If we weren't able to find a single valid uberblock, return failure. */ if (ub->ub_txg == 0) { nvlist_free(label); return (spa_vdev_err(rvd, VDEV_AUX_CORRUPT_DATA, ENXIO)); } Fortunately we had backup of the zpool available so we just restored from the backup. Thanks again for the tips which may be useful in the future (I hope we will not make such mistake again). -Youzhong On Mon, Jan 25, 2016 at 3:50 PM, Youzhong Yang wrote: > Hi all, > > Just wondering if anyone has done similar recovery using txg stuff. > > We have a zpool attached to two hosts physically, ideally at any time only > one host imports this zpool. Due to some operational mistake this zpool was > corrupted when the two hosts tried to have access to it. Here is the crash > stack: > > Jan 25 10:07:17 batfs0346 genunix: [ID 403854 kern.notice] assertion > failed: 0 == dmu_bonus_hold(spa->spa_meta_objset, obj, FTAG, &db), file: > ../../common/fs/zfs/spa.c, line: 1549 > Jan 25 10:07:17 batfs0346 unix: [ID 100000 kern.notice] > Jan 25 10:07:17 batfs0346 genunix: [ID 802836 kern.notice] > ffffff017495c920 fffffffffba6b1f8 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495c9a0 zfs:load_nvlist+e8 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495ca90 zfs:spa_load_impl+10bb () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cb30 zfs:spa_load+14e () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cb80 zfs:spa_tryimport+aa () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cbd0 zfs:zfs_ioc_pool_tryimport+51 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cc80 zfs:zfsdev_ioctl+4a7 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495ccc0 genunix:cdev_ioctl+39 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cd10 specfs:spec_ioctl+60 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cda0 genunix:fop_ioctl+55 () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cec0 genunix:ioctl+9b () > Jan 25 10:07:17 batfs0346 genunix: [ID 655072 kern.notice] > ffffff017495cf10 unix:brand_sys_sysenter+1c9 () > > Is it possible to roll back the zpool to its last known good txg? We know > when the zpool should be in good state. > > Any suggestion would be very much appreciated. We can build a kernel if > needed. > > Thanks, > > - Youzhong > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Wed Jan 27 14:46:08 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 27 Jan 2016 08:46:08 -0600 (CST) Subject: [OmniOS-discuss] NTP vulnerabilities in OmniOS Message-ID: NTP ntp-4.2.8p6 was released on 19 January 2016 (OmniOS is at 4.2.8p5). It addresses these security issues: http://support.ntp.org/bin/view/Main/SecurityNotice#January_2016_NTP_4_2_8p6_Securit For some reason the xntp announcements list still fails to deliver any release announcements since April of last year. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Wed Jan 27 15:50:06 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 27 Jan 2016 10:50:06 -0500 Subject: [OmniOS-discuss] NTP vulnerabilities in OmniOS In-Reply-To: References: Message-ID: <057075A4-6F26-4F74-B4CE-BCED5C3D26EE@omniti.com> > On Jan 27, 2016, at 9:46 AM, Bob Friesenhahn wrote: > > NTP ntp-4.2.8p6 was released on 19 January 2016 (OmniOS is at 4.2.8p5). It addresses these security issues: > > http://support.ntp.org/bin/view/Main/SecurityNotice#January_2016_NTP_4_2_8p6_Securit > > For some reason the xntp announcements list still fails to deliver any release announcements since April of last year. And I didn't see this on OSS-Security either, but that is a higher-traffic list. It'll come later this week, alongside OpenSSL 1.0.2f if not sooner. Dan From danmcd at omniti.com Wed Jan 27 16:59:37 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 27 Jan 2016 11:59:37 -0500 Subject: [OmniOS-discuss] NTP 4.2.8p6 is out, OpenSSL coming soon Message-ID: Both LTS and Stable now have the newest NTP in their repo. If you're an NTP server, please "pkg update" as soon as you can, because it's a security patch. Also, we are aware of OpenSSL 1.0.2f coming out tomorrow. There will be travel among the OmniOS team, however, and so we may be slower-than-normal in getting it out (depending on the 1.0.2f release timings, flights, layovers, delays, wifi availability, etc. etc.). It should be ready before FOSDEM begins this weekend. Speaking of FOSDEM, I will be speaking there on Sunday at 12noon, and hanging around the illumos booth the rest of the time. Dan From miniflowtrader at gmail.com Thu Jan 28 03:04:30 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Wed, 27 Jan 2016 22:04:30 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter Message-ID: Slow CIFS Writes when using Moca 2.0 Adapter. I am experiencing this only under OmniOS. I do not see this in Windows or Linux. I have a ZFS CIFS share setup which can easily do writes that would saturate a 1GBe connection. My problem appears to be related somehow to the interaction between OmniOS and ECB6200 Moca 2.0 adapters. 1. If I write to my OmniOS CIFS share using ethernet my speeds up/down are around 110 mb/sec - good 2. If I write to my share using the same source but over the adapter my speeds are around 35mb/sec - problem 3. If I read from the share using the same device over the adapter my speeds are around 110mb/sec - good 4. If I setup a share on a Windows machine and write to it from the same source using the adapter the speeds are around 110mb/sec. The Windows machine is actually a VM whos disks are backed by a ZFS NFS share on the same machine So basically the issue only takes place when writing to the OmniOS CIFS share using the adapter, if the adapter is not used than the write speed is perfect. Any ideas why/how a Moca 2.0 adapter which is just designed to convert an ethernet signal to a coax and back to ethernet would cause issues with writes on OmniOS when the exact same share has no issues when using an actual ethernet connection? More importantly, why is this happening with OmniOS CIFS and not anything else? I appreciate any help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Thu Jan 28 03:35:46 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 27 Jan 2016 21:35:46 -0600 (CST) Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: On Wed, 27 Jan 2016, Mini Trader wrote: > Slow CIFS Writes when using Moca 2.0 Adapter. > > I am experiencing this only under OmniOS.? I do not see this in Windows or Linux. > > I have a ZFS CIFS share setup which can easily do writes that would saturate a 1GBe connection. > > My problem appears to be related somehow to the interaction between OmniOS and ECB6200 Moca 2.0 adapters. > > 1. If I write to my OmniOS CIFS share using ethernet my speeds up/down are around 110 mb/sec - good > > 2. If I write to my share using the same source but over the adapter my speeds are around 35mb/sec - problem MoCA has a 3.0+ millisecond latency (I typically see 3.5ms when using ping). This latency is fairly large compared with typical hard drive latencies and vastly higher than Ethernet. There is nothing which can be done about this latency. Unbonded MoCA 2.0 throughput for streaming data is typically 500Mbit/second, and bonded (two channels) MoCA 2.0 doubles that (the claimed specs are of course higher than this and higher speeds can be measured under ideal conditions). This means that typical MoCA 2.0 (not bonded) achieves a bit less than half of what gigabit Ethernet achieves when streaming data over TCP. > 3. If I read from the share using the same device over the adapter my speeds are around 110mb/sec - good Reading is normally more of a streaming operation so the TCP will stream rather well. > 4. If I setup a share on a Windows machine and write to it from the same source using the ?adapter the speeds are > around 110mb/sec.? The Windows machine is actually a VM whos disks are backed by a ZFS NFS share on the same > machine This seems rather good. Quite a lot depends on what the server side does. If it commits each write to disk before accepting more, then the write speed would suffer. > So basically the issue only takes place when writing to the OmniOS CIFS share using the adapter, if the adapter is > not used than the write speed is perfect. If the MoCA adaptor supports bonded mode, then it is useful to know that usually bonded mode needs to be enabled. Is it possible that the Windows driver is enabling bonded mode but the OmniOS driver does not? Try running a TCP streaming benchmark (program to program) to see what the peak network throughput is in each case. > Any ideas why/how a Moca 2.0 adapter which is just designed to convert an ethernet ?signal to a coax and back to > ethernet ?would cause issues with writes on OmniOS when the exact same share has no issues when using an actual > ethernet connection?? More importantly, why is this happening with OmniOS CIFS and not anything else? Latency, synchronous writes, and possibly bonding not enabled. Also, OmniOS r151016 or later is need to get the latest CIFS implementation (based on Nexenta changes), which has been reported on this list to be quite a lot faster than the older one. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Thu Jan 28 03:40:14 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 27 Jan 2016 22:40:14 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: <9807A432-B9BB-4390-ACEE-B80540C9A872@omniti.com> And even more will be on the way with this current bloody cycle and r151018. (I.e. Smb2) Dan Sent from my iPhone (typos, autocorrect, and all) > On Jan 27, 2016, at 10:35 PM, Bob Friesenhahn wrote: > > enabled. Also, OmniOS r151016 or later is need to get the latest CIFS implementation (based on Nexenta changes), which has been reported on this list to be quite a lot faster than the older one. From miniflowtrader at gmail.com Thu Jan 28 16:34:47 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Thu, 28 Jan 2016 11:34:47 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: Thank you for all the responses! Ive run some more detailed tests using iperf 2. The results that I see are inline with the transfer rates so they describe the behavior that I am seeing. Note I used a laptop on same connection as desktop. So that there would be a basis to compare it to the Desktop. For some reason the laptop has a limit of around 500-600 mbit/sec for its downloads, regardless the test still seem to show the behavior that I am seeing. Note that Linux does not seem to have the same issues where OmniOS does. Additionally OmniOS does not have the issue when using a direct ethernet connection. One thing I can say about Linux is that its downloads on the adapters are less than its uploads which is the complete opposite as OmniOS. This Linux behavior is not seen when using ethernet. Both Linux and OmniOS are running on ESXi 6U1. OmniOS is using the vmxnet driver. The adapters being used are Adaptec ECB6200. These are bonded Moca 2.0 adapters and are running the latest firmware. Source Machine: Desktop Connection: Adapter Windows <-> OmniOS Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to storage1, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.141 port 31595 connected with 10.255.0.15 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 34.9 MBytes 293 Mbits/sec [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec [ 4] 2.0- 3.0 sec 35.2 MBytes 296 Mbits/sec [ 4] 3.0- 4.0 sec 34.4 MBytes 288 Mbits/sec [ 4] 4.0- 5.0 sec 34.5 MBytes 289 Mbits/sec [ 4] 0.0- 5.0 sec 174 MBytes 292 Mbits/sec [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 33341 [ 4] 0.0- 1.0 sec 46.2 MBytes 388 Mbits/sec [ 4] 1.0- 2.0 sec 101 MBytes 849 Mbits/sec [ 4] 2.0- 3.0 sec 104 MBytes 872 Mbits/sec [ 4] 3.0- 4.0 sec 101 MBytes 851 Mbits/sec [ 4] 4.0- 5.0 sec 102 MBytes 855 Mbits/sec [ 4] 0.0- 5.0 sec 457 MBytes 763 Mbits/sec Source Machine: Desktop Connection: Adapter Windows <-> Linux Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to media.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.141 port 31602 connected with 10.255.0.73 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 108 MBytes 902 Mbits/sec [ 4] 1.0- 2.0 sec 111 MBytes 929 Mbits/sec [ 4] 2.0- 3.0 sec 111 MBytes 928 Mbits/sec [ 4] 3.0- 4.0 sec 106 MBytes 892 Mbits/sec [ 4] 4.0- 5.0 sec 109 MBytes 918 Mbits/sec [ 4] 0.0- 5.0 sec 545 MBytes 914 Mbits/sec [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.73 port 55045 [ 4] 0.0- 1.0 sec 67.0 MBytes 562 Mbits/sec [ 4] 1.0- 2.0 sec 75.6 MBytes 634 Mbits/sec [ 4] 2.0- 3.0 sec 75.1 MBytes 630 Mbits/sec [ 4] 3.0- 4.0 sec 74.5 MBytes 625 Mbits/sec [ 4] 4.0- 5.0 sec 75.7 MBytes 635 Mbits/sec [ 4] 0.0- 5.0 sec 368 MBytes 616 Mbits/sec Machine: Laptop Connection: Adapter Windows <-> OmniOS Notice same issue with 35mb cap. ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to storage1.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.54 port 57487 connected with 10.255.0.15 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 35.5 MBytes 298 Mbits/sec [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec [ 4] 2.0- 3.0 sec 35.0 MBytes 294 Mbits/sec [ 4] 3.0- 4.0 sec 34.2 MBytes 287 Mbits/sec [ 4] 4.0- 5.0 sec 33.9 MBytes 284 Mbits/sec [ 4] 0.0- 5.0 sec 174 MBytes 291 Mbits/sec [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 40779 [ 4] 0.0- 1.0 sec 28.8 MBytes 242 Mbits/sec [ 4] 1.0- 2.0 sec 55.8 MBytes 468 Mbits/sec [ 4] 2.0- 3.0 sec 43.7 MBytes 366 Mbits/sec [ 4] 3.0- 4.0 sec 50.7 MBytes 425 Mbits/sec [ 4] 4.0- 5.0 sec 52.7 MBytes 442 Mbits/sec [ 4] 0.0- 5.0 sec 233 MBytes 389 Mbits/sec Machine: Laptop Connection: Adapter Windows <-> Linux (not issue on upload, same as desktop) Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to media.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.54 port 57387 connected with 10.255.0.73 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 110 MBytes 919 Mbits/sec [ 4] 1.0- 2.0 sec 110 MBytes 920 Mbits/sec [ 4] 2.0- 3.0 sec 110 MBytes 921 Mbits/sec [ 4] 3.0- 4.0 sec 110 MBytes 923 Mbits/sec [ 4] 4.0- 5.0 sec 110 MBytes 919 Mbits/sec [ 4] 0.0- 5.0 sec 548 MBytes 919 Mbits/sec [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52723 [ 4] 0.0- 1.0 sec 49.8 MBytes 418 Mbits/sec [ 4] 1.0- 2.0 sec 55.1 MBytes 462 Mbits/sec [ 4] 2.0- 3.0 sec 55.1 MBytes 462 Mbits/sec [ 4] 3.0- 4.0 sec 53.6 MBytes 449 Mbits/sec [ 4] 4.0- 5.0 sec 56.9 MBytes 477 Mbits/sec [ 4] 0.0- 5.0 sec 271 MBytes 454 Mbits/sec Machine: Laptop Connection: Ethernet Windows <-> OmniOS (No issues on upload) ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to storage1.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.54 port 57858 connected with 10.255.0.15 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 113 MBytes 950 Mbits/sec [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec [ 4] 2.0- 3.0 sec 109 MBytes 912 Mbits/sec [ 4] 3.0- 4.0 sec 111 MBytes 931 Mbits/sec [ 4] 4.0- 5.0 sec 106 MBytes 889 Mbits/sec [ 4] 0.0- 5.0 sec 550 MBytes 921 Mbits/sec [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 42565 [ 4] 0.0- 1.0 sec 38.4 MBytes 322 Mbits/sec [ 4] 1.0- 2.0 sec 68.9 MBytes 578 Mbits/sec [ 4] 2.0- 3.0 sec 67.7 MBytes 568 Mbits/sec [ 4] 3.0- 4.0 sec 66.7 MBytes 559 Mbits/sec [ 4] 4.0- 5.0 sec 63.2 MBytes 530 Mbits/sec [ 4] 0.0- 5.0 sec 306 MBytes 513 Mbits/sec Machine: Laptop Connection: Ethernet Windows <-> Linux (Exact same speeds this time as OmnioOS) ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to media.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.54 port 57966 connected with 10.255.0.73 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 110 MBytes 920 Mbits/sec [ 4] 1.0- 2.0 sec 111 MBytes 932 Mbits/sec [ 4] 2.0- 3.0 sec 111 MBytes 931 Mbits/sec [ 4] 3.0- 4.0 sec 108 MBytes 902 Mbits/sec [ 4] 4.0- 5.0 sec 106 MBytes 887 Mbits/sec [ 4] 0.0- 5.0 sec 545 MBytes 913 Mbits/sec [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52726 [ 4] 0.0- 1.0 sec 63.4 MBytes 532 Mbits/sec [ 4] 1.0- 2.0 sec 62.9 MBytes 528 Mbits/sec [ 4] 2.0- 3.0 sec 66.7 MBytes 560 Mbits/sec [ 4] 3.0- 4.0 sec 65.3 MBytes 548 Mbits/sec [ 4] 4.0- 5.0 sec 66.8 MBytes 560 Mbits/sec [ 4] 0.0- 5.0 sec 326 MBytes 545 Mbits/sec On Wed, Jan 27, 2016 at 10:35 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > On Wed, 27 Jan 2016, Mini Trader wrote: > > Slow CIFS Writes when using Moca 2.0 Adapter. >> >> I am experiencing this only under OmniOS. I do not see this in Windows >> or Linux. >> >> I have a ZFS CIFS share setup which can easily do writes that would >> saturate a 1GBe connection. >> >> My problem appears to be related somehow to the interaction between >> OmniOS and ECB6200 Moca 2.0 adapters. >> >> 1. If I write to my OmniOS CIFS share using ethernet my speeds up/down >> are around 110 mb/sec - good >> >> 2. If I write to my share using the same source but over the adapter my >> speeds are around 35mb/sec - problem >> > > MoCA has a 3.0+ millisecond latency (I typically see 3.5ms when using > ping). This latency is fairly large compared with typical hard drive > latencies and vastly higher than Ethernet. There is nothing which can be > done about this latency. > > Unbonded MoCA 2.0 throughput for streaming data is typically > 500Mbit/second, and bonded (two channels) MoCA 2.0 doubles that (the > claimed specs are of course higher than this and higher speeds can be > measured under ideal conditions). This means that typical MoCA 2.0 (not > bonded) achieves a bit less than half of what gigabit Ethernet achieves > when streaming data over TCP. > > 3. If I read from the share using the same device over the adapter my >> speeds are around 110mb/sec - good >> > > Reading is normally more of a streaming operation so the TCP will stream > rather well. > > 4. If I setup a share on a Windows machine and write to it from the same >> source using the adapter the speeds are >> around 110mb/sec. The Windows machine is actually a VM whos disks are >> backed by a ZFS NFS share on the same >> machine >> > > This seems rather good. Quite a lot depends on what the server side does. > If it commits each write to disk before accepting more, then the write > speed would suffer. > > So basically the issue only takes place when writing to the OmniOS CIFS >> share using the adapter, if the adapter is >> not used than the write speed is perfect. >> > > If the MoCA adaptor supports bonded mode, then it is useful to know that > usually bonded mode needs to be enabled. Is it possible that the Windows > driver is enabling bonded mode but the OmniOS driver does not? > > Try running a TCP streaming benchmark (program to program) to see what the > peak network throughput is in each case. > > Any ideas why/how a Moca 2.0 adapter which is just designed to convert an >> ethernet signal to a coax and back to >> ethernet would cause issues with writes on OmniOS when the exact same >> share has no issues when using an actual >> ethernet connection? More importantly, why is this happening with OmniOS >> CIFS and not anything else? >> > > Latency, synchronous writes, and possibly bonding not enabled. Also, > OmniOS r151016 or later is need to get the latest CIFS implementation > (based on Nexenta changes), which has been reported on this list to be > quite a lot faster than the older one. > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From miniflowtrader at gmail.com Thu Jan 28 18:39:11 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Thu, 28 Jan 2016 13:39:11 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: I also tried the following. Which seems to have improved iperf speeds. But I am still getting the same CIFS speeds. root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p recv_buf=1048576 tcp root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p send_buf=1048576 tcp root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p max_buf=4194304 tcp ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to storage1.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.141 port 33452 connected with 10.255.0.15 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 106 MBytes 892 Mbits/sec [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec [ 4] 2.0- 3.0 sec 108 MBytes 904 Mbits/sec [ 4] 3.0- 4.0 sec 109 MBytes 916 Mbits/sec [ 4] 4.0- 5.0 sec 110 MBytes 923 Mbits/sec [ 4] 5.0- 6.0 sec 110 MBytes 919 Mbits/sec [ 4] 6.0- 7.0 sec 110 MBytes 919 Mbits/sec [ 4] 7.0- 8.0 sec 105 MBytes 884 Mbits/sec [ 4] 8.0- 9.0 sec 109 MBytes 915 Mbits/sec [ 4] 9.0-10.0 sec 111 MBytes 928 Mbits/sec [ 4] 0.0-10.0 sec 1.06 GBytes 912 Mbits/sec [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 50899 [ 4] 0.0- 1.0 sec 97.5 MBytes 818 Mbits/sec [ 4] 1.0- 2.0 sec 110 MBytes 923 Mbits/sec [ 4] 2.0- 3.0 sec 49.3 MBytes 414 Mbits/sec [ 4] 3.0- 4.0 sec 98.0 MBytes 822 Mbits/sec [ 4] 4.0- 5.0 sec 96.7 MBytes 811 Mbits/sec [ 4] 5.0- 6.0 sec 99.7 MBytes 836 Mbits/sec [ 4] 6.0- 7.0 sec 103 MBytes 861 Mbits/sec [ 4] 7.0- 8.0 sec 101 MBytes 851 Mbits/sec [ 4] 8.0- 9.0 sec 104 MBytes 876 Mbits/sec [ 4] 9.0-10.0 sec 104 MBytes 876 Mbits/sec [ 4] 0.0-10.0 sec 966 MBytes 808 Mbits/sec root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p recv_buf tcp root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p send_buf tcp root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p max_buf tcp ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to storage1.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.141 port 33512 connected with 10.255.0.15 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 35.2 MBytes 296 Mbits/sec [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec [ 4] 2.0- 3.0 sec 34.2 MBytes 287 Mbits/sec [ 4] 3.0- 4.0 sec 33.4 MBytes 280 Mbits/sec [ 4] 4.0- 5.0 sec 34.1 MBytes 286 Mbits/sec [ 4] 5.0- 6.0 sec 35.2 MBytes 296 Mbits/sec [ 4] 6.0- 7.0 sec 35.4 MBytes 297 Mbits/sec [ 4] 7.0- 8.0 sec 34.4 MBytes 288 Mbits/sec [ 4] 8.0- 9.0 sec 35.0 MBytes 294 Mbits/sec [ 4] 9.0-10.0 sec 33.4 MBytes 280 Mbits/sec [ 4] 0.0-10.0 sec 346 MBytes 289 Mbits/sec [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 41435 [ 4] 0.0- 1.0 sec 57.6 MBytes 483 Mbits/sec [ 4] 1.0- 2.0 sec 87.2 MBytes 732 Mbits/sec [ 4] 2.0- 3.0 sec 99.3 MBytes 833 Mbits/sec [ 4] 3.0- 4.0 sec 99.5 MBytes 835 Mbits/sec [ 4] 4.0- 5.0 sec 100 MBytes 842 Mbits/sec [ 4] 5.0- 6.0 sec 103 MBytes 866 Mbits/sec [ 4] 6.0- 7.0 sec 100 MBytes 840 Mbits/sec [ 4] 7.0- 8.0 sec 98.7 MBytes 828 Mbits/sec [ 4] 8.0- 9.0 sec 101 MBytes 847 Mbits/sec [ 4] 9.0-10.0 sec 105 MBytes 882 Mbits/sec [ 4] 0.0-10.0 sec 954 MBytes 799 Mbits/sec On Thu, Jan 28, 2016 at 11:34 AM, Mini Trader wrote: > Thank you for all the responses! Ive run some more detailed tests using > iperf 2. The results that I see are inline with the transfer rates so they > describe the behavior that I am seeing. > > Note I used a laptop on same connection as desktop. So that there would > be a basis to compare it to the Desktop. > > For some reason the laptop has a limit of around 500-600 mbit/sec for its > downloads, regardless the test still seem to show the behavior > that I am seeing. Note that Linux does not seem to have the same issues > where OmniOS does. Additionally OmniOS does not have the issue > when using a direct ethernet connection. One thing I can say about Linux > is that its downloads on the adapters are less than its uploads which > is the complete opposite as OmniOS. This Linux behavior is not seen when > using ethernet. > > Both Linux and OmniOS are running on ESXi 6U1. OmniOS is using the vmxnet > driver. > > The adapters being used are Adaptec ECB6200. These are bonded Moca 2.0 > adapters and are running the latest firmware. > > Source Machine: Desktop > Connection: Adapter > Windows <-> OmniOS > > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to storage1, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.141 port 31595 connected with 10.255.0.15 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 34.9 MBytes 293 Mbits/sec > [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec > [ 4] 2.0- 3.0 sec 35.2 MBytes 296 Mbits/sec > [ 4] 3.0- 4.0 sec 34.4 MBytes 288 Mbits/sec > [ 4] 4.0- 5.0 sec 34.5 MBytes 289 Mbits/sec > [ 4] 0.0- 5.0 sec 174 MBytes 292 Mbits/sec > [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 33341 > [ 4] 0.0- 1.0 sec 46.2 MBytes 388 Mbits/sec > [ 4] 1.0- 2.0 sec 101 MBytes 849 Mbits/sec > [ 4] 2.0- 3.0 sec 104 MBytes 872 Mbits/sec > [ 4] 3.0- 4.0 sec 101 MBytes 851 Mbits/sec > [ 4] 4.0- 5.0 sec 102 MBytes 855 Mbits/sec > [ 4] 0.0- 5.0 sec 457 MBytes 763 Mbits/sec > > Source Machine: Desktop > Connection: Adapter > Windows <-> Linux > > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to media.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.141 port 31602 connected with 10.255.0.73 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 108 MBytes 902 Mbits/sec > [ 4] 1.0- 2.0 sec 111 MBytes 929 Mbits/sec > [ 4] 2.0- 3.0 sec 111 MBytes 928 Mbits/sec > [ 4] 3.0- 4.0 sec 106 MBytes 892 Mbits/sec > [ 4] 4.0- 5.0 sec 109 MBytes 918 Mbits/sec > [ 4] 0.0- 5.0 sec 545 MBytes 914 Mbits/sec > [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.73 port 55045 > [ 4] 0.0- 1.0 sec 67.0 MBytes 562 Mbits/sec > [ 4] 1.0- 2.0 sec 75.6 MBytes 634 Mbits/sec > [ 4] 2.0- 3.0 sec 75.1 MBytes 630 Mbits/sec > [ 4] 3.0- 4.0 sec 74.5 MBytes 625 Mbits/sec > [ 4] 4.0- 5.0 sec 75.7 MBytes 635 Mbits/sec > [ 4] 0.0- 5.0 sec 368 MBytes 616 Mbits/sec > > > Machine: Laptop > Connection: Adapter > Windows <-> OmniOS Notice same issue with 35mb cap. > > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to storage1.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.54 port 57487 connected with 10.255.0.15 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 35.5 MBytes 298 Mbits/sec > [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec > [ 4] 2.0- 3.0 sec 35.0 MBytes 294 Mbits/sec > [ 4] 3.0- 4.0 sec 34.2 MBytes 287 Mbits/sec > [ 4] 4.0- 5.0 sec 33.9 MBytes 284 Mbits/sec > [ 4] 0.0- 5.0 sec 174 MBytes 291 Mbits/sec > [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 40779 > [ 4] 0.0- 1.0 sec 28.8 MBytes 242 Mbits/sec > [ 4] 1.0- 2.0 sec 55.8 MBytes 468 Mbits/sec > [ 4] 2.0- 3.0 sec 43.7 MBytes 366 Mbits/sec > [ 4] 3.0- 4.0 sec 50.7 MBytes 425 Mbits/sec > [ 4] 4.0- 5.0 sec 52.7 MBytes 442 Mbits/sec > [ 4] 0.0- 5.0 sec 233 MBytes 389 Mbits/sec > > Machine: Laptop > Connection: Adapter > Windows <-> Linux (not issue on upload, same as desktop) > > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to media.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.54 port 57387 connected with 10.255.0.73 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 110 MBytes 919 Mbits/sec > [ 4] 1.0- 2.0 sec 110 MBytes 920 Mbits/sec > [ 4] 2.0- 3.0 sec 110 MBytes 921 Mbits/sec > [ 4] 3.0- 4.0 sec 110 MBytes 923 Mbits/sec > [ 4] 4.0- 5.0 sec 110 MBytes 919 Mbits/sec > [ 4] 0.0- 5.0 sec 548 MBytes 919 Mbits/sec > [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52723 > [ 4] 0.0- 1.0 sec 49.8 MBytes 418 Mbits/sec > [ 4] 1.0- 2.0 sec 55.1 MBytes 462 Mbits/sec > [ 4] 2.0- 3.0 sec 55.1 MBytes 462 Mbits/sec > [ 4] 3.0- 4.0 sec 53.6 MBytes 449 Mbits/sec > [ 4] 4.0- 5.0 sec 56.9 MBytes 477 Mbits/sec > [ 4] 0.0- 5.0 sec 271 MBytes 454 Mbits/sec > > Machine: Laptop > Connection: Ethernet > Windows <-> OmniOS (No issues on upload) > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to storage1.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.54 port 57858 connected with 10.255.0.15 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 113 MBytes 950 Mbits/sec > [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec > [ 4] 2.0- 3.0 sec 109 MBytes 912 Mbits/sec > [ 4] 3.0- 4.0 sec 111 MBytes 931 Mbits/sec > [ 4] 4.0- 5.0 sec 106 MBytes 889 Mbits/sec > [ 4] 0.0- 5.0 sec 550 MBytes 921 Mbits/sec > [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 42565 > [ 4] 0.0- 1.0 sec 38.4 MBytes 322 Mbits/sec > [ 4] 1.0- 2.0 sec 68.9 MBytes 578 Mbits/sec > [ 4] 2.0- 3.0 sec 67.7 MBytes 568 Mbits/sec > [ 4] 3.0- 4.0 sec 66.7 MBytes 559 Mbits/sec > [ 4] 4.0- 5.0 sec 63.2 MBytes 530 Mbits/sec > [ 4] 0.0- 5.0 sec 306 MBytes 513 Mbits/sec > > Machine: Laptop > Connection: Ethernet > Windows <-> Linux (Exact same speeds this time as OmnioOS) > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to media.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.54 port 57966 connected with 10.255.0.73 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 110 MBytes 920 Mbits/sec > [ 4] 1.0- 2.0 sec 111 MBytes 932 Mbits/sec > [ 4] 2.0- 3.0 sec 111 MBytes 931 Mbits/sec > [ 4] 3.0- 4.0 sec 108 MBytes 902 Mbits/sec > [ 4] 4.0- 5.0 sec 106 MBytes 887 Mbits/sec > [ 4] 0.0- 5.0 sec 545 MBytes 913 Mbits/sec > [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52726 > [ 4] 0.0- 1.0 sec 63.4 MBytes 532 Mbits/sec > [ 4] 1.0- 2.0 sec 62.9 MBytes 528 Mbits/sec > [ 4] 2.0- 3.0 sec 66.7 MBytes 560 Mbits/sec > [ 4] 3.0- 4.0 sec 65.3 MBytes 548 Mbits/sec > [ 4] 4.0- 5.0 sec 66.8 MBytes 560 Mbits/sec > [ 4] 0.0- 5.0 sec 326 MBytes 545 Mbits/sec > > > On Wed, Jan 27, 2016 at 10:35 PM, Bob Friesenhahn < > bfriesen at simple.dallas.tx.us> wrote: > >> On Wed, 27 Jan 2016, Mini Trader wrote: >> >> Slow CIFS Writes when using Moca 2.0 Adapter. >>> >>> I am experiencing this only under OmniOS. I do not see this in Windows >>> or Linux. >>> >>> I have a ZFS CIFS share setup which can easily do writes that would >>> saturate a 1GBe connection. >>> >>> My problem appears to be related somehow to the interaction between >>> OmniOS and ECB6200 Moca 2.0 adapters. >>> >>> 1. If I write to my OmniOS CIFS share using ethernet my speeds up/down >>> are around 110 mb/sec - good >>> >>> 2. If I write to my share using the same source but over the adapter my >>> speeds are around 35mb/sec - problem >>> >> >> MoCA has a 3.0+ millisecond latency (I typically see 3.5ms when using >> ping). This latency is fairly large compared with typical hard drive >> latencies and vastly higher than Ethernet. There is nothing which can be >> done about this latency. >> >> Unbonded MoCA 2.0 throughput for streaming data is typically >> 500Mbit/second, and bonded (two channels) MoCA 2.0 doubles that (the >> claimed specs are of course higher than this and higher speeds can be >> measured under ideal conditions). This means that typical MoCA 2.0 (not >> bonded) achieves a bit less than half of what gigabit Ethernet achieves >> when streaming data over TCP. >> >> 3. If I read from the share using the same device over the adapter my >>> speeds are around 110mb/sec - good >>> >> >> Reading is normally more of a streaming operation so the TCP will stream >> rather well. >> >> 4. If I setup a share on a Windows machine and write to it from the same >>> source using the adapter the speeds are >>> around 110mb/sec. The Windows machine is actually a VM whos disks are >>> backed by a ZFS NFS share on the same >>> machine >>> >> >> This seems rather good. Quite a lot depends on what the server side >> does. If it commits each write to disk before accepting more, then the >> write speed would suffer. >> >> So basically the issue only takes place when writing to the OmniOS CIFS >>> share using the adapter, if the adapter is >>> not used than the write speed is perfect. >>> >> >> If the MoCA adaptor supports bonded mode, then it is useful to know that >> usually bonded mode needs to be enabled. Is it possible that the Windows >> driver is enabling bonded mode but the OmniOS driver does not? >> >> Try running a TCP streaming benchmark (program to program) to see what >> the peak network throughput is in each case. >> >> Any ideas why/how a Moca 2.0 adapter which is just designed to convert an >>> ethernet signal to a coax and back to >>> ethernet would cause issues with writes on OmniOS when the exact same >>> share has no issues when using an actual >>> ethernet connection? More importantly, why is this happening with >>> OmniOS CIFS and not anything else? >>> >> >> Latency, synchronous writes, and possibly bonding not enabled. Also, >> OmniOS r151016 or later is need to get the latest CIFS implementation >> (based on Nexenta changes), which has been reported on this list to be >> quite a lot faster than the older one. >> >> Bob >> -- >> Bob Friesenhahn >> bfriesen at simple.dallas.tx.us, >> http://www.simplesystems.org/users/bfriesen/ >> GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Jan 28 19:28:53 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 28 Jan 2016 14:28:53 -0500 Subject: [OmniOS-discuss] OpenSSL now at 1.0.2f for LTS and Stable Message-ID: <3B7833EC-2C16-497C-AC12-481D6390961E@omniti.com> OpenSSL details are here: http://openssl.org/news/secadv/20160128.txt I've pushed out 1.0.2f for r151014 (LTS) and r151016 (Stable). Bloody will be getting its long-delayed total update very soon (today if my flight timings work out) so please stay tuned. Happy updating! Dan From miniflowtrader at gmail.com Thu Jan 28 19:30:03 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Thu, 28 Jan 2016 14:30:03 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: Is there a way to adjust the default Window Size for CIFS or NFS? On Thu, Jan 28, 2016 at 1:39 PM, Mini Trader wrote: > I also tried the following. Which seems to have improved iperf speeds. > But I am still getting the same CIFS speeds. > > root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p > recv_buf=1048576 tcp > root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p > send_buf=1048576 tcp > root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p > max_buf=4194304 tcp > > > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to storage1.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.141 port 33452 connected with 10.255.0.15 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 106 MBytes 892 Mbits/sec > [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec > [ 4] 2.0- 3.0 sec 108 MBytes 904 Mbits/sec > [ 4] 3.0- 4.0 sec 109 MBytes 916 Mbits/sec > [ 4] 4.0- 5.0 sec 110 MBytes 923 Mbits/sec > [ 4] 5.0- 6.0 sec 110 MBytes 919 Mbits/sec > [ 4] 6.0- 7.0 sec 110 MBytes 919 Mbits/sec > [ 4] 7.0- 8.0 sec 105 MBytes 884 Mbits/sec > [ 4] 8.0- 9.0 sec 109 MBytes 915 Mbits/sec > [ 4] 9.0-10.0 sec 111 MBytes 928 Mbits/sec > [ 4] 0.0-10.0 sec 1.06 GBytes 912 Mbits/sec > [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 50899 > [ 4] 0.0- 1.0 sec 97.5 MBytes 818 Mbits/sec > [ 4] 1.0- 2.0 sec 110 MBytes 923 Mbits/sec > [ 4] 2.0- 3.0 sec 49.3 MBytes 414 Mbits/sec > [ 4] 3.0- 4.0 sec 98.0 MBytes 822 Mbits/sec > [ 4] 4.0- 5.0 sec 96.7 MBytes 811 Mbits/sec > [ 4] 5.0- 6.0 sec 99.7 MBytes 836 Mbits/sec > [ 4] 6.0- 7.0 sec 103 MBytes 861 Mbits/sec > [ 4] 7.0- 8.0 sec 101 MBytes 851 Mbits/sec > [ 4] 8.0- 9.0 sec 104 MBytes 876 Mbits/sec > [ 4] 9.0-10.0 sec 104 MBytes 876 Mbits/sec > [ 4] 0.0-10.0 sec 966 MBytes 808 Mbits/sec > > root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p recv_buf > tcp > root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p send_buf > tcp > root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p max_buf > tcp > > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > ------------------------------------------------------------ > Client connecting to storage1.midway, TCP port 5001 > TCP window size: 977 KByte > ------------------------------------------------------------ > [ 4] local 10.255.0.141 port 33512 connected with 10.255.0.15 port 5001 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0- 1.0 sec 35.2 MBytes 296 Mbits/sec > [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec > [ 4] 2.0- 3.0 sec 34.2 MBytes 287 Mbits/sec > [ 4] 3.0- 4.0 sec 33.4 MBytes 280 Mbits/sec > [ 4] 4.0- 5.0 sec 34.1 MBytes 286 Mbits/sec > [ 4] 5.0- 6.0 sec 35.2 MBytes 296 Mbits/sec > [ 4] 6.0- 7.0 sec 35.4 MBytes 297 Mbits/sec > [ 4] 7.0- 8.0 sec 34.4 MBytes 288 Mbits/sec > [ 4] 8.0- 9.0 sec 35.0 MBytes 294 Mbits/sec > [ 4] 9.0-10.0 sec 33.4 MBytes 280 Mbits/sec > [ 4] 0.0-10.0 sec 346 MBytes 289 Mbits/sec > [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 41435 > [ 4] 0.0- 1.0 sec 57.6 MBytes 483 Mbits/sec > [ 4] 1.0- 2.0 sec 87.2 MBytes 732 Mbits/sec > [ 4] 2.0- 3.0 sec 99.3 MBytes 833 Mbits/sec > [ 4] 3.0- 4.0 sec 99.5 MBytes 835 Mbits/sec > [ 4] 4.0- 5.0 sec 100 MBytes 842 Mbits/sec > [ 4] 5.0- 6.0 sec 103 MBytes 866 Mbits/sec > [ 4] 6.0- 7.0 sec 100 MBytes 840 Mbits/sec > [ 4] 7.0- 8.0 sec 98.7 MBytes 828 Mbits/sec > [ 4] 8.0- 9.0 sec 101 MBytes 847 Mbits/sec > [ 4] 9.0-10.0 sec 105 MBytes 882 Mbits/sec > [ 4] 0.0-10.0 sec 954 MBytes 799 Mbits/sec > > > On Thu, Jan 28, 2016 at 11:34 AM, Mini Trader > wrote: > >> Thank you for all the responses! Ive run some more detailed tests using >> iperf 2. The results that I see are inline with the transfer rates so they >> describe the behavior that I am seeing. >> >> Note I used a laptop on same connection as desktop. So that there would >> be a basis to compare it to the Desktop. >> >> For some reason the laptop has a limit of around 500-600 mbit/sec for its >> downloads, regardless the test still seem to show the behavior >> that I am seeing. Note that Linux does not seem to have the same issues >> where OmniOS does. Additionally OmniOS does not have the issue >> when using a direct ethernet connection. One thing I can say about Linux >> is that its downloads on the adapters are less than its uploads which >> is the complete opposite as OmniOS. This Linux behavior is not seen when >> using ethernet. >> >> Both Linux and OmniOS are running on ESXi 6U1. OmniOS is using the >> vmxnet driver. >> >> The adapters being used are Adaptec ECB6200. These are bonded Moca 2.0 >> adapters and are running the latest firmware. >> >> Source Machine: Desktop >> Connection: Adapter >> Windows <-> OmniOS >> >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to storage1, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.141 port 31595 connected with 10.255.0.15 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 34.9 MBytes 293 Mbits/sec >> [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec >> [ 4] 2.0- 3.0 sec 35.2 MBytes 296 Mbits/sec >> [ 4] 3.0- 4.0 sec 34.4 MBytes 288 Mbits/sec >> [ 4] 4.0- 5.0 sec 34.5 MBytes 289 Mbits/sec >> [ 4] 0.0- 5.0 sec 174 MBytes 292 Mbits/sec >> [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 33341 >> [ 4] 0.0- 1.0 sec 46.2 MBytes 388 Mbits/sec >> [ 4] 1.0- 2.0 sec 101 MBytes 849 Mbits/sec >> [ 4] 2.0- 3.0 sec 104 MBytes 872 Mbits/sec >> [ 4] 3.0- 4.0 sec 101 MBytes 851 Mbits/sec >> [ 4] 4.0- 5.0 sec 102 MBytes 855 Mbits/sec >> [ 4] 0.0- 5.0 sec 457 MBytes 763 Mbits/sec >> >> Source Machine: Desktop >> Connection: Adapter >> Windows <-> Linux >> >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to media.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.141 port 31602 connected with 10.255.0.73 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 108 MBytes 902 Mbits/sec >> [ 4] 1.0- 2.0 sec 111 MBytes 929 Mbits/sec >> [ 4] 2.0- 3.0 sec 111 MBytes 928 Mbits/sec >> [ 4] 3.0- 4.0 sec 106 MBytes 892 Mbits/sec >> [ 4] 4.0- 5.0 sec 109 MBytes 918 Mbits/sec >> [ 4] 0.0- 5.0 sec 545 MBytes 914 Mbits/sec >> [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.73 port 55045 >> [ 4] 0.0- 1.0 sec 67.0 MBytes 562 Mbits/sec >> [ 4] 1.0- 2.0 sec 75.6 MBytes 634 Mbits/sec >> [ 4] 2.0- 3.0 sec 75.1 MBytes 630 Mbits/sec >> [ 4] 3.0- 4.0 sec 74.5 MBytes 625 Mbits/sec >> [ 4] 4.0- 5.0 sec 75.7 MBytes 635 Mbits/sec >> [ 4] 0.0- 5.0 sec 368 MBytes 616 Mbits/sec >> >> >> Machine: Laptop >> Connection: Adapter >> Windows <-> OmniOS Notice same issue with 35mb cap. >> >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to storage1.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.54 port 57487 connected with 10.255.0.15 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 35.5 MBytes 298 Mbits/sec >> [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec >> [ 4] 2.0- 3.0 sec 35.0 MBytes 294 Mbits/sec >> [ 4] 3.0- 4.0 sec 34.2 MBytes 287 Mbits/sec >> [ 4] 4.0- 5.0 sec 33.9 MBytes 284 Mbits/sec >> [ 4] 0.0- 5.0 sec 174 MBytes 291 Mbits/sec >> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 40779 >> [ 4] 0.0- 1.0 sec 28.8 MBytes 242 Mbits/sec >> [ 4] 1.0- 2.0 sec 55.8 MBytes 468 Mbits/sec >> [ 4] 2.0- 3.0 sec 43.7 MBytes 366 Mbits/sec >> [ 4] 3.0- 4.0 sec 50.7 MBytes 425 Mbits/sec >> [ 4] 4.0- 5.0 sec 52.7 MBytes 442 Mbits/sec >> [ 4] 0.0- 5.0 sec 233 MBytes 389 Mbits/sec >> >> Machine: Laptop >> Connection: Adapter >> Windows <-> Linux (not issue on upload, same as desktop) >> >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to media.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.54 port 57387 connected with 10.255.0.73 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 110 MBytes 919 Mbits/sec >> [ 4] 1.0- 2.0 sec 110 MBytes 920 Mbits/sec >> [ 4] 2.0- 3.0 sec 110 MBytes 921 Mbits/sec >> [ 4] 3.0- 4.0 sec 110 MBytes 923 Mbits/sec >> [ 4] 4.0- 5.0 sec 110 MBytes 919 Mbits/sec >> [ 4] 0.0- 5.0 sec 548 MBytes 919 Mbits/sec >> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52723 >> [ 4] 0.0- 1.0 sec 49.8 MBytes 418 Mbits/sec >> [ 4] 1.0- 2.0 sec 55.1 MBytes 462 Mbits/sec >> [ 4] 2.0- 3.0 sec 55.1 MBytes 462 Mbits/sec >> [ 4] 3.0- 4.0 sec 53.6 MBytes 449 Mbits/sec >> [ 4] 4.0- 5.0 sec 56.9 MBytes 477 Mbits/sec >> [ 4] 0.0- 5.0 sec 271 MBytes 454 Mbits/sec >> >> Machine: Laptop >> Connection: Ethernet >> Windows <-> OmniOS (No issues on upload) >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to storage1.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.54 port 57858 connected with 10.255.0.15 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 113 MBytes 950 Mbits/sec >> [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec >> [ 4] 2.0- 3.0 sec 109 MBytes 912 Mbits/sec >> [ 4] 3.0- 4.0 sec 111 MBytes 931 Mbits/sec >> [ 4] 4.0- 5.0 sec 106 MBytes 889 Mbits/sec >> [ 4] 0.0- 5.0 sec 550 MBytes 921 Mbits/sec >> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 42565 >> [ 4] 0.0- 1.0 sec 38.4 MBytes 322 Mbits/sec >> [ 4] 1.0- 2.0 sec 68.9 MBytes 578 Mbits/sec >> [ 4] 2.0- 3.0 sec 67.7 MBytes 568 Mbits/sec >> [ 4] 3.0- 4.0 sec 66.7 MBytes 559 Mbits/sec >> [ 4] 4.0- 5.0 sec 63.2 MBytes 530 Mbits/sec >> [ 4] 0.0- 5.0 sec 306 MBytes 513 Mbits/sec >> >> Machine: Laptop >> Connection: Ethernet >> Windows <-> Linux (Exact same speeds this time as OmnioOS) >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to media.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.54 port 57966 connected with 10.255.0.73 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 110 MBytes 920 Mbits/sec >> [ 4] 1.0- 2.0 sec 111 MBytes 932 Mbits/sec >> [ 4] 2.0- 3.0 sec 111 MBytes 931 Mbits/sec >> [ 4] 3.0- 4.0 sec 108 MBytes 902 Mbits/sec >> [ 4] 4.0- 5.0 sec 106 MBytes 887 Mbits/sec >> [ 4] 0.0- 5.0 sec 545 MBytes 913 Mbits/sec >> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52726 >> [ 4] 0.0- 1.0 sec 63.4 MBytes 532 Mbits/sec >> [ 4] 1.0- 2.0 sec 62.9 MBytes 528 Mbits/sec >> [ 4] 2.0- 3.0 sec 66.7 MBytes 560 Mbits/sec >> [ 4] 3.0- 4.0 sec 65.3 MBytes 548 Mbits/sec >> [ 4] 4.0- 5.0 sec 66.8 MBytes 560 Mbits/sec >> [ 4] 0.0- 5.0 sec 326 MBytes 545 Mbits/sec >> >> >> On Wed, Jan 27, 2016 at 10:35 PM, Bob Friesenhahn < >> bfriesen at simple.dallas.tx.us> wrote: >> >>> On Wed, 27 Jan 2016, Mini Trader wrote: >>> >>> Slow CIFS Writes when using Moca 2.0 Adapter. >>>> >>>> I am experiencing this only under OmniOS. I do not see this in Windows >>>> or Linux. >>>> >>>> I have a ZFS CIFS share setup which can easily do writes that would >>>> saturate a 1GBe connection. >>>> >>>> My problem appears to be related somehow to the interaction between >>>> OmniOS and ECB6200 Moca 2.0 adapters. >>>> >>>> 1. If I write to my OmniOS CIFS share using ethernet my speeds up/down >>>> are around 110 mb/sec - good >>>> >>>> 2. If I write to my share using the same source but over the adapter my >>>> speeds are around 35mb/sec - problem >>>> >>> >>> MoCA has a 3.0+ millisecond latency (I typically see 3.5ms when using >>> ping). This latency is fairly large compared with typical hard drive >>> latencies and vastly higher than Ethernet. There is nothing which can be >>> done about this latency. >>> >>> Unbonded MoCA 2.0 throughput for streaming data is typically >>> 500Mbit/second, and bonded (two channels) MoCA 2.0 doubles that (the >>> claimed specs are of course higher than this and higher speeds can be >>> measured under ideal conditions). This means that typical MoCA 2.0 (not >>> bonded) achieves a bit less than half of what gigabit Ethernet achieves >>> when streaming data over TCP. >>> >>> 3. If I read from the share using the same device over the adapter my >>>> speeds are around 110mb/sec - good >>>> >>> >>> Reading is normally more of a streaming operation so the TCP will stream >>> rather well. >>> >>> 4. If I setup a share on a Windows machine and write to it from the same >>>> source using the adapter the speeds are >>>> around 110mb/sec. The Windows machine is actually a VM whos disks are >>>> backed by a ZFS NFS share on the same >>>> machine >>>> >>> >>> This seems rather good. Quite a lot depends on what the server side >>> does. If it commits each write to disk before accepting more, then the >>> write speed would suffer. >>> >>> So basically the issue only takes place when writing to the OmniOS CIFS >>>> share using the adapter, if the adapter is >>>> not used than the write speed is perfect. >>>> >>> >>> If the MoCA adaptor supports bonded mode, then it is useful to know that >>> usually bonded mode needs to be enabled. Is it possible that the Windows >>> driver is enabling bonded mode but the OmniOS driver does not? >>> >>> Try running a TCP streaming benchmark (program to program) to see what >>> the peak network throughput is in each case. >>> >>> Any ideas why/how a Moca 2.0 adapter which is just designed to convert >>>> an ethernet signal to a coax and back to >>>> ethernet would cause issues with writes on OmniOS when the exact same >>>> share has no issues when using an actual >>>> ethernet connection? More importantly, why is this happening with >>>> OmniOS CIFS and not anything else? >>>> >>> >>> Latency, synchronous writes, and possibly bonding not enabled. Also, >>> OmniOS r151016 or later is need to get the latest CIFS implementation >>> (based on Nexenta changes), which has been reported on this list to be >>> quite a lot faster than the older one. >>> >>> Bob >>> -- >>> Bob Friesenhahn >>> bfriesen at simple.dallas.tx.us, >>> http://www.simplesystems.org/users/bfriesen/ >>> GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From miniflowtrader at gmail.com Thu Jan 28 19:44:54 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Thu, 28 Jan 2016 14:44:54 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: Turns out that running svcadm restart smb/server after tuning the send and receive buffers has fixed the problem. I can now transfer at nearly 1GBe both up and down! Problem has been resolved :) On Thu, Jan 28, 2016 at 2:30 PM, Mini Trader wrote: > Is there a way to adjust the default Window Size for CIFS or NFS? > > On Thu, Jan 28, 2016 at 1:39 PM, Mini Trader > wrote: > >> I also tried the following. Which seems to have improved iperf speeds. >> But I am still getting the same CIFS speeds. >> >> root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p >> recv_buf=1048576 tcp >> root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p >> send_buf=1048576 tcp >> root at storage1:/var/web-gui/data/tools/iperf# ipadm set-prop -p >> max_buf=4194304 tcp >> >> >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to storage1.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.141 port 33452 connected with 10.255.0.15 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 106 MBytes 892 Mbits/sec >> [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec >> [ 4] 2.0- 3.0 sec 108 MBytes 904 Mbits/sec >> [ 4] 3.0- 4.0 sec 109 MBytes 916 Mbits/sec >> [ 4] 4.0- 5.0 sec 110 MBytes 923 Mbits/sec >> [ 4] 5.0- 6.0 sec 110 MBytes 919 Mbits/sec >> [ 4] 6.0- 7.0 sec 110 MBytes 919 Mbits/sec >> [ 4] 7.0- 8.0 sec 105 MBytes 884 Mbits/sec >> [ 4] 8.0- 9.0 sec 109 MBytes 915 Mbits/sec >> [ 4] 9.0-10.0 sec 111 MBytes 928 Mbits/sec >> [ 4] 0.0-10.0 sec 1.06 GBytes 912 Mbits/sec >> [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 50899 >> [ 4] 0.0- 1.0 sec 97.5 MBytes 818 Mbits/sec >> [ 4] 1.0- 2.0 sec 110 MBytes 923 Mbits/sec >> [ 4] 2.0- 3.0 sec 49.3 MBytes 414 Mbits/sec >> [ 4] 3.0- 4.0 sec 98.0 MBytes 822 Mbits/sec >> [ 4] 4.0- 5.0 sec 96.7 MBytes 811 Mbits/sec >> [ 4] 5.0- 6.0 sec 99.7 MBytes 836 Mbits/sec >> [ 4] 6.0- 7.0 sec 103 MBytes 861 Mbits/sec >> [ 4] 7.0- 8.0 sec 101 MBytes 851 Mbits/sec >> [ 4] 8.0- 9.0 sec 104 MBytes 876 Mbits/sec >> [ 4] 9.0-10.0 sec 104 MBytes 876 Mbits/sec >> [ 4] 0.0-10.0 sec 966 MBytes 808 Mbits/sec >> >> root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p >> recv_buf tcp >> root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p >> send_buf tcp >> root at storage1:/var/web-gui/data/tools/iperf# ipadm reset-prop -p max_buf >> tcp >> >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> ------------------------------------------------------------ >> Client connecting to storage1.midway, TCP port 5001 >> TCP window size: 977 KByte >> ------------------------------------------------------------ >> [ 4] local 10.255.0.141 port 33512 connected with 10.255.0.15 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0- 1.0 sec 35.2 MBytes 296 Mbits/sec >> [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec >> [ 4] 2.0- 3.0 sec 34.2 MBytes 287 Mbits/sec >> [ 4] 3.0- 4.0 sec 33.4 MBytes 280 Mbits/sec >> [ 4] 4.0- 5.0 sec 34.1 MBytes 286 Mbits/sec >> [ 4] 5.0- 6.0 sec 35.2 MBytes 296 Mbits/sec >> [ 4] 6.0- 7.0 sec 35.4 MBytes 297 Mbits/sec >> [ 4] 7.0- 8.0 sec 34.4 MBytes 288 Mbits/sec >> [ 4] 8.0- 9.0 sec 35.0 MBytes 294 Mbits/sec >> [ 4] 9.0-10.0 sec 33.4 MBytes 280 Mbits/sec >> [ 4] 0.0-10.0 sec 346 MBytes 289 Mbits/sec >> [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 41435 >> [ 4] 0.0- 1.0 sec 57.6 MBytes 483 Mbits/sec >> [ 4] 1.0- 2.0 sec 87.2 MBytes 732 Mbits/sec >> [ 4] 2.0- 3.0 sec 99.3 MBytes 833 Mbits/sec >> [ 4] 3.0- 4.0 sec 99.5 MBytes 835 Mbits/sec >> [ 4] 4.0- 5.0 sec 100 MBytes 842 Mbits/sec >> [ 4] 5.0- 6.0 sec 103 MBytes 866 Mbits/sec >> [ 4] 6.0- 7.0 sec 100 MBytes 840 Mbits/sec >> [ 4] 7.0- 8.0 sec 98.7 MBytes 828 Mbits/sec >> [ 4] 8.0- 9.0 sec 101 MBytes 847 Mbits/sec >> [ 4] 9.0-10.0 sec 105 MBytes 882 Mbits/sec >> [ 4] 0.0-10.0 sec 954 MBytes 799 Mbits/sec >> >> >> On Thu, Jan 28, 2016 at 11:34 AM, Mini Trader >> wrote: >> >>> Thank you for all the responses! Ive run some more detailed tests using >>> iperf 2. The results that I see are inline with the transfer rates so they >>> describe the behavior that I am seeing. >>> >>> Note I used a laptop on same connection as desktop. So that there would >>> be a basis to compare it to the Desktop. >>> >>> For some reason the laptop has a limit of around 500-600 mbit/sec for >>> its downloads, regardless the test still seem to show the behavior >>> that I am seeing. Note that Linux does not seem to have the same issues >>> where OmniOS does. Additionally OmniOS does not have the issue >>> when using a direct ethernet connection. One thing I can say about >>> Linux is that its downloads on the adapters are less than its uploads which >>> is the complete opposite as OmniOS. This Linux behavior is not seen >>> when using ethernet. >>> >>> Both Linux and OmniOS are running on ESXi 6U1. OmniOS is using the >>> vmxnet driver. >>> >>> The adapters being used are Adaptec ECB6200. These are bonded Moca 2.0 >>> adapters and are running the latest firmware. >>> >>> Source Machine: Desktop >>> Connection: Adapter >>> Windows <-> OmniOS >>> >>> Server listening on TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> ------------------------------------------------------------ >>> Client connecting to storage1, TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> [ 4] local 10.255.0.141 port 31595 connected with 10.255.0.15 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0- 1.0 sec 34.9 MBytes 293 Mbits/sec >>> [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec >>> [ 4] 2.0- 3.0 sec 35.2 MBytes 296 Mbits/sec >>> [ 4] 3.0- 4.0 sec 34.4 MBytes 288 Mbits/sec >>> [ 4] 4.0- 5.0 sec 34.5 MBytes 289 Mbits/sec >>> [ 4] 0.0- 5.0 sec 174 MBytes 292 Mbits/sec >>> [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 33341 >>> [ 4] 0.0- 1.0 sec 46.2 MBytes 388 Mbits/sec >>> [ 4] 1.0- 2.0 sec 101 MBytes 849 Mbits/sec >>> [ 4] 2.0- 3.0 sec 104 MBytes 872 Mbits/sec >>> [ 4] 3.0- 4.0 sec 101 MBytes 851 Mbits/sec >>> [ 4] 4.0- 5.0 sec 102 MBytes 855 Mbits/sec >>> [ 4] 0.0- 5.0 sec 457 MBytes 763 Mbits/sec >>> >>> Source Machine: Desktop >>> Connection: Adapter >>> Windows <-> Linux >>> >>> Server listening on TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> ------------------------------------------------------------ >>> Client connecting to media.midway, TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> [ 4] local 10.255.0.141 port 31602 connected with 10.255.0.73 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0- 1.0 sec 108 MBytes 902 Mbits/sec >>> [ 4] 1.0- 2.0 sec 111 MBytes 929 Mbits/sec >>> [ 4] 2.0- 3.0 sec 111 MBytes 928 Mbits/sec >>> [ 4] 3.0- 4.0 sec 106 MBytes 892 Mbits/sec >>> [ 4] 4.0- 5.0 sec 109 MBytes 918 Mbits/sec >>> [ 4] 0.0- 5.0 sec 545 MBytes 914 Mbits/sec >>> [ 4] local 10.255.0.141 port 5001 connected with 10.255.0.73 port 55045 >>> [ 4] 0.0- 1.0 sec 67.0 MBytes 562 Mbits/sec >>> [ 4] 1.0- 2.0 sec 75.6 MBytes 634 Mbits/sec >>> [ 4] 2.0- 3.0 sec 75.1 MBytes 630 Mbits/sec >>> [ 4] 3.0- 4.0 sec 74.5 MBytes 625 Mbits/sec >>> [ 4] 4.0- 5.0 sec 75.7 MBytes 635 Mbits/sec >>> [ 4] 0.0- 5.0 sec 368 MBytes 616 Mbits/sec >>> >>> >>> Machine: Laptop >>> Connection: Adapter >>> Windows <-> OmniOS Notice same issue with 35mb cap. >>> >>> ------------------------------------------------------------ >>> Server listening on TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> ------------------------------------------------------------ >>> Client connecting to storage1.midway, TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> [ 4] local 10.255.0.54 port 57487 connected with 10.255.0.15 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0- 1.0 sec 35.5 MBytes 298 Mbits/sec >>> [ 4] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec >>> [ 4] 2.0- 3.0 sec 35.0 MBytes 294 Mbits/sec >>> [ 4] 3.0- 4.0 sec 34.2 MBytes 287 Mbits/sec >>> [ 4] 4.0- 5.0 sec 33.9 MBytes 284 Mbits/sec >>> [ 4] 0.0- 5.0 sec 174 MBytes 291 Mbits/sec >>> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 40779 >>> [ 4] 0.0- 1.0 sec 28.8 MBytes 242 Mbits/sec >>> [ 4] 1.0- 2.0 sec 55.8 MBytes 468 Mbits/sec >>> [ 4] 2.0- 3.0 sec 43.7 MBytes 366 Mbits/sec >>> [ 4] 3.0- 4.0 sec 50.7 MBytes 425 Mbits/sec >>> [ 4] 4.0- 5.0 sec 52.7 MBytes 442 Mbits/sec >>> [ 4] 0.0- 5.0 sec 233 MBytes 389 Mbits/sec >>> >>> Machine: Laptop >>> Connection: Adapter >>> Windows <-> Linux (not issue on upload, same as desktop) >>> >>> Server listening on TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> ------------------------------------------------------------ >>> Client connecting to media.midway, TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> [ 4] local 10.255.0.54 port 57387 connected with 10.255.0.73 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0- 1.0 sec 110 MBytes 919 Mbits/sec >>> [ 4] 1.0- 2.0 sec 110 MBytes 920 Mbits/sec >>> [ 4] 2.0- 3.0 sec 110 MBytes 921 Mbits/sec >>> [ 4] 3.0- 4.0 sec 110 MBytes 923 Mbits/sec >>> [ 4] 4.0- 5.0 sec 110 MBytes 919 Mbits/sec >>> [ 4] 0.0- 5.0 sec 548 MBytes 919 Mbits/sec >>> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52723 >>> [ 4] 0.0- 1.0 sec 49.8 MBytes 418 Mbits/sec >>> [ 4] 1.0- 2.0 sec 55.1 MBytes 462 Mbits/sec >>> [ 4] 2.0- 3.0 sec 55.1 MBytes 462 Mbits/sec >>> [ 4] 3.0- 4.0 sec 53.6 MBytes 449 Mbits/sec >>> [ 4] 4.0- 5.0 sec 56.9 MBytes 477 Mbits/sec >>> [ 4] 0.0- 5.0 sec 271 MBytes 454 Mbits/sec >>> >>> Machine: Laptop >>> Connection: Ethernet >>> Windows <-> OmniOS (No issues on upload) >>> ------------------------------------------------------------ >>> Server listening on TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> ------------------------------------------------------------ >>> Client connecting to storage1.midway, TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> [ 4] local 10.255.0.54 port 57858 connected with 10.255.0.15 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0- 1.0 sec 113 MBytes 950 Mbits/sec >>> [ 4] 1.0- 2.0 sec 111 MBytes 928 Mbits/sec >>> [ 4] 2.0- 3.0 sec 109 MBytes 912 Mbits/sec >>> [ 4] 3.0- 4.0 sec 111 MBytes 931 Mbits/sec >>> [ 4] 4.0- 5.0 sec 106 MBytes 889 Mbits/sec >>> [ 4] 0.0- 5.0 sec 550 MBytes 921 Mbits/sec >>> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.15 port 42565 >>> [ 4] 0.0- 1.0 sec 38.4 MBytes 322 Mbits/sec >>> [ 4] 1.0- 2.0 sec 68.9 MBytes 578 Mbits/sec >>> [ 4] 2.0- 3.0 sec 67.7 MBytes 568 Mbits/sec >>> [ 4] 3.0- 4.0 sec 66.7 MBytes 559 Mbits/sec >>> [ 4] 4.0- 5.0 sec 63.2 MBytes 530 Mbits/sec >>> [ 4] 0.0- 5.0 sec 306 MBytes 513 Mbits/sec >>> >>> Machine: Laptop >>> Connection: Ethernet >>> Windows <-> Linux (Exact same speeds this time as OmnioOS) >>> ------------------------------------------------------------ >>> Server listening on TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> ------------------------------------------------------------ >>> Client connecting to media.midway, TCP port 5001 >>> TCP window size: 977 KByte >>> ------------------------------------------------------------ >>> [ 4] local 10.255.0.54 port 57966 connected with 10.255.0.73 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0- 1.0 sec 110 MBytes 920 Mbits/sec >>> [ 4] 1.0- 2.0 sec 111 MBytes 932 Mbits/sec >>> [ 4] 2.0- 3.0 sec 111 MBytes 931 Mbits/sec >>> [ 4] 3.0- 4.0 sec 108 MBytes 902 Mbits/sec >>> [ 4] 4.0- 5.0 sec 106 MBytes 887 Mbits/sec >>> [ 4] 0.0- 5.0 sec 545 MBytes 913 Mbits/sec >>> [ 4] local 10.255.0.54 port 5001 connected with 10.255.0.73 port 52726 >>> [ 4] 0.0- 1.0 sec 63.4 MBytes 532 Mbits/sec >>> [ 4] 1.0- 2.0 sec 62.9 MBytes 528 Mbits/sec >>> [ 4] 2.0- 3.0 sec 66.7 MBytes 560 Mbits/sec >>> [ 4] 3.0- 4.0 sec 65.3 MBytes 548 Mbits/sec >>> [ 4] 4.0- 5.0 sec 66.8 MBytes 560 Mbits/sec >>> [ 4] 0.0- 5.0 sec 326 MBytes 545 Mbits/sec >>> >>> >>> On Wed, Jan 27, 2016 at 10:35 PM, Bob Friesenhahn < >>> bfriesen at simple.dallas.tx.us> wrote: >>> >>>> On Wed, 27 Jan 2016, Mini Trader wrote: >>>> >>>> Slow CIFS Writes when using Moca 2.0 Adapter. >>>>> >>>>> I am experiencing this only under OmniOS. I do not see this in >>>>> Windows or Linux. >>>>> >>>>> I have a ZFS CIFS share setup which can easily do writes that would >>>>> saturate a 1GBe connection. >>>>> >>>>> My problem appears to be related somehow to the interaction between >>>>> OmniOS and ECB6200 Moca 2.0 adapters. >>>>> >>>>> 1. If I write to my OmniOS CIFS share using ethernet my speeds up/down >>>>> are around 110 mb/sec - good >>>>> >>>>> 2. If I write to my share using the same source but over the adapter >>>>> my speeds are around 35mb/sec - problem >>>>> >>>> >>>> MoCA has a 3.0+ millisecond latency (I typically see 3.5ms when using >>>> ping). This latency is fairly large compared with typical hard drive >>>> latencies and vastly higher than Ethernet. There is nothing which can be >>>> done about this latency. >>>> >>>> Unbonded MoCA 2.0 throughput for streaming data is typically >>>> 500Mbit/second, and bonded (two channels) MoCA 2.0 doubles that (the >>>> claimed specs are of course higher than this and higher speeds can be >>>> measured under ideal conditions). This means that typical MoCA 2.0 (not >>>> bonded) achieves a bit less than half of what gigabit Ethernet achieves >>>> when streaming data over TCP. >>>> >>>> 3. If I read from the share using the same device over the adapter my >>>>> speeds are around 110mb/sec - good >>>>> >>>> >>>> Reading is normally more of a streaming operation so the TCP will >>>> stream rather well. >>>> >>>> 4. If I setup a share on a Windows machine and write to it from the >>>>> same source using the adapter the speeds are >>>>> around 110mb/sec. The Windows machine is actually a VM whos disks are >>>>> backed by a ZFS NFS share on the same >>>>> machine >>>>> >>>> >>>> This seems rather good. Quite a lot depends on what the server side >>>> does. If it commits each write to disk before accepting more, then the >>>> write speed would suffer. >>>> >>>> So basically the issue only takes place when writing to the OmniOS CIFS >>>>> share using the adapter, if the adapter is >>>>> not used than the write speed is perfect. >>>>> >>>> >>>> If the MoCA adaptor supports bonded mode, then it is useful to know >>>> that usually bonded mode needs to be enabled. Is it possible that the >>>> Windows driver is enabling bonded mode but the OmniOS driver does not? >>>> >>>> Try running a TCP streaming benchmark (program to program) to see what >>>> the peak network throughput is in each case. >>>> >>>> Any ideas why/how a Moca 2.0 adapter which is just designed to convert >>>>> an ethernet signal to a coax and back to >>>>> ethernet would cause issues with writes on OmniOS when the exact same >>>>> share has no issues when using an actual >>>>> ethernet connection? More importantly, why is this happening with >>>>> OmniOS CIFS and not anything else? >>>>> >>>> >>>> Latency, synchronous writes, and possibly bonding not enabled. Also, >>>> OmniOS r151016 or later is need to get the latest CIFS implementation >>>> (based on Nexenta changes), which has been reported on this list to be >>>> quite a lot faster than the older one. >>>> >>>> Bob >>>> -- >>>> Bob Friesenhahn >>>> bfriesen at simple.dallas.tx.us, >>>> http://www.simplesystems.org/users/bfriesen/ >>>> GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ >>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Jan 28 19:49:28 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 28 Jan 2016 14:49:28 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> > On Jan 28, 2016, at 2:44 PM, Mini Trader wrote: > > Problem has been resolved :) > Makes sense. Those settings are only inherited by new TCP connections. Sorry I missed a good chunk of this thread, but you pretty much figured it all out. And you should check out this bloody cycle... SMB2 is on it, and it may help you further. Or you can wait until r151018, but early testing is why we have bloody. :) Dan From miniflowtrader at gmail.com Thu Jan 28 20:15:20 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Thu, 28 Jan 2016 15:15:20 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> References: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> Message-ID: I most definitely will. Any other tunables worth looking at or can most of these issues be fixed by send/receive buffer size? This was a nice crash course on how TCP Window sizes can affect your data throughput! On Thu, Jan 28, 2016 at 2:49 PM, Dan McDonald wrote: > > > On Jan 28, 2016, at 2:44 PM, Mini Trader > wrote: > > > > Problem has been resolved :) > > > > Makes sense. Those settings are only inherited by new TCP connections. > Sorry I missed a good chunk of this thread, but you pretty much figured it > all out. > > And you should check out this bloody cycle... SMB2 is on it, and it may > help you further. Or you can wait until r151018, but early testing is why > we have bloody. :) > > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alka at hfg-gmuend.de Thu Jan 28 21:20:11 2016 From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=) Date: Thu, 28 Jan 2016 22:20:11 +0100 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> Message-ID: <56AA860B.8060506@hfg-gmuend.de> I have done some tests about different tuning options (network, disk, service, client related) - mainly with 10G ethernet in mind but this may give some ideas about options (on new 151017 bloody) http://napp-it.org/doc/downloads/performance_smb2.pdf Am 28.01.2016 um 21:15 schrieb Mini Trader: > I most definitely will. Any other tunables worth looking at or can > most of these issues be fixed by send/receive buffer size? > > This was a nice crash course on how TCP Window sizes can affect your > data throughput! > > On Thu, Jan 28, 2016 at 2:49 PM, Dan McDonald > wrote: > > > > On Jan 28, 2016, at 2:44 PM, Mini Trader > > wrote: > > > > Problem has been resolved :) > > > > Makes sense. Those settings are only inherited by new TCP > connections. Sorry I missed a good chunk of this thread, but you > pretty much figured it all out. > > And you should check out this bloody cycle... SMB2 is on it, and > it may help you further. Or you can wait until r151018, but early > testing is why we have bloody. :) > > Dan > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From daleg at omniti.com Thu Jan 28 21:40:30 2016 From: daleg at omniti.com (Dale Ghent) Date: Thu, 28 Jan 2016 16:40:30 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: <56AA860B.8060506@hfg-gmuend.de> References: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> <56AA860B.8060506@hfg-gmuend.de> Message-ID: For what it's worth, the max MTU for X540 (and X520, and X550) is 15.5k. You can nearly double the frame size that you used in your tests, switch and the MacOS ixgbe driver allowing, of course. > On Jan 28, 2016, at 4:20 PM, G?nther Alka wrote: > > I have done some tests about different tuning options (network, disk, service, client related) - > mainly with 10G ethernet in mind but this may give some ideas about options (on new 151017 bloody) > > http://napp-it.org/doc/downloads/performance_smb2.pdf > > > Am 28.01.2016 um 21:15 schrieb Mini Trader: >> I most definitely will. Any other tunables worth looking at or can most of these issues be fixed by send/receive buffer size? >> >> This was a nice crash course on how TCP Window sizes can affect your data throughput! >> >> On Thu, Jan 28, 2016 at 2:49 PM, Dan McDonald wrote: >> >> > On Jan 28, 2016, at 2:44 PM, Mini Trader wrote: >> > >> > Problem has been resolved :) >> > >> >> Makes sense. Those settings are only inherited by new TCP connections. Sorry I missed a good chunk of this thread, but you pretty much figured it all out. >> >> And you should check out this bloody cycle... SMB2 is on it, and it may help you further. Or you can wait until r151018, but early testing is why we have bloody. :) >> >> Dan >> >> >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bfriesen at simple.dallas.tx.us Thu Jan 28 21:58:39 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 28 Jan 2016 15:58:39 -0600 (CST) Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: On Thu, 28 Jan 2016, Mini Trader wrote: > Turns out that running?svcadm restart smb/server after tuning the send and receive buffers has fixed the problem.? I can now > transfer at nearly 1GBe both up and down! > Problem has been resolved :) The next problem you may encounter is that MoCA is basically half-duplex so performance will suffer with two-way traffic. MoCA is not at all like Ethernet although it passes Ethernet frames. It "bundles" multiple frames which happens to be going to the same place because it seems like it is slow to turn the pipe around. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From miniflowtrader at gmail.com Thu Jan 28 22:32:45 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Thu, 28 Jan 2016 17:32:45 -0500 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: Message-ID: You are right about this. Client connecting to storage1.midway, TCP port 5001 TCP window size: 977 KByte ------------------------------------------------------------ [ 4] local 10.255.0.141 port 14766 connected with 10.255.0.15 port 5001 [ 5] local 10.255.0.141 port 5001 connected with 10.255.0.15 port 55052 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 87.2 MBytes 732 Mbits/sec [ 5] 0.0- 1.0 sec 17.6 MBytes 147 Mbits/sec [ 4] 1.0- 2.0 sec 78.4 MBytes 657 Mbits/sec [ 5] 1.0- 2.0 sec 33.4 MBytes 280 Mbits/sec [ 4] 2.0- 3.0 sec 69.5 MBytes 583 Mbits/sec [ 5] 2.0- 3.0 sec 34.7 MBytes 291 Mbits/sec [ 5] 3.0- 4.0 sec 31.8 MBytes 267 Mbits/sec [ 4] 3.0- 4.0 sec 68.1 MBytes 571 Mbits/sec [ 4] 4.0- 5.0 sec 71.9 MBytes 603 Mbits/sec [ 5] 4.0- 5.0 sec 31.9 MBytes 267 Mbits/sec [ 4] 5.0- 6.0 sec 72.1 MBytes 605 Mbits/sec [ 5] 5.0- 6.0 sec 30.5 MBytes 256 Mbits/sec [ 4] 6.0- 7.0 sec 74.0 MBytes 621 Mbits/sec [ 5] 6.0- 7.0 sec 30.3 MBytes 254 Mbits/sec [ 5] 7.0- 8.0 sec 31.0 MBytes 260 Mbits/sec [ 4] 7.0- 8.0 sec 77.8 MBytes 652 Mbits/sec [ 4] 8.0- 9.0 sec 74.9 MBytes 628 Mbits/sec [ 5] 8.0- 9.0 sec 33.5 MBytes 281 Mbits/sec [ 4] 9.0-10.0 sec 57.1 MBytes 479 Mbits/sec [ 4] 0.0-10.0 sec 731 MBytes 613 Mbits/sec [ 5] 9.0-10.0 sec 41.5 MBytes 348 Mbits/sec [ 5] 0.0-10.0 sec 318 MBytes 266 Mbits/sec On Thu, Jan 28, 2016 at 4:58 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > On Thu, 28 Jan 2016, Mini Trader wrote: > > Turns out that running svcadm restart smb/server after tuning the send and >> receive buffers has fixed the problem. I can now >> transfer at nearly 1GBe both up and down! >> Problem has been resolved :) >> > > The next problem you may encounter is that MoCA is basically half-duplex > so performance will suffer with two-way traffic. MoCA is not at all like > Ethernet although it passes Ethernet frames. It "bundles" multiple frames > which happens to be going to the same place because it seems like it is > slow to turn the pipe around. > > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alka at hfg-gmuend.de Fri Jan 29 08:39:11 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Fri, 29 Jan 2016 09:39:11 +0100 Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: References: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> <56AA860B.8060506@hfg-gmuend.de> Message-ID: <56AB252F.5050302@hfg-gmuend.de> With the default mtu 1500 you can max out 1G networks but on 10G you are limited to about 300-400 MB/s. With mtu 9000 that is supported on all of my switches and computers the SMB2 limit is near the limit of 10G Higher mtu values may be of interest when 40G+ becomes more available. Would be a problem for my switches. More important is the question if OmniOS could be optimized per default to be better prepared to 10G like ipbuffer or some NFS settings beside hotplug support for AHCI or the timeout for disks. This would mean minimal more RAM for the OS, minimal better 1G performance but opens the potential of 10G Currently if you want to use OmniOS, you must do after setup from ISO/USB - setup networking manually at CLI (annoying, everywhere in the installer) - check nic driver config if mtu 9000 is allowed there - enable hotplug behaviour with AHCI - reduce timeouts of disks ex to the 7s of TLER (way too high per default) http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ - modify ip buffers and NFS settings for a proper NFS/SMB performance while there is no "global best setting" the current OmniOS defaults are worser than suboptimal. If someone compares a default OmniOS vs a BSD or Linux system, the OmniOS results are far below the potential. Even this MoCa problem would have been obsolete with higher ip buffers per default Gea Am 28.01.2016 um 22:40 schrieb Dale Ghent: > For what it's worth, the max MTU for X540 (and X520, and X550) is 15.5k. You can nearly double the frame size that you used in your tests, switch and the MacOS ixgbe driver allowing, of course. > > >> On Jan 28, 2016, at 4:20 PM, G?nther Alka wrote: >> >> I have done some tests about different tuning options (network, disk, service, client related) - >> mainly with 10G ethernet in mind but this may give some ideas about options (on new 151017 bloody) >> >> http://napp-it.org/doc/downloads/performance_smb2.pdf >> >> >> Am 28.01.2016 um 21:15 schrieb Mini Trader: >>> I most definitely will. Any other tunables worth looking at or can most of these issues be fixed by send/receive buffer size? >>> >>> This was a nice crash course on how TCP Window sizes can affect your data throughput! >>> >>> On Thu, Jan 28, 2016 at 2:49 PM, Dan McDonald wrote: >>> >>>> On Jan 28, 2016, at 2:44 PM, Mini Trader wrote: >>>> >>>> Problem has been resolved :) >>>> >>> Makes sense. Those settings are only inherited by new TCP connections. Sorry I missed a good chunk of this thread, but you pretty much figured it all out. >>> >>> And you should check out this bloody cycle... SMB2 is on it, and it may help you further. Or you can wait until r151018, but early testing is why we have bloody. :) >>> >>> Dan >>> >>> >>> >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss -- H f G Hochschule f?r Gestaltung university of design Schw?bisch Gm?nd Rektor-Klaus Str. 100 73525 Schw?bisch Gm?nd Guenther Alka, Dipl.-Ing. (FH) Leiter des Rechenzentrums head of computer center Tel 07171 602 627 Fax 07171 69259 guenther.alka at hfg-gmuend.de http://rz.hfg-gmuend.de From danmcd at omniti.com Fri Jan 29 11:10:01 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 29 Jan 2016 06:10:01 -0500 Subject: [OmniOS-discuss] OmniOS bloody now up to date. Message-ID: The first OmniOS bloody (r151017) update for 2016 is now finally available. You can get it at the regular place: http://omnios.omniti.com/wiki.php/Installation illumos-omnios is built from revision 2e178ba. omnios-build is built from 45cb430, but is a faster moving target. There is one incomplete feature in this update, I'll mention it in the list of what's new, and discuss it later. It affects ssh, sudo, and others. New for this release: * The ISO or USB install media now includes the "pciutils" package, so one can diagnose their hardware a bit better just by using "lspci" and friends. * Various security patches you've seen rolled out for 014 & 016 (OpenSSL, NTP, bind, ISC DHCP, libxml2, rsync) * zsh is now version 5.2. * Mozilla NSS is now version 3.21. The CA-bundle is updated correspondingly. * Merged with illumos-gate up to branch 5e11cc3. * NFSv4 deadlock fix -- illumos 6554 * NLM locking fix -- illumos 6525 * Scrub & resilver improvement -- illumos 6450 * SMF ipfilter enhancements for IPv6 * mpt_sas deadlock fix -- illumos 6256 * illumos 6057 --> new lastlog & associated PAM fixes. THIS NEEDS MORE WORK -- see below. ABOUT ILLUMOS 6057 It was backed-out of upstream, but because it was initially contributed by an OmniOS community member, I want to keep it in bloody for now in the hopes of working out all of the kinks. Some oddness you will notice: - sudo prints last login - SunSSH prints an extra line after last login, AND it prints last login when it shouldn't (scp & command execution). - There are probably other case I'm missing. I would REALLY appreciate further community help with testing so 6057 can be in good enough shape to re-upstream. Thanks, Dan p.s. I'm in Brussels for FOSDEM this weekend, and I'll be speaking at 12noon on Sunday. From bfriesen at simple.dallas.tx.us Fri Jan 29 20:09:49 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 29 Jan 2016 14:09:49 -0600 (CST) Subject: [OmniOS-discuss] Slow CIFS Writes when using Moca 2.0 Adapter In-Reply-To: <56AB252F.5050302@hfg-gmuend.de> References: <2137BF9A-A40B-43D8-81D2-069C09CDB3AB@omniti.com> <56AA860B.8060506@hfg-gmuend.de> <56AB252F.5050302@hfg-gmuend.de> Message-ID: On Fri, 29 Jan 2016, Guenther Alka wrote: > With the default mtu 1500 you can max out 1G networks but on 10G you are > limited to about 300-400 MB/s. > With mtu 9000 that is supported on all of my switches and computers the SMB2 > limit is near the limit of 10G Since this topic started about Moca 2.0, its worth mentioning that this consumer-grade networking technology might not adequately support large MTUs. A particular Moca 2.0 device might support large MTUs, but this is likely atypical. Hardware that I am working with does support a somewhat elevated MTU (e.g. 2k) with Moca 2.0 but that is because we wrote code to support it and tested it between two units of our hardware. With limited interoperability testing, we have not encountered other Moca 2.0 hardware which supports MTU over 1500 bytes. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From g_patrickb at yahoo.com Fri Jan 29 23:33:06 2016 From: g_patrickb at yahoo.com (G B) Date: Fri, 29 Jan 2016 23:33:06 +0000 (UTC) Subject: [OmniOS-discuss] Local repository References: <448598721.2496939.1454110386837.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <448598721.2496939.1454110386837.JavaMail.yahoo@mail.yahoo.com> I would like to create a local repository without a network connection.? All of the documentation for OmniOS has a repository being created via http and the Oracle and OpenIndiana documentation show to copy data from /repo mounted from an .iso.? However, when mounting the OmniOS .iso there isn't any /repo directory. What method would be used to create a local repository without a network connection using OmniOS? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Fri Jan 29 23:56:57 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 29 Jan 2016 17:56:57 -0600 (CST) Subject: [OmniOS-discuss] Local repository In-Reply-To: <448598721.2496939.1454110386837.JavaMail.yahoo@mail.yahoo.com> References: <448598721.2496939.1454110386837.JavaMail.yahoo.ref@mail.yahoo.com> <448598721.2496939.1454110386837.JavaMail.yahoo@mail.yahoo.com> Message-ID: On Fri, 29 Jan 2016, G B wrote: > What method would be used to create a local repository without a network connection using OmniOS? While I have not yet tried it myself, I imagine that you would follow the first bit of instructions (three commands) listed at http://omnios.omniti.com/wiki.php/CreatingRepos and then use pkg set-publisher -g file:///data/myrepo myrepo to reference it. I am interested to hear if this works as expected. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From vab at bb-c.de Sun Jan 31 17:14:33 2016 From: vab at bb-c.de (Volker A. Brandt) Date: Sun, 31 Jan 2016 18:14:33 +0100 Subject: [OmniOS-discuss] Local repository In-Reply-To: <448598721.2496939.1454110386837.JavaMail.yahoo@mail.yahoo.com> References: <448598721.2496939.1454110386837.JavaMail.yahoo.ref@mail.yahoo.com> <448598721.2496939.1454110386837.JavaMail.yahoo@mail.yahoo.com> Message-ID: <22190.16633.309680.153987@glaurung.bb-c.de> > I would like to create a local repository without a network connection.? All > of the documentation for OmniOS has a repository being created via http and > the Oracle and OpenIndiana documentation show to copy data from /repo > mounted from an .iso.? However, when mounting the OmniOS .iso there isn't > any /repo directory. > > What method would be used to create a local repository without a network > connection using OmniOS? There is no repository DVD available for OmniOS. So you will need a system with a network connection, running an OS that uses IPS (such as OmniOS, OpenIndiana, or Solaris). You can clone the entire repository, or just copy the latest versions of all packages. Once you have a repository copy that contains everything you want, you can burn it to a DVD or save a tar file to a USB stick. From that, you can set up a local repository on your system without any network connection. You can update your local repository via the same method whenever the "master" OmniOS repository is updated. Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Oracle Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim, GERMANY Email: vab at bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgr??e: 46 Gesch?ftsf?hrer: Rainer J.H. Brandt und Volker A. Brandt "When logic and proportion have fallen sloppy dead" From g_patrickb at yahoo.com Sun Jan 31 20:24:28 2016 From: g_patrickb at yahoo.com (G B) Date: Sun, 31 Jan 2016 20:24:28 +0000 (UTC) Subject: [OmniOS-discuss] Local repository In-Reply-To: <22190.16633.309680.153987@glaurung.bb-c.de> References: <22190.16633.309680.153987@glaurung.bb-c.de> Message-ID: <259326642.3072633.1454271868211.JavaMail.yahoo@mail.yahoo.com> Thanks! On Sunday, January 31, 2016 11:14 AM, Volker A. Brandt wrote: > I would like to create a local repository without a network connection.? All > of the documentation for OmniOS has a repository being created via http and > the Oracle and OpenIndiana documentation show to copy data from /repo > mounted from an .iso.? However, when mounting the OmniOS .iso there isn't > any /repo directory. > > What method would be used to create a local repository without a network > connection using OmniOS? There is no repository DVD available for OmniOS.? So you will need a system with a network connection, running an OS that uses IPS (such as OmniOS, OpenIndiana, or Solaris).? You can clone the entire repository, or just copy the latest versions of all packages. Once you have a repository copy that contains everything you want, you can burn it to a DVD or save a tar file to a USB stick.? From that, you can set up a local repository on your system without any network connection.? You can update your local repository via the same method whenever the "master" OmniOS repository is updated. Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt? ? ? ? ? ? ? Consulting and Support for Oracle Solaris Brandt & Brandt Computer GmbH? ? ? ? ? ? ? ? ? WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim, GERMANY? ? ? ? ? ? Email: vab at bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513? ? ? ? ? ? ? Schuhgr??e: 46 Gesch?ftsf?hrer: Rainer J.H. Brandt und Volker A. Brandt "When logic and proportion have fallen sloppy dead" -------------- next part -------------- An HTML attachment was scrubbed... URL: