<div dir="ltr">Already forwarded to illumos-discuss, and they already have the snoop, and the denied lock segment</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 28, 2015 at 10:36 AM, Youzhong Yang <span dir="ltr"><<a href="mailto:youzhong@gmail.com" target="_blank">youzhong@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I would suggest capturing packets, find out if the 'no locks available' is returned from the server. If it is, do dtrace on the server, find out where it returns <a href="http://src.illumos.org/source/s?defs=nlm4_denied_nolocks&project=illumos-gate" style="color:rgb(32,32,98);text-decoration:none" target="_blank">nlm4_denied_nolocks</a>.<div><br></div><div><a href="http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/klm/nlm_service.c#460" target="_blank">http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/klm/nlm_service.c#460</a><br></div><div><br></div><div>Again, as Dan suggested, it would be better to post on illumos-dev list.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 28, 2015 at 1:23 PM, Joe Little <span dir="ltr"><<a href="mailto:jmlittle@gmail.com" target="_blank">jmlittle@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I just set it to 1024 and still locking times out.</div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 28, 2015 at 10:13 AM, Youzhong Yang <span dir="ltr"><<a href="mailto:youzhong@gmail.com" target="_blank">youzhong@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Depending on how many active locks your system needs to handle, 80 might be a small value.<div><div><br></div><div>We use a different distro of illumos-gate and we set max threads to 1024, so far so good we are happy with the open source nlockmgr except the nlockmgr startup issue when machine reboots.<br></div></div><div><br></div><div><br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 28, 2015 at 1:02 PM, Joe Little <span dir="ltr"><<a href="mailto:jmlittle@gmail.com" target="_blank">jmlittle@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Just to answer this question, I had already bumped that up based on some suggestions on the net:<div><br></div><div>
<p><span>root@miele:/root# echo ::svc_pool nlm | mdb -k | grep 'Max threads'</span></p>
<p><span>mdb: failed to add kvm_pte_chain walker: walk name already in use</span></p>
<p><span>mdb: failed to add kvm_rmap_desc walker: walk name already in use</span></p>
<p><span>mdb: failed to add kvm_mmu_page_header walker: walk name already in use</span></p>
<p><span>mdb: failed to add kvm_pte_chain walker: walk name already in use</span></p>
<p><span>mdb: failed to add kvm_rmap_desc walker: walk name already in use</span></p>
<p><span>mdb: failed to add kvm_mmu_page_header walker: walk name already in use</span></p>
<p><span>Max threads = 80</span></p><p>Still no locking w/ v3.</p></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 28, 2015 at 9:23 AM, Youzhong Yang <span dir="ltr"><<a href="mailto:youzhong@gmail.com" target="_blank">youzhong@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">max threads of nlockmgr is set to 20 I think. Bump up this value then you can get rid of 'no locks available' error.<div><br></div><div>To confirm the current value:</div><div><br></div><div>echo ::svc_pool nlm | mdb -k | grep 'Max threads'<br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Wed, Jan 28, 2015 at 11:49 AM, Joe Little <span dir="ltr"><<a href="mailto:jmlittle@gmail.com" target="_blank">jmlittle@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">I recently switched one file server from Nexenta 4 Community (still uses closed NLM I believe) to OmniOS r151012.<div><br></div><div>Immediately, users started to complain from various Linux clients that locking was failing. Most of those clients explicitly set their NFS version to 3. I finally isolated that the locking does not fail on NFS v4 and have worked on transition where possible. But presently, no NFS v3 client and successfully lock against OmniOS NFS v3 locking service. I've confirmed that the locking service is running and is present using rpcinfo, matching one for one in services from previous OpenSolaris and Illumos variants. One example from a user:</div><div><br></div><div>
<p><span>$ strace /bin/tcsh</span></p>
<p><span>[...]</span></p>
<p><span>open("/home/REDACTED/.history", O_RDWR|O_CREAT, 0600) = 0</span></p>
<p><span>dup(0) = 1</span></p>
<p><span>dup(1) = 2</span></p>
<p><span>dup(2) = 3</span></p>
<p><span>dup(3) = 4</span></p>
<p><span>dup(4) = 5</span></p>
<p><span>dup(5) = 6</span></p>
<p><span>close(5) = 0</span></p>
<p><span>close(4) = 0</span></p>
<p><span>close(3) = 0</span></p>
<p><span>close(2) = 0</span></p>
<p><span>close(1) = 0</span></p>
<p><span>close(0) = 0</span></p>
<p><span>fcntl(6, F_SETFD, FD_CLOEXEC) = 0</span></p>
<p><span>fcntl(6, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0})</span></p>
<p><span></span><br></p>
<p><span>HERE fcntl hangs for 1-2 min and finally returns with "-1 ENOLCK (No</span></p>
<p><span>locks available)"</span></p></div></div>
<br></div></div>_______________________________________________<br>
OmniOS-discuss mailing list<br>
<a href="mailto:OmniOS-discuss@lists.omniti.com" target="_blank">OmniOS-discuss@lists.omniti.com</a><br>
<a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>