<div dir="ltr">I think I found the root cause. At least, I'm 90% certain what follows is correct and is the cause of the problem.<div><br></div><div><span style="font-size:12.8px"> getconf("CHILD_MAX") ends up calling sysconf(_SC_CHILD_MAX) which can also be read from a shell with ulimit -u.</span><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Here's what I get from a few different machines:<br></span><div><br></div><div><div># uname -a ; ulimit -u<br></div></div><div><div>SunOS omnios 5.11 omnios-r151020-4151d05 i86pc i386 i86pc</div><div>29995</div></div><div>#</div><div><br></div><div><div># uname -a ; ulimit -u</div><div>Linux opensuse 4.4.27-2-default #1 SMP Thu Nov 3 14:59:54 UTC 2016 (5c21e7c) x86_64 x86_64 x86_64 GNU/Linux</div><div>1200</div></div><div>#</div><div><br></div><div><div><div># uname -a ; ulimit -u</div><div>Linux debian-8 4.4 BrandZ virtual linux x86_64 GNU/Linux</div><div>2147483647</div></div><div>#</div><div><br></div><div>Apparently, ksh isn't very happy when <span style="font-size:12.8px">CHILD_MAX equals to MAX_INT, but that's probably a ksh bug.</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">If my understanding of the LX code is correct, </span><span style="font-size:12.8px">sysconf(_SC_CHILD_MAX)</span><span style="font-size:12.8px"> ends up being translated to lx_getrlimit() which would return the value of zone.max-lwps. Looks like an odd default to me, but I can't say for sure. Since I haven't configured any rctl on my lx zone, apparently the default is MAX_INT. I assume smartos uses a different default, but I wish I could double-check that.</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Now, I'm not sure how this could or should be fixed.</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">--</span></div><div><span style="font-size:12.8px">Ludovic</span></div><div><br></div><div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 10, 2017 at 10:32 PM, Dan McDonald <span dir="ltr"><<a href="mailto:danmcd@omniti.com" target="_blank">danmcd@omniti.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div>Again, thank you for the digging. I'd be interested in why ksh's getconf() fails as well.</div><div id="m_2090701291403095790AppleMailSignature"><br></div><div id="m_2090701291403095790AppleMailSignature">Thanks,</div><span class=""><div id="m_2090701291403095790AppleMailSignature">Dan<br><br>Sent from my iPhone (typos, autocorrect, and all)</div></span><div><div class="h5"><div><br>On May 10, 2017, at 3:44 PM, Ludovic Orban <<a href="mailto:lorban@bitronix.be" target="_blank">lorban@bitronix.be</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div>Okay, I found what causes ksh to misbehave. It's in sh_init(), when shgd->lim.child_max is initialized with the results of getconf("CHILD_MAX"), see: <a href="https://github.com/att/ast/blob/master/src/cmd/ksh93/sh/init.c#L1289" target="_blank">https://github.com/att/ast/<wbr>blob/master/src/cmd/ksh93/sh/<wbr>init.c#L1289</a></div><div><br></div><div>I've commented out that line, hardcoded shgd->lim.child_max to 128, rebuilt and voila: ksh works as it should.</div><div><br></div><div>Now I have to dig into that getconf() method to figure out what the returned value is and where it's coming from. Sounds trivial, but my C is *very* rusty, the asm gcc generates doesn't look at all what the JVM's JIT generates (which gives me wrong reflexes as I'm used to the latter) and I'm not very familiar with mdb.</div><div><br></div><div>Oh well, that turned into a nice debugging re-training session which I very much needed. That reminds me the good old days at my first job when I was porting Linux apps to Solaris.</div><div><br></div><div>Thank you for maintaining such a well-designed and pleasant to use OS!</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 10, 2017 at 3:59 PM, Dan McDonald <span dir="ltr"><<a href="mailto:danmcd@omniti.com" target="_blank">danmcd@omniti.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Wow, thank you for the further deep-diving.<br>
<span><br>
> On May 10, 2017, at 5:21 AM, Ludovic Orban <<a href="mailto:lorban@bitronix.be" target="_blank">lorban@bitronix.be</a>> wrote:<br>
><br>
> Looking at ksh' sources, my understanding is that job_post is stuck in that else clause:<br>
> else<br>
> {<br>
> /* create a new job */<br>
> while((pw->p_job = job_alloc()) < 0)<br>
> job_wait((pid_t)1);<br>
> pw->p_nxtjob = job.pwlist;<br>
> pw->p_nxtproc = 0;<br>
> }<br>
><br>
> Digging into the sources and stepping though the instructions of job_alloc and job_byjid it looks like ksh cannot allocate a job id as it believes they're all reserved. But so far, all this code is purely working on internal structures of ksh so a LX bug would have no impact.<br>
><br>
> I'll continue looking into this as time permits and I'll post an update if I find anything worth mentioning.<br>
><br>
<br>
</span>Be careful of narrowing your focus too far. I see some things worth considering:<br>
<br>
1.) If the "if" you're not showing me dependent on something in global state that may have been mis-initialized by an LX emulation bug?<br>
<br>
2.) Same question as #1, but applied to job_alloc() and job_wait().<br>
<br>
I'm guessing LX in OmniOS is failing because I mismerged or plain forgot something, given that Nahum says he can run ksh93 on SmartOS just fine.<br>
<br>
<br>
Please make sure you're looking at the bigger picture, but THANK YOU for the further investigation.<br>
<span class="m_2090701291403095790HOEnZb"><font color="#888888"><br>
Dan<br>
<br>
</font></span></blockquote></div><br></div>
</div></blockquote></div></div></div></blockquote></div><br></div>