[OmniOS-discuss] Ang: Re: Ang: Re: Ang: Re: CoreOS

Fri Jan 2 08:33:13 UTC 2015

Hi Jorge and list!

Haven't been active during this time of christmas and new year, but I'm back now...

Thanks, Jorge, for digging into this!

I will do some more investigations....

About SmartOS and LX branded zones: Well, if I could use them on OmniOS I would be interested, because I'd like a fully working server OS in the bottom, not a crippled OS just developed for running zones on.

Do you know if there are possibilities to run these LX zones on OmniOS as well?

But generally, I'd prefer to have CoreOS as a KVM guest, since the CoreOS model is very interesting, imo.

I guess this discussion will continue in one way or another, now when it turns out that interesting solutions like CoreOS can't be run because of lack of features/old implementation in our KVM...

Rgrds Johan

-----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>
Från: Jorge Schrauwen <sjorge+ml at blackdot.be>
Datum: 2014-12-20 14:37
Kopia: omnios-discuss at lists.omniti.com
Ärende: Re: Ang: Re: Ang: Re: [OmniOS-discuss] CoreOS

Hey Johan,

I just poked at the qemu image... it seems it wants some stuff not in 
our old qemu-kvm fork. e.g. fsdev (mouting a filesystem from host to 
guest).

But let's try anyway!

# convert qcow2 to raw
qemu-img convert coreos_production_qemu_image.img 
coreos_production_qemu_image.dd
# dump this on our zvol
dd if=coreos_production_qemu_image.dd 
of=/dev/zvol/rdsk/core/vms/hosts/coreos/disk0

We now have the correctly formatted data on our zvol...

On the plus side it does output nicely to ttya if added to a vm :)

So... here is where the kernel dies: (oh it does some kexec bits which 
are a PITA)
---
[    0.001000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.17.2 #2
[    0.001000] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[    0.001000]  0000000000000008 ffff88007a3e7db8 ffffffff814e8915 
000000000000e
[    0.001000]  ffffffff81798190 ffff88007a3e7e38 ffffffff814e4c97 
0000000000006
[    0.001000]  0000000000000008 ffff88007a3e7e48 ffff88007a3e7de8 
00000000fffb0
[    0.001000] Call Trace:
[    0.001000]  [<ffffffff814e8915>] dump_stack+0x46/0x58
[    0.001000]  [<ffffffff814e4c97>] panic+0xc1/0x1f5
[    0.001000]  [<ffffffff81b10205>] setup_IO_APIC+0x7d6/0x83d
[    0.001000]  [<ffffffff81b0d0cb>] native_smp_prepare_cpus+0x2bc/0x337
[    0.001000]  [<ffffffff81afc1a1>] kernel_init_freeable+0xcd/0x212
[    0.001000]  [<ffffffff814daf30>] ? rest_init+0x80/0x80
[    0.001000]  [<ffffffff814daf3e>] kernel_init+0xe/0xf0
[    0.001000]  [<ffffffff814ef8fc>] ret_from_fork+0x7c/0xb0
[    0.001000]  [<ffffffff814daf30>] ? rest_init+0x80/0x80
[    0.001000] Rebooting in 60 seconds..
---

I actually also have this on a ubuntu vm I am using, it needs noapic 
kernel option... on the grub prompt (really nice is coreos seems to have 
grub + console on both tty0 (vga) and ttyS0 (serial ttya).

Woo hoo we got past that bit where it fails on IO-APIC, now we just hang 
on smpboot :(

---
[    0.001000] CPU: Physical Processor ID: 0
[    0.001000] CPU: Processor Core ID: 0
[    0.001000] mce: CPU supports 10 MCE banks
[    0.001000] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.001000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.001000] Freeing SMP alternatives memory: 20K (ffffffff82fa1000 - 
fffffff)
[    0.001000] ftrace: allocating 19518 entries in 77 pages
[    0.001000] smpboot: CPU0: Intel QEMU Virtual CPU version 0.14.1 
(fam: 06, m
---

Pretty much stuck here... I tried some variations of cpu type (qemu64, 
Nehalem and host) I also tried using one vcpu but still stuck.

Let's just cripple the entire thing and plow are way through: adding 
'nosmp noapic noacpi'

So yeah at this point coreos is pretty useless... but we fly past 
smpboot!
And... land here:

---
[    0.239823] scsi host0: ata_piix
[    0.239823] scsi host1: ata_piix
[    0.239823] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 
irq 14
[    0.239823] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 
irq 15
---

If it is docker you want, you may as well look at SmartOS's LX Brand 
stuff, they are racing towards workable docker zones.

But since I came this far, lets see if I can make it to the finish...
I am using virtio... lets try scsi... nothing... ide... nothing...

So this is were it ends.

Our qemu-kvm fork is probably just too old.

Regards

Jorge

On 2014-12-20 12:47, Johan Kragsterman wrote:
> Hi, Jorge and all!
> 
> 
> I would be interested in discussing this further, but perhaps
> omnios-discuss isn't the right place? Since I don't know if this is
> omnios/illumos/coreos specific...
> 
> I did some experimenting:
> 
> I only used CoreOS stable in my tests.
> 
> I tried the iso, but the iso isn't full featured, and doesn't run
> docker out of the box. And the docker implementation is of coarse what
> everybody is interested in. I got it to boot without problems, but I
> had big problems with VNC keymapping due to my Swedish keyboard and
> perhaps my Swedish client computer. So I could actually never do
> something with it, and since it is not full featured, it is not what I
> want to use.
> 
> So instead, I downloaded the img file for qemu, created a volume, and
> dd'ed the image to the volume, and then set this volume as boot. That
> went fine, to get it to boot. But then, with the default boot option
> in grub, it panicked, and restarted every 60 seconds.
> 
> I stopped the grub booting, and chosed the B option. That didn't
> panic, but it didn't work either, it was too much that didn't work.
> But option A went fine, no panic, and everything seem to work more or
> less without problems. The only problem here seem to be that I can't
> log in, due to the "first log in"-principles they seem to have: It is
> only possible to log in via ssh, which means the network have to be
> up, and I couldn't get the network to come up....so there I am right
> now...
> 
> Regards Johan
> 
> 
> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
> Från: Jorge Schrauwen <sjorge+ml at blackdot.be>
> Datum: 2014-12-18 18:07
> Kopia: omnios-discuss at lists.omniti.com
> Ärende: Re: Ang: Re: [OmniOS-discuss] CoreOS
> 
> 
> On 2014-12-18 17:57, Johan Kragsterman wrote:
>> Jorge, I was thinking about you when I posted this! I thought you
>> would be a possible contributor to this thread...  More furhter
>> down...
>> 
>> 
>> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
>> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
>> Från: Jorge Schrauwen <sjorge+ml at blackdot.be>
>> Datum: 2014-12-18 17:38
>> Kopia: omnios-discuss at lists.omniti.com
>> Ärende: Re: [OmniOS-discuss] CoreOS
>> 
>> Something like this will probably work:
>> 
>> 
>>   /usr/bin/qemu-system-x86_64
>>    -name coreos \
>>    -enable-kvm \
>>    -no-hpet \
>>    -m 4096
>>    -cpu Nehalem \
>>    -smp sockets=1,cores=4,threads=2 \
>>    -rtc base=utc,driftfix=slew \
>>    -pidfile /tank/coreo/coreos.pid  \
>>    -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay  \
>>    -vga std  \
>>    -vnc :1  \
>>    -nographic \
>>    -drive
>> file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
>>    -drive
>> file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on
>> \
>>    -boot order=cd,once=d \
>>    -device
>> virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0
>> \
>>    -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
>>    -chardev
>> socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
>>    -serial chardev:serial0 \
>>    -usb \
>>    -usbdevice tablet \
>>    -daemonize
>> 
>> You should get vnc at port 5901, seemed to boot for me but I did not
>> complete the install.
>> 
>> 
>> 
>> At the CoreOS site they say: Start like this:
>> 
>> ./coreos_production_qemu.sh -nographic
>> 
>> and they pass on that string -nographic  ...?
>> 
>> It makes me wonder, because they tell you to connect with the instans
>> only over ssh with: ssh -l core -p 2222 localhost   ...
>> 
>> So I'm not sure if it is possible to connect via VNC...did you
>> actually check VNC, to confirm you had a VNC connection?
>> 
>> It should boot and run from the image r/o, so perhaps you just need
>> one "disk"? I can see you got two configured, or at least the iso
>> file, and then a disk. Don't you think it would be enough with just
>> the image file?
>> 
>> Perhaps I just try...
> I used the install iso to see if it booted.
> -nographic just mean don't spawn a graphical console AKA SDL or simular
> window. It does not prevent '-vnc :1' from working.
> 
>> 
>> 
>> 
>> Regards
>> 
>> Jorge
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> On 2014-12-18 16:57, Johan Kragsterman wrote:
>>> Hi!
>>> 
>>> 
>>>  I've been looking at CoreOS and finds it interesting! Since I'd like
>>> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
>>> Haven't tested yet, but I downloaded the startscript for qemu, and it
>>> looks a little bit "too much" for Illumos KVM...
>>> 
>>>  It would be nice to get some views on people that have been
>>> considering this as well, perhaps some already tested or already
>>> running...?
>>> 
>>>  I've seen that Frederic Alix on this list been blogging about it, 
>>> but
>>> haven't seen if he managed to run it as a KVM guest on OmniOS.
>>> 
>>>  For me it seems to be some complications at first startup, mainly. 
>>> It
>>> doesn't seem to be reachable by VNC...
>>> 
>>>  Hope to get some input from you guys...
>>> 
>>> 
>>> Best regards from/Med vänliga hälsningar från
>>> 
>>> Johan Kragsterman
>>> 
>>> Capvert
>>> 
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss