<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div>So I was still having issues with virtio performance. I’ve finally determined that its the child zone that is capping the throughput at 85mbps. </div>
<div><br>
</div>
<div>If I halt the zone and launch the same VM from the GZ I get 955mbps. </div>
<div><br>
</div>
<div>Another thing…the virtio driver in Centos6.6 does not work well with OmniOS kvm. </div>
<div><br>
</div>
<div>I can boot Centos from either the GZ or a CZ and I’m actually getting results in the Kb now instead of mbps with iperf. May have something to do with the tcp window being 19.5 kb on CentOS vs 85kb on Ubuntu. Assuming this is a driver problem. </div>
<div><br>
</div>
<div>The only OS I get good speeds with are Ubuntu server 14.04 running the Global Zone. (Have only tested two though :))</div>
<div><br>
</div>
<div>So my recipe for decent virtio performance on OmniOS:</div>
<div><br>
</div>
<div>Ubuntu Linux Server 14.04 running in Global Zone. </div>
<div><br>
</div>
<div>Does anyone have any idea why the child zone is capping my throughput? </div>
<div><br>
</div>
<div>Am I missing a zone cfg parameter to allow the child zone to have full 1GB bandwidth?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Theo Schlossnagle<br>
<span style="font-weight:bold">Date: </span>Wednesday, March 25, 2015 at 6:56 AM<br>
<span style="font-weight:bold">To: </span>John Barfield<br>
<span style="font-weight:bold">Cc: </span>Phil Harman, "<a href="mailto:omnios-discuss@lists.omniti.com">omnios-discuss@lists.omniti.com</a>"<br>
<span style="font-weight:bold">Subject: </span>Re: [OmniOS-discuss] Potential KVM Virtio Performance Issues<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">+1 John. That documentation would be very welcome.</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Mar 24, 2015 at 9:50 PM, John Barfield <span dir="ltr">
<<a href="mailto:john.barfield@bissinc.com" target="_blank">john.barfield@bissinc.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Actually the numbers I sent for the SmartOS VM to VM test were on a switch with Jumbo frames (switch = 9216 mtu...SmartOS GZ MTU = 9000) (Extreme Networks Summit X440-48t release 15.2.3 patch12) Theyre also sitting in Q-in-Q tagged VLANs. Admin tagged nic sits
in Vman (provider bridge) 10 the VMs were tagged in VLAN 1674. (not bad :) really)<br>
<br>
As far as everyone who is wondering how I got 952 Mbps on OmnisOS KVM while running in a zone I plan to write up a how-to that can be posted to the core site if you'd like. There are several caveats that are not documented today for running KVM in a zone. Not
that I didnt reverse engineer some of Joyents work of course.<br>
<span class="im HOEnZb"><br>
<br>
<br>
Thanks and have a great day,<br>
<br>
John Barfield<br>
<br>
</span>
<div class="HOEnZb">
<div class="h5">> On Mar 24, 2015, at 7:40 PM, Phil Harman <<a href="mailto:phil.harman@gmail.com">phil.harman@gmail.com</a>> wrote:<br>
><br>
> John,<br>
><br>
> Interesting work and data. Thanks for sharing.<br>
><br>
> I've also been playing with Oracle Solaris 11.2 vs Linux vs FreeBSD on SmartOS vs ESX5.5 (free edition) both VM2VM and VM to remote host over a couple of Intel 10GBASE-T cards.<br>
><br>
> As far as I can tell, there remains no virtio-net driver for Solaris / Illumos guests, so I've been using e1000g, which really sucks.<br>
><br>
> I found virtio-net works ok under KVM, but was blown away by vmxnet3 under ESX performance (for which a Solaris / Illumos drivers do exist), being able to get close to 8gbps from the guest over the wire!<br>
><br>
> To achieve this I had to use jumbo frames (something the current Solaris 11.2 e1000g appears unable to do at all any more).<br>
><br>
> So I was wondering, while you are there, whether you've got (or can get) any data for KVM virtio-net VM2VM using jumbo frames?<br>
><br>
> My expectation would be at least 2x for MTU 9000 vs 1500.<br>
><br>
> I also wonder whether like for like comparison with ESX might encourage further improvements?<br>
><br>
> As someone used to say at Sun "If Linux is faster, it's a Solaris bug!". It would be great if the community could agree to the same for ESX vs KVM :)<br>
><br>
> Cheers,<br>
> Phil<br>
><br>
><br>
>> On 24 Mar 2015, at 23:45, John Barfield <<a href="mailto:john.barfield@bissinc.com">john.barfield@bissinc.com</a>> wrote:<br>
>><br>
>> Btw I did go ahead and test both virtio methods...I gave a vm the -device argument on one interface and the -net argument for another the results where....<br>
>><br>
>> -device = eth0 = 952mbps<br>
>> -net = eth1 = 199 mbps<br>
>><br>
>> Thanks and have a great day,<br>
>><br>
>> John Barfield<br>
>><br>
>>> On Mar 24, 2015, at 6:12 PM, Dan McDonald <<a href="mailto:danmcd@omniti.com">danmcd@omniti.com</a>> wrote:<br>
>>><br>
>>><br>
>>>> On Mar 24, 2015, at 7:04 PM, Dominik Hassler <<a href="mailto:hasslerd@gmx.li">hasslerd@gmx.li</a>> wrote:<br>
>>>><br>
>>>> Dan,<br>
>>>><br>
>>>>>> After further testing I achieved 952 MBytes on a VM-2-VM<br>
>>>>>> connection...1<br>
>>>>>> linux Ubuntu 12.04 vm to another CentOS 6.6 VM running on two<br>
>>>>>> different SmartOS host machines (through an extreme networks switch).<br>
>>>><br>
>>>> if I got John correctly, he was running his second test on SmartOS hosts...<br>
>>>><br>
>>>> We did a lot of testing on OmniOS with -net vnic and -device<br>
>>>> virtio-net-pci but sadly to no avail...<br>
>>>><br>
>>>> I think we have to hope that SmartOS kvm improvements will get<br>
>>>> upstreamed sooner or later.<br>
>>><br>
>>> Ahh yes.<br>
>>><br>
>>> I was hoping to have them ready for 014, but it's a complicated process to upstream larger projects, and Joyent was in the middle of getting their new Triton release out the door.<br>
>>><br>
>>> Dan<br>
>> _______________________________________________<br>
>> OmniOS-discuss mailing list<br>
>> <a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>
>> <a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">
http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
_______________________________________________<br>
OmniOS-discuss mailing list<br>
<a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>
<a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">
<p>Theo Schlossnagle</p>
<p><a href="http://omniti.com/is/theo-schlossnagle" target="_blank">http://omniti.com/is/theo-schlossnagle</a></p>
</div>
</div>
</div>
</div>
</span>
</body>
</html>