<div dir="ltr"><div><div><div>Thanks Richard for your help. </div>My problem is that i have a network ISCSI traffic of 2 MB/s, each 5 seconds i need to write on disks 10 MB of network traffic but on pool filervm2 I am writing much more that, approximatively 60 MB each 5 seconds. Each ssd of filervm2 is writting 15 MB every 5 second. When i check with smartmootools every ssd is writing approximatively 250 GB of data each day. </div>How can i reduce amont of data writting on each ssd ? i have try to reduce block size of zvol but it change nothing. </div>Anthony <div> <div> </div></div></div><div class="gmail_extra"> <div class="gmail_quote">2017-09-28 1:29 GMT+02:00 Richard Elling <<a href="mailto:richard.elling@richardelling.com" target="_blank">richard.elling@richardelling.com</a>>: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Comment below... <div><div class="h5"> > On Sep 27, 2017, at 12:57 AM, anthony omnios <<a href="mailto:icoomnios@gmail.com">icoomnios@gmail.com</a>> wrote: > > Hi, > > i have a problem, i used many ISCSI zvol (for each vm), network traffic is 2MB/s between kvm host and filer but i write on disks many more than that. I used a pool with separated mirror zil (intel s3710) and 8 ssd samsung 850 evo 1To > > zpool status > pool: filervm2 > state: ONLINE > scan: resilvered 406G in 0h22m with 0 errors on Wed Sep 20 15:45:48 2017 > config: > > NAME STATE READ WRITE CKSUM > filervm2 ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > c7t5002538D41657AAFd0 ONLINE 0 0 0 > c7t5002538D41F85C0Dd0 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > c7t5002538D41CC7105d0 ONLINE 0 0 0 > c7t5002538D41CC7127d0 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > c7t5002538D41CD7F7Ed0 ONLINE 0 0 0 > c7t5002538D41CD83FDd0 ONLINE 0 0 0 > mirror-4 ONLINE 0 0 0 > c7t5002538D41CD7F7Ad0 ONLINE 0 0 0 > c7t5002538D41CD7F7Dd0 ONLINE 0 0 0 > logs > mirror-1 ONLINE 0 0 0 > c4t2d0 ONLINE 0 0 0 > c4t4d0 ONLINE 0 0 0 > > i used correct ashift of 13 for samsung 850 evo > zdb|grep ashift : > > ashift: 13 > ashift: 13 > ashift: 13 > ashift: 13 > ashift: 13 > > But i write a lot on ssd every 5 seconds (many more than the network traffic of 2 MB/s) > > iostat -xn -d 1 : > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > 11.0 3067.5 288.3 153457.4 6.8 0.5 2.2 0.2 5 14 filervm2 </div></div>filervm2 is seeing 3067 writes per second. This is the interface to the upper layers. These writes are small. > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 rpool > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c4t0d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c4t1d0 > 0.0 552.6 0.0 17284.0 0.0 0.1 0.0 0.2 0 8 c4t2d0 > 0.0 552.6 0.0 17284.0 0.0 0.1 0.0 0.2 0 8 c4t4d0 The log devices are seeing 552 writes per second and since sync=standard that means that the upper layers are requesting syncs. > 1.0 233.3 48.1 10051.6 0.0 0.0 0.0 0.1 0 3 c7t5002538D41657AAFd0 > 5.0 250.3 144.2 13207.3 0.0 0.0 0.0 0.1 0 3 c7t5002538D41CC7127d0 > 2.0 254.3 24.0 13207.3 0.0 0.0 0.0 0.1 0 4 c7t5002538D41CC7105d0 > 3.0 235.3 72.1 10051.6 0.0 0.0 0.0 0.1 0 3 c7t5002538D41F85C0Dd0 > 0.0 228.3 0.0 16178.7 0.0 0.0 0.0 0.2 0 4 c7t5002538D41CD83FDd0 > 0.0 225.3 0.0 16210.7 0.0 0.0 0.0 0.2 0 4 c7t5002538D41CD7F7Ed0 > 0.0 282.3 0.0 19991.1 0.0 0.0 0.0 0.2 0 5 c7t5002538D41CD7F7Dd0 > 0.0 280.3 0.0 19871.0 0.0 0.0 0.0 0.2 0 5 c7t5002538D41CD7F7Ad0 The pool disks see 1989 writes per second total or 994 writes per second logically. It seems to me that reducing 3067 requested writes to 994 logical writes is the opposite of amplification. What do you expect? -- richard <div class="HOEnZb"><div class="h5"> > > I used zvol of 64k, i try with 8k and problem is the same. > > zfs get all filervm2/hdd-110022a : > > NAME PROPERTY VALUE SOURCE > filervm2/hdd-110022a type volume - > filervm2/hdd-110022a creation Tue May 16 10:24 2017 - > filervm2/hdd-110022a used 5.26G - > filervm2/hdd-110022a available 2.90T - > filervm2/hdd-110022a referenced 5.24G - > filervm2/hdd-110022a compressratio 3.99x - > filervm2/hdd-110022a reservation none default > filervm2/hdd-110022a volsize 25G local > filervm2/hdd-110022a volblocksize 64K - > filervm2/hdd-110022a checksum on default > filervm2/hdd-110022a compression lz4 local > filervm2/hdd-110022a readonly off default > filervm2/hdd-110022a copies 1 default > filervm2/hdd-110022a refreservation none default > filervm2/hdd-110022a primarycache all default > filervm2/hdd-110022a secondarycache all default > filervm2/hdd-110022a usedbysnapshots 15.4M - > filervm2/hdd-110022a usedbydataset 5.24G - > filervm2/hdd-110022a usedbychildren 0 - > filervm2/hdd-110022a usedbyrefreservation 0 - > filervm2/hdd-110022a logbias latency default > filervm2/hdd-110022a dedup off default > filervm2/hdd-110022a mlslabel none default > filervm2/hdd-110022a sync standard local > filervm2/hdd-110022a refcompressratio 3.99x - > filervm2/hdd-110022a written 216K - > filervm2/hdd-110022a logicalused 20.9G - > filervm2/hdd-110022a logicalreferenced 20.9G - > filervm2/hdd-110022a snapshot_limit none default > filervm2/hdd-110022a snapshot_count none default > filervm2/hdd-110022a redundant_metadata all default > > Sorry for my bad english. > > What can be the problem ? thanks > > Best regards, > > Anthony > > </div></div><div class="HOEnZb"><div class="h5">> _______________________________________________ > OmniOS-discuss mailing list > <a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a> > <a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" rel="noreferrer" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a> </div></div></blockquote></div> </div>