<div dir="ltr"><div>You haven't enabled dedup on any zfs volumes? That will drop performance by 30x to 300x especially on an array that size unless you have an insane amount of memory.<br><br></div><br></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Tue, Feb 12, 2013 at 3:52 AM, Ram Chander <span dir="ltr"><<a href="mailto:ramquick@gmail.com" target="_blank">ramquick@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">cp of 1Gb file takes 20min to the pool whereas on normal disk it takes 35sec.<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Feb 11, 2013 at 10:06 PM, Eric Sproul <span dir="ltr"><<a href="mailto:esproul@omniti.com" target="_blank">esproul@omniti.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">You still haven't said how you know that performance has dropped 30x.<br>
Where are the numbers?<br>
<div><div><br>
On Mon, Feb 11, 2013 at 11:33 AM, Ram Chander <<a href="mailto:ramquick@gmail.com" target="_blank">ramquick@gmail.com</a>> wrote:<br>
> I am not sure what happened on Jan 5. I havent run scrub or replaced devices<br>
> for more than 3 months. The underlying hardware is Dell Md1200 which has 48<br>
> disks.<br>
> How to recover from this ? I had tried rebooting but issue comes back.<br>
><br>
><br>
> On Mon, Feb 11, 2013 at 8:14 PM, Eric Sproul <<a href="mailto:esproul@omniti.com" target="_blank">esproul@omniti.com</a>> wrote:<br>
>><br>
>> On Mon, Feb 11, 2013 at 7:48 AM, Ram Chander <<a href="mailto:ramquick@gmail.com" target="_blank">ramquick@gmail.com</a>> wrote:<br>
>> > Hi,<br>
>> ><br>
>> > My OI box is expreiencing slow zfs writes ( around 30 times slower ).<br>
>> > iostat<br>
>> > reports below error though pool is healthy. This is happening in past 4<br>
>> > days<br>
>> > though no change was done to system. Is the hard disks faulty ?<br>
>> > Please help.<br>
>><br>
>> How have you measured this 30x drop in performance? You haven't<br>
>> provided any data.<br>
>><br>
>><br>
>> > c4t0d0 Soft Errors: 0 Hard Errors: 5 Transport Errors: 0<br>
>> > Vendor: iDRAC Product: Virtual CD Revision: 0323 Serial No:<br>
>> > Size: 0.00GB <0 bytes><br>
>> > Media Error: 0 Device Not Ready: 5 No Device: 0 Recoverable: 0<br>
>> > Illegal Request: 1 Predictive Failure Analysis: 0<br>
>><br>
>> I wouldn't worry about errors here. This is a virtual device provided<br>
>> by your server's lights-out management system.<br>
>><br>
>><br>
>> > root@host:~# fmadm faulty<br>
>> > --------------- ------------------------------------ --------------<br>
>> > ---------<br>
>> > TIME EVENT-ID MSG-ID<br>
>> > SEVERITY<br>
>> > --------------- ------------------------------------ --------------<br>
>> > ---------<br>
>> > Jan 05 08:21:09 7af1ab3c-83c2-602d-d4b9-f9040db6944a ZFS-8000-HC<br>
>> > Major<br>
>> ><br>
>> > Host : host<br>
>> > Platform : PowerEdge-R810<br>
>> > Product_sn :<br>
>> ><br>
>> > Fault class : fault.fs.zfs.io_failure_wait<br>
>> > Affects : zfs://pool=test<br>
>> > faulted but still in service<br>
>> > Problem in : zfs://pool=test<br>
>> > faulted but still in service<br>
>> ><br>
>> > Description : The ZFS pool has experienced currently unrecoverable I/O<br>
>> > failures. Refer to<br>
>> > <a href="http://illumos.org/msg/ZFS-8000-HC" target="_blank">http://illumos.org/msg/ZFS-8000-HC</a><br>
>> > for<br>
>> > more information.<br>
>> ><br>
>> > Response : No automated response will be taken.<br>
>> ><br>
>> > Impact : Read and write I/Os cannot be serviced.<br>
>> ><br>
>> > Action : Make sure the affected devices are connected, then run<br>
>> > 'zpool clear'.<br>
>><br>
>> What has happened since January 5? The pool appears fine now. Did<br>
>> you run a scrub? Replace devices? Reboot? It looks like ZFS<br>
>> encountered an underlying problem with the hardware.<br>
>><br>
>> Eric<br>
>> _______________________________________________<br>
>> OmniOS-discuss mailing list<br>
>> <a href="mailto:OmniOS-discuss@lists.omniti.com" target="_blank">OmniOS-discuss@lists.omniti.com</a><br>
>> <a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
><br>
><br>
_______________________________________________<br>
OmniOS-discuss mailing list<br>
<a href="mailto:OmniOS-discuss@lists.omniti.com" target="_blank">OmniOS-discuss@lists.omniti.com</a><br>
<a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OmniOS-discuss mailing list<br>
<a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>
<a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
<br></blockquote></div><br></div>