<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> I have added a similar solution with full service and storage failover of two identical storage servers last year into my napp-it web ui as an experimental solution. Main difference is that I do nor use single disks as targets but the whole ZFS pools on two servers to mirror them over the network and share the resulting network mirror. This will allow a head to function with full redundancy if the other server fails. As you only need one LUN per server this reduces complexity. I also switch target ip and NFS/SMB service ip for an instant service failover, see my solution under <a class="moz-txt-link-freetext" href="http://www.napp-it.org/doc/downloads/z-raid.pdf">http://www.napp-it.org/doc/downloads/z-raid.pdf</a> Gea @napp-it.org <div class="moz-cite-prefix">Am 30.09.2017 um 08:45 schrieb Stephan Budach: </div> <blockquote type="cite" cite="mid:150975111.1847.1506753957992.JavaMail.stephan.budach@budy.stephanbudach.de"> <pre wrap="">Hi Sergey, ----- Ursprüngliche Mail ----- </pre> <blockquote type="cite"> <pre wrap="">Von: "sergey ivanov" <a class="moz-txt-link-rfc2396E" href="mailto:sergey57@gmail.com"><sergey57@gmail.com></a> An: "Stephan Budach" <a class="moz-txt-link-rfc2396E" href="mailto:stephan.budach@jvm.de"><stephan.budach@jvm.de></a> CC: "omnios-discuss" <a class="moz-txt-link-rfc2396E" href="mailto:omnios-discuss@lists.omniti.com"><omnios-discuss@lists.omniti.com></a> Gesendet: Freitag, 29. September 2017 22:30:31 Betreff: Re: [OmniOS-discuss] OmniOS based redundant NFS Thanks, Stephan. I did a simple test with creating lu over physical disks for use as ISSI targets, and it worked well. I am going to directly connect 2 servers and export their disks as separate I SCSI targets. Or maybe different LUNs in a target. And then on the active server start initiator to get these targets, and combine them into a pool of 2-way mirrors so that it stays degraded but working if one of the servers dies. </pre> </blockquote> <pre wrap=""> Well, different targets mean, that you will be able to service single disks on one node, without having to degrade the whole zpool, but only the affected vdevs. On the other hand, there is more complexity since, you will have of course quite a big number of iSCSI targets to login to. This may be ok, if the number doesn't get too hight, but going with hundreds of disks, I chose to use fewer targets with more LUNs. One thing to keep in mind is, that stmfad allows you to create the guuid to your liking. That is that you can freely choose the last 20 bytes to be anything you want. I used that to ascii-code the node name and slot into the guid, such as that it displays on my NFS heads, when running format. This helps a lot in mapping the LUNs to drives. </pre> <blockquote type="cite"> <pre wrap="">So, manual failover for this configuration will be the following. If the server to be disabled is still active, stop NFS, export zpool on it, stop iscsiadm, release shared IP. On the other server: import zpool and start NFS, activate shared IP. </pre> </blockquote> <pre wrap=""> I am using the sharenfs properties of ZFS, but you will likely have to run zpool export -f <zpool> if you want to fail over the service, since the zpool is still busy. Also, you'd better set zpool failmode to panic instead of wait, such as that an issue triggers a reboot, rather than keeping you NFS head waiting. </pre> <blockquote type="cite"> <pre wrap="">I read once there are some tricks which make clients do not recognize NFS server is changed underneath all mounts, but I never tried it. </pre> </blockquote> <pre wrap=""> The only issue I came across was, when I deliberatley failed over the NFS service forth and back within the a too short period, which causes the NFSd on the former primary node to re-use the tcp packets numbers, insisting on reusing it's old NFS connections to the clients. I solved that by resetting the NFSd each time a service starts on any NFS head. The currently connected NFS clients are not affected by that and this solved this particular issue for me. Cheers, Stephan </pre> <blockquote type="cite"> <pre wrap="">-- Regards, Sergey Ivanov. Regards, Sergey Ivanov On Thu, Sep 28, 2017 at 12:49 AM, Stephan Budach <a class="moz-txt-link-rfc2396E" href="mailto:stephan.budach@jvm.de"><stephan.budach@jvm.de></a> wrote: </pre> <blockquote type="cite"> <pre wrap="">Hi Sergey, ----- Ursprüngliche Mail ----- </pre> <blockquote type="cite"> <pre wrap="">Von: "sergey ivanov" <a class="moz-txt-link-rfc2396E" href="mailto:sergey57@gmail.com"><sergey57@gmail.com></a> An: "Stephan Budach" <a class="moz-txt-link-rfc2396E" href="mailto:stephan.budach@jvm.de"><stephan.budach@jvm.de></a> CC: "omnios-discuss" <a class="moz-txt-link-rfc2396E" href="mailto:omnios-discuss@lists.omniti.com"><omnios-discuss@lists.omniti.com></a> Gesendet: Mittwoch, 27. September 2017 23:15:49 Betreff: Re: [OmniOS-discuss] OmniOS based redundant NFS Thanks, Stephan! Please, explain "The reason to use two x two separate servers is, that the mirrored zpool's vdevs look the same on each NFS head". I understand that, if I want to have the same zpool based on iscsi devices, I should not mix local disks with iscsi target disks. But I think I can have 2 computers, each exporting a set of local disks as iscsi targets. And to have iscsi initiators on the same computers importing these targets to build zpools. Also, looking at sbdadm, I think I can 'create lu /dev/rdsk/c0t0d3s2'. Ok, I think I would better try it and report how it goes. </pre> </blockquote> <pre wrap=""> Actually, things can become quite complex, I'd like to reduce the "mental" involvement to the absolute minimum, mainly because we often faced a situation where something would suddenly break, which had been running for a long time without problems. This is when peeple start… well maybe not panicking, but having to recap what the current setup was like and what they had to do to tackle this. So, uniformity is a great deal of help on such systems - at least for us. Technically, there is no issue with mixing local and remote iSCST targets on the same node, which serves as an iSCSI target and a NFS head. Also, if one of the nodes really goes down, you will be loosing your failover NFS head as well, maybe not a big deal and depending on your requirements okay. I do have such a setup as well, although only for an archive ZPOOL, where I can tolerate this reduced redundancy for the benefit of a more lightweight setup. Cheers, Stephan </pre> </blockquote> <pre wrap=""> </pre> <fieldset class="mimeAttachmentHeader"></fieldset> <pre wrap="">_______________________________________________ OmniOS-discuss mailing list <a class="moz-txt-link-abbreviated" href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a> <a class="moz-txt-link-freetext" href="http://lists.omniti.com/mailman/listinfo/omnios-discuss">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a> </pre> </blockquote> </blockquote> </body> </html>