Linux CIFS Transparent Failover

Introduction

Starting from 5.11 the cifs kernel client supports transparent failover for clients connected to Windows clusters by using the witness protocol [MS-SWN].

Usage

To mount a share with witness protocol support, just add witness to the mount options:

mount -tcifs {service} {mount-point} -owitness.


Implementation details

Most of the implementation is done in user-space, the kernel just sends and receives messages from the swnd daemon over a netlink socket.

When cifs mount a share having the continuous availability flag set in the tree connect response it sends a message to swnd to register for notifications for the share.

When swnd receive a notification from the cluster, it transforms the message to a generic representation and forwards it to the kernel. The kernel then decide what to do.

When the share is unmounted, a new message is sent to swnd to unregister and stop receiving notifications.

The swnd daemon

This binary is part of cifs-utils and communicates with the kernel over a netlink socket. Its main task is to handle the netlink communication, as most of the protocol is implemented in a shared library provided by samba. This daemon is intended to be run as a systemd unit and logs to the system journal.

The swnclient shared library

The client side of the witness protocol is implemented as a shared library provided by samba (libswnclient.so). This shared library abstract its consumers of opening the RPC connection, authentication, NDR marshaling and unmarshaling and some witness protocol complexity, providing an easy to use API which consists of the following functions:

  • swnc_init_state
  • swnc_set_debug_callback
  • swnc_register_[send,recv]
  • swnc_wait_notification_[send,recv]
  • swnc_unregister_[send,recv]

Demo

To setup a Windows cluster you can have a look to the Setting up a Windows failover cluster page.

Move cluster role

Cifs witness move role.gif

Move the cifs client to a different node

Cifs witness move client.gif