Difference between revisions of "Bidirectional Rsync/Unison based SysVol replication workaround"

(DONE)
Line 1: Line 1:
 
= Introduction =
 
= Introduction =
 
WORK in Progress
 
 
 
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.
 
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.
  
Line 24: Line 21:
 
Please thank L.v. Belle for creating this
 
Please thank L.v. Belle for creating this
  
Make Change to the following to fix your need:
+
Make Change to the following to fit your need:
  
 
  SETSAMBASYSVOLBASE=/var/lib/samba
 
  SETSAMBASYSVOLBASE=/var/lib/samba
Line 37: Line 34:
  
 
== Manually ==
 
== Manually ==
 +
Some assumption:
 +
You are running all command as root.
 +
rsync is located  /usr/bin/rsync
 +
sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
 +
unison is located /usr/bin/unison
 +
DC1 is at DC1
 +
DC2 is at DC2
 +
sysvolsync log is located /var/log/sysvol-sync.log
 +
 +
Change the path if that don't fit your setup.
 +
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
 
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 
* We don't need to setup rsync server.
 
* We don't need to setup rsync server.
Line 81: Line 88:
 
  root = /var/lib/samba
 
  root = /var/lib/samba
 
  # Note that 2 x / behind DC2, it is required
 
  # Note that 2 x / behind DC2, it is required
  root = ssh://root@$DC2//var/lib/samba  
+
  root = ssh://root@DC2//var/lib/samba  
 
  #  
 
  #  
 
  # Paths to synchronize
 
  # Paths to synchronize
Line 105: Line 112:
 
=== Setup SysVol on DC2 ===
 
=== Setup SysVol on DC2 ===
 
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
* On DC2 Install unison by using your package manager or compile from source.
+
* On DC2 Install unison by using your package manager or compile from source. (Gentoo need to do <tt>eselect unison</tt> to create the link)
 
* Run the following command on '''DC1'''
 
* Run the following command on '''DC1'''
  
 
  ssh DC2 "cp -R --preserve=all /var/lib/samba/sysvol /var/lib/samba/sysvol"
 
  ssh DC2 "cp -R --preserve=all /var/lib/samba/sysvol /var/lib/samba/sysvol"
  
 
+
== 1st Trial ==  
== Let us try ==  
 
 
What happen is we use rsync to create the directory structure with extended attributes
 
What happen is we use rsync to create the directory structure with extended attributes
 
Than unison setup copies only the extened attributes on files.
 
Than unison setup copies only the extened attributes on files.
  
crontab -e
+
<BR>Please make a '''backup''' on you sysvol just in case as we are really '''TRYING''' there are no <tt>dry-run</tt>
*/5 * * * * root  ${SETRSYNC} -XAavz --log-file ${SETSYNCLOG} --delete-after -f"+ */" -f"- *"  ${SETSAMBASYSVOLPATH} root@${SETIPDC2}:${SETSAMBASYSVOLBASE}  &&  ${SETUNISON}  &> /dev/null
+
EOF
+
  /usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@$DC2:/var/lib/samba  &&  /usr/bin/unison  &> /dev/null
 
 
 
 
 
 
 
 
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and removes everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!'''
 
 
 
* If everything looks sane, run the command without the <tt>--dry-run</tt> option and let rsync do the replication.
 
 
 
* To automate synchronisation, you can run the command via cron (e. g. every 5 minutes).
 
 
 
  */5 * * * *          rsync -XAavz --delete-after --password-file=/usr/local/samba/etc/rsync-sysvol.secret rsync://sysvol-replication@{IP-of-you-PDC}/SysVol/ /path/to/your/sysvol/folder/
 
 
 
* Repeat these steps on every DC (except your PDC!).
 
 
 
 
 
  
 +
== Add to Crontab on DC1 ==
 +
On DC1 run the following:
 +
crontab -e
 +
*/5 * * * * root  /sbin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@$DC2://var/lib/samba  &&  /usr/bin/unison  &> /dev/null
  
 +
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!'''
  
 
= FAQ =
 
= FAQ =
 
* How can I get multi-direction replication ?
 
** There are some testing using unsion with rsync to archive this.
 
Please check this installation script
 
https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh
 
 
  
 
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
 
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
 
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.
 
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.

Revision as of 17:13, 7 October 2014

Introduction

Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.

Information on unison + rsync replication

This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.

It have the following advantages:

  • setup is fast done
  • configuration is very easy
  • Can work with windows (Please add in)

We will use rsync through a SSH tunnel.

Setup the SysVol replication

Installation script

If you are using apt package manger you can try this script [1]

Please thank L.v. Belle for creating this

Make Change to the following to fit your need:

SETSAMBASYSVOLBASE=/var/lib/samba
SETSAMBASYSVOLPATH=/var/lib/samba/sysvol
## in Minutes
SETCRONTIME=5
SETCRONFILE=/etc/cron.d/sysvol-sync
SETIPDC1=""
SETIPDC2=""

Run it ./3-setup-sysvol-bidirectional.sh

Manually

Some assumption:

You are running all command as root.
rsync is located  /usr/bin/rsync
sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
unison is located /usr/bin/unison
DC1 is at DC1
DC2 is at DC2
sysvolsync log is located /var/log/sysvol-sync.log

Change the path if that don't fit your setup.

Setup on the Domain Controller with the PDC Emulator FSMO role

  • Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • We don't need to setup rsync server.
  • Install unison by using your package manager or compile from source. (Gentoo need to do eselect unison to create the link)

Creating SSH Public Key and ssh-copy to DC2

ssh-keygen -t dsa
ssh-copy-id -i ~/.ssh/id_dsa.pub root@DC2

You can try to access DC2 via ssh

ssh DC2

Setup ssh Controle

If the remote system enforces rate limits on incoming ssh connections, unison wil fail if you try to run it this way. So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.

mkdir ~/.ssh/ctl
cat << EOF > ~/.ssh/ctl/config
Host *
ControlMaster auto
ControlPath ~/.ssh/ctl/%h_%p_%r
ControlPersist 1
EOF

Setup Sysvolsync Log files

Do the following on DC1 so that we can check what happen on the sync. Please include this files into to log rotate as we didn't control the log size here.

touch /var/log/sysvol-sync.log
chmod 640 /var/log/sysvol-sync.log

Setup Unison defaults running parameters

Please run the following on DC1

install -o root -g root -m 0750 -d /root/.unison
cat << EOF > /root/.unison/default.prf
# Unison preferences file
# Roots of the synchronization
#
# copymax & maxthreads params were set to 1 for easier troubleshooting.
# Have to experiment to see if they can be increased again.
root = /var/lib/samba
# Note that 2 x / behind DC2, it is required
root = ssh://root@DC2//var/lib/samba 
# 
# Paths to synchronize
path = sysvol
#
#ignore = Path stats    ## ignores /var/www/stats
auto=true
batch=true
perms=0
rsync=true
maxthreads=1
retry=3
confirmbigdeletes=false
servercmd=/usr/bin/unison
copythreshold=0
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
copyquoterem = true
copymax = 1
logfile = ${SETSYNCLOG} 
EOF

Setup SysVol on DC2

  • On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • On DC2 Install unison by using your package manager or compile from source. (Gentoo need to do eselect unison to create the link)
  • Run the following command on DC1
ssh DC2 "cp -R --preserve=all /var/lib/samba/sysvol /var/lib/samba/sysvol"

1st Trial

What happen is we use rsync to create the directory structure with extended attributes Than unison setup copies only the extened attributes on files.


Please make a backup on you sysvol just in case as we are really TRYING there are no dry-run

/usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@$DC2:/var/lib/samba  &&  /usr/bin/unison  &> /dev/null

Add to Crontab on DC1

On DC1 run the following:

crontab -e 
*/5 * * * * root  /sbin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@$DC2://var/lib/samba  &&  /usr/bin/unison  &> /dev/null
Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!

FAQ

  • Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
    • A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.