Bidirectional Rsync/Unison based SysVol replication workaround: Difference between revisions

From SambaWiki
m (Mmuehlfeld moved page SysVol Bidirectional Replication to Bidirectional Rsync/Unison based SysVol replication workaround: Rename page for new, better wiki structure)
(7 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Introduction =
= Introduction =
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.


= Information on Osync replication =
= Information on unison + rsync replication =


This HowTo describes a solution for SysVol replication, that is based on Osync, which is two way sync system based on rsync. This howto covers the bidirectional synchronization of two DC servers.
This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.


This solution has the following advantages:
It have the following advantages:
* setup is fast
* setup is fast done
* configuration is very easy
* configuration is very easy
* Can work with windows (Windows can be used as master using the MinGW environment)
* Can work with windows (Please add in)


Osnyc uses rsync through a SSH tunnel.
We will use rsync through a SSH tunnel.


= Setup the SysVol replication =
= Setup the SysVol replication =


Some assumptions:
== Installation ==
You are running all commands as root.
rsync is located /usr/bin/rsync
sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
unison is located /usr/bin/unison
DC1 is at DC1
DC2 is at DC2
sysvolsync log is located /var/log/sysvol-sync.log


Change the path if that don't fit your setup.
There aren't any deb or rpm packages as osync is a script, but you can get a copy here:
[http://www.netpower.fr/projects/osync/osync.v1.00a.tar.gz]http://www.netpower.fr/projects/osync/osync.v1.00a.tar.gz

You can then install osync with the following commands
tar xvf osync.v1.00a.tar.gz
cd ./osync
sh ./install.sh

This will copy osync to /usr/local/bin and create an example configuration file at /etc/osync/sync.conf


=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
Some assumption:
* You are running all command as root.
* rsync is installed on both servers
* sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
* osync is located /usr/local/bin/osync
* sync.conf is located /etc/osync/sync.conf
* DC1 is at DC1
* DC2 is at DC2 (And already join DC1)
* sysvolsync log is located /var/log/osync_*.log
* rsync must support extended ACLs
* Install rsync by using your package manager (eg: yum install rsync or apt-get install rsync) or compile from source. Make sure, that you use a version that supports extended ACLs (most versions do)!
* sysvol must be on disk which is mounted to support acl and also xattr
* We don't need to setup rsync server.
* We don't need to setup rsync server.
* Install unison by using your package manager or compile from source. (Gentoo need to do <tt>eselect unison</tt> to create the link)

Change the path if that don't fit your setup.


==== Creating SSH Public Key and ssh-copy to DC2====
==== Creating SSH Public Key and ssh-copy to DC2====
ssh-keygen -t dsa
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_dsa.pub root@DC2
ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2


You can try to access DC2 via ssh
You can try to access DC2 via ssh
ssh DC2
ssh DC2


=== Osync Configuration Setup on DC1 ===
==== Setup ssh Control ====
If the remote system enforces rate limits on incoming ssh connections, unison wil fail if you try to run it this way.
Edit the /etc/osync/sync.conf and make some changes
So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.


mkdir ~/.ssh/ctl
#!/usr/bin/env bash
cat << EOF > ~/.ssh/ctl/config
SYNC_ID="sysvol_sync"
Host *
MASTER_SYNC_DIR="/var/lib/samba/sysvol"
ControlMaster auto
SLAVE_SYNC_DIR="ssh://root@DC2:22//var/lib/samba/sysvol"
ControlPath ~/.ssh/ctl/%h_%p_%r
SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa"
ControlPersist 1
PRESERVE_ACL=yes
EOF
PRESERVE_XATTR=yes
SOFT_DELETE=yes
DESTINATION_MAILS="your@email.com"


==== Setup Sysvolsync Log files ====
Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir.
Do the following on DC1 so that we can check what happen on the sync.
Unless you have specific reasons to do so, leave these options enabled.
Please include this files into to log rotate as we didn't control the log size here.
Osync has one benefit as it will only send email alert if there is problem.


touch /var/log/sysvol-sync.log
=== Setup on DC2 ===
chmod 640 /var/log/sysvol-sync.log
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!


==== Setup Unison defaults running parameters ====
* Shutdown DC2 Samba AD DC
Please run the following on DC1
mv /var/lib/samba/private/idmap.ldb /var/lib/samba/private/idmap.ldb.backup"
rm /var/cache/samba/gencache.tdb"


install -o root -g root -m 0750 -d /root/.unison
* Copy idmap.ldb from DC1 to sync the idmap.
cat << EOF > /root/.unison/default.prf
# Unison preferences file
# Roots of the synchronization
#
# copymax & maxthreads params were set to 1 for easier troubleshooting.
# Have to experiment to see if they can be increased again.
root = /var/lib/samba
# Note that 2 x / behind DC2, it is required
root = ssh://root@DC2//var/lib/samba
#
# Paths to synchronize
path = sysvol
#
#ignore = Path stats ## ignores /var/www/stats
auto=true
batch=true
perms=0
rsync=true
maxthreads=1
retry=3
confirmbigdeletes=false
servercmd=/usr/bin/unison
copythreshold=0
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
copyquoterem = true
copymax = 1
logfile = /var/log/sysvol-sync.log
EOF

=== Setup SysVol on DC2 ===
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
* On DC2 Install unison by using your package manager or compile from source. (Gentoo need to do <tt>eselect unison</tt> to create the link)
* Run the following command on '''DC1'''
* Run the following command on '''DC1'''
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/


ssh DC2 "cp -R --preserve=all /var/lib/samba/sysvol /var/lib/samba/"
== 1st Try ==

== 1st Trial ==
What happen is we use rsync to create the directory structure with extended attributes
What happen is we use rsync to create the directory structure with extended attributes
Than unison setup copies only the extened attributes on files.
Than unison setup copies only the extened attributes on files.


<BR>Before starting, make a '''backup''' of your sysvol just in case.
<BR>Please make a '''backup''' on you sysvol just in case as we are really '''TRYING''' there are no <tt>dry-run</tt>
/usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *" /var/lib/samba/sysvol root@DC2:/var/lib/samba && /usr/bin/unison


:'''Note: The path on DC2 are just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!'''
/usr/local/bin/osync.sh /etc/osync/sync.conf --dry --verbose

If this runs successfully, you may remove the --dry option and run again.

/usr/local/bin/osync.sh /etc/osync/sync.conf --verbose


== Add to Crontab on DC1 ==
== Add to Crontab on DC1 ==
On DC1 run the following:
On DC1 run the following:
crontab -e
crontab -e
*/5 * * * * root /usr/local/bin/osync.sh /etc/osync/sync.conf --silent
*/5 * * * * /usr/bin/unison -silent

:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully to see if the replication is doing, what you expect!'''


= When you try to resync the folder =
= When you try to resync the folder =
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
# Disable Cron on DC1, like Add a "#" on the line with <tt>crontab -e</tt>
# Disable Cron on DC1, like Add a "#" on the line with <tt>crontab -e</tt>
# Check is any rsync or osync are currently running in <tt>ps -aux | grep sync</tt> if yes, wait for it to finished OR kill it (if it is zombie)
# Check is any rsync or unison are currently running in <tt>ps -aux</tt> if yes, wait for it to finished OR kill it (if it is zombie)
# Remove the .osync_workdir hash files on both DC1 and DC2 on <tt>MASTER/SLAVE_SYNC_DIR "/var/lib/samba/sysvol"</tt>
# Remove the hash files on both DC1 and DC2 on <tt>/root/.unison</tt>
# Now check your sysvol and resync
# Now check your sysvol and resync
# Confirm that everything is working again
# Confirm that everything is ok again
# Re-enable the Cron on DC1 again
# Re-enable the Cron on DC1 again


Line 110: Line 125:


* How can I do this on windows?
* How can I do this on windows?
** I don't have an answer, please post on the mailing list
** Install the MSYS bash environment (read the osync documentation), then you may run osync as master from Windows.



* What to do if I've more than one DC?
* What to do if I've more than one DC?
** By Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server. (Not tested)
** In Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server.
** Something like:
** DC1 <> DC2
** DC1 <> DC3
** DC1 <> DC2





* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible. Please check on the [https://lists.samba.org/archive/samba/2014-February/178546.html reply] from Samba Developer on the mailing list .
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.





----
[[Category:Active Directory]]

Revision as of 16:31, 22 January 2020

Introduction

Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.

Information on unison + rsync replication

This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.

It have the following advantages:

  • setup is fast done
  • configuration is very easy
  • Can work with windows (Please add in)

We will use rsync through a SSH tunnel.

Setup the SysVol replication

Some assumptions:

You are running all commands as root.
rsync is located  /usr/bin/rsync
sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
unison is located /usr/bin/unison
DC1 is at DC1
DC2 is at DC2
sysvolsync log is located /var/log/sysvol-sync.log

Change the path if that don't fit your setup.

Setup on the Domain Controller with the PDC Emulator FSMO role

  • Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • We don't need to setup rsync server.
  • Install unison by using your package manager or compile from source. (Gentoo need to do eselect unison to create the link)

Creating SSH Public Key and ssh-copy to DC2

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2

You can try to access DC2 via ssh

ssh DC2

Setup ssh Control

If the remote system enforces rate limits on incoming ssh connections, unison wil fail if you try to run it this way. So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.

mkdir ~/.ssh/ctl
cat << EOF > ~/.ssh/ctl/config
Host *
ControlMaster auto
ControlPath ~/.ssh/ctl/%h_%p_%r
ControlPersist 1
EOF

Setup Sysvolsync Log files

Do the following on DC1 so that we can check what happen on the sync. Please include this files into to log rotate as we didn't control the log size here.

touch /var/log/sysvol-sync.log
chmod 640 /var/log/sysvol-sync.log

Setup Unison defaults running parameters

Please run the following on DC1

install -o root -g root -m 0750 -d /root/.unison
cat << EOF > /root/.unison/default.prf
# Unison preferences file
# Roots of the synchronization
#
# copymax & maxthreads params were set to 1 for easier troubleshooting.
# Have to experiment to see if they can be increased again.
root = /var/lib/samba
# Note that 2 x / behind DC2, it is required
root = ssh://root@DC2//var/lib/samba 
# 
# Paths to synchronize
path = sysvol
#
#ignore = Path stats    ## ignores /var/www/stats
auto=true
batch=true
perms=0
rsync=true
maxthreads=1
retry=3
confirmbigdeletes=false
servercmd=/usr/bin/unison
copythreshold=0
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
copyquoterem = true
copymax = 1
logfile = /var/log/sysvol-sync.log
EOF

Setup SysVol on DC2

  • On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • On DC2 Install unison by using your package manager or compile from source. (Gentoo need to do eselect unison to create the link)
  • Run the following command on DC1
ssh DC2 "cp -R --preserve=all /var/lib/samba/sysvol /var/lib/samba/"

1st Trial

What happen is we use rsync to create the directory structure with extended attributes Than unison setup copies only the extened attributes on files.


Please make a backup on you sysvol just in case as we are really TRYING there are no dry-run

/usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@DC2:/var/lib/samba  &&  /usr/bin/unison
Note: The path on DC2 are just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!

Add to Crontab on DC1

On DC1 run the following:

crontab -e 
*/5 * * * * /usr/bin/unison -silent

When you try to resync the folder

Warning: Please follow the steps below OR you can end up with an empty sysvol folder.
  1. Disable Cron on DC1, like Add a "#" on the line with crontab -e
  2. Check is any rsync or unison are currently running in ps -aux if yes, wait for it to finished OR kill it (if it is zombie)
  3. Remove the hash files on both DC1 and DC2 on /root/.unison
  4. Now check your sysvol and resync
  5. Confirm that everything is ok again
  6. Re-enable the Cron on DC1 again

FAQ

  • How can I do this on windows?
    • I don't have an answer, please post on the mailing list


  • What to do if I've more than one DC?
    • In Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server.


  • Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
    • A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.