Difference between revisions of "Bidirectional Rsync/Unison based SysVol replication workaround"

(Initial Edition)
 
m (/* minor update and grammar)
 
(27 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
= Introduction =
 
= Introduction =
 
+
Samba AD currently doesn't provide support for SysVol replication. To achieve this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.
WORK in Progress
 
 
 
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.
 
  
 
= Information on unison + rsync replication =
 
= Information on unison + rsync replication =
  
This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.
+
This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. This howto only covers a two DC setup.
  
It have the following advantages:  
+
It has the following advantages:  
* setup is fast done
+
* Quick setup
* configuration is very easy
+
* Configuration is very easy
 
* Can work with windows (Please add in)
 
* Can work with windows (Please add in)
  
Line 18: Line 15:
 
= Setup the SysVol replication =
 
= Setup the SysVol replication =
  
== Installation script ==
+
Some assumptions:
If you are using apt package manger you can try this script
+
You are running all commands as root.
[https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh]
+
rsync location  /usr/bin/rsync
 
+
  sysvol is located at /var/lib/samba/sysvol on both DC1 and DC2
Please thank L.v. Belle for creating this
+
  unison location /usr/bin/unison
 
+
  The first DC is DC1
Make Change to the following to fix your need:
+
  The second DC is DC2
 
+
  sysvolsync log location /var/log/sysvol-sync.log
  SETSAMBASYSVOLBASE=/var/lib/samba
 
  SETSAMBASYSVOLPATH=/var/lib/samba/sysvol
 
  ## in Minutes
 
  SETCRONTIME=5
 
  SETCRONFILE=/etc/cron.d/sysvol-sync
 
SETIPDC1=""
 
SETIPDC2=""
 
  
Run it <tt>./3-setup-sysvol-bidirectional.sh</tt>
+
Change the paths if your setup is different.
  
== Manually ==
 
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
 
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
* We don't need to setup rsync server.
+
* You don't need to setup the rsync server.
* Install unison by using your package manager or compile from source.
+
* Install unison by using your package manager or compile from source. (On Gentoo you need to do <code>eselect unison</code> to create the link)
  
 
==== Creating SSH Public Key and ssh-copy to DC2====
 
==== Creating SSH Public Key and ssh-copy to DC2====
  ssh-keygen -t dsa
+
  ssh-keygen -t rsa
  ssh-copy-id -i ~/.ssh/id_dsa.pub root@dc2
+
  ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2
  
 
You can try to access DC2 via ssh  
 
You can try to access DC2 via ssh  
ssh $SETIPDC2
+
ssh DC2
 +
 
 +
==== Setup ssh Control ====
 +
If the remote system enforces rate limits on incoming ssh connections, unison will fail if you try to run it this way.
 +
So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.
 +
 
 +
mkdir ~/.ssh/ctl
 +
cat << EOF > ~/.ssh/ctl/config
 +
Host *
 +
ControlMaster auto
 +
ControlPath ~/.ssh/ctl/%h_%p_%r
 +
ControlPersist 1
 +
EOF
  
 +
==== Setup Sysvolsync Log files ====
 +
Do the following on DC1 so that you can check what happens during the sync.
 +
Please include this file into logrotate as the log size is not controlled.
  
 +
touch /var/log/sysvol-sync.log
 +
chmod 640 /var/log/sysvol-sync.log
  
== Setup on all other Domain Controller(s) ==
+
==== Setup Unison defaults running parameters ====
 +
Please run the following on DC1
  
* Install rsync by using your paket manager or compile from source. Make sure, that you use a version that supports extended ACLs!
+
install -o root -g root -m 0750 -d /root/.unison
 +
cat << EOF > /root/.unison/default.prf
 +
# Unison preferences file
 +
# Roots of the synchronization
 +
#
 +
# copymax & maxthreads params were set to 1 for easier troubleshooting.
 +
# Have to experiment to see if they can be increased again.
 +
root = /var/lib/samba
 +
# Note that 2 x / behind DC2, it is required
 +
root = ssh://root@DC2//var/lib/samba
 +
#
 +
# Paths to synchronize
 +
path = sysvol
 +
#
 +
#ignore = Path stats    ## ignores /var/www/stats
 +
auto=true
 +
batch=true
 +
perms=0
 +
rsync=true
 +
maxthreads=1
 +
retry=3
 +
confirmbigdeletes=false
 +
servercmd=/usr/bin/unison
 +
copythreshold=0
 +
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
 +
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
 +
copyquoterem = true
 +
copymax = 1
 +
logfile = /var/log/sysvol-sync.log
 +
EOF
  
* Create a password file /usr/local/samba/etc/rsync-sysvol.secret</tt> and fill it with the password you set on the PDC for the <tt>sysvol-replication</tt> rsync account (permissions of that file must not be world-readable!):
+
=== Setup SysVol on DC2 ===
 +
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 +
* On DC2 Install unison by using your package manager or compile from source. (On Gentoo you need to do <code>eselect unison</code> to create the link)
 +
* Make sure, that you have [[Joining_a_Samba_DC_to_an_Existing_Active_Directory#Built-in_User_.26_Group_ID_Mappings|identical IDs of built-in groups on all DCs]].
  
pa$$w0rd
 
  
* For replicating the SysVol folder, run the following command (<tt>--dry-run</tt> means that no modifications are actually made):
+
== 1st Trial ==
 +
You now use rsync to create the directory structure with extended attributes
 +
Then the unison setup will only copy the extended attributes on files.
  
# rsync --dry-run -XAavz --delete-after --password-file=/usr/local/samba/etc/rsync-sysvol.secret rsync://sysvol-replication@{IP-of-you-PDC}/SysVol/ /path/to/your/sysvol/folder/
+
<BR>Please make a '''backup''' of your sysvol, just in case, this is because there is no <code>dry-run</code>
 
   
 
   
  receiving file list ... done                                                                                                                                                                                 
+
  /usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *" /var/lib/samba/sysvol root@DC2:/var/lib/samba  &&  /usr/bin/unison
created directory /usr/local/samba/var/locks/sysvol                                                                                                                                                           
+
 
./                                                                                                                                                                                                           
+
:'''Note: The path on DC2 is just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!'''
samdom.example.com/                                                                                                                                                                                           
 
samdom.example.com/Policies/                                                                                                                                                                                 
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/                                                                                                                                          
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INI                                                                                                                                   
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/MACHINE/                                                                                                                                  
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/MACHINE/Registry.pol                                                                                                                       
 
  samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/USER/                                                                                                                                      
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/USER/Documents & Settings/                                                                                                                 
 
  samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/USER/Scripts/                                                                                                                              
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/USER/Scripts/Logoff/
 
samdom.example.com/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/USER/Scripts/Logon/
 
samdom.example.com/Policies/{6AC1786C-016F-11D2-945F-00C04FB984F9}/
 
samdom.example.com/Policies/{6AC1786C-016F-11D2-945F-00C04FB984F9}/GPT.INI
 
samdom.example.com/Policies/{6AC1786C-016F-11D2-945F-00C04FB984F9}/MACHINE/
 
samdom.example.com/Policies/{6AC1786C-016F-11D2-945F-00C04FB984F9}/USER/
 
samdom.example.com/scripts/
 
samdom.example.com/scripts/logonscript.bat
 
 
sent 124 bytes  received 1064 bytes  2376.00 bytes/sec
 
total size is 1207  speedup is 1.02 (DRY RUN)
 
  
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and removes everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!'''
+
== Add to Crontab on DC1 ==
 +
On DC1 run the following:
 +
crontab -e
 +
*/5 * * * * /usr/bin/unison -silent
  
* If everything looks sane, run the command without the <tt>--dry-run</tt> option and let rsync do the replication.
+
= When you try to resync the folder =
 +
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
 +
# Disable Cron on DC1, like Add a "#" on the line with <code>crontab -e</code>
 +
# Check if rsync or unison are currently running in <code>ps -aux</code> if yes, wait for it to finish OR kill it (if it is zombie)
 +
# Remove the hash files on both DC1 and DC2 on <code>/root/.unison</code>
 +
# Now check your sysvol and resync
 +
# Confirm that everything is ok again
 +
# Re-enable the cron on DC1 again
  
* To automate synchronisation, you can run the command via cron (e. g. every 5 minutes).
+
= FAQ =
  
*/5 * * * *          rsync -XAavz --delete-after --password-file=/usr/local/samba/etc/rsync-sysvol.secret rsync://sysvol-replication@{IP-of-you-PDC}/SysVol/ /path/to/your/sysvol/folder/
+
* How can I do this on windows?
 +
** I don't have an answer, please post on the mailing list
  
* Repeat these steps on every DC (except your PDC!).
 
  
 +
* What to do if I've more than two DC's?
 +
** In Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server.
  
  
 +
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
 +
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.
  
  
= FAQ =
 
  
* How can I get multi-direction replication ?
 
** There are some testing using unsion with rsync to archive this.
 
Please check this installation script
 
https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh
 
  
  
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
+
----
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.
+
[[Category:Active Directory]]

Latest revision as of 10:51, 5 May 2020

Introduction

Samba AD currently doesn't provide support for SysVol replication. To achieve this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.

Information on unison + rsync replication

This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. This howto only covers a two DC setup.

It has the following advantages:

  • Quick setup
  • Configuration is very easy
  • Can work with windows (Please add in)

We will use rsync through a SSH tunnel.

Setup the SysVol replication

Some assumptions:

You are running all commands as root.
rsync location  /usr/bin/rsync
sysvol is located at /var/lib/samba/sysvol on both DC1 and DC2
unison location /usr/bin/unison
The first DC is DC1
The second DC is DC2
sysvolsync log location /var/log/sysvol-sync.log

Change the paths if your setup is different.

Setup on the Domain Controller with the PDC Emulator FSMO role

  • Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • You don't need to setup the rsync server.
  • Install unison by using your package manager or compile from source. (On Gentoo you need to do eselect unison to create the link)

Creating SSH Public Key and ssh-copy to DC2

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2

You can try to access DC2 via ssh

ssh DC2

Setup ssh Control

If the remote system enforces rate limits on incoming ssh connections, unison will fail if you try to run it this way. So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.

mkdir ~/.ssh/ctl
cat << EOF > ~/.ssh/ctl/config
Host *
ControlMaster auto
ControlPath ~/.ssh/ctl/%h_%p_%r
ControlPersist 1
EOF

Setup Sysvolsync Log files

Do the following on DC1 so that you can check what happens during the sync. Please include this file into logrotate as the log size is not controlled.

touch /var/log/sysvol-sync.log
chmod 640 /var/log/sysvol-sync.log

Setup Unison defaults running parameters

Please run the following on DC1

install -o root -g root -m 0750 -d /root/.unison
cat << EOF > /root/.unison/default.prf
# Unison preferences file
# Roots of the synchronization
#
# copymax & maxthreads params were set to 1 for easier troubleshooting.
# Have to experiment to see if they can be increased again.
root = /var/lib/samba
# Note that 2 x / behind DC2, it is required
root = ssh://root@DC2//var/lib/samba 
# 
# Paths to synchronize
path = sysvol
#
#ignore = Path stats    ## ignores /var/www/stats
auto=true
batch=true
perms=0
rsync=true
maxthreads=1
retry=3
confirmbigdeletes=false
servercmd=/usr/bin/unison
copythreshold=0
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
copyquoterem = true
copymax = 1
logfile = /var/log/sysvol-sync.log
EOF

Setup SysVol on DC2

  • On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • On DC2 Install unison by using your package manager or compile from source. (On Gentoo you need to do eselect unison to create the link)
  • Make sure, that you have identical IDs of built-in groups on all DCs.


1st Trial

You now use rsync to create the directory structure with extended attributes Then the unison setup will only copy the extended attributes on files.


Please make a backup of your sysvol, just in case, this is because there is no dry-run

/usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@DC2:/var/lib/samba  &&  /usr/bin/unison
Note: The path on DC2 is just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!

Add to Crontab on DC1

On DC1 run the following:

crontab -e 
*/5 * * * * /usr/bin/unison -silent

When you try to resync the folder

Warning: Please follow the steps below OR you can end up with an empty sysvol folder.
  1. Disable Cron on DC1, like Add a "#" on the line with crontab -e
  2. Check if rsync or unison are currently running in ps -aux if yes, wait for it to finish OR kill it (if it is zombie)
  3. Remove the hash files on both DC1 and DC2 on /root/.unison
  4. Now check your sysvol and resync
  5. Confirm that everything is ok again
  6. Re-enable the cron on DC1 again

FAQ

  • How can I do this on windows?
    • I don't have an answer, please post on the mailing list


  • What to do if I've more than two DC's?
    • In Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server.


  • Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
    • A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.