Difference between revisions of "Bidirectional Rsync/Unison based SysVol replication workaround"

m (wrong path)
(Change unison to osync)
Line 1: Line 1:
 
= Introduction =
 
= Introduction =
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.
+
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.
  
= Information on unison + rsync replication =
+
= Information on Osync replication =
  
This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.
+
This HowTo describes a solution for SysVol replication, that is based on Osync (rsync is required). As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.
  
 
It have the following advantages:  
 
It have the following advantages:  
* setup is fast done
+
* setup is fast
 
* configuration is very easy
 
* configuration is very easy
* Can work with windows (Please add in)
+
* Can work with windows (Todo)
  
We will use rsync through a SSH tunnel.
+
In this setup we will use rsync through a SSH tunnel.
  
 
= Setup the SysVol replication =
 
= Setup the SysVol replication =
  
== Installation script ==
+
== Installation ==
If you are using apt package manger you can try this script
 
[https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh]
 
  
Please thank L.v. Belle for creating this
+
I don't see any package but osync available on
 +
[https://github.com/deajan/osync]https://github.com/deajan/osync
  
Make Change to the following to fit your need:
+
== Getting Osync ==
 +
At the time of writing this Osync is still at v.100pre.
 +
However, you should always get a stable branch when it is available.
 +
We are getting it done using master branch.
 +
wget https://raw.githubusercontent.com/deajan/osync/master/osync.sh
 +
chmod +x osync.sh
 +
Move osync to /usr/bin/ or your preferred dir
 +
mv osync.sh /usr/bin/
  
  SETSAMBASYSVOLBASE=/var/lib/samba
+
== Getting Osync Configuration ==
SETSAMBASYSVOLPATH=/var/lib/samba/sysvol
+
  wget https://raw.githubusercontent.com/deajan/osync/master/sync.conf
  ## in Minutes
+
Move it to /etc/osync/
  SETCRONTIME=5
+
  mkdir /etc/osync/
SETCRONFILE=/etc/cron.d/sysvol-sync
+
  mv sync.conf /etc/osync/sync.conf
SETIPDC1=""
 
SETIPDC2=""
 
  
Run it <tt>./3-setup-sysvol-bidirectional.sh</tt>
+
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
 
== Manually ==
 
 
Some assumption:
 
Some assumption:
You are running all command as root.
+
* You are running all command as root.
rsync is located  /usr/bin/rsync
+
* rsync is located  /usr/bin/rsync
sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
+
* sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
unison is located /usr/bin/unison
+
* osync is located /usr/bin/osync
DC1 is at DC1
+
* osync.conf is located /etc/osync/sync.conf
DC2 is at DC2
+
* DC1 is at DC1
sysvolsync log is located /var/log/sysvol-sync.log
+
* DC2 is at DC2 (And already join DC1)
 +
* sysvolsync log is located /var/log/osync_*.log
 +
* rsync must support extended ACLs
 +
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 +
* sysvol must be on disk which is mounted to support acl and also xattr
 +
* We don't need to setup rsync server.
  
 
Change the path if that don't fit your setup.
 
Change the path if that don't fit your setup.
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 
* We don't need to setup rsync server.
 
* Install unison by using your package manager or compile from source. (Gentoo need to do <tt>eselect unison</tt> to create the link)
 
  
 
==== Creating SSH Public Key and ssh-copy to DC2====
 
==== Creating SSH Public Key and ssh-copy to DC2====
Line 57: Line 59:
 
  ssh DC2
 
  ssh DC2
  
==== Setup ssh Controle ====
+
=== Osync Configuration Setup on DC1 ===
If the remote system enforces rate limits on incoming ssh connections, unison wil fail if you try to run it this way.
+
Edit the /etc/osync/sync.conf and make some changes
So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.
 
  
  mkdir ~/.ssh/ctl
+
  #!/usr/bin/env bash
  cat << EOF > ~/.ssh/ctl/config
+
  SYNC_ID="sysvol_sync"
  Host *
+
MASTER_SYNC_DIR="/var/lib/samba/sysvol"
  ControlMaster auto
+
  SLAVE_SYNC_DIR="ssh://root@DC2:22//var/lib/samba/sysvol"
ControlPath ~/.ssh/ctl/%h_%p_%r
+
  SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa"
  ControlPersist 1
+
PRESERVE_ACL=yes
  EOF
+
PRESERVE_XATTR=yes
 +
  SOFT_DELETE=no
 +
  DESTINATION_MAILS="your@email.com"
  
==== Setup Sysvolsync Log files ====
+
I'm having some soft_delete bugs with Osync as files/folder conflict when move the deleted folder.  
Do the following on DC1 so that we can check what happen on the sync.
+
(Checking with the developer, might get fix soon)
Please include this files into to log rotate as we didn't control the log size here.
+
So Before that "SOFT_DELETE=no"
  
touch /var/log/sysvol-sync.log
+
Osync have one benefit as it will only send email alert if there is problem.
chmod 640 /var/log/sysvol-sync.log
 
  
==== Setup Unison defaults running parameters ====
+
=== Setup on DC2 ===
Please run the following on DC1
+
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  
install -o root -g root -m 0750 -d /root/.unison
+
* Shutdown DC2 Samba AD DC
  cat << EOF > /root/.unison/default.prf
+
  mv /var/lib/samba/private/idmap.ldb /var/lib/samba/private/idmap.ldb.backup"
# Unison preferences file
+
  rm /var/cache/samba/gencache.tdb"
# Roots of the synchronization
 
#
 
# copymax & maxthreads params were set to 1 for easier troubleshooting.
 
# Have to experiment to see if they can be increased again.
 
root = /var/lib/samba
 
# Note that 2 x / behind DC2, it is required
 
root = ssh://root@DC2//var/lib/samba  
 
#
 
# Paths to synchronize
 
path = sysvol
 
#
 
#ignore = Path stats    ## ignores /var/www/stats
 
auto=true
 
batch=true
 
perms=0
 
rsync=true
 
maxthreads=1
 
retry=3
 
  confirmbigdeletes=false
 
servercmd=/usr/bin/unison
 
copythreshold=0
 
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
 
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
 
copyquoterem = true
 
copymax = 1
 
logfile = /var/log/sysvol-sync.log
 
EOF
 
  
=== Setup SysVol on DC2 ===
 
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
 
* On DC2 Install unison by using your package manager or compile from source. (Gentoo need to do <tt>eselect unison</tt> to create the link)
 
 
* Run the following command on '''DC1'''
 
* Run the following command on '''DC1'''
 +
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/
  
ssh DC2 "cp -R --preserve=all /var/lib/samba/sysvol /var/lib/samba/"
+
== 1st Try ==  
 
 
== 1st Trial ==  
 
 
What happen is we use rsync to create the directory structure with extended attributes
 
What happen is we use rsync to create the directory structure with extended attributes
 
Than unison setup copies only the extened attributes on files.
 
Than unison setup copies only the extened attributes on files.
  
<BR>Please make a '''backup''' on you sysvol just in case as we are really '''TRYING''' there are no <tt>dry-run</tt>
+
<BR>Please make a '''backup''' on you sysvol just in case.
+
 
  /usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@DC2:/var/lib/samba  &&  /usr/bin/unison  &> /dev/null
+
  /usr/bin/osync.sh /etc/osync/sync.conf --dry --verbose
 +
 
 +
If this run successfully let remove the --dry and execute it.
  
:'''Note: The path on DC2 are just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!'''
+
/usr/bin/osync.sh /etc/osync/sync.conf --verbose
  
 
== Add to Crontab on DC1 ==
 
== Add to Crontab on DC1 ==
 
On DC1 run the following:
 
On DC1 run the following:
 
  crontab -e  
 
  crontab -e  
  */5 * * * * root  /usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root@DC2:/var/lib/samba  &&  /usr/bin/unison  &> /dev/null
+
  */5 * * * * root  /usr/bin/osync.sh /etc/osync/sync.conf  --silent
  
 
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!'''
 
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!'''
Line 137: Line 110:
 
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
 
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
 
# Disable Cron on DC1, like Add a "#" on the line with <tt>crontab -e</tt>
 
# Disable Cron on DC1, like Add a "#" on the line with <tt>crontab -e</tt>
# Check is any rsync or unison are currently running in <tt>ps -aux</tt> if yes, wait for it to finished OR kill it (if it is zombie)
+
# Check is any rsync or osync are currently running in <tt>ps -aux</tt> if yes, wait for it to finished OR kill it (if it is zombie)
# Remove the hash files on both DC1 and DC2 on <tt>/root/.unison</tt>
+
# Remove the .osync_workdir hash files on both DC1 and DC2 on <tt>MASTER/SLAVE_SYNC_DIR "/var/lib/samba/sysvol"</tt>
 
# Now check your sysvol and resync
 
# Now check your sysvol and resync
 
# Confirm that everything is ok again
 
# Confirm that everything is ok again

Revision as of 18:22, 24 June 2015

Introduction

Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.

Information on Osync replication

This HowTo describes a solution for SysVol replication, that is based on Osync (rsync is required). As Compare to the rsync method, it is bidirectional. But this howto only cover two DC setup.

It have the following advantages:

  • setup is fast
  • configuration is very easy
  • Can work with windows (Todo)

In this setup we will use rsync through a SSH tunnel.

Setup the SysVol replication

Installation

I don't see any package but osync available on [1]https://github.com/deajan/osync

Getting Osync

At the time of writing this Osync is still at v.100pre. However, you should always get a stable branch when it is available. We are getting it done using master branch.

wget https://raw.githubusercontent.com/deajan/osync/master/osync.sh
chmod +x osync.sh

Move osync to /usr/bin/ or your preferred dir

mv osync.sh /usr/bin/

Getting Osync Configuration

wget https://raw.githubusercontent.com/deajan/osync/master/sync.conf

Move it to /etc/osync/

mkdir /etc/osync/
mv sync.conf /etc/osync/sync.conf

Setup on the Domain Controller with the PDC Emulator FSMO role

Some assumption:

  • You are running all command as root.
  • rsync is located /usr/bin/rsync
  • sysvol is located /var/lib/samba/sysvol on both DC1 and DC2
  • osync is located /usr/bin/osync
  • osync.conf is located /etc/osync/sync.conf
  • DC1 is at DC1
  • DC2 is at DC2 (And already join DC1)
  • sysvolsync log is located /var/log/osync_*.log
  • rsync must support extended ACLs
  • Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • sysvol must be on disk which is mounted to support acl and also xattr
  • We don't need to setup rsync server.

Change the path if that don't fit your setup.

Creating SSH Public Key and ssh-copy to DC2

ssh-keygen -t dsa
ssh-copy-id -i ~/.ssh/id_dsa.pub root@DC2

You can try to access DC2 via ssh

ssh DC2

Osync Configuration Setup on DC1

Edit the /etc/osync/sync.conf and make some changes

#!/usr/bin/env bash
SYNC_ID="sysvol_sync"
MASTER_SYNC_DIR="/var/lib/samba/sysvol"
SLAVE_SYNC_DIR="ssh://root@DC2:22//var/lib/samba/sysvol"
SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa"
PRESERVE_ACL=yes
PRESERVE_XATTR=yes
SOFT_DELETE=no
DESTINATION_MAILS="your@email.com"

I'm having some soft_delete bugs with Osync as files/folder conflict when move the deleted folder. (Checking with the developer, might get fix soon) So Before that "SOFT_DELETE=no"

Osync have one benefit as it will only send email alert if there is problem.

Setup on DC2

  • On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
  • Shutdown DC2 Samba AD DC
mv /var/lib/samba/private/idmap.ldb /var/lib/samba/private/idmap.ldb.backup"
rm /var/cache/samba/gencache.tdb"
  • Run the following command on DC1
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/

1st Try

What happen is we use rsync to create the directory structure with extended attributes Than unison setup copies only the extened attributes on files.


Please make a backup on you sysvol just in case.

/usr/bin/osync.sh /etc/osync/sync.conf --dry --verbose

If this run successfully let remove the --dry and execute it.

/usr/bin/osync.sh /etc/osync/sync.conf --verbose

Add to Crontab on DC1

On DC1 run the following:

crontab -e 
*/5 * * * * root  /usr/bin/osync.sh /etc/osync/sync.conf  --silent
Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!

When you try to resync the folder

Warning: Please follow the steps below OR you can end up with an empty sysvol folder.
  1. Disable Cron on DC1, like Add a "#" on the line with crontab -e
  2. Check is any rsync or osync are currently running in ps -aux if yes, wait for it to finished OR kill it (if it is zombie)
  3. Remove the .osync_workdir hash files on both DC1 and DC2 on MASTER/SLAVE_SYNC_DIR "/var/lib/samba/sysvol"
  4. Now check your sysvol and resync
  5. Confirm that everything is ok again
  6. Re-enable the Cron on DC1 again

FAQ

  • How can I do this on windows?
    • I don't have an answer, please post on the mailing list


  • What to do if I've more than one DC?
    • By Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server.


  • Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
    • A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.