Difference between revisions of "Bidirectional Rsync/osync based SysVol replication workaround"

m (/* first update before testing, fixed download url etc)
m (/* Commented out 'REMOTE_RUN_AFTER_CMD' in sample conf, sysvolreset is broken.)
 
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
= Introduction =
 
= Introduction =
 
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.
 
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.
 +
 +
  
 
= Information on Osync replication =
 
= Information on Osync replication =
Line 12: Line 14:
  
 
Osnyc uses rsync through a SSH tunnel.
 
Osnyc uses rsync through a SSH tunnel.
 +
 +
  
 
= Setup the SysVol replication =
 
= Setup the SysVol replication =
 +
  
 
== Installation ==
 
== Installation ==
  
There are no deb or rpm packages available this is because osync is a script, but you can get a copy here: [https://github.com/deajan/osync/releases]https://github.com/deajan/osync/releases
+
There are no deb or rpm packages available, this is because osync is a script, but you can get a copy here: [https://github.com/deajan/osync/releases]https://github.com/deajan/osync/releases
  
 
Once you have downloaded the tarball, you can install osync with the following commands
 
Once you have downloaded the tarball, you can install osync with the following commands
  tar xvf osync.v1.1.1.tar.gz
+
  tar xvf osync.v1.1.3.tar.gz
 
  cd ./osync
 
  cd ./osync
 
  sh ./install.sh
 
  sh ./install.sh
  
 
This will copy osync to /usr/local/bin and create an example configuration file at /etc/osync/sync.conf
 
This will copy osync to /usr/local/bin and create an example configuration file at /etc/osync/sync.conf
 +
 +
  
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
=== Setup on the Domain Controller with the PDC Emulator FSMO role ===
 
Some assumptions:
 
Some assumptions:
 
* You will run all commands as root.
 
* You will run all commands as root.
 +
* Samba is installed at /usr/local/samba
 
* sysvol is located at /usr/local/samba/var/locks/sysvol on both DC1 and DC2
 
* sysvol is located at /usr/local/samba/var/locks/sysvol on both DC1 and DC2
 
* osync is located /usr/local/bin/osync
 
* osync is located /usr/local/bin/osync
 
* sync.conf is located /etc/osync/sync.conf
 
* sync.conf is located /etc/osync/sync.conf
 
* The first DC is called DC1
 
* The first DC is called DC1
* The second DC is called DC2 (And already join DC1)
+
* The second DC is called DC2 (And already joined to the domain)
 
* sysvolsync log is located /var/log/osync_*.log
 
* sysvolsync log is located /var/log/osync_*.log
 
* rsync is installed on both servers
 
* rsync is installed on both servers
Line 42: Line 50:
  
 
Change path's if they don't fit in with your setup.
 
Change path's if they don't fit in with your setup.
 +
 +
  
 
==== Creating SSH Public Key and ssh-copy to DC2====
 
==== Creating SSH Public Key and ssh-copy to DC2====
  ssh-keygen -t dsa
+
  ssh-keygen -t rsa
  ssh-copy-id -i ~/.ssh/id_dsa.pub root@DC2
+
  ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2
  
 
You can try to access DC2 via ssh  
 
You can try to access DC2 via ssh  
 
  ssh DC2
 
  ssh DC2
 +
 +
  
 
=== Osync Configuration Setup on DC1 ===
 
=== Osync Configuration Setup on DC1 ===
Edit the /etc/osync/sync.conf and make some changes
+
Edit the /etc/osync/sync.conf and make some changes.
 +
 
 +
'''Note:''' if the conf file doesn't exist, manually copy it from the tarball.
  
 
  #!/usr/bin/env bash
 
  #!/usr/bin/env bash
Line 58: Line 72:
 
  TARGET_SYNC_DIR="ssh://root@DC2:22//usr/local/samba/var/locks/sysvol"
 
  TARGET_SYNC_DIR="ssh://root@DC2:22//usr/local/samba/var/locks/sysvol"
 
  SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa"
 
  SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa"
 +
REMOTE_3RD_PARTY_HOSTS=""
 
  PRESERVE_ACL=yes
 
  PRESERVE_ACL=yes
 
  PRESERVE_XATTR=yes
 
  PRESERVE_XATTR=yes
 
  SOFT_DELETE=yes
 
  SOFT_DELETE=yes
 
  DESTINATION_MAILS="your@email.com"
 
  DESTINATION_MAILS="your@email.com"
 +
#REMOTE_RUN_AFTER_CMD="/usr/local/samba/bin/samba-tool ntacl sysvolreset"
  
 
Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir.  
 
Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir.  
 
Unless you have specific reasons to do so, leave these options enabled.
 
Unless you have specific reasons to do so, leave these options enabled.
 
Osync will an send email alert if there is a problem.
 
Osync will an send email alert if there is a problem.
 +
 +
  
 
=== Setup on DC2 ===
 
=== Setup on DC2 ===
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
+
* On DC2 Install rsync by using your package manager or compile from source. Make sure you use a version that supports extended ACLs!
  
* Shutdown DC2 Samba AD DC
 
mv /var/lib/samba/private/idmap.ldb /var/lib/samba/private/idmap.ldb.backup"
 
rm /var/cache/samba/gencache.tdb"
 
  
* Copy idmap.ldb from DC1 to sync the idmap.
 
* Run the following command on '''DC1'''
 
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/
 
  
 
== 1st Try ==
 
== 1st Try ==
 
FIXME: this is something to do with Unison
 
 
What happen is we use rsync to create the directory structure with extended attributes
 
Than unison setup copies only the extened attributes on files.
 
  
 
<BR>Before starting, make a '''backup''' of your sysvol just in case.
 
<BR>Before starting, make a '''backup''' of your sysvol just in case.
Line 92: Line 99:
  
 
  /usr/local/bin/osync.sh /etc/osync/sync.conf --verbose
 
  /usr/local/bin/osync.sh /etc/osync/sync.conf --verbose
 +
 +
  
 
== Add to Crontab on DC1 ==
 
== Add to Crontab on DC1 ==
Line 100: Line 109:
 
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully to see if the replication is doing, what you expect!'''
 
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully to see if the replication is doing, what you expect!'''
  
= When you try to resync the folder =
+
 
 +
 
 +
= Problems when you try to sync the folder =
 
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
 
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.'''
# Disable Cron on DC1, like Add a "#" on the line with <tt>crontab -e</tt>
+
# Disable the Cron command on DC1 by running 'crontab -e' and adding a "#" at the start of the osync line.
# Check is any rsync or osync are currently running in <tt>ps -aux | grep sync</tt> if yes, wait for it to finished OR kill it (if it is zombie)
+
# Check if rsync or osync is currently running with 'ps -aux | grep sync', if yes, wait for it to finish OR kill it (if it is zombie)
# Remove the .osync_workdir hash files on both DC1 and DC2 on <tt>MASTER/SLAVE_SYNC_DIR "/var/lib/samba/sysvol"</tt>
+
# Remove the .osync_workdir hash files on both DC1 and DC2 at '/usr/local/samba/var/locks/sysvol'
 
# Now check your sysvol and resync
 
# Now check your sysvol and resync
 
# Confirm that everything is working again
 
# Confirm that everything is working again
 
# Re-enable the Cron on DC1 again
 
# Re-enable the Cron on DC1 again
 +
 +
  
 
= FAQ =
 
= FAQ =
Line 113: Line 126:
 
* How can I do this on windows?
 
* How can I do this on windows?
 
** Install the MSYS bash environment (read the osync documentation), then you may run osync as master from Windows.
 
** Install the MSYS bash environment (read the osync documentation), then you may run osync as master from Windows.
 +
  
 
* What to do if I've more than one DC?
 
* What to do if I've more than one DC?
** By Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server. (Not tested)
+
** By Theory, You would just make a cron job on DC1 for every other DC you have and the complete sync will be perform next sync to all other servers. (Not tested)
 
** Something like:
 
** Something like:
 
** DC1 <> DC2
 
** DC1 <> DC2
 
** DC1 <> DC3
 
** DC1 <> DC3
** DC1 <> DC2
+
** DC1 <> DC4
 
+
** DC1 <> Any other DCs you may have
  
  
 
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
 
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
 
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible. Please check on the [https://lists.samba.org/archive/samba/2014-February/178546.html reply] from Samba Developer on the mailing list .
 
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible. Please check on the [https://lists.samba.org/archive/samba/2014-February/178546.html reply] from Samba Developer on the mailing list .
 +
 +
 +
* osync logs that an error email was sent, but it wasn't received.
 +
**This is being looked into and appears to be because the smtp server on the DC isn't set up correctly.
 +
 +
 +
* Why is the 'REMOTE_RUN_AFTER_CMD' commented out in the sample conf above ?
 +
** There is a problem with sysvolreset at present, once this is fixed, the comment will be removed, along with this question.
 +
 +
 +
----
 +
[[Category:Active Directory]]

Latest revision as of 07:29, 28 August 2017

Introduction

Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.


Information on Osync replication

This HowTo describes a solution for SysVol replication based on Osync, which uses rsync to give two way sync between systems. This howto covers the bidirectional synchronization of two DC servers.

This solution has the following advantages:

  • setup is fast
  • configuration is very easy
  • Can work with windows (Windows can be used as master using the MinGW environment)

Osnyc uses rsync through a SSH tunnel.


Setup the SysVol replication

Installation

There are no deb or rpm packages available, this is because osync is a script, but you can get a copy here: [1]https://github.com/deajan/osync/releases

Once you have downloaded the tarball, you can install osync with the following commands

tar xvf osync.v1.1.3.tar.gz
cd ./osync
sh ./install.sh

This will copy osync to /usr/local/bin and create an example configuration file at /etc/osync/sync.conf


Setup on the Domain Controller with the PDC Emulator FSMO role

Some assumptions:

  • You will run all commands as root.
  • Samba is installed at /usr/local/samba
  • sysvol is located at /usr/local/samba/var/locks/sysvol on both DC1 and DC2
  • osync is located /usr/local/bin/osync
  • sync.conf is located /etc/osync/sync.conf
  • The first DC is called DC1
  • The second DC is called DC2 (And already joined to the domain)
  • sysvolsync log is located /var/log/osync_*.log
  • rsync is installed on both servers
  • rsync must support extended ACLs
  • Install rsync by using your package manager (eg: yum install rsync or apt-get install rsync) or compile from source. Make sure that you use a version that supports extended ACLs (most versions do)!
  • The partition holding sysvol must be mounted on a filesystem that supports acl and also xattr
  • You haven't setup rsync as a deamon.

Change path's if they don't fit in with your setup.


Creating SSH Public Key and ssh-copy to DC2

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2

You can try to access DC2 via ssh

ssh DC2


Osync Configuration Setup on DC1

Edit the /etc/osync/sync.conf and make some changes.

Note: if the conf file doesn't exist, manually copy it from the tarball.

#!/usr/bin/env bash
INSTANCE_ID="sysvol_sync"
INITIATOR_SYNC_DIR="/usr/local/samba/var/locks/sysvol"
TARGET_SYNC_DIR="ssh://root@DC2:22//usr/local/samba/var/locks/sysvol"
SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa"
REMOTE_3RD_PARTY_HOSTS=""
PRESERVE_ACL=yes
PRESERVE_XATTR=yes
SOFT_DELETE=yes
DESTINATION_MAILS="your@email.com"
#REMOTE_RUN_AFTER_CMD="/usr/local/samba/bin/samba-tool ntacl sysvolreset"

Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir. Unless you have specific reasons to do so, leave these options enabled. Osync will an send email alert if there is a problem.


Setup on DC2

  • On DC2 Install rsync by using your package manager or compile from source. Make sure you use a version that supports extended ACLs!


1st Try


Before starting, make a backup of your sysvol just in case.

/usr/local/bin/osync.sh /etc/osync/sync.conf --dry --verbose

If this runs successfully, you may remove the --dry option and run again.

/usr/local/bin/osync.sh /etc/osync/sync.conf --verbose


Add to Crontab on DC1

On DC1 run the following:

crontab -e 
*/5 * * * * root  /usr/local/bin/osync.sh /etc/osync/sync.conf --silent
Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully to see if the replication is doing, what you expect!


Problems when you try to sync the folder

Warning: Please follow the steps below OR you can end up with an empty sysvol folder.
  1. Disable the Cron command on DC1 by running 'crontab -e' and adding a "#" at the start of the osync line.
  2. Check if rsync or osync is currently running with 'ps -aux | grep sync', if yes, wait for it to finish OR kill it (if it is zombie)
  3. Remove the .osync_workdir hash files on both DC1 and DC2 at '/usr/local/samba/var/locks/sysvol'
  4. Now check your sysvol and resync
  5. Confirm that everything is working again
  6. Re-enable the Cron on DC1 again


FAQ

  • How can I do this on windows?
    • Install the MSYS bash environment (read the osync documentation), then you may run osync as master from Windows.


  • What to do if I've more than one DC?
    • By Theory, You would just make a cron job on DC1 for every other DC you have and the complete sync will be perform next sync to all other servers. (Not tested)
    • Something like:
    • DC1 <> DC2
    • DC1 <> DC3
    • DC1 <> DC4
    • DC1 <> Any other DCs you may have


  • Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
    • A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible. Please check on the reply from Samba Developer on the mailing list .


  • osync logs that an error email was sent, but it wasn't received.
    • This is being looked into and appears to be because the smtp server on the DC isn't set up correctly.


  • Why is the 'REMOTE_RUN_AFTER_CMD' commented out in the sample conf above ?
    • There is a problem with sysvolreset at present, once this is fixed, the comment will be removed, along with this question.