Bidirectional Rsync/osync based SysVol replication workaround: Difference between revisions
(/* added content to osync sysvol page) |
m (/* first update before testing, fixed download url etc) |
||
Line 4: | Line 4: | ||
= Information on Osync replication = |
= Information on Osync replication = |
||
This HowTo describes a solution for SysVol replication |
This HowTo describes a solution for SysVol replication based on Osync, which uses rsync to give two way sync between systems. This howto covers the bidirectional synchronization of two DC servers. |
||
This solution has the following advantages: |
This solution has the following advantages: |
||
Line 17: | Line 17: | ||
== Installation == |
== Installation == |
||
There |
There are no deb or rpm packages available this is because osync is a script, but you can get a copy here: [https://github.com/deajan/osync/releases]https://github.com/deajan/osync/releases |
||
[http://www.netpower.fr/projects/osync/osync.v1.00a.tar.gz]http://www.netpower.fr/projects/osync/osync.v1.00a.tar.gz |
|||
⚫ | |||
Get the latest version at github : [https://github.com/deajan/osync/releases]https://github.com/deajan/osync/releases |
|||
⚫ | |||
⚫ | |||
⚫ | |||
cd ./osync |
cd ./osync |
||
sh ./install.sh |
sh ./install.sh |
||
Line 30: | Line 27: | ||
=== Setup on the Domain Controller with the PDC Emulator FSMO role === |
=== Setup on the Domain Controller with the PDC Emulator FSMO role === |
||
Some |
Some assumptions: |
||
* You |
* You will run all commands as root. |
||
⚫ | |||
⚫ | |||
⚫ | |||
* osync is located /usr/local/bin/osync |
* osync is located /usr/local/bin/osync |
||
* sync.conf is located /etc/osync/sync.conf |
* sync.conf is located /etc/osync/sync.conf |
||
* |
* The first DC is called DC1 |
||
* |
* The second DC is called DC2 (And already join DC1) |
||
* sysvolsync log is located /var/log/osync_*.log |
* sysvolsync log is located /var/log/osync_*.log |
||
⚫ | |||
* rsync must support extended ACLs |
* rsync must support extended ACLs |
||
* Install rsync by using your package manager (eg: yum install rsync or apt-get install rsync) or compile from source. Make sure |
* Install rsync by using your package manager (eg: yum install rsync or apt-get install rsync) or compile from source. Make sure that you use a version that supports extended ACLs (most versions do)! |
||
* sysvol must be on |
* The partition holding sysvol must be mounted on a filesystem that supports acl and also xattr |
||
* |
* You haven't setup rsync as a deamon. |
||
Change |
Change path's if they don't fit in with your setup. |
||
==== Creating SSH Public Key and ssh-copy to DC2==== |
==== Creating SSH Public Key and ssh-copy to DC2==== |
||
Line 57: | Line 54: | ||
#!/usr/bin/env bash |
#!/usr/bin/env bash |
||
INSTANCE_ID="sysvol_sync" |
|||
INITIATOR_SYNC_DIR="/usr/local/samba/var/locks/sysvol" |
|||
TARGET_SYNC_DIR="ssh://root@DC2:22//usr/local/samba/var/locks/sysvol" |
|||
SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa" |
SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa" |
||
PRESERVE_ACL=yes |
PRESERVE_ACL=yes |
||
Line 68: | Line 65: | ||
Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir. |
Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir. |
||
Unless you have specific reasons to do so, leave these options enabled. |
Unless you have specific reasons to do so, leave these options enabled. |
||
Osync |
Osync will an send email alert if there is a problem. |
||
=== Setup on DC2 === |
=== Setup on DC2 === |
||
Line 81: | Line 78: | ||
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/ |
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/ |
||
== 1st Try == |
== 1st Try == |
||
FIXME: this is something to do with Unison |
|||
What happen is we use rsync to create the directory structure with extended attributes |
What happen is we use rsync to create the directory structure with extended attributes |
||
Than unison setup copies only the extened attributes on files. |
Than unison setup copies only the extened attributes on files. |
Revision as of 18:23, 26 August 2016
Introduction
Samba AD currently doesn't provide support for SysVol replication. To achive this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on Osync.
Information on Osync replication
This HowTo describes a solution for SysVol replication based on Osync, which uses rsync to give two way sync between systems. This howto covers the bidirectional synchronization of two DC servers.
This solution has the following advantages:
- setup is fast
- configuration is very easy
- Can work with windows (Windows can be used as master using the MinGW environment)
Osnyc uses rsync through a SSH tunnel.
Setup the SysVol replication
Installation
There are no deb or rpm packages available this is because osync is a script, but you can get a copy here: [1]https://github.com/deajan/osync/releases
Once you have downloaded the tarball, you can install osync with the following commands
tar xvf osync.v1.1.1.tar.gz cd ./osync sh ./install.sh
This will copy osync to /usr/local/bin and create an example configuration file at /etc/osync/sync.conf
Setup on the Domain Controller with the PDC Emulator FSMO role
Some assumptions:
- You will run all commands as root.
- sysvol is located at /usr/local/samba/var/locks/sysvol on both DC1 and DC2
- osync is located /usr/local/bin/osync
- sync.conf is located /etc/osync/sync.conf
- The first DC is called DC1
- The second DC is called DC2 (And already join DC1)
- sysvolsync log is located /var/log/osync_*.log
- rsync is installed on both servers
- rsync must support extended ACLs
- Install rsync by using your package manager (eg: yum install rsync or apt-get install rsync) or compile from source. Make sure that you use a version that supports extended ACLs (most versions do)!
- The partition holding sysvol must be mounted on a filesystem that supports acl and also xattr
- You haven't setup rsync as a deamon.
Change path's if they don't fit in with your setup.
Creating SSH Public Key and ssh-copy to DC2
ssh-keygen -t dsa ssh-copy-id -i ~/.ssh/id_dsa.pub root@DC2
You can try to access DC2 via ssh
ssh DC2
Osync Configuration Setup on DC1
Edit the /etc/osync/sync.conf and make some changes
#!/usr/bin/env bash INSTANCE_ID="sysvol_sync" INITIATOR_SYNC_DIR="/usr/local/samba/var/locks/sysvol" TARGET_SYNC_DIR="ssh://root@DC2:22//usr/local/samba/var/locks/sysvol" SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa" PRESERVE_ACL=yes PRESERVE_XATTR=yes SOFT_DELETE=yes DESTINATION_MAILS="your@email.com"
Osync also does backups and soft deletes which will keep a copy of deleted files and folders in a subfolder named .osync_workdir. Unless you have specific reasons to do so, leave these options enabled. Osync will an send email alert if there is a problem.
Setup on DC2
- On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
- Shutdown DC2 Samba AD DC
mv /var/lib/samba/private/idmap.ldb /var/lib/samba/private/idmap.ldb.backup" rm /var/cache/samba/gencache.tdb"
- Copy idmap.ldb from DC1 to sync the idmap.
- Run the following command on DC1
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/
1st Try
FIXME: this is something to do with Unison
What happen is we use rsync to create the directory structure with extended attributes Than unison setup copies only the extened attributes on files.
Before starting, make a backup of your sysvol just in case.
/usr/local/bin/osync.sh /etc/osync/sync.conf --dry --verbose
If this runs successfully, you may remove the --dry option and run again.
/usr/local/bin/osync.sh /etc/osync/sync.conf --verbose
Add to Crontab on DC1
On DC1 run the following:
crontab -e */5 * * * * root /usr/local/bin/osync.sh /etc/osync/sync.conf --silent
- Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully to see if the replication is doing, what you expect!
When you try to resync the folder
- Warning: Please follow the steps below OR you can end up with an empty sysvol folder.
- Disable Cron on DC1, like Add a "#" on the line with crontab -e
- Check is any rsync or osync are currently running in ps -aux | grep sync if yes, wait for it to finished OR kill it (if it is zombie)
- Remove the .osync_workdir hash files on both DC1 and DC2 on MASTER/SLAVE_SYNC_DIR "/var/lib/samba/sysvol"
- Now check your sysvol and resync
- Confirm that everything is working again
- Re-enable the Cron on DC1 again
FAQ
- How can I do this on windows?
- Install the MSYS bash environment (read the osync documentation), then you may run osync as master from Windows.
- What to do if I've more than one DC?
- By Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server. (Not tested)
- Something like:
- DC1 <> DC2
- DC1 <> DC3
- DC1 <> DC2
- Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
- A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible. Please check on the reply from Samba Developer on the mailing list .