Bidirectional Rsync/Unison based SysVol replication workaround: Difference between revisions
(update Soft delete issue update) |
|||
(14 intermediate revisions by 7 users not shown) | |||
Line 1: | Line 1: | ||
= Introduction = |
= Introduction = |
||
Samba AD currently doesn't provide support for SysVol replication. To |
Samba AD currently doesn't provide support for SysVol replication. To achieve this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison. |
||
= Information on |
= Information on unison + rsync replication = |
||
This HowTo describes a solution for SysVol replication, that is based on |
This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. This howto only covers a two DC setup. |
||
It |
It has the following advantages: |
||
* setup |
* Quick setup |
||
* |
* Configuration is very easy |
||
* Can work with windows ( |
* Can work with windows (Please add in) |
||
We will use rsync through a SSH tunnel. |
|||
= Setup the SysVol replication = |
= Setup the SysVol replication = |
||
Some assumptions: |
|||
== Installation == |
|||
You are running all commands as root. |
|||
rsync location /usr/bin/rsync |
|||
sysvol is located at /var/lib/samba/sysvol on both DC1 and DC2 |
|||
unison location /usr/bin/unison |
|||
The first DC is DC1 |
|||
The second DC is DC2 |
|||
sysvolsync log location /var/log/sysvol-sync.log |
|||
Change the paths if your setup is different. |
|||
I don't see any package but osync available on |
|||
[https://github.com/deajan/osync]https://github.com/deajan/osync |
|||
== Getting Osync == |
|||
At the time of writing this Osync is still at v.100pre. |
|||
However, you should always get a stable branch when it is available. |
|||
We are getting it done using master branch. |
|||
wget https://raw.githubusercontent.com/deajan/osync/master/osync.sh |
|||
chmod +x osync.sh |
|||
Move osync to /usr/bin/ or your preferred dir |
|||
mv osync.sh /usr/bin/ |
|||
== Getting Osync Configuration == |
|||
wget https://raw.githubusercontent.com/deajan/osync/master/sync.conf |
|||
Move it to /etc/osync/ |
|||
mkdir /etc/osync/ |
|||
mv sync.conf /etc/osync/sync.conf |
|||
=== Setup on the Domain Controller with the PDC Emulator FSMO role === |
=== Setup on the Domain Controller with the PDC Emulator FSMO role === |
||
Some assumption: |
|||
* You are running all command as root. |
|||
* rsync is installed on both server |
|||
* sysvol is located /var/lib/samba/sysvol on both DC1 and DC2 |
|||
* osync is located /usr/bin/osync |
|||
* osync.conf is located /etc/osync/sync.conf |
|||
* DC1 is at DC1 |
|||
* DC2 is at DC2 (And already join DC1) |
|||
* sysvolsync log is located /var/log/osync_*.log |
|||
* rsync must support extended ACLs |
|||
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs! |
* Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs! |
||
* You don't need to setup the rsync server. |
|||
* sysvol must be on disk which is mounted to support acl and also xattr |
|||
* Install unison by using your package manager or compile from source. (On Gentoo you need to do <code>eselect unison</code> to create the link) |
|||
* We don't need to setup rsync server. |
|||
Change the path if that don't fit your setup. |
|||
==== Creating SSH Public Key and ssh-copy to DC2==== |
==== Creating SSH Public Key and ssh-copy to DC2==== |
||
ssh-keygen -t |
ssh-keygen -t rsa |
||
ssh-copy-id -i ~/.ssh/ |
ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2 |
||
You can try to access DC2 via ssh |
You can try to access DC2 via ssh |
||
ssh DC2 |
ssh DC2 |
||
=== |
==== Setup ssh Control ==== |
||
If the remote system enforces rate limits on incoming ssh connections, unison will fail if you try to run it this way. |
|||
Edit the /etc/osync/sync.conf and make some changes |
|||
So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection. |
|||
mkdir ~/.ssh/ctl |
|||
#!/usr/bin/env bash |
|||
cat << EOF > ~/.ssh/ctl/config |
|||
SYNC_ID="sysvol_sync" |
|||
Host * |
|||
MASTER_SYNC_DIR="/var/lib/samba/sysvol" |
|||
ControlMaster auto |
|||
SLAVE_SYNC_DIR="ssh://root@DC2:22//var/lib/samba/sysvol" |
|||
ControlPath ~/.ssh/ctl/%h_%p_%r |
|||
SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa" |
|||
ControlPersist 1 |
|||
PRESERVE_ACL=yes |
|||
EOF |
|||
PRESERVE_XATTR=yes |
|||
SOFT_DELETE=yes |
|||
DESTINATION_MAILS="your@email.com" |
|||
==== Setup Sysvolsync Log files ==== |
|||
Osync have also backup and soft delete which will kept a copy of deleted files and folder. |
|||
Do the following on DC1 so that you can check what happens during the sync. |
|||
You can choose to enable them. |
|||
Please include this file into logrotate as the log size is not controlled. |
|||
:'''Branch Fix#22 fix the soft delete issue you can get them from the url below, however, it should be check in to stable branch soon.''' |
|||
touch /var/log/sysvol-sync.log |
|||
wget https://raw.githubusercontent.com/deajan/osync/fix%2322/osync.sh |
|||
chmod 640 /var/log/sysvol-sync.log |
|||
==== Setup Unison defaults running parameters ==== |
|||
Osync have one benefit as it will only send email alert if there is problem. |
|||
Please run the following on DC1 |
|||
install -o root -g root -m 0750 -d /root/.unison |
|||
=== Setup on DC2 === |
|||
cat << EOF > /root/.unison/default.prf |
|||
# Unison preferences file |
|||
# Roots of the synchronization |
|||
# |
|||
# copymax & maxthreads params were set to 1 for easier troubleshooting. |
|||
# Have to experiment to see if they can be increased again. |
|||
root = /var/lib/samba |
|||
# Note that 2 x / behind DC2, it is required |
|||
root = ssh://root@DC2//var/lib/samba |
|||
# |
|||
# Paths to synchronize |
|||
path = sysvol |
|||
# |
|||
#ignore = Path stats ## ignores /var/www/stats |
|||
auto=true |
|||
batch=true |
|||
perms=0 |
|||
rsync=true |
|||
maxthreads=1 |
|||
retry=3 |
|||
confirmbigdeletes=false |
|||
servercmd=/usr/bin/unison |
|||
copythreshold=0 |
|||
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress |
|||
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress |
|||
#copyquoterem = true |
|||
copymax = 1 |
|||
logfile = /var/log/sysvol-sync.log |
|||
EOF |
|||
=== Setup SysVol on DC2 === |
|||
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs! |
* On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs! |
||
* On DC2 Install unison by using your package manager or compile from source. (On Gentoo you need to do <code>eselect unison</code> to create the link) |
|||
* Make sure, that you have [[Joining_a_Samba_DC_to_an_Existing_Active_Directory#Built-in_User_.26_Group_ID_Mappings|identical IDs of built-in groups on all DCs]]. |
|||
* Shutdown DC2 Samba AD DC |
|||
mv /var/lib/samba/private/idmap.ldb /var/lib/samba/private/idmap.ldb.backup" |
|||
rm /var/cache/samba/gencache.tdb" |
|||
== 1st Trial == |
|||
* Copy idmap.ldb from DC1 to sync the idmap. |
|||
You now use rsync to create the directory structure with extended attributes |
|||
* Run the following command on '''DC1''' |
|||
Then the unison setup will only copy the extended attributes on files. |
|||
scp /var/lib/samba/private/idmap.ldb root@DC2:/var/lib/samba/private/ |
|||
<BR>Please make a '''backup''' of your sysvol, just in case, this is because there is no <code>dry-run</code> |
|||
== 1st Try == |
|||
What happen is we use rsync to create the directory structure with extended attributes |
|||
/usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *" /var/lib/samba/sysvol root@DC2:/var/lib/samba && /usr/bin/unison |
|||
Than unison setup copies only the extened attributes on files. |
|||
:'''Note: The path on DC2 is just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!''' |
|||
<BR>Please make a '''backup''' on you sysvol just in case. |
|||
/usr/bin/osync.sh /etc/osync/sync.conf --dry --verbose |
|||
If this run successfully let remove the --dry and execute it. |
|||
/usr/bin/osync.sh /etc/osync/sync.conf --verbose |
|||
== Add to Crontab on DC1 == |
== Add to Crontab on DC1 == |
||
On DC1 run the following: |
On DC1 run the following: |
||
crontab -e |
crontab -e |
||
*/5 * * * * |
*/5 * * * * /usr/bin/unison -silent |
||
:'''Warning: Make sure that the destination folder is really your SysVol folder, because the command will replicate to the given directory and sync everything in it that isn't also on the source! You could damage your system! So check the output carefully if the replication is doing, what you expect!''' |
|||
= When you try to resync the folder = |
= When you try to resync the folder = |
||
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.''' |
:'''Warning: Please follow the steps below OR you can end up with an empty sysvol folder.''' |
||
# Disable Cron on DC1, like Add a "#" on the line with < |
# Disable Cron on DC1, like Add a "#" on the line with <code>crontab -e</code> |
||
# Check |
# Check if rsync or unison are currently running in <code>ps -aux</code> if yes, wait for it to finish OR kill it (if it is zombie) |
||
# Remove the |
# Remove the hash files on both DC1 and DC2 on <code>/root/.unison</code> |
||
# Now check your sysvol and resync |
# Now check your sysvol and resync |
||
# Confirm that everything is ok again |
# Confirm that everything is ok again |
||
# Re-enable the |
# Re-enable the cron on DC1 again |
||
= FAQ = |
= FAQ = |
||
Line 125: | Line 127: | ||
* What to do if I've more than |
* What to do if I've more than two DC's? |
||
** |
** In Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server. Follow below steps: |
||
:# Copy '''default.prf''' to another file called, eg: '''root1.prf''' |
|||
:# Open '''root1.prf''' and named '''root = ...''' line to '''root1 = ...''' and save the file |
|||
:# Repeat step 1 and 2 for all DCs |
|||
:# Change cronjob to: |
|||
* * * * * /usr/bin/unison root1 -silent |
|||
* * * * * /usr/bin/unison root2 -silent |
|||
and so on. |
|||
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol? |
* Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol? |
||
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible |
** A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible. |
||
---- |
|||
[[Category:Active Directory]] |
Latest revision as of 08:13, 1 January 2024
Introduction
Samba AD currently doesn't provide support for SysVol replication. To achieve this important feature in a Multi-DC environment, until it's implemented, workarounds are necessary to keep it in sync. This HowTo provides a basic workaround solution based on rsync and unison.
Information on unison + rsync replication
This HowTo describes a solution for SysVol replication, that is based on rsync and unison. As Compare to the rsync method, it is bidirectional. This howto only covers a two DC setup.
It has the following advantages:
- Quick setup
- Configuration is very easy
- Can work with windows (Please add in)
We will use rsync through a SSH tunnel.
Setup the SysVol replication
Some assumptions:
You are running all commands as root. rsync location /usr/bin/rsync sysvol is located at /var/lib/samba/sysvol on both DC1 and DC2 unison location /usr/bin/unison The first DC is DC1 The second DC is DC2 sysvolsync log location /var/log/sysvol-sync.log
Change the paths if your setup is different.
Setup on the Domain Controller with the PDC Emulator FSMO role
- Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
- You don't need to setup the rsync server.
- Install unison by using your package manager or compile from source. (On Gentoo you need to do
eselect unison
to create the link)
Creating SSH Public Key and ssh-copy to DC2
ssh-keygen -t rsa ssh-copy-id -i ~/.ssh/id_rsa.pub root@DC2
You can try to access DC2 via ssh
ssh DC2
Setup ssh Control
If the remote system enforces rate limits on incoming ssh connections, unison will fail if you try to run it this way. So we create the first ssh connection as a controlpath file in the location specified, all subsequent connections will reuse on the first connection.
mkdir ~/.ssh/ctl cat << EOF > ~/.ssh/ctl/config Host * ControlMaster auto ControlPath ~/.ssh/ctl/%h_%p_%r ControlPersist 1 EOF
Setup Sysvolsync Log files
Do the following on DC1 so that you can check what happens during the sync. Please include this file into logrotate as the log size is not controlled.
touch /var/log/sysvol-sync.log chmod 640 /var/log/sysvol-sync.log
Setup Unison defaults running parameters
Please run the following on DC1
install -o root -g root -m 0750 -d /root/.unison cat << EOF > /root/.unison/default.prf # Unison preferences file # Roots of the synchronization # # copymax & maxthreads params were set to 1 for easier troubleshooting. # Have to experiment to see if they can be increased again. root = /var/lib/samba # Note that 2 x / behind DC2, it is required root = ssh://root@DC2//var/lib/samba # # Paths to synchronize path = sysvol # #ignore = Path stats ## ignores /var/www/stats auto=true batch=true perms=0 rsync=true maxthreads=1 retry=3 confirmbigdeletes=false servercmd=/usr/bin/unison copythreshold=0 copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress #copyquoterem = true copymax = 1 logfile = /var/log/sysvol-sync.log EOF
Setup SysVol on DC2
- On DC2 Install rsync by using your package manager or compile from source. Make sure, that you use a version that supports extended ACLs!
- On DC2 Install unison by using your package manager or compile from source. (On Gentoo you need to do
eselect unison
to create the link) - Make sure, that you have identical IDs of built-in groups on all DCs.
1st Trial
You now use rsync to create the directory structure with extended attributes Then the unison setup will only copy the extended attributes on files.
Please make a backup of your sysvol, just in case, this is because there is no dry-run
/usr/bin/rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *" /var/lib/samba/sysvol root@DC2:/var/lib/samba && /usr/bin/unison
- Note: The path on DC2 is just /var/lib/samba which is different from DC1, it is by design, there is nothing wrong!
Add to Crontab on DC1
On DC1 run the following:
crontab -e */5 * * * * /usr/bin/unison -silent
When you try to resync the folder
- Warning: Please follow the steps below OR you can end up with an empty sysvol folder.
- Disable Cron on DC1, like Add a "#" on the line with
crontab -e
- Check if rsync or unison are currently running in
ps -aux
if yes, wait for it to finish OR kill it (if it is zombie) - Remove the hash files on both DC1 and DC2 on
/root/.unison
- Now check your sysvol and resync
- Confirm that everything is ok again
- Re-enable the cron on DC1 again
FAQ
- How can I do this on windows?
- I don't have an answer, please post on the mailing list
- What to do if I've more than two DC's?
- In Theory, We would just make more cron jobs on DC1 and the complete sync will be perform next sync to all server. Follow below steps:
- Copy default.prf to another file called, eg: root1.prf
- Open root1.prf and named root = ... line to root1 = ... and save the file
- Repeat step 1 and 2 for all DCs
- Change cronjob to:
* * * * * /usr/bin/unison root1 -silent * * * * * /usr/bin/unison root2 -silent
and so on.
- Why can't I simply use a distributed filesystem like GlusterFS, Lustre, etc. for SysVol?
- A cluster file system with Samba requires CTDB to be able to do it safely. And CTDB and AD DC are incompatible.