Rotating LVM snapshots for shadow copy
From SambaWiki
(Redirected from Shadow Copies with Snapshots Script)
Purpose
Use this script to utilize rotating LVM snapshots. The snapshots can then be exported via Samba and the Volume Shadow Copy interface to Windows clients (remember to install the client software) to access the content of the snapshots.
Usage
- Install the script into /root/bin/smbsnap
- Create a crontab like this:
0 12 * * 1-5 root /root/bin/smbsnap autosnap all 0 0 7 * * 2-5 root /root/bin/smbsnap autosnap all 1 0 7 * * 1 root /root/bin/smbsnap autosnap all 2 0 8 * * * root /root/bin/smbsnap clean all 3,33 * * * * root /root/bin/smbsnap autoresize all
- Create /etc/samba/smbsnap.conf like this:
SnapVolumes=('/dev/prod0/source;2000;500;1000;,nouuid')
SnapSets=(2 5 20)
OffDays="Sat Sun"
This is assuming that your LVM volume is /dev/prod0/source and it is formatted with XFS. For other filesystems you should omit the ";,nouuid" part.
- Add this to some boot script (e.g. /etc/init.d/boot.local on SuSE):
# mount smbsnap snapshots /root/bin/smbsnap mount all
- Ensure your configuration is correct in smb.conf. Example:
# This example was verified on Ubuntu 18.04 LTS on 2019-09-05 # Assuming your Samba shares are all on an LVM partition /srv # and you use the "userhomes" at /srv/samba/userhomes # and you have a share exampleshare at /srv/samba/shares/exampleshare # then smbsnap.conf will create your snapshots at /srv/ such as # /srv/@GMT-2019.09.05-13.00.00 # in which case you could add the following to your smb.conf configuration: [global] vfs objects = shadow_copy2 shadow:mountpoint = /srv shadow:snapdir = /srv [userhomes] path = /srv/samba/userhomes shadow:snapsharepath = samba/userhomes [exampleshare] path = /srv/samba/shares/exampleshare shadow:snapsharepath = samba/shares/exampleshare # note that any share where you do not configure "shadow:snapsharepath" will not use shadow copy
- Restart the smbd process after adding configuration to smb.conf
- Enjoy
Notes
- Too many LVM snapshots severly degrade storage performance, the above mentioned setup worked fine for me
- If your system gets messed up you will have to remove broken snapshots manually with lvremove
- You don't want your snapshots to overflow (they just get disabled), so set the growth parameter to a reasonable large size corresponding to the maximum data intake of your server
The Script
#!/bin/bash
#
# written by Christian Schwamborn
# bugs and suggestions to:
# christian.schwamborn[you-know-what-comes-here]nswit.de
#
# published under GNU General Public License v.2
# version 1.0.3 (2007-12-13)
#
# Authors: Christian Schwamborn [CS]
# Schlomo Schapiro [GSS] sschapiro[you-know-what-comes-here]probusiness.de
#
# History:
# 1.0.1 (2006-11-21) CS initial release
# 1.0.2 (2007-01-08) CS snapshottime now in GMT
# 1.0.3 (2007-12-13) GSS added support for extra mount options (for XFS)
#
# You are using this scrip at your own risk, the author is not responsible
# for data damage or losses in any way.
#
# What this is:
# You can use this script to create and manage snapshots for the Samba
# VFS module shadow_copy.
#
# How to use:
# The script provides some commanline parameters which are usefull for
# start/stop scrips (i.e. mount and unmount). Other parameters are usefull
# for cronjobs - add this, for a usual snapshot scenario (without trailing #)
# to your crontab:
#
# 0 12 * * 1-5 root /usr/local/sbin/smbsnap autosnap 0 0
# 0 7 * * 2-5 root /usr/local/sbin/smbsnap autosnap 0 1
# 0 7 * * 1 root /usr/local/sbin/smbsnap autosnap 0 2
# 0 8 * * * root /usr/local/sbin/smbsnap clean all
# 3,33 * * * * root /usr/local/sbin/smbsnap autoresize all
#
# This takes snapshots at 7:00 and 12:00 every workday and checks every hour
# if a snapshot needs to be resized.
#
# The script has some flaws:
# -This script currently works only with LVM2, no EVMS support yet
# -XFS should be easy to implement, but it isn't yet
# -You must not use dashes in your volumegroups or logical volumes
# -Be carefull with the configuration, the parameters are not completely
# checked right now, as the same for the command line parameters
# -You have to keep track of the freespace of your volumegroups
# -Be aware, that if your snapshots grow faster than you assumed, they will
# become unusable. With the configuration shown above, this script checks
# every 30 minutes if the snapshots are in the need of a resize. If
# someone has a better idea how to check the snapshots than periodical,
# let me know plaese.
#
# This script is written for the bash, other shells might work, it also uses
# some external commands: mount, umount, grep, date, bc, logger, lvcreate,
# lvremove, lvresize
#
# There are currently three variables that have to be configured:
# -SnapVolumes is an array, every element of that array represents a logical
# volume that is configured for snapshots. Each element is a comma seperated
# list, which consists of the logical volume itself (i.e. /dev/GROUP1/foo),
# the start size of the snapshot (in megabytes), the freespace which should
# be maintained (in megabytes), the space added, when a snapshot is
# resized (also in megabytes) and optionally additional mount options required
# for mounting the snapshot, like ",nouuid" for XFS. Please add the leading ","
# because this parameter will be appended to "mount -o ro" *verbatim*.
# The number of an element is used as a reference when calling the script
# -SnapSets is also an array, currently every element just represents the
# age (in days) of a snapshot of the specific snapshot-set.
# -OffDays is a simple string with the none work days.
#
# The script will figure out by itself where to mount the snapshots, but the
# original logical volumes has to be mounted fist.
#
# Copy and adjust the following three variables (without #) to a blank file in
# /etc/samba and name it smbsnap.conf. If you place the configuration file
# elsewhere, make sure to adjust the path below.
#
# SnapVolumes=('/dev/GROUP/foo;2000;500;1000;,nouuid' '/dev/GROUP/bar;3000;1000;2000')
# SnapSets=(2 5 20)
# OffDays="Sat Sun"
#
# NOTE TO USERS OF PREVIOUS VERSION !!
#
# The delimiter changed from , to ; to support adding multiple mount options
#
# please convert your smbsnap.conf with sed -e 's/,/;/g' -i /etc/samba/smbsnap.conf
#
# Sorry for the invonvenience ...
#
###############################################################################
. /etc/samba/smbsnap.conf
export LANG=en_US.UTF-8
export LANGUAGE=en_US:en
SnapDate=$(date -u +%Y.%m.%d-%H.%M).00
[ -z "${1}" ] || Command=${1}
[ -z "${2}" ] || LVolume=${2}
[ -z "${3}" ] || SnapSet=${3}
ExtraMountOptions=
# process a single snapshot
# arguments: Command
# needs: SnapShot, VolumePath, SnapSets, OffDays, FreeSize, ReSize
# provides: na.
# local: cmd, SnapShotPath, CurrSnapSets, Count, Expire, Parameters, SnapState, CurrSize, FillPercet, CurrFreeSize
function DoSnap()
{
cmd=${1}
SnapShotPath=${VolumePath}/@GMT-$(echo ${SnapShot##*/} | cut -f3-4 -d\-)
case ${cmd} in
# to mount snapshots
mount)
[ -d ${SnapShotPath} ] || mkdir ${SnapShotPath} || \
logger "${0}: ***error*** - unable to create mountpoint for ${SnapShot}"
if mount | grep -q ${SnapShotPath}; then
logger "${0}: ***error*** - snapshot ${SnapShot} is allready mounted to ${SnapShotPath}"
else
mount ${SnapShot} ${SnapShotPath} -o ro$ExtraMountOptions >/dev/null 2>&1 || \
logger "${0}: ***error*** - can not mount ${SnapShot} to ${SnapShotPath}"
fi
;;
# to unmount a snapshots
umount)
if mount | grep -q ${SnapShotPath}; then
umount -f -l ${SnapShotPath} >/dev/null 2>&1 || \
logger "${0}: ***error*** - can not unmount ${SnapShot} mounted to ${SnapShotPath}"
else
logger "${0}: ***error*** - snapshot ${SnapShot} is not mounted to ${SnapShotPath}"
fi
;;
# to remove expired snapshots
clean)
CurrSnapSet=$(echo ${SnapShot##*/} | cut -f2 -d\-)
if [ ${CurrSnapSet} -ge 0 ] && [ ${CurrSnapSet} -lt ${#SnapSets[@]} ]; then
Expire=$(echo ${SnapSets[${CurrSnapSet}]} | cut -f1 -d,)
# add off-days, if any, to the expire time; we just count work-days
declare -i Count=1
while [ ${Expire} -ge ${Count} ];do
echo ${OffDays} | grep -q $(date -d "-${Count} day" +%a) && Expire=$((${Expire} + 1))
Count=$((${Count} + 1))
done
# compare date now minus expire-time with the snapshot-date
if [ $(( $(date +%s) - ${Expire}*24*60*60 - 12*60*60)) -gt \
$(date -d "$(echo ${SnapShot##*/} | cut -f3 -d\- | tr \. \-) \
$(echo ${SnapShot##*/} | cut -f4 -d\- | tr \. \:)" +%s) ]; then
# unmount snapshot
if mount | grep -q ${SnapShotPath}; then
umount -f ${SnapShotPath} >/dev/null 2>&1 || \
logger "${0}: ***error*** - can not unmount ${SnapShot} mounted to ${SnapShotPath}"
fi
if ! mount | grep -q ${SnapShotPath}; then
# remove mount-directory
if [ -d ${SnapShotPath} ]; then
rmdir ${SnapShotPath} || \
logger "${0}: ***error*** - unable to remove mountpoint for ${SnapShot}"
fi
# finally remove snapshot
if lvremove -f ${SnapShot} >/dev/null 2>&1;then
logger "${0}: successfully removed outdated snapshot ${SnapShot}"
else
logger "${0}: ***error*** - can not remove logical volume ${SnapShot}"
fi
fi
fi
else
logger "${0}: ***error*** - snapshot-set #${CurrSnapSet} of snaphot ${SnapShot} is not configured"
fi
;;
# to check periodical if the snapshots have to be resized
autoresize)
Parameters="--options lv_size,snap_percent --noheadings --nosuffix --separator , --unbuffered --units m"
SnapState=$(lvs ${Parameters} ${SnapShot})
CurrSize=$(echo ${SnapState} | cut -f1 -d,)
FillPercet=$(echo ${SnapState} | cut -f2 -d,)
CurrFreeSize=$(echo "${CurrSize}-${CurrSize}/100*${FillPercet}" | bc)
if ! [ $(echo "${CurrFreeSize} > ${FreeSize}" | bc) -eq 1 ]; then
if lvresize -L +${ReSize}M ${SnapShot} >/dev/null 2>&1; then
logger "${0}: successfully resized snapshot ${SnapShot}"
else
logger "${0}: ***error*** - an error occurred while resizing ${SnapShot}"
fi
fi
;;
esac
}
# invoked if all snapshots of a volume are processed
# arguments: Command
# needs: VolumeDevice, VolumePath, SnapSet & functions: DoSnap
# provides: SnapShot
# local: snapset_tmp, cmd
function DoAllSnaps()
{
cmd=${1}
[ -z "${SnapSet}" ] || snapset_tmp="${SnapSet}-"
# checkout if the configured volume exists and is mounted
if [ -b ${VolumeDevice} ]; then
if mount | grep -q "${VolumePath} "; then
# process all snapshots of the volume and, if given, of a specific snapshot-set
for SnapShot in ${VolumeDevice}-${snapset_tmp}*; do
if [ ${SnapShot} = "${VolumeDevice}-${snapset_tmp}*" ]; then
logger "${0}: ***error*** - no backupset #${SnapSet} found for ${VolumeDevice}"
else
DoSnap ${cmd}
fi
done
else
logger "${0}: ***error*** - logical volume ${VolumeDevice} not mounted to ${VolumePath}"
fi
else
logger "${0}: ***error*** - logical volume ${VolumeDevice} does not exist"
fi
}
# creates a new snapshot and mounts it
# arguments: na.
# needs: VolumeDevice, VolumePath, SnapSet, SnapSize, SnapDate & functions: DoAllSnaps, DoSnap
# provides: SnapShot
# local: na.
function MakeSnap ()
{
case ${SnapSet} in
[0-9])
if [ "${Command}" = "autosnap" ]; then DoAllSnaps "clean"; fi
SnapShot=${VolumeDevice}-${SnapSet}-${SnapDate}
if lvcreate -L${SnapSize}M -s -n ${SnapShot##*/} ${VolumeDevice} >/dev/null 2>&1; then
logger "${0}: successfully created new snapshot ${SnapShot}"
else
logger "${0}: ***error*** - an error occurred while creating snapshot ${SnapShot}"
fi
DoSnap "mount"
;;
*)
echo "usage: ${0} snap|autosnap <LV number | all> <Snap-Set Number>"
;;
esac
}
# sets some variables and splits the way for certain commands
# arguments: one object of the array SnapVolumes
# needs: Command, & functions: DoAllSnaps MakeSnap
# provides: SnapVolume, VolumeDevice, PVGroupName, LVolumeName, VolumePath, SnapSize, FreeSize, ReSize
# local: na.
function SecondChoice ()
{
SnapVolume=${1}
VolumeDevice=$(echo ${SnapVolume} | cut -f1 -d\;)
PVGroupName=$(echo ${VolumeDevice} | cut -f3 -d/)
LVolumeName=$(echo ${VolumeDevice} | cut -f4 -d/)
VolumePath=$(mount | grep ^/dev[[:alnum:]/]*${PVGroupName}.${LVolumeName}[\ ] | cut -f3 -d' ')
SnapSize=$(echo ${SnapVolume} | cut -f2 -d\;)
FreeSize=$(echo ${SnapVolume} | cut -f3 -d\;)
ReSize=$(echo ${SnapVolume} | cut -f4 -d\;)
ExtraMountOptions=$(echo ${SnapVolume} | cut -f5 -d\;)
case ${Command} in
mount|umount|clean|autoresize)
DoAllSnaps ${Command}
;;
snap|autosnap)
MakeSnap
;;
esac
}
# decides if all configured volumes are processed or just a specific one
# arguments: na.
# needs: Command, LVolume, SnapVolumes & functions: SecondChoice
# provides: na.
# local: snp
case ${Command} in
mount|umount|snap|clean|autosnap|autoresize)
case ${LVolume} in
all)
for snp in ${SnapVolumes[@]}; do
SecondChoice ${snp}
done
;;
[0-9])
if [ ${LVolume} -ge 0 ] && [ ${LVolume} -lt ${#SnapVolumes[@]} ]; then
SecondChoice ${SnapVolumes[LVolume]}
else
logger "${0}: ***error*** - there is no configured logical volume #${LVolume} for snapshots"
fi
;;
*)
echo "usage: ${0} <command> <LV number | all> [<Snap-Set Number>]"
;;
esac
;;
*)
echo "usage: ${0} <command> <LV number | all> [<Snap-Set Number>]"
echo
echo " valid commands are:"
echo " mount - to mount snapshots"
echo " umount - to unmount snapshots"
echo " snap - to make a new snapshot"
echo " clean - to cleanup outdated snapshots"
echo " autosnap - normally used for cronjobs to cleanup"
echo " outdates snapshots an create a new one"
echo " autoresize - for a periodical check if snapshots"
echo " needs to be resized"
echo
echo " <LV number> is the number of a logical volume, configured for"
echo " snapshots in SnapVolumes, or simply 'all' for all volumes"
echo " <Snap-Set Number> is the number of the snapshot-set, configured"
echo " in SnapSet. It is optional, except for the commands 'snap' and"
echo " 'autosnap'"
echo
;;
esac