Revisión actual del 17:47 22 jul 2010

short selection of rsync parameters

-a, --archive

-a, --archive               archive mode

"-a" is the same as -rlptgoD (no -H)

 -r, --recursive             recurse into directories
 -l, --links                 copy symlinks as symlinks
 -p, --perms                 preserve permissions
 -t, --times                 preserve times
 -g, --group                 preserve group
 -o, --owner                 preserve owner (super-user only)
 -D                          same as --devices --specials

commons

-v, --verbose               increase verbosity
-u, --update                skip files that are newer on the receiver
-z, --compress              compress file data during the transfer
-e, --rsh=COMMAND           specify the remote shell to use
-h, --human-readable        output numbers in a human-readable format Only for Etch version
    --progress              show progress during transfer
-f, --filter=RULE           add a file-filtering RULE

for incrementals

-b, --backup                destination files are renamed
  --backup-dir=DIR          for incremental backups

limits

--max-size=SIZE         don't transfer any file larger than SIZE
--bwlimit=KBPS          limit I/O bandwidth; KBytes per second

Entire host excluding some directories (ssh without using root) RECOMENDED

At host to backup (as root)

mkdir /var/backups/.ssh
chmod 750 /var/backups/.ssh
chown backup:root /var/backups/.ssh
echo "backup ALL = NOPASSWD: /usr/bin/rsync" >> /etc/sudoers

At backup server (as root)

ssh-keygen -f id_dsa
scp id_dsa.pub server_to_backup:/var/backups/.ssh/authorized_keys

script (ssh)

HOST=debian

dir=full_backup
date=`date +%F_%T | tr \: _`
incremental=incremental/incremental_$date

mkdir -p $dir
mkdir -p $incremental

rsync -auzv \
  -e "ssh -l backup -i id_rsa" \
  --rsync-path="sudo /usr/bin/rsync"  \
  --delete \
  --filter='- /sys/' \
  --filter='- /tmp/' \
  --filter='- /dev/' \
  --filter='- /mnt/' \
  --filter='- /media/' \
  --filter='- /proc/' \
  --filter='- /var/log/mysql/' \
  --filter='- /var/lib/mysql/' \
  --filter='- /var/cache/apt/archives/' \
  --filter='- /var/tmp/' \
  --filter='- /nobackup/' \
  --filter='- /opt/lampp/var/mysql/' \
  --backup --backup-dir=../$incremental \
  $HOST:/ $dir

Backing up host configuration and selected files (ssh with using root user)

Commands to run

At host to backup (as root)

If root ssh access is enable (/etc/ssh/sshd_config with "PermitRootLogin yes") you can create the user with UID=0, and no problems with permisions:

useradd -g 0 -o --uid 0 rbackup

If not, for other servers with no ssh root access enabled, add user to system groups or use rsyncd

useradd -g 0 -G daemon,dip,lpadmin,shadow,sasl rbackup

passwd rbackup
su - rbackup
mkdir -p /home/rbackup/.ssh
chmod 700 /home/rbackup/.ssh
chown -R rbackup.root /home/rbackup

At backup server (as root)

ssh-keygen -t dsa -f diskarray.gacq.com-id_dsa
scp diskarray.gacq.com-id_dsa.pub rbackup@diskarray.gacq.com:.ssh/authorized_keys

Test the key:

ssh -i diskarray.gacq.com-id_dsa.pub rbackup@diskarray.gacq.com

script (ssh)

#!/bin/sh
#
# Configuration:
HOST=mynotebook.gacq.com
USER=gacq
MYSQLPASS=MYSQL_ROOT_PASS

#
dir=full_backup
date=`date +%F_%T | tr \: _`
incremental=incremental/incremental_$date
dumps=dumps/dump_$date

mkdir -p $dir
mkdir -p $incremental
mkdir -p $dumps

rsync -auzv \
  -e "ssh -l $USER -i $HOST-id_dsa" \
  --delete \
  --filter='+ /etc/' \
  --filter='+ /etc/**' \
  --filter='+ /root/' \
  --filter='+ /root/**' \
  --filter='+ /usr/' \
  --filter='+ /usr/local/' \
  --filter='+ /usr/local/scripts/' \
  --filter='+ /usr/local/scripts/**' \
  --filter='- /**' \
  --backup --backup-dir=../$incremental \
  $HOST:/ $dir

ssh -l $USER -i $HOST-id_dsa $HOST "/usr/bin/dpkg --get-selections" > $dumps/dpkg--get-selections.txt

ssh -l $USER -i $HOST-id_dsa $HOST "mysqldump --all-databases --password=$MYSQLPASS | gzip" > $dumps/all-databases.sql.gz

Backup remote server (rsyncd - no root ssh)

(Without root ssh access available - /etc/ssh/sshd_config with "PermitRootLogin no")

/etc/rsyncd.conf

[all]
        comment = from root
        path = /
        read only = yes
        list = yes
        uid = root
        gid = root
        strict modes = yes
        hosts allow = gacq.com
        ignore errors = no
        ignore nonreadable = yes
        transfer logging = no
        timeout = 600
        refuse options = checksum dry-run
        dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

script (rsyncd)

#!/bin/sh
#
# Configuration:
HOST=mycustomer.gacq.com
USER=gacq
MYSQLPASS=MYSQL_ROOT_PASS

#
dir=full_backup
date=`date +%F_%T | tr \: _`
incremental=incremental/incremental_$date
dumps=dumps/dump_$date

mkdir -p $dir
mkdir -p $incremental
mkdir -p $dumps

rsync -azv \
  --delete \
  --max-size=50M \
  --bwlimit=200 \
  --progress \
  --filter='+ /etc/' \
  --filter='+ /etc/**' \
  --filter='+ /root/' \
  --filter='+ /root/**' \
  --filter='+ /home/' \
  --filter='+ /home/**' \
  --filter='+ /usr/' \
  --filter='+ /usr/local/' \
  --filter='+ /usr/local/scripts/' \
  --filter='+ /usr/local/scripts/**' \
  --filter='- /**' \
  --backup --backup-dir=../$incremental \
  $HOST::all $dir

ssh -l $USER -i $HOST-id_dsa $HOST "/usr/bin/dpkg --get-selections" > $dumps/dpkg--get-selections.txt

ssh -l $USER -i $HOST-id_dsa $HOST "mysqldump --all-databases --password=$MYSQLPASS | gzip" > $dumps/all-databases.sql.gz

NO root ssh access (and) NO rsyncd installed on remote host

Use non root user and put it in the necesaries systems groups, for example daemon,dip,lpadmin,shadow,sasl

Diferences between ryncd and ssh

In the above scripts:

add/remove this

-e "ssh -l $USER -i $HOST-id_dsa"

and change the source

for rsyncd use:

$HOST::all $dir

for ssh use:

$HOST:/ $dir

duplicating systems with sync

Not tested

rsync -v -r -p -o -g -D -t -S -l -H \
--exclude /mnt/   \
--exclude /proc/ \
--exclude /tmp/ \
--exclude /home/ / /mnt/fireball/

Boot from a CD-ROM and mount the target-disk. Fix fstab and lilo.conf. Create /mnt and /proc and set the right permissions (for this directories). Then run lilo and reboot. Next time you run rsync, add these option to the command-line:

--exclude /etc/fstab \
--exclude /etc/lilo.conf \
--delete 

Working files syncronization / desktop <--> notebook

• IMPORTANT: If target not exist will be deleted, use with caution.
• Remember to syncronize always after a change.
• Use these scripts from your home directory

gacq.com-get

# Run this at the notebook
# Download new files from server, also remove files not present on server
#
HOST=gacq.com
USER=anysyncuser
SSH_PORT=22

rsync -avuze 'ssh -p $SSH_PORT' --delete $USER@$HOST:/srv/gacq/rsync/ rsync

gacq.com-put

# Run this at the notebook
# Upload new files to server, also remove files deleted here
#
HOST=gacq.com
USER=anysyncuser
SSH_PORT=22

rsync -avuze 'ssh -p $SSH_PORT' --delete rsync $USER@$HOST:/srv/gacq/