Did say 'it sounds like'! ;)
Hmmmm ... no Vulcan Mind Meld possible ... perl script? If you haven't seen it, then I certainly have no knowledge of it either.
Are you asking for an example rsync command?
man rsync is your friend there ... but ... will share one I used this week. Dry run shows what it would have done - so brief look at what files/folders it was working with and also important the summary at the end ... how long it took, total of files/sizes, etc.
[root@sos backup]# cat syncbucketdry
rsync -avzh --dry-run --progress --delete ./ /root/gcloud* /mnt/gbucket/
command + options rsync -avzh --dry-run --progress --delete
man rsync will show all the options/switches, etc. ... no sense re-inventing the wheel here.
source ./ /root/gcloud*
I was issuing from /home/backup/ [the ./] that contained tar balls of code/data and sql dumps in m## directories and etc. ... ie, local server backups. Also dry run rsync'd /root/gcloud* files and directories.
[root@sos home]# du -h ./backup
destination /mnt/gbucket/ (a Google Bucket - 1.0P - that's Pentabyte)
So let's say I ran backups of a moodle35 site which would then put new tar balls and sql dump in /home/backup/m35/. The --delete would pick up new files not present before and xfer them to the bucket. So if I deleted from /home/backup/m35/ all previous tar balls and sql dump, the rsync would remove those deleted files in /mnt/gbucket/m35 and transfer only the new.
That's clear as mud, isn't it! :| Again, man rsync is your friend.
And rsync does require some study and dry runs to get it right.
One doesn't need multiple copies ... I run an rsync to dup a moodledata directory on another Moodle site once a day. It's a school district site so almost all summer long there were hardly any changes. With school about to begin another academic year, teachers have been into their courses and adding/deleting, etc. so here recently there has been more changes.
How is one assured that rsync is working ... comparisons of what was rsync'd to the location of the archive?
An example command was provided:
[root@sos home]# du -h ./backup show me totals on the server to be backed up - shown above.
Do the same command on destination location.
Now if one insist on daily/weekly/monthly copies ... the destination better be 1.0P at least ... i would think. And maybe 3 rsync scripts (maybe with different options) ... one for daily, one for weekly, one for monthly.
That help any?
'spirit of sharing', Ken