Synchronizing Directory Trees With Rsync

My Top 3

My thanks must go to a former colleague for introducing me to rsync. Up until that point, I had only ever used FTP and scp, neither of which ever seemed an elegant method of copying entire directories from one location to another.

These days, rsync is installed by default on pretty much all of the Unix-based systems I’ve logged into. This, combined with SSH public key authentication makes sychronizing directories whether they be on the same server or remote, makes for an incredibly effective and easy way to get the information you want to where you want with the minimum of fuss.

The typical command line options to copy across a directory, and all files and sub-directories within it that I would use on a regular basis would be as follows:

The options given are:

• a - archive mode, makes it recursively copy, copies across permissions, owner, group etc as is,
• v - verbose mode
• x - do not cross filesystem boundaries - important so that it doesn’t follow symlinks, for example
• t - preserves timestamp information
• progress - shows speed of transfer and progress on a per-file basis

It should be noted that the trailing slash when specifying the source directory, or lack of one, is a subtle, but quite important consideration. If included, it signifies that all the source directories contents are to be copied over, if omitted, it signifies to create a directory with the same name as the source over onto the destination and then copy the source directories contents into that directory.

For my usage, the ability to mirror entire directory structures from server to server is a highly regarded feature that I would use rsync for alone. The other major benefit however, is that transferring all the data across only takes a time equal to the products of the available bandwidth and directory contents size for the first time. If the command used to transfer over the files the first time is run again, then only any changes that have been made to the files will be transferred across. So if only two small files were changed within a directory structure of hundreds of directories and multiple gigabytes, then only those two small files would be transferred across to the location in order to keep the source and destination locations synchronized.

The two major features described are the reasons I almost exlusively use rsync to copy any file sets other than single small textual files for which I would still use scp. For anything more complex, there is not a simpler way that I have yet come across.