Synchronizing Files with rsync

Submitted by Christopher Ingram on Tue, 2005-11-15 03:07.

Have you ever needed to have a copy of a file in more than one location? Perhaps you need copies of the files on your desktop machine on your laptop and persistent remote access isn't an acceptable solution? Maybe you're an administrator and would like to consolidate log files from multiple hosts for your review. rsync is a good solution for both of the above scenarios as well as just about any situation where you would like to keep updated copies of files in multiple locations.

rsync has the ability to copy files across networks (or between two locations on the same host, if you like) with two very special features:

  • It will only copy files that need to be updated.
  • It will only transmit the differences between two files, reducing the amount of data that needs to be sent over the network.

What I’m about to show you is how to synchronize two directories on separate machines using SSH as a transport. rsync can run as a daemon but this requires a lot of additional setup. This method will be quick and has four big advantages over an rsync daemon:

  • Transfers are encrypted
  • SSH will handle authentication so we don’t have to replicate account information
  • We have the option of using public key encryption with SSH for unattended operation
  • Synchronization from remote locations requires only SSH access which already exists in many locations. This means less to set up, which means less possibility for new security issues.

Let’s begin building an rsync command to synchronize the directory ~/Documents on our remote machine with a directory of the same name on the local machine. The first parameter we will pass to rsync is -avv. a is archive mode, which implies several other options. Basically, a means that the copy will be recursive and preserve as much information (like owners, permissions, and time stamps) as possible. vv increases the level of verbosity twice. By default rsync operates silently but this way we see what files are being copied (one v) and what files are being passed over (the second v).

Next we will tell rsync to use ssh as its remote shell. You could use other applications like rsh if you don’t want to use SSH. We can do this with the --rsh= parameter.

Finally we will need to supply source and destination directories. Remote directories are in the form host:path where host is the remote machine to interact with and path is the path on the remote machine to use. When specifying paths there is one very important thing to note: If the path ends with a forward slash ( / ) rsync will copy all files within that directory, otherwise it will copy the directory. If your remote directory is Documents and your local directory is ~ the directory Documents will be created in your home directory. If the remote directory is Documents/ the contents of Documents will be created in ~.

Below we have a finished command line for rsync which synchronizes the directory Documents between two home directories. The first directory is the source directory, and the second is the destination:

rsync -avv --rsh=ssh washu:/home/chris/Documents ~

The remote directory Documents will be created and synchronized with the local directory ~/Documents.

rsync is one way. If you plan on preserving changes to both directories you will have to call rsync with the source and destination directories reversed:

rsync -avv --rsh=ssh ~ washu:/home/chris/Documents

If you have multiple directories you would like to synchronize (or you want to synchronize in both directions) create a small shell script and place the rsync commands inside. Easy, isn’t it?


( categories: Articles | Networking | Software | Tutorials )