OT: system migrations or backups and rsync

[Bill Campbell]

On Wed, Feb 06, 2008, Fairlight wrote:
>Just so nobody else scratches their head over this inordinately, I would
>advise that anyone using rsync to do system migrations and/or backups who
>notices things taking an inordinate amount of time (and not finishing)
>should really look at their versions of rsync.
>
>Currently, I'm doing a migration between SuSE 9.0 Professional to SLES10,
>and the version of rsync on the 9.0 box was inadequate and froze in the
>middle of tbe "Building file list..." process.  Further checks in more
>verbose modes after ruling out other factors showed that rsync itself was
>hanging for no apparent reason.  Got and built a new version of rsync that
>matched the SLES10 version (actually it supercedes it a bit, although SuSE
>may have made equivalent patches in theirs), and it started working as it
>should.
>
>I'm dealing with 38GB of data (much of it fP and fP-derived, hence the
>relevance) in this one run, so it was a bit of an educated guess as to
>whether it was just taking an appropriately long time or getting stuck.  It
>-was- getting stuck, and I can confirm that.
>
>I've had issues with rsync versions not being compatible enough and doing
>odd things before.  This time was really annoying, however.  Figured I'd
>save others the aggravation if I could.

The only place where I've had concerns with rsync versions is on
Apple OS X where Apple has modified rsync to grok resource forks.

I've always figured that hanging during the file list building
was most likely a RAM problem as rsync sucks memory as it is
comparing the source and destination.  I use rsync extensively to
back up fairly large systems, and usually break it into
individual file systems.  On a few occassions when backing up
large servers, I've broken the job down into subdirectories of
file systems.

It's probably a Good Idea(tm) to do the first copy using the
secure scp command as there will be nothing to compare, then use
rsync for any subsequent transfers.

Bill
--
INTERNET:   bill at celestial.com  Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
FAX:            (206) 232-9186  Mercer Island, WA 98040-0820; (206) 236-1676

"If taxation without consent is robbery, the United States government
has never had, has not now, and is never likely to have, a single honest
dollar in its treasury." -- Lysander Spooner, Letter to Grover Cleveland 1886