I occasionally use RSYNC to synchronize large directories of files between servers. This is especially useful if you’re moving a client from one server to another and they have alot of static files that are always changing. You can copy the files and sync them up, all with RSYNC and if your connection gets cut off, it will start where it left off. It will also grab changes to files that have already been RSYNCd.

I ran into an issue with RSYNC recently, wherein the RSYNC process was running in the background; but was terminating due to errors similar to the following. These connections were probably related to the slow and unstable connection to the remote server.

rsync: writefd_unbuffered failed to write 998 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (888092 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]

Given that I was transferring files through a relatively bad internet connection and received this error a half dozen times over a couple of days, I decided the best way to handle it, would be to write a cron script. This cron script should check for the RSYNC process and start it if it isn’t running.

rsync_check.sh

Customize this script for your own purpose, to check for your RSYNC process and start it if it isn’t running.

#!/bin/bash
echo "checking for active rsync process"
COUNT=`ps ax | grep rsync | grep -v grep | grep -v rsync_check.sh | wc -l` # see how many are running
echo "there are $COUNT rsync related processes running";
if [ $COUNT -eq 0 ] 
then
	echo "no rsync processes running, restarting process"
	killall rsync  # prevent RSYNCs from piling up, if by some unforeseen reason there are already processes running
	rsync -avz -e "ssh" user@host.com:/mnt/syncdirectory/ /home/ccase/syncdirectory/ 
fi

Crontab Entry

Save the script in the appropriate cron directory, or add it to the cron.d directory and put a crontab entry in, to run it at the desired interval. This will have it run every 10 minutes.

*/10 * * * * ccase /etc/cron.d/rsync_check.sh

No More Worries

Now you can move onto other things, with the knowledge that your RSYNC will not just fail and leave the work undone. It probably wouldn’t hurt to check on it at first and from time to time; but there’s alot less to worry about!

[amazon_link asins=’B00PKTGLWM,B0043GXMUM,1593273894,B01BNPT1EG,B00LB0E9B4′ template=’ProductCarousel’ store=’openmindspace-20′ marketplace=’US’ link_id=’b0b28615-d376-11e6-9827-b5269e98ea0d’]