In order to rsync a huge chunk of data (containing considerably large number of smaller files), the best option one can have, is to run multiple instances of
rsyncs in parallel. This seems to be pretty effective, but at the cost of high load average, more I/O oparations and network bandwidth utilization.
So as to parallelize multiple
rsynccommands, one might use
xargsor a series of
rsynccommands run in the background using &. But, over all of those alternatives, I would prefer GNU Parallel, a utility used to execute jobs in parallel. It is a single command that can replace certain loops in your code or a sequence of commands run in background.
Read more at YourOwnLinux