Copy Large File From Local to Remote Using Rsync

In this post we'll learn how to use rsync to copy large file of data, from local to remote and vice versa.

Copy File Using scp

Copying file using scp is easy. Below is an example of how to transfer files between local and remote.

# copy from local to remote
$ scp ~/file.txt root@123.456.789.1:/home/novalagung/

# copy from remote to local
$ scp root@123.456.789.1:/home/novalagung/file.txt ~/

Basically by using scp, copying file is pretty much done. But, if the connection is not good enough and the file size is quite big, it'll be a problem.

Just imagine, you are transferring a file from your local with size of 10GB, and after waiting for few hours, the transfer disconnected.

There is a solution to solve this kind of problem, by using rsync.

Copy File Using rsync

rsync is a utility for efficiently transferring and synchronizing files across computer systems, by checking the timestamp and size of files. It is commonly found on Unix-like systems and functions as both a file synchronization and file transfer program

rsync is used for synchronizing and transferring files. It has lot of advantages compared to scp.

  1. Good for copying multiple and large files, the process is faster compared to scp. It's because rsync use algorithm called delta compression, that's designed for minimizing network usage.

  2. Because the main purpose of rsync is for both transferring and synchronizing files, it has many useful features. One if them is the capability to resume previous unsuccessful transfers.

rsync depends on ssh to make the transferring process secure.

rysnc is standard utility for linux & macOS, so no need to install anything

Below is an example of rsync transfer file command.

# copy from local to remote
$ rsync -avz ~/file.txt root@123.456.789.1:/home/novalagung/

# copy from remote to local
$ rsync -avz root@123.456.789.1:/home/novalagung/file.txt ~/

The -avz argument is required for faster transfer process. -a is short of archive, -v is verbose, and -z is for compression.

Resume Previous Download

To be able to resume the previous transfer (in case the current transfer is failing, so we don't have to start over again) use --partial. Also put --progress as argument to show the transfer progress.

$ rsync -avz --partial --progress root@123.456.789.1:/home/novalagung/file.txt ~/

By using --partial, rsync will write the data that's being transferred into a temporary file, then later will be moved into destination place when it's complete. The temporary path directory can be changed by adding --partial-dir argument.

$ rsync -avz \
        --partial \
        --partial-dir=/homee/novalagung/temp/ \
        --progress \
        root@123.456.789.1:/home/novalagung/file.txt ~/

There is another alternative, use --inplace, it will make rsync to not write the transferred data into temporary file, instead it'll be written directly on the destination file.

$ rsync -avz --inplace --progress root@123.456.789.1:/home/novalagung/file.txt ~/