11 rsync command Linux practical examples to synchronise data
Rsync is known as “Remote Synchronisation“. Rsync command to used for sharing and synchronising data (files and directories) from one location to another location. It is also used for backup operation in UNIX/Linux machines. In this article we are going to see 11 rsync command Linux with practical examples.
Features of Rsync Command:
- It’s more secure.
- Does not require root privileges to install and execute rsync.
- Its more speed than Secure copy(SCP)
- It Supports for copying links, devices, owners, groups, and permissions
- Exclude and exclude-from options similar to GNU tar
- rsync allows encryption of data using ssh protocol during transfer.
- Can use any transparent remote shell, including ssh or rsh
- Pipelining of file or directories transfers to minimise latency costs.
- Support for anonymous rsync daemons
- Rsync consumes less bandwidth.
Install rsync command:
# yum install rsync
Basic syntax:
# rsync <options> <source> <destination>
- r – Recursive
- v – verbose
- l – Transfer any symlinks encountered
- h – Human-readable
- p – display progress during transfer
- e – specify the remote shell use
- a – archive mode
- z – compress file data
- o – preserve ownership
- g – preserve groups
- t – preserve time stamps
- D – preserve block and character devices
- d – Directory tree will be created in destination
What is an 11 rsync command Linux..?
Rsync is a command, which is used to synchronise data from one directory to another directory within local machine and client to server. Rsync consider local machine as client and remote machine as Server.
1. Synchronise from one directory to another directory within Local machine
To copy / synchronise directories with in Local host we can make use of below command example.
[root@TechTutorials ~]# rsync -av /rsynctest/*.txt /tmp/ sending incremental file list file1.txt file10.txt file2.txt file3.txt file4.txt file5.txt file6.txt file7.txt file8.txt file9.txt sent 523 bytes received 202 bytes 1450.00 bytes/sec total size is 0 speedup is 0.00
Above example will copy only matching *.txt pattern from /rsynctest directory to remote directory /tmp/
[root@TechTutorials ~]# rsync -av /rsynctest /tmp/ sending incremental file list rsynctest/ rsynctest/document1.doc rsynctest/document10.doc rsynctest/document2.doc rsynctest/document3.doc rsynctest/document4.doc rsynctest/document5.doc rsynctest/document6.doc rsynctest/document7.doc rsynctest/document8.doc rsynctest/document9.doc rsynctest/file1.txt rsynctest/file10.txt rsynctest/file2.txt rsynctest/file3.txt rsynctest/file4.txt rsynctest/file5.txt rsynctest/file6.txt rsynctest/file7.txt rsynctest/file8.txt rsynctest/file9.txt sent 1072 bytes received 396 bytes 2936.00 bytes/sec total size is 0 speedup is 0.00
above example will copy an data and it creates a directory in remote directory because we did not used trailing slash / after directory name. If we use trailing slash after directory name it will not create directory in remote path it only copies files from source
[root@TechTutorials ~]# ls /tmp/ file10.txt file5.txt rsynctest
2. Synchronise data from directory to remote machine
This would transfer all files matching the pattern *.doc from the current directory to the directory /tmp/ on the machine 192.168.4.200. If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences.
[root@TechTutorials ~]# rsync -tv /rsynctest/*.doc 192.168.4.200:/tmp/ sent 133 bytes received 12 bytes 96.67 bytes/sec total size is 0 speedup is 0.00
rsync source and destination has same files with same time stamp
3. Copy data to remote host with compression
To compress the data while sending we have to use option ‘-z’ along with 11 rsync command Linux.
Let’s see an example below
[root@TechTutorials ~]# rsync -azv /rsynctest/ 192.168.4.200:/root/ root@192.168.4.200's password: sending incremental file list ./ document1.doc document10.doc document2.doc document3.doc document4.doc document5.doc document6.doc document7.doc document8.doc document9.doc file1.txt file10.txt file2.txt file3.txt file4.txt file5.txt file6.txt file7.txt file8.txt file9.txt sent 993 bytes received 395 bytes 925.33 bytes/sec total size is 0 speedup is 0.00 [root@TechTutorials ~]# du -sh /rsynctest/ 4.0K /rsynctest/
Copy data to remote host using compression
4. Synchronise data from remote host to Local machine
Not only to sync data from local machine to remote machine. we can also do in reverse way that we can simply sync data from remote machine to local host. see below example:
[root@TechTutorials ~]# rsync -azv 192.168.4.200:/root/reversesync/ /root/ root@192.168.4.200's password: receiving incremental file list ./ sync1 sync10 sync2 sync3 sync4 sync5 sync6 sync7 sync8 sync9 sent 204 bytes received 472 bytes 1352.00 bytes/sec total size is 0 speedup is 0.00
As per above example data synchronisation has been done from remote machine 192.168.4.200 to local machine /root/ directory. Here we have taken the first path (Source) as remote machine and destination has local machine.
5. Do not Overwrite Destination Path files
In some cases after the synchronisation destination files has been modified / Updated with latest data in this situation we don’t want to overwrite them with old data. To do this we have to use -u option along with rsync command
Before the Update i have modified an file called file1.txt with some text in destination
[root@desktop ~]# cat file1.txt Testing DAta not update
Now synchronise and test whether data will overwrite or not
[root@TechTutorials ~]# rsync -azuv /rsynctest/ /root/ root@192.168.4.200's password: sending incremental file list ./ sent 273 bytes received 15 bytes 576.00 bytes/sec total size is 0 speedup is 0.00 [root@TechTutorials ~]# cd /rsynctest/ [root@TechTutorials rsynctest]# cat file1.txt
After the synchronisation lets see the data status. Nothing has been changed its 100% intact.
[root@desktop ~]# cat file1.txt Testing DAta not update
6. See Rsync progress while synchronising
As of now in all above rsync commands we did not see any progress while copy is in progress, Now lets see copy progress so that we know how much copy has been done. Simple we have add option –progress which will show current copy progress information.
[root@TechTutorials ~]# rsync -azv --progress /rsynctest/ 192.168.4.200:/tmp/ root@192.168.4.200's password: sending incremental file list ./ file1.txt 0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=10/22) file10.txt 0 100% 0.00kB/s 0:00:00 (xfer#2, to-check=9/22) file2.txt 0 100% 0.00kB/s 0:00:00 (xfer#3, to-check=8/22) file3.txt 0 100% 0.00kB/s 0:00:00 (xfer#4, to-check=7/22) file4.txt 0 100% 0.00kB/s 0:00:00 (xfer#5, to-check=6/22) file5.txt 0 100% 0.00kB/s 0:00:00 (xfer#6, to-check=5/22) file6.txt 0 100% 0.00kB/s 0:00:00 (xfer#7, to-check=4/22) file7.txt 0 100% 0.00kB/s 0:00:00 (xfer#8, to-check=3/22) file8.txt 0 100% 0.00kB/s 0:00:00 (xfer#9, to-check=2/22) file9.txt 0 100% 0.00kB/s 0:00:00 (xfer#10, to-check=1/22) linux-nrpe-agent.tar.gz 7003923 100% 19.61MB/s 0:00:00 (xfer#11, to-check=0/22) sent 7005639 bytes received 224 bytes 14011726.00 bytes/sec total size is 7003923 speedup is 1.00
7. Delete files in destination which are not required (Files does not exists in source)
This is good idea to keep file intact source and destination this is very useful when you want to make exact copy of source
Before rsync command in destination below files are exists
[root@desktop reversesync]# ls sync1 sync10 sync2 sync3 sync4 sync5 sync6 sync7 sync8 sync9 [root@desktop reversesync]# pwd /root/reversesync
Run rsync command with –delete option delete destination files
[root@TechTutorials ~]# rsync -azv --delete /rsynctest/ 192.168.4.200:/root/reversesync/ root@192.168.4.200's password: sending incremental file list ./ deleting sync9 deleting sync8 deleting sync7 deleting sync6 deleting sync5 deleting sync4 deleting sync3 deleting sync2 deleting sync10 deleting sync1 document1.doc document10.doc document2.doc document3.doc document4.doc document5.doc document6.doc document7.doc document8.doc document9.doc file1.txt file10.txt file2.txt file3.txt file4.txt file5.txt file6.txt file7.txt file8.txt file9.txt linux-nrpe-agent.tar.gz sent 7006003 bytes received 414 bytes 4670944.67 bytes/sec total size is 7003923 speedup is 1.00
After rsync has been successful lets see destination
[root@desktop reversesync]# ls document10.doc document2.doc document4.doc document6.doc document8.doc file10.txt file2.txt file4.txt file6.txt file8.txt linux-nrpe-agent.tar.gz document1.doc document3.doc document5.doc document7.doc document9.doc file1.txt file3.txt file5.txt file7.txt file9.txt [root@desktop reversesync]# pwd /root/reversesync
8. Check Changes in between Source and Destination
This option will help in comparing the files and there changes, If you want to compare source and destination manually it is a TDS job. Lets see how this amazing command option works for us
[root@TechTutorials rsynctest]# rsync -azvi /rsynctest/ 192.168.4.200:/root/reversesync/ root@192.168.4.200's password: sending incremental file list .d..t...... ./ .f....og... document4.doc <f.st...... document7.doc sent 426 bytes received 43 bytes 938.00 bytes/sec total size is 7003923 speedup is 14933.74
As we see source and destination has changes in document4.doc and document7.doc files before file names we are able to see alphabet symbols right what they mean
- f = That is a file
- o = Ownership changed
- g = Group Ownership Changed
- s = Size Variation
- t = Time stamp changed
using this option -i will sync files from source to destination and make the changes in destination
9. Just checking changes in between source and destination before sync
If you afraid to run rsync command in your environment don’t worry here is an example where you can compare changes before running rsync command
[root@TechTutorials ~]# rsync -vh --dry-run /rsynctest/document7.doc 192.168.4.200:/root/reversesync/document7.doc root@192.168.4.200's password: document7.doc sent 37 bytes received 15 bytes 104.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN) [root@TechTutorials ~]# rsync --dry-run /rsynctest/document4.doc 192.168.4.200:/root/reversesync/document4.doc root@192.168.4.200's password:
As per above example i have used option –dry-run which will tell us the changes in destination. File document7.doc has changes that’s why it is shown send and received bytes whereas file document4.doc doesn’t have any changes compare to source that’s why sent and receive bytes are zero.
10. Avoid transfer large files
In some of the cases we don’t want to transfer/copy/sync larger files to destination in such a case also we have an option to stop transfer of larger files.
[root@TechTutorials ~]# rsync -azv --max-size='1000k' /rsynctest/ 192.168.4.200:/root/reversesync/ sending incremental file list ./ document1.doc document10.doc document2.doc document3.doc document4.doc document5.doc document6.doc document7.doc document8.doc document9.doc file1.txt file10.txt file2.txt file3.txt file4.txt file5.txt file6.txt file7.txt file8.txt file9.txt test.rpm test1.nothing test2.nothing test3.nothing test4.nothing test5.nothing sent 14260 bytes received 509 bytes 9846.00 bytes/sec total size is 9695775 speedup is 656.50 [root@TechTutorials ~]# ls -ltr /rsynctest/ total 9472 -rw-r--r--. 1 root root 0 Aug 3 19:55 file3.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file2.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file1.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file7.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file6.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file5.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file4.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file9.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file8.txt -rw-r--r--. 1 root root 0 Aug 3 19:55 file10.txt -rw-r--r--. 1 root root 0 Aug 3 19:56 document4.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document3.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document2.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document1.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document9.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document8.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document7.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document6.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document5.doc -rw-r--r--. 1 root root 0 Aug 3 19:56 document10.doc -rw-r--r--. 1 root root 0 Aug 4 15:03 test5.nothing -rw-r--r--. 1 root root 0 Aug 4 15:03 test4.nothing -rw-r--r--. 1 root root 0 Aug 4 15:03 test3.nothing -rw-r--r--. 1 root root 0 Aug 4 15:03 test2.nothing -rw-r--r--. 1 root root 0 Aug 4 15:03 test1.nothing -rw-r--r--. 1 root root 7003923 Aug 4 15:36 linux-nrpe-agent.tar.gz -rw-r--r--. 1 root root 14500 Aug 4 15:36 test.rpm -rw-r--r--. 1 root root 2677352 Aug 4 15:36 nagios-plugins-2.1.1.tar.gz
As per above command option is copy files which are not more than 1000KB in size. Below tar files are more than 1000KB in size that’s why they did not copied to destination.
11. Send whole-file as-is
With this option rsync’s delta-transfer algorithm is not used and the whole file is sent as-is instead. The transfer may be faster if this option is used when the bandwidth between the source and destination machines is higher than the bandwidth to disk (especially when the “disk” is actually a networked filesystem). This is the default when both the source and destination are specified as local paths, but only if no batch-writing option is in effect. Below is the example.
[root@TechTutorials rsynctest]# rsync -azvW /rsynctest/file2.txt 192.168.4.200:/root/ sending incremental file list file2.txt sent 114 bytes received 31 bytes 290.00 bytes/sec total size is 40 speedup is 0.28
Thanks for the read.
11 rsync command Linux 11 rsync command Linux 11 rsync command Linux 11 rsync command Linux 11 rsync command Linux 11 rsync command Linux
11 rsync command Linux 11 rsync command Linux
Related Articles
15 scp commands to securely copy data to remote host
How to reset root user forgotten password in RHEL 7
Practical Grep commands useful in real time
Thanks for your wonderful Support and Encouragement