In this article, we cover how to copy files and directories locally in Ubuntu and AlmaLinux using rsync command-line utility. With the help of rsync command-line utility, we can copy files/directories both locally as well as between local and remote hosts. The article is about copying files/directories locally.
At this stage, one is right to ask – We already have cp command-line utility to copy files or directories then, why use rsync?
rsync mainly uses Delta encoding, so it only transmits the difference between source and destination file rather than the entire file itself. So, when only the difference is transferred – it saves both time and bandwidth over which the file is sent. The same also applies to directories as well.
It is worth mentioning here that, by default delta-transmission is disabled for local transfer.
Apart from that, we also use it as a backup tool. Since, it only transmits the difference therefore it works faster than the conventional method of taking backups.
Install rsync in Ubuntu
The package which is required to use rsync also goes by the same name – rsync. Although, it comes installed as default in Ubuntu. Issue the following in terminal to check if its there or not –
rsync -V
Note: Following operations require administrative rights. If you don’t have the rights then contact your System Administrator for assistance.
But, for some reason, if its not there then for Ubuntu –
sudo apt update sudo apt install rsync
Install rsync in AlmaLinux
To install it in AlmaLinux (package is a part of BaseOS repository) –
# dnf install rsync
rsync syntax
rsync [options] {source-file-dir} {destination-file-dir}
Copy files and directories locally using rsync
We should have required file and directory permissions to copy them locally. Let’s understand it with the help of examples.
Case I. If you want to copy contents of directory – directory1/ to destination directory – /dev/shm/directory2
rsync -avvz directory1/ /dev/shm/directory2/
Here,
-a or –archive option is equivalent to using the option -rlptgoD. This option would preserve file permissions, modification times, file/group ownership and devices as devices. Furthermore, it also copies symlinks and recursively copies the source directory.
-z option is used for data compression.
-vv option is to increase the level of verbosity. We can go ahead with -v as well. But, increasing it with -vv is something we prefer. If you would like to know more about the transfer then increase the verbosity level by -vvv option.
Case II. If you want to copy the directory itself then –
rsync -avvz directory1 /dev/shm/directory2/
This time around, we haven’t used slash(/) after directory1. And, the above command will create a new directory – directory2/ (if it doesn’t exists). And, in directory2/ it would copy the entire directory1/
Or, if you don’t want to create a new directory2/ then,
rsync -avvz directory1 /dev/shm/
Dry run
Important – Though, some of us would want to test things before implementing them. Do we have an option to simulate a command without executing it? Yes, we do. Perform a trial with –dry-run or -n option. Continuing with the example as mentioned in Case II with –dry-run option.
rsync --dry-run -avvz directory1 /dev/shm/
No changes are made to the directory structure. Notice, (DRY RUN) at the end of output. Clearly, we performed a trial and it only shows how things would have changed if we had gone otherwise which gets reflected on the standard output. If you are satisfied with the outcome then, move ahead without the –dry-run option.
Case III. To copy only specific file types
If we want to copy specific file types like .txt then,
rsync -avvz directory1/*.txt /dev/shm/directory2
Case IV. See transfer progress
Use –progress option to display the transfer progress. It is particularly useful where we have to transmit large files or directories. With this option you would at least have an idea about the time it would take to complete transfer. Again, we continue with the example in Case II.
rsync --progress -avvz directory1 /dev/shm/
Case V. Exclude a pattern of files
If we don’t want to copy certain types of files then, we can exclude them through –exclude option.
rsync --exclude="*.txt" -avvz directory1 /dev/shm/
Case VI. Only include error messages in the output
Use -q option to exclude non-error messages from standard output –
rsync --progress -azq directory1 /dev/shm/
So, we get to see only the error messages.
Case VII. Specify size of files to be transferred
We can also specify maximum and minimum size of files we intend to transfer through –max-size and –min-size options.
If we don’t wish to transfer files larger than a particular size then –
rsync --max-size=5M --progress -avvz directory1 /dev/shm/
On the other hand, if we don’t wish to transfer files smaller than a specific size then –
rsync --min-size=1M --progress -avvz directory1 /dev/shm/
But, can both of above options be used to limit file transfers on the basis of size? Yes, we can. For instance, if we wish to transfer only those files which are larger than 1MB but smaller than 5MB in size then –
rsync --min-size=1M --max-size=5M --progress -avvz directory1 /dev/shm/
Case VIII. Keep logs for the transfers
It is necessary to keep logs for the file/directory transfers. So, the operations we execute with rsync can be stored in a user-defined log file. This we can do through –log-file option. For instance,
rsync --log-file=/dev/shm/rsync-logs -avvz directory1 /dev/shm/
The above command would keep logs in rsync-logs file which is saved in /dev/shm directory.
Case IX. To get transfer data in human readable format
If we want standard output, log file data etc. in human readable format then use -h option.
rsync -h -avvz directory1 /dev/shm/
The above command was specifically for standard output. Besides, if you wish to save log file data in human readable format as well then –
rsync -h --log-file=/dev/shm/rsync-logs -avvz directory1 /dev/shm/
Case X. List source files and directories
If you want to list only source file and directories then use –list-only option.
rsync --list-only dir1/
If we remove slash (/) from dir1/ then notice the difference –
rsync --list-only dir1
Apart from that, we can also use it with –
rsync --list-only -avvz dir1 /dev/shm/
It won’t initiate the file transfer but only list source files or directories.
Case XI. Set maximum file or directory transfer rate
With the help of –bwlimit option, we can specify maximum data transfer rate.
If we don’t use a suffix then, the transfer rate value will be in units of 1024 bytes.
rsync --bwlimit=5000 -avvz directory1 /dev/shm/
Case XII. Keep partially transferred files
Though all the options we have covered till now are quite useful. But, what we are about to cover has a special place. Reason: We initiate a larger file transfer. And, everything is working as intended. But, due to some external events the file transfer got interrupted. What if the 90% of the file was already transferred? And, just because some event interrupted it- we would have to initiate the file transfer again. And, it starts from zero this time.
rsync, by default, delete the files which were being transferred but got interrupted before reaching the destination. So, no matter if 10% transfer happened or 99%. It would delete the partially transferred file at the destination.
But, there is a workaround with –partial option. Though we may not use it for all files. With –partial option, we mainly change the default behavior of rsync. Thus, it keeps the partially transferred files. And, when we run the command again it would get things done for us faster.
rsync --partial -avvz directory1/file /dev/shm/filetrf
Apart from what we have covered here, rsync can be utilized to copy files and directories over a network as well. Then there is Rsync Daemon and GTK frontend for rsync – Grsync. We will cover each of those in subsequent articles.
![Fixed: [ Error writing: Permission denied ] with Nano](https://techpiezo.com/wp-content/uploads/2023/06/fixed-error-writing-permission-denied-with-nano-768x432.webp)
