As rsync will not accept to be given a custom port on the destination address, a way around it is to initiate an ssh connection that will handle it for you:
rsync -e 'ssh -p 2222' $SOURCE_DIR $DEST_DIR;
The rsync command is a powerful tool for file synchronization and transfer in Linux and Unix-like operating systems. It provides a robust and efficient way to copy, backup, and mirror files and directories both locally and remotely. In this article, we will explore the technical details of the following rsync command:
rsync -avh --delete --progress "path/to/source/" "path/to/destination/";
We will explain each option and argument used in the command and their respective functionality.
Options and Arguments
The rsync command is a versatile tool with many options and arguments to customize its behavior. The options used in the above command are:
The arguments used in the command are:
Explanation
The rsync command is a powerful tool that synchronizes the source and destination directories or files. The -a option enables the archive mode, which ensures that the file metadata is preserved during the transfer, including permissions, ownership, timestamps, and symbolic links. The -v option enables the verbose mode, which provides detailed output of the rsync operation, including the transferred files and their sizes.
The -h option enables the human-readable mode, which displays file sizes in a more readable format. The –delete option tells rsync to delete any files at the destination that do not exist at the source, which ensures that the destination is an exact copy of the source. Finally, the –progress option displays real-time progress information during the rsync operation, including the percentage of completion and the estimated time remaining.
Conclusion
The rsync command is a powerful tool for file synchronization and transfer in Linux and Unix-like operating systems. The command discussed in this article synchronizes the source and destination directories or files, preserves file metadata, displays detailed output, deletes any files that do not exist at the destination, and displays real-time progress information. With the proper use of rsync options and arguments, file synchronization and transfer can be made easy, efficient, and reliable.
We have two external hard disks that we use to keep backups of our data.
The way we do that is by using the command rsync that makes our life easy.
Specifically, we use the following command to synchronize the first hard disk with the second one: rsync -avh –delete –progress “path/to/source/” “path/to/destination/”;
rsync is a fast, versatile, remote (and local) file-copying tool, it is available in almost every system (GNU/Linux, Unix (MacOS as well) and Windows).
The parameters we use are the following:
-a, --archive enables archive mode which is equal to -rlptgoD (no -H,-A,-X)-r, --recursive recurse into directories-l, --links copy symlinks as symlinks-p, --perms preserve permissions-t, --times preserve modification times-g, --group preserve group-o, --owner preserve owner (super-user only)-D same as --devices --specials--devices preserve device files (super-user only)--specials preserve special files-v, --verbose it increases verbosity of the output-h, --human-readable outputs numbers in a human-readable format--delete deletes extraneous files from destination directories--progress shows progress during transferOur Jenkins setup has a lot of cool features and configuration.
It has ‘project-based security’, it has parametrized projects, multiple source code management blocks per project and fairly extensive tests implemented with several build steps.
Of course, we do not want to lose them, so we make backups often.
The commands we use for the backup are the following.
jenkins_folder="/var/lib/jenkins/"; backup_folder="$HOME/jenkins/`date +%F`"; mkdir -p "$backup_folder"; (cd "$jenkins_folder"/jobs/; find . -mindepth 3 -type d -regex '.*/[0-9]*$' -print) | sed 's|./|jobs/|' | sudo rsync --archive --exclude 'workspace/*' --exclude-from=- "$jenkins_folder" "$backup_folder";
Explanation of commands:
backup_folder="$HOME/jenkins/`date +%F`"; we used the $HOME variable instead of the tilde ~ as this would create a folder in the current directory called ~ instead of creating a new folder called jenkins in the home directory.mkdir -p "$backup_folder"; instructs mkdir to create all parent folders needed to create our destination folder.(cd "$jenkins_folder"/jobs/; find . -mindepth 3 -type d -regex '.*/[0-9]*$' -print) navigates to the directory of jenkins before performing the search, this way the result file names will be relative to the installation location which we need later to pass to rsync.sed 's|./|jobs/|' replaces the prefix ./ with jobs/ to match the relative path from where rsync will work fromsudo rsync --archive --exclude 'workspace/*' --exclude-from=- "$jenkins_folder" "$backup_folder"; it will copy everything from $jenkins_folder to the folder $backup_folder while excluding the data in workspace and the folders matched from find (the job build folders).--exclude-from=- instructs rsync to read from stdin the list of files to exclude.
