recursive


How to find differences between two directories using diff

Gotta love diff!

Finding all files and folders that are different in two locations is extremely easy. Using only two parameters we can exhaustively compare two directories, including their sub-directories and produce a list of their differences as such:

diff -qr folder-1/ folder-2/;

The -q parameter instructs diff to print only the files that are different and thus not spam us with thousands of files that are the same.

The -r parameter turns on the recursive feature which instructs diff to check all sub-folders and their files in the two directories under investigation.

Example run:

diff -qr Desktop/source/ /media/tux/My\ Disk/backup\ A/

Files Desktop/source/Camera/20191023_171328.jpg and /media/tux/My\ Disk/backup\ A/Camera/20191023_171328.jpg differ

Files Desktop/source/Camera/VID_20191011_115231.mp4 and /media/tux/My\ Disk/backup\ A/Camera/VID_20191011_115231.mp4 differ

diff: /media/xeirwn//media/tux/My\ Disk/backup\ A/Camera/IMG_20191225_165939.jpg: Input/output error
Advertisements

Automatically download possibly a whole public website using wget recursively

The following command will try to download a full website with all pages it can find through public links.

wget --wait=2 --limit-rate=200K --recursive --no-clobber --page-requisites --convert-links --domains example.com http://example.com/

Parameters:

  • --wait Wait the specified number of seconds between the retrievals.  We use this option to lighten the server load by making the requests less frequent.
  • --limit-rate Limit the download speed to amount bytes per second. We use this option to lighten the server load and to reduce the bandwidth we consume on our own network.
  • --recursive Turn on recursive retrieving.
  • --no-clobber If a file is downloaded more than once in the same directory, we prevent multiple version saving.
  • --page-requisites This option causes Wget to download all the files that are necessary to properly display a given HTML page.
  • --convert-links After the download is complete, convert the links in the document to make them suitable for local viewing.
  • --domains Set domains to be followed.  It accepts a domain-list as a comma-separated list of domains.