folder


How to find differences between two directories using diff

Gotta love diff!

Finding all files and folders that are different in two locations is extremely easy. Using only two parameters we can exhaustively compare two directories, including their sub-directories and produce a list of their differences as such:

diff -qr folder-1/ folder-2/;

The -q parameter instructs diff to print only the files that are different and thus not spam us with thousands of files that are the same.

The -r parameter turns on the recursive feature which instructs diff to check all sub-folders and their files in the two directories under investigation.

Example run:

diff -qr Desktop/source/ /media/tux/My\ Disk/backup\ A/

Files Desktop/source/Camera/20191023_171328.jpg and /media/tux/My\ Disk/backup\ A/Camera/20191023_171328.jpg differ

Files Desktop/source/Camera/VID_20191011_115231.mp4 and /media/tux/My\ Disk/backup\ A/Camera/VID_20191011_115231.mp4 differ

diff: /media/xeirwn//media/tux/My\ Disk/backup\ A/Camera/IMG_20191225_165939.jpg: Input/output error

How to add automatically all empty folders in git repository 4

Since you are searching for this issue, you must have realised that git does not support storing empty folders/directories.

Currently the design of the Git index (staging area) only permits files to be listed, and nobody competent enough to make the change to allow empty directories has cared enough about this situation to remedy it.

Directories are added automatically when adding files inside them. That is, directories never have to be added to the repository, and are not tracked on their own.
— From https://git.wiki.kernel.org/index.php/Git_FAQ#Can_I_add_empty_directories.3F

All the content is stored as tree and blob objects, with trees corresponding to UNIX directory entries and blobs corresponding more or less to inodes or file contents. A single tree object contains one or more tree entries, each of which contains a SHA-1 pointer to a blob or subtree with its associated mode, type, and filename.
— From https://git-scm.com/book/en/v2/Git-Internals-Git-Objects

Below we propose two solutions, depending on how you want to use those empty folders.

Solution A – The folders will always be empty

There are scenarios where the empty folders should always remain empty on git no matter what the local copy has inside them.
Such a scenario would be, wanting to add on git folders where you will build your objects and/or put temporary/cached data.
In such scenarios it is important to have the structure available but never to add those files in git.

To achieve that, we add a .gitignore file in every empty folder containing the following:

# git does not allow empty directories.
# Yet, we need to add this empty directory on git.
# To achieve that, we created this .gitignore file, so that the directory will not be empty thus enabling us to commit it.
# Since we want all generated files/folders in this directory to be ignored by git, we add a rule for this.
*
# And then add an exception for this specifc file (so that we can commit it).
!.gitignore

[download id=”2322″]

The above .gitignore file, instructs git to add this file on the repository, and thus add the folder itself while ignoring ALL other files.
You can always update your .gitignore file to allow additional files to be added on the repository at any time.
This way you will never get prompted for temporary files that they were modified/created as part of the status of your repository.

Automation

A way to achieve this automatically, and place a copy of the .gitignore file in every empty folder would be to use the following command to copy an existing .gitignore file in all empty folders.

find $PATH_TO_REPOSITORY -type d ! -path "*.git*" -empty -exec cp .gitignore '{}'/ \;

The above command assumes that there is a .gitignore file in the folder that we are executing from, which it will copy in every empty directory inside the folder that the variable $PATH_TO_REPOSITORY is pointing to.

[download id=”2322″]

Solution B – Files will be added eventually to the folders

There are scenarios where the empty folders will be filled at a later stage and we want allow those files on git.
Such a scenario would be, wanting to add on git folders where right now are empty but in some time we will add new source code or resources there.

To achieve that, we add an empty .gitkeep file in every empty folder.

The above .gitkeep file is nothing more than a placeholder.
It is not documented, because it’s not a feature of Git.
It’s a dummy file, so git will not process the empty directory, since git tracks only files.
Once you add other files to the folder, you can safely delete it.
This way you will always get prompted for files that they were modified/created as part of the status of your repository.

Automation

A way to achieve this automatically, and place a copy of the .gitkeep file in every empty folder would be to use the following command to create an empty .gitkeep file in all empty folders.

find $PATH_TO_REPOSITORY -type d ! -path "*.git*" -empty -exec touch '{}'/.gitkeep \;

The above command will create in every empty directory inside the folder that the variable $PATH_TO_REPOSITORY is pointing to a new .gitkeep file.

Finally, push the changes to the git repository

After you create/copy the files, navigate to the repository, add all the new files to the commit, commit them and push them to the repository.

cd $PATH_TO_REPOSITORY;
# Create a new branch
git checkout -b empty_folders;
# Add all modified files to the next commit.
git add .;
git commit -m "Minor change: Adding all empty folders to the repository.";
git push -u origin empty_folders;

Bash: Determine state of file

Following you will find some tests one can perform on a file to identify its state

Check if file $FILE does not exist

if [ ! -f "$FILE" ]; then
    echo "File $FILE does not exist";
fi

Check if file $FILE exists and is a directory

if [ -d "$FILE" ]; then
    echo "File $FILE exists and is a directory";
fi

Check if file $FILE exists and is a regular file (not a directory)

if [ -f "$FILE" ]; then
    echo "File $FILE exists and is a regular file (not a directory)";
fi

Check if file $FILE exists, we do not know what type it is (if it is a directory, socket, node, etc.)

if [ -e "$FILE" ]; then
    echo "File $FILE exists, we do not know what type it is (if it is a directory, socket, node, etc.)";
fi

Check if file $FILE exists and is a symbolic link

if [ -L "$FILE" ]; then
    echo "File $FILE exists and is a symbolic link";
fi

Check if file $FILE exists and is a socket

if [ -S "$FILE" ]; then
    echo "File $FILE exists and is a socket";
fi

Check if file $FILE exists and is not empty

if [ -s "$FILE" ]; then
    echo "File $FILE exists and is not empty";
fi

Check if file $FILE exists and is readable

if [ -r "$FILE" ]; then
    echo "File $FILE exists and is readable";
fi

Check if file $FILE exists and is writable

if [ -w "$FILE" ]; then
    echo "File $FILE exists and is writable";
fi

Check if file $FILE exists and is executable

if [ -x "$FILE" ]; then
    echo "File $FILE exists and is executable";
fi