Programming


Android Studio – No space left on device 1

Recently we tried to install some packages from the Android SDK through Android Studio 3.0.1 and we got the error that the PC ran out of space while downloading the necessary packages and could not perform the operation.

Full Error Log:

To install:
- Google APIs Intel x86 Atom System Image (system-images;android-27;google_apis;x86)
Preparing "Install Google APIs Intel x86 Atom System Image (revision: 2)".
Downloading https://dl.google.com/android/repository/sys-img/google_apis/x86-27_r02.zip
An error occurred while preparing SDK package Google APIs Intel x86 Atom System Image: Cannot download 'https://dl.google.com/android/repository/sys-img/google_apis/x86-27_r02.zip': No space left on device
, response: 200 OK.
"Install Google APIs Intel x86 Atom System Image (revision: 2)" failed.
Failed packages:
- Google APIs Intel x86 Atom System Image (system-images;android-27;google_apis;x86)

Although the disk had a lot of free space, we got this error because the /tmp partition on our GNU/Linux Fedora 27 workstation was too small (~4GB).

$ df -h;
Filesystem Size Used Avail Use% Mounted on
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 107M 3.8G 3% /dev/shm
tmpfs 3.9G 1.9M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/mapper/fedora-root 50G 35G 12G 75% /
tmpfs 3.9G 2.1M 3.9G 1% /tmp
/dev/sdb1 477M 179M 269M 41% /boot
/dev/mapper/fedora-home 178G 37G 132G 22% /home
tmpfs 794M 16K 794M 1% /run/user/42
tmpfs 794M 9.2M 785M 2% /run/user/1000

/tmp was not explicitly set so it was automatically configured to have half of the size of the RAM.
We didn’t not want to change the download path for the Java environment (and hence Android Studio and Android SDK tools) as after leaving the tmpfs folder it could mean that we would have to manually maintain the new path and clean it up. So we ended up in temporarily increasing the size of /tmp partition which did the trick and the virtual device was installed successfully.

Solution – temporarily increase the size of the /tmp partition:

The command we used to increase the size of the /tmp partition on Fedora 27 was the following:

sudo mount -o remount,size=8G,noatime /tmp;

After executing we, it we immediately saw that the results were applied without the need for a restart or any other operation and we could proceed to use the PC as normal.

$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 89M 3.8G 3% /dev/shm
tmpfs 3.9G 1.9M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/mapper/fedora-root 50G 35G 12G 75% /
tmpfs 8.0G 2.0M 8.0G 1% /tmp
/dev/sdb1 477M 179M 269M 41% /boot
/dev/mapper/fedora-home 178G 37G 132G 22% /home
tmpfs 794M 16K 794M 1% /run/user/42
tmpfs 794M 6.9M 787M 1% /run/user/1000

The information for tmpfs we got it from: https://wiki.archlinux.org/index.php/Tmpfs

Full Correct Execution Log:

To install:
- Google APIs Intel x86 Atom System Image (system-images;android-27;google_apis;x86)
Preparing "Install Google APIs Intel x86 Atom System Image (revision: 2)".
Downloading https://dl.google.com/android/repository/sys-img/google_apis/x86-27_r02.zip
"Install Google APIs Intel x86 Atom System Image (revision: 2)" ready.
Installing Google APIs Intel x86 Atom System Image in /home/xeirwn/Android/Sdk/system-images/android-27/google_apis/x86
"Install Google APIs Intel x86 Atom System Image (revision: 2)" complete.
"Install Google APIs Intel x86 Atom System Image (revision: 2)" finished.


Manually set the CMake output folder

If you want to manually set the global output folder for you whole CMake project and depending on the output you expect add the following configuration lines in the root CMakeLists.txt file of your project:

set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)

In case you wan to specify those folders per target, you can update them as follows:

set_target_properties( target_or_targets
  PROPERTIES
  ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
  LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
  RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin"
)

Please note that we are setting the same properties using different variables.

The CMAKE_ARCHIVE_OUTPUT_DIRECTORY variable is used to initialize the ARCHIVE_OUTPUT_DIRECTORY property on all the targets.
ARCHIVE_OUTPUT_DIRECTORY property specifies the directory into which archive target files should be built.
An archive output artifact of a buildsystem target may be:

  • The static library file (e.g. .lib or .a) of a static library target created by the add_library() command with the STATIC option.
  • On DLL platforms: the import library file (e.g. .lib) of a shared library target created by the add_library() command with the SHARED option.
  • On DLL platforms: the import library file (e.g. .lib) of an executable target created by the add_executable() command when its ENABLE_EXPORTS target property is set.

 

The CMAKE_LIBRARY_OUTPUT_DIRECTORY variable is used to initialize the LIBRARY_OUTPUT_DIRECTORY property on all the targets.
LIBRARY_OUTPUT_DIRECTORY property specifies the directory into which library target files should be built.
A library output artifact of a buildsystem target may be:
The loadable module file (e.g. .dll or .so) of a module library target created by the add_library() command with the MODULE option.
On non-DLL platforms: the shared library file (e.g. .so or .dylib) of a shared shared library target created by the add_library() command with the SHARED option.

 

The CMAKE_RUNTIME_OUTPUT_DIRECTORY variable is used to initialize the RUNTIME_OUTPUT_DIRECTORY property on all the targets.
RUNTIME_OUTPUT_DIRECTORY property specifies the directory into which runtime target files should be built.
A runtime output artifact of a buildsystem target may be:

  • The executable file (e.g. .exe) of an executable target created by the add_executable() command.
  • On DLL platforms: the executable file (e.g. .dll) of a shared library target created by the add_library() command with the SHARED option.

From: https://cmake.org/documentation/


Two ways to append a new argument to CMAKE_ARGS list for ExternalProject_Add

Recently, we wanted to pass a new cached value to an external project in CMake via the CMAKE_ARGS variable.

CMAKE_ARGS holds various types of arguments. From which, the arguments in the form -Dname:type=value are passed to the CMake command line and cannot be changed by the user.

We found two ways to add a new argument pair to the CMAKE_ARGS of the external project.

The first method uses the set function:

set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR}${CACHE_VAR_TYPE}=\"${${CACHE_VAR}}\"")

where basically we create a new string that starts with the existing value of CMAKE_ARGS and then we append to its end the new pair.

The second method uses the list function:

list(APPEND CMAKE_ARGS "-D${CACHE_VAR}${CACHE_VAR_TYPE}=${${CACHE_VAR}}")

that treats CMAKE_ARGS as a list and it appends to its end the new pair.

To verify by hand that they were equivalent we did the following small test with success

#Copy the original value of ${CMAKE_ARGS}
set(CMAKE_ARGS_ORIGINAL ${CMAKE_ARGS})

#Define the Name, Type and Value for the new argument pairs
set(CACHE_VAR_P_NAME PRODUCTS_DIR)
set(CACHE_VAR_P_TYPE string)
set(CACHE_VAR_P_VALUE "someplace/with spaces")

set(CACHE_VAR_C_NAME CLIENTS_DIR)
set(CACHE_VAR_C_TYPE string)
set(CACHE_VAR_C_VALUE "another/place")

#Print the original value of ${CMAKE_ARGS}
MESSAGE(STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Append the two pairs to ${CMAKE_ARGS} using set
set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR_P_NAME}:${CACHE_VAR_P_TYPE}=\"${CACHE_VAR_P_VALUE}\"")
set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR_C_NAME}:${CACHE_VAR_C_TYPE}=\"${CACHE_VAR_C_VALUE}\"")

#Print the modified value of ${CMAKE_ARGS}
MESSAGE(STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Reset ${CMAKE_ARGS} to its original value
set(CMAKE_ARGS ${CMAKE_ARGS_ORIGINAL})

#Print the original value of ${CMAKE_ARGS} for verification
MESSAGE( STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Append the two pairs to ${CMAKE_ARGS} using list
list(APPEND CMAKE_ARGS "-D${CACHE_VAR_P_NAME}:${CACHE_VAR_P_TYPE}=\"${CACHE_VAR_P_VALUE}\"")
list(APPEND CMAKE_ARGS "-D${CACHE_VAR_C_NAME}:${CACHE_VAR_C_TYPE}=\"${CACHE_VAR_C_VALUE}\"")

#Print the modified value of ${CMAKE_ARGS}
#Notice that here we surrounded ${CMAKE_ARGS} with quotes so that it is printed as list of delimiter separated values
#Between each element the character ';' will be added because ${CMAKE_ARGS} is surrounded with quotes
MESSAGE(STATUS "CMAKE_ARGS: '" "${CMAKE_ARGS}" "'")

Which resulted in the following output

-- CMAKE_ARGS: ''
-- CMAKE_ARGS: ' -DPRODUCTS_DIR:string="someplace/with spaces" -DCLIENTS_DIR:string="another/place"'
-- CMAKE_ARGS: ''
-- CMAKE_ARGS: '-DPRODUCTS_DIR:string="someplace/with spaces";-DCLIENTS_DIR:string="another/place"'

From the output we can see that the two options treat the variable in a different way as using set just created a huge string containing both pairs while list created a list of two elements.
Both methods seem to work properly but we chose the list method for our external project.

The CMakeLists.txt file that we used to include the external project using custom CMAKE_ARGS resulted as follows:

cmake_minimum_required (VERSION 3.2.2)
include (ExternalProject)

set (TARGET c-banana-eat-banana)
project (${TARGET} C)

set (BANANA_ROOT ${CMAKE_CURRENT_BINARY_DIR} CACHE INTERNAL "")

#Add custom CMake arguments to be passed to the CMake command line of the external project
list(APPEND CMAKE_ARGS "-DARTIFACTS_DIR:string=\"${LIBRARIES_DIR}/${TARGET}\"")

ExternalProject_Add (${TARGET}
 URL ${CMAKE_CURRENT_SOURCE_DIR}/${TARGET}.zip
 URL URL_HASH MD5=34734a678729967f426931d913326112
 CMAKE_ARGS "${CMAKE_ARGS}"
 BUILD_IN_SOURCE 1)

JavaSript: Remove all non printable and all non ASCII characters from text

According to the ASCII character encoding, there are 95 printable characters in total.
Those characters are in the range [0x20 to 0x7E] ([32 to 126] in decimal) and they represent letters, digits, punctuation marks, and a few miscellaneous symbols.
Character 0x20 (or 32 in decimal) is the space character ' ' and
character 0x7E (or 126 in decimal) is the tilde character '~'.
Source: https://en.wikipedia.org/wiki/ASCII#Printable_characters

Since all the printable characters of ASCII are conveniently in one continuous range, we used the following to filter all other characters out of our string in JavaScript.

printable_ASCII_only_string = input_string.replace(/[^ -~]+/g, "");

What the above code does is that it passes the input string through a regular expression which will match all characters out of the printable range and replace them with nothing (hence, delete them).
In case you do not like writing your regular expression with the space character to it, you can re-write the above regular expression using the hex values of the two characters as follows:

printable_ASCII_only_string = input_string.replace(/[^\x20-\x7E]+/g, "");


My .gitignore file is ignored by git and it does not work

Some times, even if you haven’t added some files to the repository, git seems to monitor them even after you add them to the .gitignore file.

This is a caching issue that can occur and to fix it, you need to clear your cache.

NOTE : Before proceeding with this solution, commit all changes you do not want to lose!

.. then execute the following commands from the root folder of your repository:
The following, will untrack every file that is in your .gitignore:

git rm -r --cached .;
git add .;
git commit -m "Untracked files issue resolved to fix .gitignore";

git-rm removes files from the index, or from the working tree and the index. git rm will not remove a file from just your working directory. (There is no option to remove a file only from the working tree and yet keep it in the index; use /bin/rm if you want to do that.) The files being removed have to be identical to the tip of the branch, and no updates to their contents can be staged in the index, though that default behavior can be overridden with the -f option. When --cached is given, the staged content has to match either the tip of the branch or the file on disk, allowing the file to be removed from just the index.

-r allows recursive removal when a leading directory name is given.

--cached unstages and removes paths only from the index. Working tree files, whether modified or not, will be left alone.

From: git-rm

To stop tracking a single file file but not delete it from your filesystem use the following:

git rm --cached <file>;

Another issue: file removed from .gitignore filters does not appear to be tracked

When you remove something from .gitignore file and the file does not appear to be tracked, you can add it manually as follows:

git add -f <file>;
git commit -m "Re-Adding ignored file by force";


How to add untracked files to a git patch

Recently, we had to create a git patch for the deployment of a 3rd party repository in our code.
Some of the changes we had to apply using the patch mechanism was the creation of a few new files.
We did not want to have an external script to copy the new files to the appropriate locations, so we had to include those new files in the git patch somehow.
The git diff command (with the parameter -p or --patch) that generates the patch, it ignored the untracked files and so they did not appear in the patch.
To make the untracked files visible to the git diff command, we staged them (using git add) and then used the following command to create the patch:

git diff --patch --staged;

git diff [--options] --cached [<commit>] [--] [<path>...]
git diff [--options] --staged [<commit>] [--] [<path>...]
Adding the parameter --staged or --cached allows you to view the changes you staged for the next commit relative to the named <commit>. Typically you would want comparison with the latest commit, so if you do not give <commit>, it defaults to HEAD. If HEAD does not exist (e.g. unborned branches) and <commit> is not given, it shows all staged changes. --staged is a synonym of --cached.
From: man git-diff

In the end our commands to create the patch with the new files and apply it on a new clone of the 3rd party repository was as follows:

#In the folder of the modified repository, where the new files are staged
git diff -p --staged > ~/new.file.patch.diff;
#In the folder of the new clone of the repository, where the new files need to be created
git apply ~/new.file.patch.diff;


Ignore all edits to a file that is committed in git

Recently, we were working on a project that had committed in the source code a configuration file. That configuration file had hard-coded the production system values, so we had to modify them to the development system values before using it.

To avoid committing the configuration file with the development parameters by accident, we instructed git to ignore any changes that were made to it using the following command.

git update-index --assume-unchanged <file>;

By doing so, git assumed that the file was always unchanged and it never showed up in the git status results nor was staged when git add . was used etc.

After we were done with development (and whenever we needed to pull the branch for changes or checkout another branch) we removed the file from the list of ignored files using the following command.

git update-index --no-assume-unchanged <file>;

Using this command, git would start again to monitor changes to the file and merge it or update it or push it when needed as it would normally do for any file not included in the .gitignore file. The best part of this trick is that you do not have to update the .gitignore file to achieve the task of ignoring a file.

More information

git update-index modifies the index or directory cache. Each file mentioned is updated into the index and any unmerged or needs updating state is cleared.

--[no-]assume-unchanged When these flags are specified, the object names recorded for the paths are not updated. Instead, these options set and unset the “assume unchanged” bit for the paths. When the “assume unchanged” bit is on, Git stops checking the working tree files for possible modifications, so you need to manually unset the bit to tell Git when you change the working tree file. This is sometimes helpful when working with a big project on a filesystem that has very slow lstat(2) system call (e.g. cifs).

This option can be also used as a coarse file-level mechanism to ignore uncommitted changes in tracked files (akin to what .gitignore does for untracked files). Git will fail (gracefully) in case it needs to modify this file in the index e.g. when merging in a commit; thus, in case the assumed-untracked file is changed upstream, you will need to handle the situation manually.

From: man git-update-index

Bonus

In case you are wondering on how to see which files are currently ignored in your local repository copy by the git update-index --assume-unchanged <file>; command, you can use the following code:

git ls-files -v | grep -e '^[[:lower:]]';

git ls-files -v will print out all objects that git knows and the -v parameter will print all flags associated with them. The files that are ignored because of the
git update-index --assume-unchanged <file>; command will be printed each one on a different line that starts with a lower case character. So, to get all files that are ignored by the git update-index --assume-unchanged <file>; command, we need to grep the results of git ls-files -v for lines that start with a lower case.

git-ls-files shows information about files in the index and the working tree.
It merges the file listing in the directory cache index with the actual working directory list, and shows different combinations of the two.

-v Similar to -t (below), but use lowercase letters for files that are marked as assume unchanged (see git-update-index(1)).

-t This feature is semi-deprecated. For scripting purpose, git-status(1)–porcelain and git-diff-files(1)–name-status are almost always superior alternatives, and users should look at git-status(1)–short or git-diff(1)–name-status for more user-friendly alternatives.
This option identifies the file status with the following tags (followed by a space) at the start of each line:

An additional interesting parameter for git ls-files is
-i, --ignored Shows only ignored files in the output. When showing files in the index, it prints only those matched by an exclude pattern. When showing “other” files, it shows only those matched by an exclude pattern.

--exclude-standard Add the standard Git exclusions: .git/info/exclude, .gitignore in each directory, and the user’s global exclusion file.
From: man git-ls-files

Examples for git ls-files -i, –ignored and –exclude-standard

# Show files in the index that are ignored because of patterns in .gitignore
git ls-files --ignored --exclude-from=.gitignore;
# Show other (i.e. untracked) files that are ignored because of patterns in .gitignore
git ls-files --ignored --other --exclude-from=.gitignore;
# Show files in the index that are ignored because of patterns in any of the standard git exclusions.
git ls-files --ignored --exclude-standard;
# Show other (i.e. untracked) files that are ignored because of patterns in any of the standard git exclusions.
git ls-files --ignored --exclude-standard --other;


Alternative to sched_yield()

In case sched_yield() does not seem to work for you or it is not available, you can try calling usleep(0) as a workaround.

int sched_yield(void); from (#include <sched.h>) causes the calling thread to relinquish the CPU. The thread is moved to the end of the queue for its static priority and a new thread gets to run.
From: man 3 sched_yield

int usleep(useconds_t usec); from (#include <unistd.h>) suspends execution of the calling thread for (at least) usec microseconds. The sleep may be lengthened slightly by any system activity or by the time spent processing the call or by the granularity of system timers.
From: man 3 usleep

Notes

  • Do not use sleep(0) as a workaround as in some older versions of glibc it will not have any effect at all! In those older versions there is a check if the input parameter is set to 0 and if it is then it will do nothing at all. Specifically the code is as follows:
    if (seconds == 0)
     return 0;

    If you want to review the code changes in sleep.c or see how usleep.c is code, download this archive: All versions of sleep.c up to the 17th of November 2017 (12 downloads)
    It contains all versions of sleep.c up to today and the latest version of usleep.c.

  • If the calling thread is the only thread in the highest priority list at that time, it will continue to run after a call to sched_yield().
  • Strategic calls to sched_yield() can improve performance by giving other threads or processes a chance to run when (heavily) contended resources (e.g., mutexes) have been released by the caller. Avoid calling sched_yield() unnecessarily or inappropriately (e.g., when resources needed by other schedulable threads are still held by the caller), since doing so will result in unnecessary context switches, which will degrade system performance.

C/C++: Hack to use an array of 4 characters as one unsigned 32 bit integer and increment their value by one

In this scenario, we had 4 unsigned char variables.
The first variable was counting ‘something’.
The second variable was counting how many times the first variable reached its maximum value and then overflowed.
The third variable was counting how many times the second variable reached its maximum value and then overflowed.
Finally, the fourth variable was counting how many times the third variable reached its maximum value and then overflowed.

What we wanted to achieve was to perform this process without using several if statements that check when each variable overflowed and then perform the corrective actions.
The trick that we did to achieve the expected result works on the principle that an array in C/C++ always holds consecutive memory spaces.

What we did was cast the array pointer as a pointer of an unsigned int variable, this allowed us to operate on all 4 bytes at the same time as if they were one.
We can do the same for an array composed of two unsigned short integers.
This trick can of course work on 64bit variables and arrays that contain 8 unsigned char elements or 4 unsigned short elements or 2 unsigned int elements.

++(*((unsigned int *)&t));

Full Example

array-of-char-as-integer.c (compressed) (7 downloads)

#include <stdio.h>
#include <time.h>
#include <stdlib.h>

static inline void increment(unsigned char t[4]) {
  if (t[0] == 0xFF) {
    if (t[1] == 0xFF) {
      if (t[2] == 0xFF) {
        ++t[3];
      }
      ++t[2];
    }
    ++t[1];
  }
  ++t[0];
}

int main() {
  const unsigned int final_limit = (unsigned int) -1; //or 0xFFFFFFFF
  const unsigned int intermediate_limit = 0xABCDEF00;

  {
    unsigned char t[4] = {0, 0, 0, 0};
    printf("%02X %02X %02X %02X - Initial Values\n", t[3], t[2], t[1], t[0]);
    const clock_t start = clock();

    unsigned int i;
    for (i = 0; i < intermediate_limit; ++i) {
      ++(*((unsigned int *)&t));
    }

    printf("%02X %02X %02X %02X - Intermediate Values\n", t[3], t[2], t[1], t[0]);

    for (; i < final_limit; ++i) {
      ++(*((unsigned int *)&t));
    }

    const clock_t end = clock();
    const float seconds = (float) (end - start) / CLOCKS_PER_SEC;
    printf("%02X %02X %02X %02X - Final Values\n", t[3], t[2], t[1], t[0]);
    printf("Seconds elapsed %f\t by casting array of unsigned chars to an unsigned int\n", seconds);
  }

  {
    unsigned char t[4] = {0, 0, 0, 0};
    printf("%02X %02X %02X %02X - Initial Values\n", t[3], t[2], t[1], t[0]);
    const clock_t start = clock();

    unsigned int i;
    for (i = 0; i < intermediate_limit; ++i) {
      increment(t);
    }

    printf("%02X %02X %02X %02X - Intermediate Values\n", t[3], t[2], t[1], t[0]);

    for (; i < final_limit; ++i) {
      increment(t);
    }

    const clock_t end = clock();
    const float seconds = (float) (end - start) / CLOCKS_PER_SEC;
    printf("%02X %02X %02X %02X - Final Values\n", t[3], t[2], t[1], t[0]);
    printf("Seconds elapsed %f\t by using various if statements to control overflow\n", seconds);
  }

  return 0;
}

array-of-char-as-integer.c (compressed) (7 downloads)

Output

00 00 00 00 - Initial Values
AB CD EF 00 - Intermediate Values
FF FF FF FF - Final Values
Seconds elapsed 7.938144 by casting array of unsigned chars to an unsigned int
00 00 00 00 - Initial Values
AB CD EF 00 - Intermediate Values
FF FF FF FF - Final Values
Seconds elapsed 12.820061 by using various if statements to control overflow