Daily Archives: 22 February 2017


About restoring a deleted Git branch

Recently, a branch was deleted from the server without it being merged.
Luckily for us, we had a local copy.

We used used the command git reflog to get access to the reference logs of the branch.
The command returned results similar to the below:

271f0084 [email protected]{0}: pull: Merge made by the 'recursive' strategy.
 0e71c0b [email protected]{1}: commit: Minor change: Cracked NSA systems
 c81624b [email protected]{2}: pull: Fast-forward
 ef1f281 [email protected]{3}: commit (amend): Deployed satelite
 1ed1a5a [email protected]{4}: commit (amend): Deployed satelite
 a8682cb [email protected]{5}: commit (amend): Deployed satelite
 e4560c8 [email protected]{6}: commit (amend): Deployed satelite
 1679a90 [email protected]{7}: commit (amend): Deployed satelite
 d27d2c9 [email protected]{8}: commit: Deployed satelite
 aaf8261 [email protected]{9}: checkout: moving from master to crack-nsa-systems
 2500e11 [email protected]{10}: clone: from ssh://[email protected]:7999/secret/bananas.git

From this information we got the hash value in front of the commit which we wanted to use to restore, which was 271f0084.

Then, we checked out that version using

git checkout 271f0084;

When we tried to push the branch back to the server the Git pre commit hooks blocked the operation.
Based on the rsa key used, we could only submit changes that were committed by the owner of that key.
The option to disable temporarily the pre commit hooks was unfortunately out of the question.
So we had to replace all author names and emails with the name of the one holding the key.

To do so we used the following command before pushing to the server once more:

git filter-branch --commit-filter '
   if [ "$GIT_COMMITTER_NAME" = "Doe, John" ];
   then
     GIT_COMMITTER_NAME="Squarepants, Bob";
     GIT_AUTHOR_NAME="Squarepants, Bob";
     GIT_COMMITTER_EMAIL="[email protected]";
     GIT_AUTHOR_EMAIL="[email protected]";
     git commit-tree "[email protected]";
   else
     git commit-tree "[email protected]";
   fi' HEAD

git filter-branch lets you rewrite Git revision history by rewriting the branches mentioned, in our case it was HEAD, applying custom filters on each revision. Those filters can modify each tree (e.g. removing a file or running a perl rewrite on all files) or information about each commit. Otherwise, all information (including original commit times or merge information) will be preserved.

HEAD is a reference to the currently checked out commit. In normal states, it’s actually a symbolic reference to the branch you have checked out.
Looking at the contents of .git/HEAD you’ll see something similar to ref: refs/heads/master.
The branch itself is a reference to the commit at the tip of the branch.


Rough notes for DJI Phantom 2 Vision +

After you connect to the Wi-Fi hot spot of the device (SSID: Phantom_XXXXXX):

System 1

ssh [email protected]
#Password for root is 19881209
[email protected]'s password: 


BusyBox v1.19.4 (2013-04-22 22:41:57 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

 -----------------------------------------------------
 DJI-INNOVATIONS
 -----------------------------------------------------
  * FC200-Vision+        
  * [email protected]
  * 02/20/2014
  * Version 1.01
  * ART / DHCP down / ping
 -----------------------------------------------------



System is based on OpenWrt.

System 2

ssh [email protected]
#Password for root is 19881209
[email protected]'s password: 


BusyBox v1.19.4 (2013-04-22 22:41:57 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

 -----------------------------------------------------
 DJI-INNOVATIONS
 -----------------------------------------------------
  * FC200-Vision+ RE   
  * [email protected]
  * 02/21/2014
  * Version 1.01
  * ART / DHCP down / ping
 -----------------------------------------------------

ssh -L 8080:downloads.openwrt.org:80 [email protected]

cat /etc/opkg.conf 
src/gz barrier_breaker http://downloads.openwrt.org/snapshots/trunk/ar71xx/packages
dest root /
dest ram /tmp
lists_dir ext /var/opkg-lists
option overlay_root /overlay

cat /etc/opkg.conf 
src/gz barrier_breaker http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base
dest root /
dest ram /tmp
lists_dir ext /var/opkg-lists
option overlay_root /overlay
option http_proxy http://127.0.0.1:8080/

[email protected]:~# opkg update
Downloading http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base/Packages.gz.
Updated list of available packages in /var/opkg-lists/barrier_breaker.
[email protected]:~# opkg install tcpdump-mini
Installing tcpdump-mini (4.5.1-4) to root...
Downloading http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base/tcpdump-mini_4.5.1-4_ar71xx.ipk.
Installing libpcap (1.5.3-1) to root...
Downloading http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base/libpcap_1.5.3-1_ar71xx.ipk.
Configuring libpcap.
Configuring tcpdump-mini.




BAD TRUNC
opkg install tcpdump-mini
Installing tcpdump-mini (4.5.1-4) to root...
Downloading http://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/base/tcpdump-mini_4.5.1-4_ar71xx.ipk.
Installing libpcap (1.7.4-1) to root...
Downloading http://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/base/libpcap_1.7.4-1_ar71xx.ipk.
Configuring libpcap.
//usr/lib/opkg/info/libpcap.postinst: line 4: default_postinst: not found
Configuring tcpdump-mini.
//usr/lib/opkg/info/tcpdump-mini.postinst: line 4: default_postinst: not found
Collected errors:
 * pkg_run_script: package "libpcap" postinst script returned status 127.
 * opkg_configure: libpcap.postinst returned 127.
 * pkg_run_script: package "tcpdump-mini" postinst script returned status 127.
 * opkg_configure: tcpdump-mini.postinst returned 127.


From host machine:
ssh [email protected] tcpdump -i br-lan -s0 -w - port not 22 > dji-wifi-traffic.pcap

ssh [email protected] tcpdump -n -i br-lan -s0 -w - port 2001 > dji.2001.pcap


ip.src==192.168.1.1 || ip.dst==192.168.1.1 ||ip.src==192.168.1.2 || ip.dst==192.168.1.2 || ip.src==192.168.1.10 || ip.dst==192.168.1.10

System is based on OpenWrt.


C Bit Fields: Full example code

The following two examples demonstrate the use of bit fields to reduce memory consumption of certain applications.

In the first example we create a compressed ‘bit’ struct and on the second we create a weird struct representation for bytes to show that the size of that struct is significantly less that the original one.

First example: Using bit fields to create a ‘bit’ structure

bits_bit_fields.c (compressed) (193 downloads)
#include <stdio.h>

// Weird structure to represent a 'bit'
typedef struct {
    unsigned char value;
} bit;

// 'bit' structure using bit fields
typedef struct {
    unsigned char value : 1;
} bit_bit_field;

int main( ) {

    printf( "Memory size occupied by 'bit' struct : %zu bytes\n", sizeof(bit));
    printf( "Memory size occupied by 'bit_bit_field' struct : %zu bytes\n", sizeof(bit_bit_field));

    bit bits[8];
    bit_bit_field bits_bit_field[8];

    printf( "Memory size occupied by 'bits' array : %zu bytes\n", sizeof(bits));
    printf( "Memory size occupied by 'bits_bit_field' array : %zu bytes\n", sizeof(bits_bit_field));

    // Setting the value of the first 'bit' and then printing it.
    // We will use various values for this test to show that when you set a value to a bit field
    // that is greater that the allowed size it will fill it using the last bits only.
    // It will not spill data though to neighbouring 'bits'.
    unsigned char value;
    for (value = 0; value < 4; value++)
    {
        printf("Input Value: %d\n", value);
        int bits_i;
        const int bits_length = (sizeof(bits) / sizeof(bit));
        for (bits_i = 0; bits_i < bits_length; bits_i++)
        {
            if (bits_i % 2)
            {
                bits[bits_i].value = 0;
            }
            else
            {
                bits[bits_i].value = value;
            }
            printf("%d", bits[bits_i].value);
        }
        printf("\n");

        int bits_bit_field_i;
        const int bits_bit_field_length = (sizeof(bits_bit_field) / sizeof(bit_bit_field));
        for (bits_bit_field_i = 0; bits_bit_field_i < bits_bit_field_length; bits_bit_field_i++)
        {
            if (bits_bit_field_i % 2)
            {
                bits_bit_field[bits_bit_field_i].value = 0;
            }
            else
            {
                bits_bit_field[bits_bit_field_i].value = value;
            }
            printf("%d", bits_bit_field[bits_bit_field_i].value);
        }
        printf("\n");
    }
    return 0;
}
bits_bit_fields.c (compressed) (193 downloads)

Execution output

Memory size occupied by 'bit' struct : 1 bytes
Memory size occupied by 'bit_bit_field' struct : 1 bytes
Memory size occupied by 'bits' array : 8 bytes
Memory size occupied by 'bits_bit_field' array : 8 bytes
Input Value: 0
00000000
00000000
Input Value: 1
10101010
10101010
Input Value: 2
20202020
00000000
Input Value: 3
30303030
10101010

Second example: Using bit fields to create a ‘byte’ structure where each ‘bit’ is another named member

bytes_bit_fields.c (compressed) (186 downloads)
#include <stdio.h>

// Weird structure to represent a 'byte'
typedef struct {
    unsigned char bit_0;
    unsigned char bit_1;
    unsigned char bit_2;
    unsigned char bit_3;
    unsigned char bit_4;
    unsigned char bit_5;
    unsigned char bit_6;
    unsigned char bit_7;
} byte;

// 'byte' structure using bit fields
// Unfortunately we cannot declare an array where the values are bit fields,
// so we have to declare each member separately.
// We instruct the compiler to use only one bit per element.
typedef struct {
    unsigned char bit_0 : 1;
    unsigned char bit_1 : 1;
    unsigned char bit_2 : 1;
    unsigned char bit_3 : 1;
    unsigned char bit_4 : 1;
    unsigned char bit_5 : 1;
    unsigned char bit_6 : 1;
    unsigned char bit_7 : 1;
} byte_bit_field;

int main( ) {

    printf( "Memory size occupied by 'byte' struct : %zu bytes\n", sizeof(byte));
    printf( "Memory size occupied by 'byte_bit_field' struct : %zu bytes\n", sizeof(byte_bit_field));

    byte bytes[8];
    byte_bit_field bytes_bit_field[8];

    printf( "Memory size occupied by 'bytes' array : %zu bytes\n", sizeof(bytes));
    printf( "Memory size occupied by 'bytes_bit_field' array : %zu bytes\n", sizeof(bytes_bit_field));

    // Setting the value of the first 'bit' and then printing it.
    // We will use various values for this test to show that when you set a value to a bit field
    // that is greater that the allowed size it will fill it using the last bits only.
    // It will not spill data though to neighbouring 'bits'.
    unsigned char value;
    for (value = 0; value < 4; value++)
    {
        printf("Input Value: %d\n", value);
        int bytes_i;
        const int bytes_length = (sizeof(bytes) / sizeof(byte));
        for (bytes_i = 0; bytes_i < bytes_length; bytes_i++)
        {
            if (bytes_i % 2)
            {
                bytes[bytes_i].bit_3 = 0;
            }
            else
            {
                bytes[bytes_i].bit_3 = value;
            }
            printf(" %d  ", bytes[bytes_i].bit_3);
        }
        printf("\n");

        int bytes_bit_field_i;
        const int bytes_bit_field_length = (sizeof(bytes_bit_field) / sizeof(byte_bit_field));
        for (bytes_bit_field_i = 0; bytes_bit_field_i < bytes_bit_field_length; bytes_bit_field_i++)
        {
            if (bytes_bit_field_i % 2)
            {
                bytes_bit_field[bytes_bit_field_i].bit_3 = 0;
            }
            else
            {
                bytes_bit_field[bytes_bit_field_i].bit_3 = value;
            }
            printf("%d%d%d ",
                   bytes_bit_field[bytes_bit_field_i].bit_2,
                   bytes_bit_field[bytes_bit_field_i].bit_3,
                   bytes_bit_field[bytes_bit_field_i].bit_4);
        }
        printf("\n");
    }
    return 0;
}
bytes_bit_fields.c (compressed) (186 downloads)

Execution output

Memory size occupied by 'byte' struct : 8 bytes
Memory size occupied by 'byte_bit_field' struct : 1 bytes
Memory size occupied by 'bytes' array : 64 bytes
Memory size occupied by 'bytes_bit_field' array : 8 bytes
Input Value: 0
 0   0   0   0   0   0   0   0  
000 000 000 000 000 000 000 000 
Input Value: 1
 1   0   1   0   1   0   1   0  
010 000 010 000 010 000 010 000 
Input Value: 2
 2   0   2   0   2   0   2   0  
000 000 000 000 000 000 000 000 
Input Value: 3
 3   0   3   0   3   0   3   0  
010 000 010 000 010 000 010 000

grep: How to match lines using any of multiple patterns

Recently, we needed to filter the results of ps x using two different patterns.
The first pattern was ./ where we needed to match that exact character sequence.
The . period character is treated as a special character in regular expressions (it matches a single character of any value, except for the end of line), so we decided to use the -F parameter to remove this special handling.
Doing this change prevented us from writing a regular expression that uses the OR | operator.

-F (or --fixed-strings) is a matching control option that instructs grep to interpret the patterns as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched.
We tried assigning the different patterns as different lines to a variable and then using them on the pipe, like in the following example:

patterns="./
banana";
ps x | grep -F $patterns;

..but it failed.

Solution

grep supports a matching control option -e that allows us to define multiple patterns using different strings.

-e PATTERN (or --regexp=PATTERN) uses the value PATTERN as the pattern. If this option is used  multiple times or it is combined with the -f (--file) option, grep will search for all patterns given.

In the end, our command was transformed to the following, which worked just fine!

ps x | grep -F -e "./" -e "banana";

A small note on how we use valgrind

valgrind is a suite of tools for debugging and profiling programs.
We use it for debugging and profiling Linux executable files.

Despite the fact that it can do a whole lot of stuff, usually we use it as follows (when we do not forget) to test our applications for memory leaks:

valgrind --show-leak-kinds=all --leak-check=full $application $application_arguments;

The options we chose are the following:

  • --show-leak-kinds=all It will show all leaks (definite, indirect, possible, reachable) in the full leak search (see next bullet)
  • --leak-check=full this option instructs valgrind to search for memory leaks when the client program finishes.
    Each individual leak will be shown in detail and be counted as an error.

These options are extremely useful as they will catch a lot of little leaks that you might have missed (e.g. closing a file, freeing some memory, …)