Yearly Archives: 2017


About restoring a deleted Git branch

Recently, a branch was deleted from the server without it being merged.
Luckily for us, we had a local copy.

We used used the command git reflog to get access to the reference logs of the branch.
The command returned results similar to the below:

271f0084 HEAD@{0}: pull: Merge made by the 'recursive' strategy.
 0e71c0b HEAD@{1}: commit: Minor change: Cracked NSA systems
 c81624b HEAD@{2}: pull: Fast-forward
 ef1f281 HEAD@{3}: commit (amend): Deployed satelite
 1ed1a5a HEAD@{4}: commit (amend): Deployed satelite
 a8682cb HEAD@{5}: commit (amend): Deployed satelite
 e4560c8 HEAD@{6}: commit (amend): Deployed satelite
 1679a90 HEAD@{7}: commit (amend): Deployed satelite
 d27d2c9 HEAD@{8}: commit: Deployed satelite
 aaf8261 HEAD@{9}: checkout: moving from master to crack-nsa-systems
 2500e11 HEAD@{10}: clone: from ssh://[email protected]:7999/secret/bananas.git

From this information we got the hash value in front of the commit which we wanted to use to restore, which was 271f0084.

Then, we checked out that version using

git checkout 271f0084;

When we tried to push the branch back to the server the Git pre commit hooks blocked the operation.
Based on the rsa key used, we could only submit changes that were committed by the owner of that key.
The option to disable temporarily the pre commit hooks was unfortunately out of the question.
So we had to replace all author names and emails with the name of the one holding the key.

To do so we used the following command before pushing to the server once more:

git filter-branch --commit-filter '
   if [ "$GIT_COMMITTER_NAME" = "Doe, John" ];
   then
     GIT_COMMITTER_NAME="Squarepants, Bob";
     GIT_AUTHOR_NAME="Squarepants, Bob";
     GIT_COMMITTER_EMAIL="[email protected]";
     GIT_AUTHOR_EMAIL="[email protected]";
     git commit-tree "$@";
   else
     git commit-tree "$@";
   fi' HEAD

git filter-branch lets you rewrite Git revision history by rewriting the branches mentioned, in our case it was HEAD, applying custom filters on each revision. Those filters can modify each tree (e.g. removing a file or running a perl rewrite on all files) or information about each commit. Otherwise, all information (including original commit times or merge information) will be preserved.

HEAD is a reference to the currently checked out commit. In normal states, it’s actually a symbolic reference to the branch you have checked out.
Looking at the contents of .git/HEAD you’ll see something similar to ref: refs/heads/master.
The branch itself is a reference to the commit at the tip of the branch.


Rough notes for DJI Phantom 2 Vision +

After you connect to the Wi-Fi hot spot of the device (SSID: Phantom_XXXXXX):

System 1

ssh [email protected]
#Password for root is 19881209
[email protected]'s password: 


BusyBox v1.19.4 (2013-04-22 22:41:57 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

 -----------------------------------------------------
 DJI-INNOVATIONS
 -----------------------------------------------------
  * FC200-Vision+        
  * [email protected]
  * 02/20/2014
  * Version 1.01
  * ART / DHCP down / ping
 -----------------------------------------------------



System is based on OpenWrt.

System 2

ssh [email protected]
#Password for root is 19881209
[email protected]'s password: 


BusyBox v1.19.4 (2013-04-22 22:41:57 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

 -----------------------------------------------------
 DJI-INNOVATIONS
 -----------------------------------------------------
  * FC200-Vision+ RE   
  * [email protected]
  * 02/21/2014
  * Version 1.01
  * ART / DHCP down / ping
 -----------------------------------------------------

ssh -L 8080:downloads.openwrt.org:80 [email protected]

cat /etc/opkg.conf 
src/gz barrier_breaker http://downloads.openwrt.org/snapshots/trunk/ar71xx/packages
dest root /
dest ram /tmp
lists_dir ext /var/opkg-lists
option overlay_root /overlay

cat /etc/opkg.conf 
src/gz barrier_breaker http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base
dest root /
dest ram /tmp
lists_dir ext /var/opkg-lists
option overlay_root /overlay
option http_proxy http://127.0.0.1:8080/

root@Phantom:~# opkg update
Downloading http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base/Packages.gz.
Updated list of available packages in /var/opkg-lists/barrier_breaker.
root@Phantom:~# opkg install tcpdump-mini
Installing tcpdump-mini (4.5.1-4) to root...
Downloading http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base/tcpdump-mini_4.5.1-4_ar71xx.ipk.
Installing libpcap (1.5.3-1) to root...
Downloading http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base/libpcap_1.5.3-1_ar71xx.ipk.
Configuring libpcap.
Configuring tcpdump-mini.




BAD TRUNC
opkg install tcpdump-mini
Installing tcpdump-mini (4.5.1-4) to root...
Downloading http://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/base/tcpdump-mini_4.5.1-4_ar71xx.ipk.
Installing libpcap (1.7.4-1) to root...
Downloading http://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/base/libpcap_1.7.4-1_ar71xx.ipk.
Configuring libpcap.
//usr/lib/opkg/info/libpcap.postinst: line 4: default_postinst: not found
Configuring tcpdump-mini.
//usr/lib/opkg/info/tcpdump-mini.postinst: line 4: default_postinst: not found
Collected errors:
 * pkg_run_script: package "libpcap" postinst script returned status 127.
 * opkg_configure: libpcap.postinst returned 127.
 * pkg_run_script: package "tcpdump-mini" postinst script returned status 127.
 * opkg_configure: tcpdump-mini.postinst returned 127.


From host machine:
ssh [email protected] tcpdump -i br-lan -s0 -w - port not 22 > dji-wifi-traffic.pcap

ssh [email protected] tcpdump -n -i br-lan -s0 -w - port 2001 > dji.2001.pcap


ip.src==192.168.1.1 || ip.dst==192.168.1.1 ||ip.src==192.168.1.2 || ip.dst==192.168.1.2 || ip.src==192.168.1.10 || ip.dst==192.168.1.10

System is based on OpenWrt.


C Bit Fields: Full example code

The following two examples demonstrate the use of bit fields to reduce memory consumption of certain applications.

In the first example we create a compressed ‘bit’ struct and on the second we create a weird struct representation for bytes to show that the size of that struct is significantly less that the original one.

First example: Using bit fields to create a ‘bit’ structure

[download id=”2693″]

#include <stdio.h>

// Weird structure to represent a 'bit'
typedef struct {
    unsigned char value;
} bit;

// 'bit' structure using bit fields
typedef struct {
    unsigned char value : 1;
} bit_bit_field;

int main( ) {

    printf( "Memory size occupied by 'bit' struct : %zu bytes\n", sizeof(bit));
    printf( "Memory size occupied by 'bit_bit_field' struct : %zu bytes\n", sizeof(bit_bit_field));

    bit bits[8];
    bit_bit_field bits_bit_field[8];

    printf( "Memory size occupied by 'bits' array : %zu bytes\n", sizeof(bits));
    printf( "Memory size occupied by 'bits_bit_field' array : %zu bytes\n", sizeof(bits_bit_field));

    // Setting the value of the first 'bit' and then printing it.
    // We will use various values for this test to show that when you set a value to a bit field
    // that is greater that the allowed size it will fill it using the last bits only.
    // It will not spill data though to neighbouring 'bits'.
    unsigned char value;
    for (value = 0; value < 4; value++)
    {
        printf("Input Value: %d\n", value);
        int bits_i;
        const int bits_length = (sizeof(bits) / sizeof(bit));
        for (bits_i = 0; bits_i < bits_length; bits_i++)
        {
            if (bits_i % 2)
            {
                bits[bits_i].value = 0;
            }
            else
            {
                bits[bits_i].value = value;
            }
            printf("%d", bits[bits_i].value);
        }
        printf("\n");

        int bits_bit_field_i;
        const int bits_bit_field_length = (sizeof(bits_bit_field) / sizeof(bit_bit_field));
        for (bits_bit_field_i = 0; bits_bit_field_i < bits_bit_field_length; bits_bit_field_i++)
        {
            if (bits_bit_field_i % 2)
            {
                bits_bit_field[bits_bit_field_i].value = 0;
            }
            else
            {
                bits_bit_field[bits_bit_field_i].value = value;
            }
            printf("%d", bits_bit_field[bits_bit_field_i].value);
        }
        printf("\n");
    }
    return 0;
}

[download id=”2693″]

Execution output

Memory size occupied by 'bit' struct : 1 bytes
Memory size occupied by 'bit_bit_field' struct : 1 bytes
Memory size occupied by 'bits' array : 8 bytes
Memory size occupied by 'bits_bit_field' array : 8 bytes
Input Value: 0
00000000
00000000
Input Value: 1
10101010
10101010
Input Value: 2
20202020
00000000
Input Value: 3
30303030
10101010

Second example: Using bit fields to create a ‘byte’ structure where each ‘bit’ is another named member

[download id=”2692″]

#include <stdio.h>

// Weird structure to represent a 'byte'
typedef struct {
    unsigned char bit_0;
    unsigned char bit_1;
    unsigned char bit_2;
    unsigned char bit_3;
    unsigned char bit_4;
    unsigned char bit_5;
    unsigned char bit_6;
    unsigned char bit_7;
} byte;

// 'byte' structure using bit fields
// Unfortunately we cannot declare an array where the values are bit fields,
// so we have to declare each member separately.
// We instruct the compiler to use only one bit per element.
typedef struct {
    unsigned char bit_0 : 1;
    unsigned char bit_1 : 1;
    unsigned char bit_2 : 1;
    unsigned char bit_3 : 1;
    unsigned char bit_4 : 1;
    unsigned char bit_5 : 1;
    unsigned char bit_6 : 1;
    unsigned char bit_7 : 1;
} byte_bit_field;

int main( ) {

    printf( "Memory size occupied by 'byte' struct : %zu bytes\n", sizeof(byte));
    printf( "Memory size occupied by 'byte_bit_field' struct : %zu bytes\n", sizeof(byte_bit_field));

    byte bytes[8];
    byte_bit_field bytes_bit_field[8];

    printf( "Memory size occupied by 'bytes' array : %zu bytes\n", sizeof(bytes));
    printf( "Memory size occupied by 'bytes_bit_field' array : %zu bytes\n", sizeof(bytes_bit_field));

    // Setting the value of the first 'bit' and then printing it.
    // We will use various values for this test to show that when you set a value to a bit field
    // that is greater that the allowed size it will fill it using the last bits only.
    // It will not spill data though to neighbouring 'bits'.
    unsigned char value;
    for (value = 0; value < 4; value++)
    {
        printf("Input Value: %d\n", value);
        int bytes_i;
        const int bytes_length = (sizeof(bytes) / sizeof(byte));
        for (bytes_i = 0; bytes_i < bytes_length; bytes_i++)
        {
            if (bytes_i % 2)
            {
                bytes[bytes_i].bit_3 = 0;
            }
            else
            {
                bytes[bytes_i].bit_3 = value;
            }
            printf(" %d  ", bytes[bytes_i].bit_3);
        }
        printf("\n");

        int bytes_bit_field_i;
        const int bytes_bit_field_length = (sizeof(bytes_bit_field) / sizeof(byte_bit_field));
        for (bytes_bit_field_i = 0; bytes_bit_field_i < bytes_bit_field_length; bytes_bit_field_i++)
        {
            if (bytes_bit_field_i % 2)
            {
                bytes_bit_field[bytes_bit_field_i].bit_3 = 0;
            }
            else
            {
                bytes_bit_field[bytes_bit_field_i].bit_3 = value;
            }
            printf("%d%d%d ",
                   bytes_bit_field[bytes_bit_field_i].bit_2,
                   bytes_bit_field[bytes_bit_field_i].bit_3,
                   bytes_bit_field[bytes_bit_field_i].bit_4);
        }
        printf("\n");
    }
    return 0;
}

[download id=”2692″]

Execution output

Memory size occupied by 'byte' struct : 8 bytes
Memory size occupied by 'byte_bit_field' struct : 1 bytes
Memory size occupied by 'bytes' array : 64 bytes
Memory size occupied by 'bytes_bit_field' array : 8 bytes
Input Value: 0
 0   0   0   0   0   0   0   0  
000 000 000 000 000 000 000 000 
Input Value: 1
 1   0   1   0   1   0   1   0  
010 000 010 000 010 000 010 000 
Input Value: 2
 2   0   2   0   2   0   2   0  
000 000 000 000 000 000 000 000 
Input Value: 3
 3   0   3   0   3   0   3   0  
010 000 010 000 010 000 010 000

grep: How to match lines using any of multiple patterns

Recently, we needed to filter the results of ps x using two different patterns.
The first pattern was ./ where we needed to match that exact character sequence.
The . period character is treated as a special character in regular expressions (it matches a single character of any value, except for the end of line), so we decided to use the -F parameter to remove this special handling.
Doing this change prevented us from writing a regular expression that uses the OR | operator.

-F (or --fixed-strings) is a matching control option that instructs grep to interpret the patterns as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched.
We tried assigning the different patterns as different lines to a variable and then using them on the pipe, like in the following example:

patterns="./
banana";
ps x | grep -F $patterns;

..but it failed.

Solution

grep supports a matching control option -e that allows us to define multiple patterns using different strings.

-e PATTERN (or --regexp=PATTERN) uses the value PATTERN as the pattern. If this option is used  multiple times or it is combined with the -f (--file) option, grep will search for all patterns given.

In the end, our command was transformed to the following, which worked just fine!

ps x | grep -F -e "./" -e "banana";