Monthly Archives: November 2017

19 posts

How to process tcpdump live data stream from a remote machine on a local WireShark

Recently we needed to process the results of a tcpdump command using the GUI version of WireShark on machine that did not have a window manager installed. That device was an embedded device, for which it did not make sense to even consider installing a window manager on it. So, in order to process the results of the tcpdump command we decided to use another machine that had a full working window manager installed and was able to operate the GUI version of WireShark.

For our solution to work some requirements were expected to be met by the embedded device (a.k.a. remote machine).

  1. tcpdump was installed on the remote machine
  2. ssh server was installed on the remote machine and allowed us to connect to it remotely
  3. there was a user that had remote ssh rights on the remote machine that also had the rights to execute tcpdump on the needed interfaces

Synopsis of our solution:

Just execute the following on the machine with the GUI (a.k.a. local machine)

mkfifo /tmp/board;
wireshark -k -i /tmp/board &
ssh [email protected] "tcpdump -s 0 -U -n -w - -i lo not port 22" > /tmp/board;

Explanation of our solution:

Following are the steps that we performed on the local machine to pipe the results of tcpdump on the remote machine on the wireshark on the local machine.

  1. First we created a named pipe as follows:
    mkfifo /tmp/board;
    You can name your pipe anyway you like and place it in any folder you wish. We used /tmp as our pipe is a temporary construct that we do not care to preserve across time/restarts.
  2. Then we started wireshark from a terminal so that we could pass as capture interface the named pipe we just created using the -i /tmp/board parameter. The -k parameter instructs wireshark to start the capture session immediately.
    wireshark -k -i /tmp/board &
    Since this operation was going to execute for a long time, we sent it to the background to release the terminal for further use by placing the & symbol at the end of the command.
  3. Finally, we started tcpdump over ssh on a board and redirected its output to our named pipe.
    ssh [email protected] "tcpdump -s 0 -U -n -w - -i lo not port 22" > /tmp/board;
    The parameters we used on tcpdump have the following effects:
    -s 0 instructs tcpdump to set the snapshot length of data from each packet to the default value of 262144 bytes.
    -U Since the -w option is not specified, make the printed packet output packet-buffered. Which means that it will print the description of the contents of each packet without waiting for the output buffer to get full.
    -n Does not convert host addresses to names. This can be used to avoid DNS lookups.
    -w - Write the raw packets to Standard Output rather than parsing them.
    -i lo Defines which interface to listen on. We wanted the loopback interface to listen to everything.
    not port 22 Since we used ssh to start this command, we do not want to listen to the data that we produce as well and flood the inputs.

 

Enable C++11 standard for GCC on Eclipse CDT

When using Eclipse CDT to write C++, we noticed that it did not enable by default the C++11 standard. Following the steps below, we added the -std=c++11 flag on the GCC C++ Compiler command line arguments enabling the standard for our use.

  1. From the main window of Eclipse, on the list on the left, where your projects are listed, right click on your project and then click Properties from the new menu
  2. In the new window, navigate from the list on the left and expand the C/C++ Build option to view its children and then click the Settings item
  3. In the middle of the window, you will see a new list, expand (if needed) the item GCC C++ Compiler and click on the Miscellaneous child
  4. On the right, a text box named Other Flags will appear, append -std=c++11 to the list of tokens in the box as seen in the image below
  5. Click on the Apply button for the effects to take place and then the OK button to close the properties window

Next time you compile, the -std=c++11 flag will be present on your compiler command line and the C++11 standard will be used.

Google Hash Code 2018 Nicosia Cyprus – Call for participation

We’ll be hosting a hub at the University of Cyprus for the Online Qualification Round of Hash Code, a team-based programming competition created by Google for university students and industry professionals. The Online Qualification Round takes place on the 1st of March at 19:30 EET and registered teams from Cyprus are invited to participate from our hub, which will take place at the Computer Science Department. Top scoring teams from the Online Qualification Round will then be invited to Google’s Paris office to compete in the Final Round of the competition in April.

If you’re interested in joining our hub, find a team (two to four people) and register at g.co/hashcode. Make sure to select University of Cyprus from the list of hubs in the Judge System.

For more information about this and other hubs in Cyprus (including the twin event in Limassol) visit https://goo.gl/uuRspx

Hash Code 2018 Nicosia Cyprus – Facebook Event

Thanks!

Address:

Rooms: 101, 102, 103
Department of Computer Science,
Pure and Applied Sciences (FST-01)
University of Cyprus
1 University Avenue
2109 Aglantzia, CYPRUS

Date and Time:

1st March 2018
From: 19:30 EET
To: 23:30 EET

Free Amenities Offered

High speed Internet access
Wi-Fi access to the Internet for your mobile devices (personal computers and smart phones)
Lab computers will be available for use by the participants
Food in the form of snacks and beverages will be available outside the labs

Google Hash Code 2018 – Online Qualification Round Schedule

18:30 EET:

  • The hub will open to the public
  • People can view the live stream on the video projector
  • Teams can set themselves up with the help of the volunteers

19:30 EET:

  • Live stream starts

19:45 EET:

  • Task will be made available, competition starts
  • Scoreboard will be displayed on the video projector
  • Participating teams will be confirmed in the Judge System

23:30 EET:

  • End of the competition
  • Announcement of the score for the local teams

00:00 EET:

  • The hub will close

Start Arduino IDE as root on Fedora / Allow root to start an X application

Solution

Execute the following as a normal user

xhost +si:localuser:root;
sudo ./arduino;

Background Story and More Information

Recently we needed to start the official Arduino IDE as root on Fedora to allow the application to take control of the serial port.
We were getting the following error whenever we tried to upload the application to the board:

processing.app.debug.RunnerException
 at cc.arduino.packages.uploaders.SerialUploader.uploadUsingPreferences(SerialUploader.java:160)
 at cc.arduino.UploaderUtils.upload(UploaderUtils.java:78)
 at processing.app.SketchController.upload(SketchController.java:713)
 at processing.app.SketchController.exportApplet(SketchController.java:686)
 at processing.app.Editor$DefaultExportHandler.run(Editor.java:2168)
 at java.lang.Thread.run(Thread.java:748)
Caused by: processing.app.SerialException: Error touching serial port '/dev/ttyACM0'.
 at processing.app.Serial.touchForCDCReset(Serial.java:107)
 at cc.arduino.packages.uploaders.SerialUploader.uploadUsingPreferences(SerialUploader.java:144)
 ... 5 more
Caused by: jssc.SerialPortException: Port name - /dev/ttyACM0; Method name - openPort(); Exception type - Permission denied.
 at jssc.SerialPort.openPort(SerialPort.java:170)
 at processing.app.Serial.touchForCDCReset(Serial.java:101)
 ... 6 more

So, we tried to start the arduino IDE using root and got another error:

[[email protected] bin]$ sudo ./arduino;
[sudo] password for george: 
No protocol specified
Picked up JAVA_TOOL_OPTIONS: 
No protocol specified
java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
	at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
	at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65)
	at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:115)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.awt.X11GraphicsEnvironment.(X11GraphicsEnvironment.java:74)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at java.awt.GraphicsEnvironment.createGE(GraphicsEnvironment.java:103)
	at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:82)
	at sun.awt.X11.XToolkit.(XToolkit.java:126)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at java.awt.Toolkit$2.run(Toolkit.java:860)
	at java.awt.Toolkit$2.run(Toolkit.java:855)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:854)
	at java.awt.SystemColor.updateSystemColors(SystemColor.java:473)
	at java.awt.SystemColor.(SystemColor.java:465)
	at processing.app.Theme.init(Theme.java:84)
	at processing.app.Base.(Base.java:219)
	at processing.app.Base.main(Base.java:144)

This error occurred because the default configuration of the X server permissions did not allow the root to connect to it.
To verify this, we used xhost (the X server access control program) to check the permissions.
Executing xhost with no command line arguments gave us a message indicating whether or not access control was currently enabled, followed by the list of those users allowed to connect.
For example in our case the output was as follows:

[[email protected] bin]$ xhost
access control enabled, only authorized clients can connect
SI:localuser:george

To add root to the list of users that was allowed to start an X application we executed the following command:

[[email protected] bin]$ xhost +si:localuser:root
localuser:root being added to access control list

Executing xhost again, we got the updated list which included the root

[[email protected] bin]$ xhost
access control enabled, only authorized clients can connect
SI:localuser:root
SI:localuser:george

After this, we were able to start arduino IDE using sudo with no problems.

[[email protected] bin]$ sudo ./arduino;

Note: This patch is not permanent, we actually execute it once at every restart of the machine.

Manually set the CMake output folder

If you want to manually set the global output folder for you whole CMake project and depending on the output you expect add the following configuration lines in the root CMakeLists.txt file of your project:

set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)

In case you wan to specify those folders per target, you can update them as follows:

set_target_properties( target_or_targets
  PROPERTIES
  ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
  LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
  RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin"
)

Please note that we are setting the same properties using different variables.

The CMAKE_ARCHIVE_OUTPUT_DIRECTORY variable is used to initialize the ARCHIVE_OUTPUT_DIRECTORY property on all the targets.
ARCHIVE_OUTPUT_DIRECTORY property specifies the directory into which archive target files should be built.
An archive output artifact of a buildsystem target may be:

  • The static library file (e.g. .lib or .a) of a static library target created by the add_library() command with the STATIC option.
  • On DLL platforms: the import library file (e.g. .lib) of a shared library target created by the add_library() command with the SHARED option.
  • On DLL platforms: the import library file (e.g. .lib) of an executable target created by the add_executable() command when its ENABLE_EXPORTS target property is set.

 

The CMAKE_LIBRARY_OUTPUT_DIRECTORY variable is used to initialize the LIBRARY_OUTPUT_DIRECTORY property on all the targets.
LIBRARY_OUTPUT_DIRECTORY property specifies the directory into which library target files should be built.
A library output artifact of a buildsystem target may be:
The loadable module file (e.g. .dll or .so) of a module library target created by the add_library() command with the MODULE option.
On non-DLL platforms: the shared library file (e.g. .so or .dylib) of a shared shared library target created by the add_library() command with the SHARED option.

 

The CMAKE_RUNTIME_OUTPUT_DIRECTORY variable is used to initialize the RUNTIME_OUTPUT_DIRECTORY property on all the targets.
RUNTIME_OUTPUT_DIRECTORY property specifies the directory into which runtime target files should be built.
A runtime output artifact of a buildsystem target may be:

  • The executable file (e.g. .exe) of an executable target created by the add_executable() command.
  • On DLL platforms: the executable file (e.g. .dll) of a shared library target created by the add_library() command with the SHARED option.

From: https://cmake.org/documentation/

Two ways to append a new argument to CMAKE_ARGS list for ExternalProject_Add

Recently, we wanted to pass a new cached value to an external project in CMake via the CMAKE_ARGS variable.

CMAKE_ARGS holds various types of arguments. From which, the arguments in the form -Dname:type=value are passed to the CMake command line and cannot be changed by the user.

We found two ways to add a new argument pair to the CMAKE_ARGS of the external project.

The first method uses the set function:

set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR}${CACHE_VAR_TYPE}=\"${${CACHE_VAR}}\"")

where basically we create a new string that starts with the existing value of CMAKE_ARGS and then we append to its end the new pair.

The second method uses the list function:

list(APPEND CMAKE_ARGS "-D${CACHE_VAR}${CACHE_VAR_TYPE}=${${CACHE_VAR}}")

that treats CMAKE_ARGS as a list and it appends to its end the new pair.

To verify by hand that they were equivalent we did the following small test with success

#Copy the original value of ${CMAKE_ARGS}
set(CMAKE_ARGS_ORIGINAL ${CMAKE_ARGS})

#Define the Name, Type and Value for the new argument pairs
set(CACHE_VAR_P_NAME PRODUCTS_DIR)
set(CACHE_VAR_P_TYPE string)
set(CACHE_VAR_P_VALUE "someplace/with spaces")

set(CACHE_VAR_C_NAME CLIENTS_DIR)
set(CACHE_VAR_C_TYPE string)
set(CACHE_VAR_C_VALUE "another/place")

#Print the original value of ${CMAKE_ARGS}
MESSAGE(STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Append the two pairs to ${CMAKE_ARGS} using set
set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR_P_NAME}:${CACHE_VAR_P_TYPE}=\"${CACHE_VAR_P_VALUE}\"")
set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR_C_NAME}:${CACHE_VAR_C_TYPE}=\"${CACHE_VAR_C_VALUE}\"")

#Print the modified value of ${CMAKE_ARGS}
MESSAGE(STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Reset ${CMAKE_ARGS} to its original value
set(CMAKE_ARGS ${CMAKE_ARGS_ORIGINAL})

#Print the original value of ${CMAKE_ARGS} for verification
MESSAGE( STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Append the two pairs to ${CMAKE_ARGS} using list
list(APPEND CMAKE_ARGS "-D${CACHE_VAR_P_NAME}:${CACHE_VAR_P_TYPE}=\"${CACHE_VAR_P_VALUE}\"")
list(APPEND CMAKE_ARGS "-D${CACHE_VAR_C_NAME}:${CACHE_VAR_C_TYPE}=\"${CACHE_VAR_C_VALUE}\"")

#Print the modified value of ${CMAKE_ARGS}
#Notice that here we surrounded ${CMAKE_ARGS} with quotes so that it is printed as list of delimiter separated values
#Between each element the character ';' will be added because ${CMAKE_ARGS} is surrounded with quotes
MESSAGE(STATUS "CMAKE_ARGS: '" "${CMAKE_ARGS}" "'")

Which resulted in the following output

-- CMAKE_ARGS: ''
-- CMAKE_ARGS: ' -DPRODUCTS_DIR:string="someplace/with spaces" -DCLIENTS_DIR:string="another/place"'
-- CMAKE_ARGS: ''
-- CMAKE_ARGS: '-DPRODUCTS_DIR:string="someplace/with spaces";-DCLIENTS_DIR:string="another/place"'

From the output we can see that the two options treat the variable in a different way as using set just created a huge string containing both pairs while list created a list of two elements.
Both methods seem to work properly but we chose the list method for our external project.

The CMakeLists.txt file that we used to include the external project using custom CMAKE_ARGS resulted as follows:

cmake_minimum_required (VERSION 3.2.2)
include (ExternalProject)

set (TARGET c-banana-eat-banana)
project (${TARGET} C)

set (BANANA_ROOT ${CMAKE_CURRENT_BINARY_DIR} CACHE INTERNAL "")

#Add custom CMake arguments to be passed to the CMake command line of the external project
list(APPEND CMAKE_ARGS "-DARTIFACTS_DIR:string=\"${LIBRARIES_DIR}/${TARGET}\"")

ExternalProject_Add (${TARGET}
 URL ${CMAKE_CURRENT_SOURCE_DIR}/${TARGET}.zip
 URL URL_HASH MD5=34734a678729967f426931d913326112
 CMAKE_ARGS "${CMAKE_ARGS}"
 BUILD_IN_SOURCE 1)

JavaSript: Remove all non printable and all non ASCII characters from text

According to the ASCII character encoding, there are 95 printable characters in total.
Those characters are in the range [0x20 to 0x7E] ([32 to 126] in decimal) and they represent letters, digits, punctuation marks, and a few miscellaneous symbols.
Character 0x20 (or 32 in decimal) is the space character ' ' and
character 0x7E (or 126 in decimal) is the tilde character '~'.
Source: https://en.wikipedia.org/wiki/ASCII#Printable_characters

Since all the printable characters of ASCII are conveniently in one continuous range, we used the following to filter all other characters out of our string in JavaScript.

printable_ASCII_only_string = input_string.replace(/[^ -~]+/g, "");

What the above code does is that it passes the input string through a regular expression which will match all characters out of the printable range and replace them with nothing (hence, delete them).
In case you do not like writing your regular expression with the space character to it, you can re-write the above regular expression using the hex values of the two characters as follows:

printable_ASCII_only_string = input_string.replace(/[^\x20-\x7E]+/g, "");

How to find lines that contain only lowercase characters

To print all lines that contain only lower case characters, we used the following regular expression in grep:

egrep '^[[:lower:]]+$' <file>;
#If you do not have egrep, use
grep -e '^[[:lower:]]+$' <file>;

Breakdown of the above regular expression:

  • ^ instructs the regular expression parser that the pattern should always start with the beginning of the line
  • [[:lower:]] this special instruction informs us that only lower case characters can match it
  • + the plus sign causes the preceding token to be matched one or more times
  • $ signifies the end of the line