Bash

NetCat (nc) as a webserver

 in Bash / GNU/Linux  tagged nc / NetCat / webserver by

Recently, we needed to perform some tests in a network. Specifically, we wanted to check the configuration of a firewall and see what IP are blocked and/or which ports are allowed to go through. To do so, we used NetCat to setup a small web-server to perform our tests.

Netcat (often abbreviated to nc) is a computer networking utility for reading from and writing to network connections using TCP or UDP. Netcat is designed to be a dependable back-end that can be used directly or easily driven by other programs and scripts. At the same time, it is a feature-rich network debugging and investigation tool, since it can produce almost any kind of connection its user could need and has a number of built-in capabilities.

From: https://en.wikipedia.org/wiki/Netcat


while true;
do
  echo -e "HTTP/1.1 200 OK\r\n\r\n<h1>$(hostname) is live</h1>$(date)" | nc -vl -p 5555;
done

or in one line


while true; do echo -e "HTTP/1.1 200 OK\r\n\r\n<h1>$(hostname) is live</h1>$(date)" | nc -vl -p 5555; done

Explanation of code:

  • The above code creates an infinite loop that calls nc in listening mode, we had to do this as nc will terminate as soon as it serves one client.
  • Using echo we create an html 200 response along with a small “webpage”.
  • While generating the webpage, echo -e will execute the commands hostname and date to get the current system values adding them to the resulted text.
  • The resulted text is then piped to nc to be served as a response to any incoming clients.
  • The date and time that nc will show to the client is not the current date and time when visiting the webpage but the one that was when echo was executed.

nc parameters:

  • -v, --verbose Sets the verbosity level and it can be used several times to increase it even further
  • -l, --listen Instructs nc to bind and listen for incoming connections (just like a web-server)
  • -p, --source-portwith port parameter specifies the source port to be used by nc

Custom terminator layout with multiple tabs and terminals

 in Bash / GNU/Linux  tagged alias / bash / bashrc / config / ssh / template / terminator by

The following terminator layout ([download id=”4658″]) opens 3 different tabs, the first two tabs contain only one terminal each and the third one has 4 terminals in a 2×2 matrix.
Each of these tabs have their own custom name set and following each terminal has its name set to make it easier for the user to recognize the purpose of each one.

[download id=”4658″]

After opening these terminals, the configuration file, contains specific commands to be executed by each terminal, allowing you to automate a some trivial part of your day to day operations.
In this example, each terminal will navigate to a specific project or connect via ssh to some server, then it will perform some operation like performing a git pull and finally it will preserve the connection for you by starting a new bash instance to continue using that terminal.

Feel free to edit the layout and create a custom configuration for your tabs / the terminals and the commands.

Installation / Usage

  1. Replace the config ([download id=”4658″]) file in your home user folder ~/.config/terminator/  with the one we provide
    (In nautilus press Ctrl+H to view hidden files and folders if you cannot find the .config folder)
  2. Open terminator and execute the following:
    terminator -l init;

If you want to create an alias for this command:
Open .bashrc file at your home user folder and add the following

alias my-init="terminator -l init"

For any new terminal in terminator, executing my-init will spawn a new window of terminator that has all the configuration from the file loaded into it.

Contents of [download id=”4658″]

[global_config]
[keybindings]
[layouts]
  [[default]]
    [[[child1]]]
      parent = window0
      type = Terminal
    [[[window0]]]
      parent = ""
      type = Window
  [[init]]
    [[[child0]]]
      fullscreen = False
      last_active_window = True
      maximised = True
      order = 0
      parent = ""
      position = 0:26
      size = 1918, 1002
      title = /bin/bash
      type = Window
    [[[child1]]]
      active_page = 0
      labels = www, MA, all other, dev logs, staging logs, live logs
      last_active_term = d3c317d7-964a-4625-96d0-39deb5166072, 93ce7874-059e-4794-b337-7b640654a3d6, db090e6f-07e4-431e-ad86-a8b6cb965b5e, 906a5f4d-a3af-4da8-8385-673b132e7edd, 1b48b3b9-216c-470b-be53-ec1e8c6fdc0b, cb7d737c-a064-4e0e-ad5e-59c47d7bdd3b
      order = 0
      parent = child0
      type = Notebook
    [[[child11]]]
      order = 3
      parent = child1
      position = 956
      ratio = 0.500261643119
      type = HPaned
    [[[child14]]]
      order = 4
      parent = child1
      position = 956
      ratio = 0.500261643119
      type = HPaned
    [[[child17]]]
      order = 5
      parent = child1
      position = 956
      ratio = 0.500261643119
      type = HPaned
    [[[child4]]]
      order = 2
      parent = child1
      position = 956
      ratio = 0.500261643119
      type = HPaned
    [[[child5]]]
      order = 0
      parent = child4
      position = 481
      ratio = 0.500520291363
      type = VPaned
    [[[child8]]]
      order = 1
      parent = child4
      position = 481
      ratio = 0.500520291363
      type = VPaned
    [[[terminal10]]]
      command = cd /vhosts/www.example.com/; git pull; bash
      order = 1
      parent = child8
      profile = default
      title = www.example.com
      type = Terminal
      uuid = db090e6f-07e4-431e-ad86-a8b6cb965b5e
    [[[terminal12]]]
      command = "ssh -t git 'cd vhosts/www.bytefreaks.net/ci_applications/registration_forms/logs/; ll; bash'"
      directory = ""
      order = 0
      parent = child11
      profile = default
      title = WWW dev logs
      type = Terminal
      uuid = 4c08356b-b516-4286-8b6d-ba071f1394f3
    [[[terminal13]]]
      command = "ssh -t git 'cd vhosts/my.bytefreaks.net/symfony/var/logs/; ll; bash'"
      directory = ""
      order = 1
      parent = child11
      profile = default
      title = MA dev logs
      type = Terminal
      uuid = 906a5f4d-a3af-4da8-8385-673b132e7edd
    [[[terminal15]]]
      command = "ssh -t web13 'cd /data/var/www/vhosts/staging-www.bytefreaks.net/htdocs/ci_applications/registration_forms/logs/; ls -la; bash'"
      order = 0
      parent = child14
      profile = default
      title = WWW staging logs
      type = Terminal
      uuid = 1b48b3b9-216c-470b-be53-ec1e8c6fdc0b
    [[[terminal16]]]
      command = "ssh -t web13 'cd /data/var/www/vhosts/staging-my.bytefreaks.net/htdocs/symfony/var/logs/; ls -la; bash'"
      order = 1
      parent = child14
      profile = default
      title = MA staging logs
      type = Terminal
      uuid = e26e94cd-855a-44ff-9c67-66b1c03bac56
    [[[terminal18]]]
      command = "ssh -t web13 'cd /data/var/www/vhosts/www.bytefreaks.net/htdocs/ci_applications/registration_forms/logs/; ls -la; bash'"
      directory = ""
      order = 0
      parent = child17
      profile = default
      title = WWW Live logs
      type = Terminal
      uuid = 70466609-2d01-45d2-84b4-e377b111e540
    [[[terminal19]]]
      command = "ssh -t web13 'cd /data/var/www/vhosts/my.bytefreaks.net/htdocs/symfony/var/logs/; ls -la; bash'"
      directory = ""
      order = 1
      parent = child17
      profile = default
      title = MA Live logs
      type = Terminal
      uuid = cb7d737c-a064-4e0e-ad5e-59c47d7bdd3b
    [[[terminal2]]]
      command = cd /vhosts/www.bytefreaks.net/; git pull; bash
      order = 0
      parent = child1
      profile = default
      title = www.bytefreaks.net
      type = Terminal
      uuid = d3c317d7-964a-4625-96d0-39deb5166072
    [[[terminal3]]]
      command = cd /vhosts/my.bytefreaks.net/; git pull; bash
      order = 1
      parent = child1
      profile = default
      title = my.bytefreaks.net
      type = Terminal
      uuid = 93ce7874-059e-4794-b337-7b640654a3d6
    [[[terminal6]]]
      command = cd /vhosts/www.michanicos.com/; git pull; bash
      order = 0
      parent = child5
      profile = default
      title = www.michanicos.com
      type = Terminal
      uuid = 2f204209-0c0b-4fab-b883-95f95f5d38e9
    [[[terminal7]]]
      command = cd /vhosts/www.etea.com.cy/; git pull; bash
      order = 1
      parent = child5
      profile = default
      title = www.etea.com.cy
      type = Terminal
      uuid = 6f801914-5225-4e1f-b54c-f48540274614
    [[[terminal9]]]
      command = cd /vhosts/www.ieee.org/; git pull; bash
      order = 0
      parent = child8
      profile = default
      title = www.ieee.org
      type = Terminal
      uuid = 3dbfe3a7-2e25-4e7d-bb02-dc4aeeeda47f
[plugins]
[profiles]
  [[default]]
    background_darkness = 0.8
    cursor_color = "#ffffff"
    foreground_color = "#ffffff"

How we create bootable GNU/Linux USB flash drives from terminal

 in Bash  tagged boot / bootable / fdisk / gnome-disks / live / liveUSB / lsblk by

A very important tool in our everyday life are the LiveUSB GNU/Linux flash drives.
We keep an updated collection of several GNU/Linux flavors/distributions (Fedora, CentOS, (L/X)Ubuntu, Kali etc.) that are used depending on the scenario.

The command we use is the following:


sudo dd bs=4M if=path/to/OS.iso of=/dev/sdX conv=fdatasync;

dd allows you to convert and copy a file and we use it to copy the ISO file of the operating system onto the USB flash drive.

Notes:

  1. You need to unmount the USB flash drive before formatting it, e.g.:
    sudo umount /dev/sdXY;
  2. You need to use the device filename and not a partition filename:
    e.g. You need to use /dev/sdX and NOT /dev/sdX1
  3. You need to use either the root account or execute the command with sudo
  4. If you do not know the filename associated with your flash drive, use an application like the following ones to determine which /dev file is mapped to the USB flash drive:
    gnome-disks; or
    lsblk; or
    sudo fdisk -l;

The parameters we use are the following:

  • bs=SIZE_IN_BYTES defines up to how many bytes should be read and written at a time.
    In our case we used 4 Megabytes (4M).
  • if=INPUT_FILE defines the file to be read, we use this parameter to point to the OS ISO file that we want to write on the USB drive.
  • of=OUTPUT_FILE defines the filename where the data is to be written in.
    In GNU/Linux, devices are accessible like files as well so we used /dev/sdX here that happened to be the device file assigned to our USB device.
  • conv=CONVS converts the file as per the comma separated symbol list
    fdatasync physically writes output file data before finishing, we use this parameter to be sure that all I/O operations are done well before dd terminates, this way we are certain that our USB device will be ready to use as soon as the application is done.

Change date in GoPro video files 1

 in Bash / GNU/Linux  tagged date / EXIF / exiftool / GoPro / metadata by

Recently, we made some time-lapses with our GoPro camera which was not used for some time and discharged. Due to that, the date and time configuration of the camera was wrong and the resulting video files had wrong metadata information in their EXIF.

To update the dates in the produced video files we used the exiftool that is shipped in the perl-Image-ExifTool package. We use Fedora, so we used dnf to install the application with all needed dependencies as follows:


sudo dnf install -y perl-Image-ExifTool;

After installing exiftool, we used the following commands to change the dates and times of the video named GOPR6426 (and its preview / thumbnail video):


$ mydate='2018:02:23 15:20:00';
$ exiftool "-AllDates=$mydate" "-TrackCreateDate=$mydate" "-TrackModifyDate=$mydate" "-MediaCreateDate=$mydate" "-MediaModifyDate=$mydate" -overwrite_original GOPR4626.*;

Notes:

  • You need extra disk space to update the file as it creates a copy of it even if you overwrite the original
  • We used the wildcard character * in our command to make sure we update not only the main video but also the preview / thumbnail videos that GoPro produces.
  • If you skip -overwrite_original, then a new file will be created
  • We had to update several date fields in order to be sure that we covered all of them that were visible in the EXIF data of the media

Link to a newer post that handles folders of pictures and video using the time difference between the real date and the camera date.