GNU/Linux


Rough notes on setting up an Ubuntu server with docker

Static IP

First, we set up a static IP to our Ubuntu server using netplan. To do so, we created the following file:

/etc/netplan/01-netcfg.yaml

using the following command

sudo nano /etc/netplan/01-netcfg.yaml;

and added the following content to it:

# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0f0:
      dhcp4: no
      addresses: [192.168.45.13/24]
      gateway4: 192.168.45.1
      nameservers:
          addresses: [1.1.1.1,8.8.8.8]

To apply the changes, we executed the following:

sudo netplan apply;

Update everything (the operating system and all packages)

Usually, it is a good idea to update your system before making significant changes to it:

sudo apt update -y; sudo apt upgrade -y; sudo apt autoremove -y;

Install docker

In this setup we did not use the docker version that is available on the Ubuntu repositories, we went for the official ones from docker.com. To install it, we used the following commands:

sudo apt-get install ca-certificates curl gnupg lsb-release;
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg;
echo   "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null;
sudo apt-get update;
sudo apt-get install docker-ce docker-ce-cli containerd.io;

Install docker-compose

Again, we installed the official docker-compose from github.com instead of the one available in the Ubuntu repositories. At the time that this post was created, version 1.29.2 was the recommended one:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose;
sudo chmod +x /usr/local/bin/docker-compose;

Increase network pool for docker daemon

To handle the following problem:

ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

We created the following file,

/etc/docker/daemon.json

using the command:

sudo nano /etc/docker/daemon.json;

and added the following content to it:

{
  "default-address-pools": [
    {
      "base": "172.80.0.0/16",
      "size": 24
    },
    {
      "base": "172.90.0.0/16",
      "size": 24
    }
  ]
}

We executed the following command to restart the docker daemon and get the network changes applied:

sudo systemctl restart docker;

Gave access to our user to manage docker

We added our user to the docker group so that we could manage the docker daemon without sudo rights.

sudo usermod -aG docker $USER;

Deploying

After we copied everything in place, we executed the following command to create our containers and start them with the appropriate networks and volumes:

export COMPOSE_HTTP_TIMEOUT=120;
docker-compose up -d --remove-orphans;

We had to increase the timeout as we were getting the following error:

ERROR: for container_a  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

Stopping all containers using a filter on the name

docker container stop $(docker container ls -q --filter name=_web);

The above command will find all containers whose names contain _web and stop them. That command is actually two commands where one is nested inside the other.

#This command finds all containers that their name contains _web, using the -q parameter, we only get back the container ID and not all information about them.
docker container ls -q --filter name=_web;
#The second command takes as input the output of the nested command and stops all containers that are returned.
docker container stop $(docker container ls -q --filter name=_web);

413 Request Entity Too Large

We tried to upload a large file to a WordPress site and got the following error:

413 Request Entity Too Large

The WordPress installation was behind an Nginx reverse proxy.

To fix this, we added the following line in the /etc/nginx/nginx.conf configuration file inside the http section/context:

client_max_body_size 64M;
http {
    ...

    client_max_body_size 64M;

    ...
}

Syntax: client_max_body_size size;
When client_max_body_size is not set, it defaults to the value of one megabyte;
It can be set to any of the three following contexts: http, server, location
client_max_body_size sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, the 413 (Request Entity Too Large) error is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.

Source: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size

After making the change to the configuration file, we restarted Nginx to apply the changes.


ImageMagick apply blur to photo using a black and white mask

Recently, we were trying to apply blurriness to the frames of a video using a custom mask. Our needs would not be short of describing using geometric shapes, so we created the following image (blur.png) as a template for the blurring effect:

The above mask applies a blur effect to all black pixels and leaves all white pixels in the original image intact.

The command that we used was the following:

convert "${FILE}" -mask blur.png -blur 0x8 +mask "blur/${FILE}";

This command creates a new copy of the input file and places it into the folder named blur, so be sure to make the folder before using the above command (e.g., using the command mkdir blur).

Parameters and other information

  • -mask this flag assosiates the filename that is given with the mask of the command.
  • -blur defines the geometry that is used reduce image noise and reduce detail levels.
    To increase the blurriness you can increase the number in this variable 0x8.
  • +mask The ‘plus’ form of the operator +mask removes the mask from the input image.

The version of convert that we used for this example was the following:

Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC

Below is a result frame from a video that we processed:

Additional material

To apply it to all video frames in the folder, we used the following command to make our life easier:

find . -maxdepth 1 -type f -name "*.ppm" -exec bash -c 'FILE="$1"; convert "${FILE}" -mask blur.png -blur 0x8 +mask "blur/${FILE}";' _ '{}' \;

The above command finds all frames in the current folder and executes the convert command described above. Since FFmpeg names the frames as PPM, we used that to filter our search. The blur folder is in the same folder as the original images. To avoid processing the pictures in that folder again, we defined the -maxdepth parameter in find that prevents it from navigating into child folders of the one we are working in.


How to encrypt data using the PGP Public Key of an organization/entity

We used this batch of notes to encrypt email communication between us and the https://www.offensive-security.com/ website contact. Precisely, we needed to encrypt some email attachments with sensitive data.

First of all, we tried to get their PGP Public Key from https://www.offensive-security.com/registrar.asc using curl.

curl -O https://www.offensive-security.com/registrar.asc;

We soon realized that the data were binary because their webserver or CDN compressed the response.

$ file registrar.asc 
registrar.asc: gzip compressed data, from Unix, original size modulo 2^32 7487

So we modified our curl command to decompress the response automatically:

curl --compressed -O https://www.offensive-security.com/registrar.asc;

After receiving the plaintext version of the registrar.asc file, we were able to proceed with the encryption steps. The first thing we did was to import their key:

gpg --import registrar.asc;
$ gpg --import registrar.asc 
gpg: key 6C12FFD0BFCBFAE2: 9 signatures not checked due to missing keys
gpg: key 6C12FFD0BFCBFAE2: public key "Offensive Security (Offensive Security Registrar) <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: next trustdb check due at 2023-12-13

Using the following command, we were able to encrypt the sensitive data and send them to via mail:

gpg --recipient [email protected] --encrypt sensitive.mp4;

The PGP command automatically used the public key that we imported in the previous step to perform the encryption. PGP named the encrypted file sensitive.mp4.gpg. We only needed to send that file, and the corresponding party had all other information to decrypt it.

Bonus: Create our own public Key so that people can contact you with encryption

gpg --gen-key;

Executing the above command asked us to provide a name, an email, and a password to encrypt the data. Below is the sample output generated for us:

$ gpg --gen-key
gpg (GnuPG) 2.2.19; Copyright (C) 2019 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Note: Use "gpg --full-generate-key" for a full featured key generation dialog.

GnuPG needs to construct a user ID to identify your key.

Real name: John Doe
Email address: [email protected]
You selected this USER-ID:
    "John Doe <[email protected]>"

Change (N)ame, (E)mail, or (O)kay/(Q)uit? O
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key A53FEA7768D67D2A marked as ultimately trusted
gpg: revocation certificate stored as '/home/john/.gnupg/openpgp-revocs.d/D1660B83341AEF2852A2A4C6A53FEA7768D67D2A.rev'
public and secret key created and signed.

pub   rsa3072 2021-12-13 [SC] [expires: 2023-12-13]
      D1660B83341AEF2852A2A4C6A53FEA7768D67D2A
uid                      John Doe <[email protected]>
sub   rsa3072 2021-12-13 [E] [expires: 2023-12-13]

Then, we exported our public key using the command below.

gpg --export --armor --output john.asc [email protected];

Sending this file to other people or putting it on a public key server allows people to encrypt data just for you to read.