GNU/Linux

ewf-tools and Ubuntu 1

 in Applications / GNU/Linux  tagged bash / e01 / ewf / ewf-tools / ewfmount / forensic / GNU/Linux / ubuntu by

Recently, we installed the ewf-tools package from the Ubuntu repositories:

sudo apt-get install ewf-tools;

When we tried to use it, we got the following errors:

ewfmount ./DISK.E01 /tmp/disk/
ewfmount 20140807

Unable to open source image(s)
libcdata_internal_array_resize: invalid entries size value exceeds maximum.
libcdata_array_resize: unable to resize array.
libmfdata_list_resize: unable to resize elements array.
libewf_segment_file_read_volume_section: unable to resize chunk table list.
libewf_handle_open_read_segment_files: unable to read section: volume.
libewf_handle_open_file_io_pool: unable to read segment files.
libewf_handle_open: unable to open handle using a file IO pool.
mount_handle_open: unable to open file(s).

To fix the issue, we uninstalled ewf-tools then installed the following packages:

sudo apt remove ewf-tools;
sudo apt-get install libfuse-dev libfuse2 uuid-dev lbzip2 python3-wchartype;
sudo apt-get install ewf-tools;

Finally, we reinstalled ewf-tools , and this time they worked!

Note

We also downloaded the latest version from the repository, built the code, and tried to use that package with the same result. The code from the repository had the same problem, which worked after we installed the packages mentioned above. For this reason, we believe the problem is not a matter of the version but rather a matter of configuration and dependencies.

Rough notes on setting up an Ubuntu server with docker

 in Docker / GNU/Linux  tagged bash / container / docker / docker-compose / GNU/Linux / IP / netcfg / netplan / static / static ip / systemctl / ubuntu / usermod by

Static IP

First, we set up a static IP to our Ubuntu server using netplan. To do so, we created the following file:

/etc/netplan/01-netcfg.yaml

using the following command

sudo nano /etc/netplan/01-netcfg.yaml;

and added the following content to it:

# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0f0:
      dhcp4: no
      addresses: [192.168.45.13/24]
      gateway4: 192.168.45.1
      nameservers:
          addresses: [1.1.1.1,8.8.8.8]

To apply the changes, we executed the following:

sudo netplan apply;

Update everything (the operating system and all packages)

Usually, it is a good idea to update your system before making significant changes to it:

sudo apt update -y; sudo apt upgrade -y; sudo apt autoremove -y;

Install docker

In this setup we did not use the docker version that is available on the Ubuntu repositories, we went for the official ones from docker.com. To install it, we used the following commands:

sudo apt-get install ca-certificates curl gnupg lsb-release;
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg;
echo   "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null;
sudo apt-get update;
sudo apt-get install docker-ce docker-ce-cli containerd.io;

Install docker-compose

Again, we installed the official docker-compose from github.com instead of the one available in the Ubuntu repositories. At the time that this post was created, version 1.29.2 was the recommended one:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose;
sudo chmod +x /usr/local/bin/docker-compose;

Increase network pool for docker daemon

To handle the following problem:

ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

We created the following file,

/etc/docker/daemon.json

using the command:

sudo nano /etc/docker/daemon.json;

and added the following content to it:

{
  "default-address-pools": [
    {
      "base": "172.80.0.0/16",
      "size": 24
    },
    {
      "base": "172.90.0.0/16",
      "size": 24
    }
  ]
}

We executed the following command to restart the docker daemon and get the network changes applied:

sudo systemctl restart docker;

Gave access to our user to manage docker

We added our user to the docker group so that we could manage the docker daemon without sudo rights.

sudo usermod -aG docker $USER;

Deploying

After we copied everything in place, we executed the following command to create our containers and start them with the appropriate networks and volumes:

export COMPOSE_HTTP_TIMEOUT=120;
docker-compose up -d --remove-orphans;

We had to increase the timeout as we were getting the following error: 

ERROR: for container_a  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

Stopping all containers using a filter on the name

docker container stop $(docker container ls -q --filter name=_web);

The above command will find all containers whose names contain _web and stop them. That command is actually two commands where one is nested inside the other.

#This command finds all containers that their name contains _web, using the -q parameter, we only get back the container ID and not all information about them.
docker container ls -q --filter name=_web;
#The second command takes as input the output of the nested command and stops all containers that are returned.
docker container stop $(docker container ls -q --filter name=_web);

413 Request Entity Too Large

 in GNU/Linux  tagged 413 / client_max_body_size / nginx / nginx.conf / reverse proxy / wordpress by

We tried to upload a large file to a WordPress site and got the following error:

413 Request Entity Too Large

The WordPress installation was behind an Nginx reverse proxy.

To fix this, we added the following line in the /etc/nginx/nginx.conf configuration file inside the http section/context:

client_max_body_size 64M;
http {
    ...

    client_max_body_size 64M;

    ...
}

Syntax: client_max_body_size size;
When client_max_body_size is not set, it defaults to the value of one megabyte;
It can be set to any of the three following contexts: http, server, location
client_max_body_size sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, the 413 (Request Entity Too Large) error is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.

Source: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size

After making the change to the configuration file, we restarted Nginx to apply the changes.

ImageMagick apply blur to photo using a black and white mask

 in Bash / GNU/Linux / ImageMagick  tagged blur / blurred / convert / exec / filter / find / imagemagick / mask / maxdepth / PPM by

Recently, we were trying to apply blurriness to the frames of a video using a custom mask. Our needs would not be short of describing using geometric shapes, so we created the following image (blur.png) as a template for the blurring effect:

The above mask applies a blur effect to all black pixels and leaves all white pixels in the original image intact.

The command that we used was the following:

convert "${FILE}" -mask blur.png -blur 0x8 +mask "blur/${FILE}";

This command creates a new copy of the input file and places it into the folder named blur, so be sure to make the folder before using the above command (e.g., using the command mkdir blur).

Parameters and other information

  • -mask this flag assosiates the filename that is given with the mask of the command.
  • -blur defines the geometry that is used reduce image noise and reduce detail levels.
    To increase the blurriness you can increase the number in this variable 0x8.
  • +mask The ‘plus’ form of the operator +mask removes the mask from the input image.

The version of convert that we used for this example was the following:

Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC

Below is a result frame from a video that we processed:

Additional material

To apply it to all video frames in the folder, we used the following command to make our life easier:

find . -maxdepth 1 -type f -name "*.ppm" -exec bash -c 'FILE="$1"; convert "${FILE}" -mask blur.png -blur 0x8 +mask "blur/${FILE}";' _ '{}' \;

The above command finds all frames in the current folder and executes the convert command described above. Since FFmpeg names the frames as PPM, we used that to filter our search. The blur folder is in the same folder as the original images. To avoid processing the pictures in that folder again, we defined the -maxdepth parameter in find that prevents it from navigating into child folders of the one we are working in.