GNU/Linux


How to access VMFS Datastore from Ubuntu GNU/Linux

Suppose the ESXi host fails, but the server’s local disk or disks are still operational. In that case, it is always possible to copy the virtual machine files (both data drives and configuration files) from the VMFS datastore and run the VM on a different server. This is true even if the ESXi host fails (even on VMware Workstation or Hyper-V). The most significant issue is that the widely used operating systems, such as Windows and Linux, do not have a VMFS driver, which causes them to be unable to recognize a partition that automatically has the VMFS file system.

To mount a VMFS file system on an Ubuntu, we will need to install the vmfs-tools package.

sudo apt-get install vmfs-tools;

Then, we need to create a folder where we will perform the mount later on:

# The folders does not have to be in the /mnt path, it can be anywhere on your file system where you have access.
sudo mkdir /mnt/vmfs;

Following that, we need to identify the disk we want to mount. There are two popular ways to do so, and the first is by executing the command fdisk -l on the terminal, which will show all physical disks attached to your system. You will get results that are similar to the ones below:

sudo fdisk -l;

...

Disk /dev/loop51: 884,85 GiB, 950075898880 bytes, 1855616990 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

From these results, the drive’s path is essential information for us. In this case, it was /dev/loop51.

The second method is to use the Gnome Disks Utility:

When you start the application, you will get an image similar to this:

If you have a physical VMFS datastore hard drive, it will appear on the list on the left. You will get more information on the right panel by clicking on it. The critical information is the value after the label Device. In this case, the vital value was /dev/loop51.

If you do not have a physical drive but an image of a drive, you can attach it by clicking on the menu button with the three lines on the top left and then selecting the Attach Disk Image... (.iso, .img) option. A new window will open, allowing you to navigate and find your image file.

After we acquire the path to the physical drive or the image file, we can mount it using the following command:

sudo vmfs-fuse /dev/loop51 /mnt/vmfs;

In case you get the following error:

VMFS: Unsupported version 6
Unable to open device/file "/dev/loop51".
Unable to open filesystem

Then, you need to install the package that can handle VMFS version 6. To install, use the following command:

sudo apt-get install vmfs6-tools;

Trying again to mount, this time with the tools that are appropriate for version 6, should do the trick:

sudo vmfs6-fuse /dev/loop51 /mnt/vmfs;

To unmount, we need to execute the following:

sudo umount /mnt/vmfs;

GNU Linux/Bash: A function that splits a word in half

The following function takes one argument – a text file.
The text file should contain one word on each line.
The function reads the text file (argument) line by line.
Then it checks if the line has one word; if this is true, it splits the word in half.
Finally, it prints the two new words with a space between them.

#!/bin/bash

splitWordsInHalf () {
  # This function takes one argument - a text file.
  # The text file contains one word on each line.
  # It reads the text file (argument) line by line.
  # Then it checks if the line contains one word, if this is true, it splits the word in half.
  # Finally, it prints the two new words with a space between them
  while read line
  do
    words=( $line )
    if [ ${#words[@]} == 1 ]
    then
      echo ${line:0:${#line}/2} ${line:${#line}/2}
    fi
  done < $1
}

splitWordsInHalf input.txt

Example

Using the following input file:

banana
apple
ball
car
door

We will get the following output when we execute splitWordsInHalf input.txt:

ban ana
ap ple
ba ll
c ar
do or

Notes

The following parts of the code are in charge of looping on the data of the incoming file. The parameter (the input file) given to the function is translated into the variable $1. The while loop gets one line of text on each iteration and assigns the text to the variable that is named line. You could have chosen any other name that suits you instead of the word line.

splitWordsInHalf () {
  while read line
  do
    ...
  done < $1
}

The next part of the code (words=( $line )) converts the string value that is contained in the line variable into an array of words, and it assigns that array to the variable named words. Then, it counts the number of elements in the array (the number of words in the line) using the following ${#words[@]} and it checks that there is only one item.

words=( $line )
if [ ${#words[@]} == 1 ]
then
  ...
fi

The following line will print two strings. The first string is a sub-string of variable line that is composed by the first half of the value. The second sub-string is the second half of the value contained in the variable named line.

echo ${line:0:${#line}/2} ${line:${#line}/2}

The ${#line} will return the length of the string contained in the variable.

The structure ${VARIABLE:START:END} defines the slice of the string that we want returned.


ewf-tools and Ubuntu 1

Recently, we installed the ewf-tools package from the Ubuntu repositories:

sudo apt-get install ewf-tools;

When we tried to use it, we got the following errors:

ewfmount ./DISK.E01 /tmp/disk/
ewfmount 20140807

Unable to open source image(s)
libcdata_internal_array_resize: invalid entries size value exceeds maximum.
libcdata_array_resize: unable to resize array.
libmfdata_list_resize: unable to resize elements array.
libewf_segment_file_read_volume_section: unable to resize chunk table list.
libewf_handle_open_read_segment_files: unable to read section: volume.
libewf_handle_open_file_io_pool: unable to read segment files.
libewf_handle_open: unable to open handle using a file IO pool.
mount_handle_open: unable to open file(s).

To fix the issue, we uninstalled ewf-tools then installed the following packages:

sudo apt remove ewf-tools;
sudo apt-get install libfuse-dev libfuse2 uuid-dev lbzip2 python3-wchartype;
sudo apt-get install ewf-tools;

Finally, we reinstalled ewf-tools , and this time they worked!

Note

We also downloaded the latest version from the repository, built the code, and tried to use that package with the same result. The code from the repository had the same problem, which worked after we installed the packages mentioned above. For this reason, we believe the problem is not a matter of the version but rather a matter of configuration and dependencies.


Rough notes on setting up an Ubuntu server with docker

Static IP

First, we set up a static IP to our Ubuntu server using netplan. To do so, we created the following file:

/etc/netplan/01-netcfg.yaml

using the following command

sudo nano /etc/netplan/01-netcfg.yaml;

and added the following content to it:

# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0f0:
      dhcp4: no
      addresses: [192.168.45.13/24]
      gateway4: 192.168.45.1
      nameservers:
          addresses: [1.1.1.1,8.8.8.8]

To apply the changes, we executed the following:

sudo netplan apply;

Update everything (the operating system and all packages)

Usually, it is a good idea to update your system before making significant changes to it:

sudo apt update -y; sudo apt upgrade -y; sudo apt autoremove -y;

Install docker

In this setup we did not use the docker version that is available on the Ubuntu repositories, we went for the official ones from docker.com. To install it, we used the following commands:

sudo apt-get install ca-certificates curl gnupg lsb-release;
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg;
echo   "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null;
sudo apt-get update;
sudo apt-get install docker-ce docker-ce-cli containerd.io;

Install docker-compose

Again, we installed the official docker-compose from github.com instead of the one available in the Ubuntu repositories. At the time that this post was created, version 1.29.2 was the recommended one:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose;
sudo chmod +x /usr/local/bin/docker-compose;

Increase network pool for docker daemon

To handle the following problem:

ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

We created the following file,

/etc/docker/daemon.json

using the command:

sudo nano /etc/docker/daemon.json;

and added the following content to it:

{
  "default-address-pools": [
    {
      "base": "172.80.0.0/16",
      "size": 24
    },
    {
      "base": "172.90.0.0/16",
      "size": 24
    }
  ]
}

We executed the following command to restart the docker daemon and get the network changes applied:

sudo systemctl restart docker;

Gave access to our user to manage docker

We added our user to the docker group so that we could manage the docker daemon without sudo rights.

sudo usermod -aG docker $USER;

Deploying

After we copied everything in place, we executed the following command to create our containers and start them with the appropriate networks and volumes:

export COMPOSE_HTTP_TIMEOUT=120;
docker-compose up -d --remove-orphans;

We had to increase the timeout as we were getting the following error:

ERROR: for container_a  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

Stopping all containers using a filter on the name

docker container stop $(docker container ls -q --filter name=_web);

The above command will find all containers whose names contain _web and stop them. That command is actually two commands where one is nested inside the other.

#This command finds all containers that their name contains _web, using the -q parameter, we only get back the container ID and not all information about them.
docker container ls -q --filter name=_web;
#The second command takes as input the output of the nested command and stops all containers that are returned.
docker container stop $(docker container ls -q --filter name=_web);