lvm


Extend LVM space to the rest of the free space on the disk

Recently, we formatted a server with Ubuntu 22.04 LTS. While selecting the disk settings, we selected the encrypted LVM partition scheme, and even though we selected the whole disk, we did not notice that the LVM would only allocate, by default, 100GB out of the 600GB available on the raid volume.

So, we proceeded with the installation, and at some point, we noticed that we ran out of space which should not have happened.

Using the command df -h we quickly spotted the problem:

$ df -h
Filesystem                 Size  Used Avail Use% Mounted on
tmpfs                      3,2G  3,9M  3,2G   1% /run
/dev/mapper/vgubuntu-root  100G   83G   17G  83% /
tmpfs                       16G   40M   16G   1% /dev/shm
tmpfs                      5,0M  4,0K  5,0M   1% /run/lock
/dev/sda5                  703M  257M  395M  40% /boot
/dev/sda1                  511M   24K  511M   1% /boot/efi
tmpfs                       16G     0   16G   0% /run/qemu
tmpfs                      3,2G  156K  3,2G   1% /run/user/1000

/dev/mapper/vgubuntu-root was only 100GB instead of the 600GB that we would expect it to be.

Using the command vgdisplay we verified that the space allocated to the logical volume group was not what we wanted.

To fix the problem, we issued the following commands:

lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv;
resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv;

lvextend instructed our logical volume to consume all the available space on the hosting disk.

Then resize2fs allocated all the available space to our partition.


How to mount a qcow2 disk image that does not contain an Ubuntu LVM installation

Mounting a qcow2 disk image on your host server can be accomplished with the help of this fast method. Thanks to this feature, it is possible to reset passwords, alter files, or recover data even while the virtual machine is not running. This specific method does not allow mounting disks with LVM as they are not properly recognized the volume group tools (e.g. vgdisplay).

Enable Network block device (NBD) module on the host

sudo modprobe nbd max_part=8;

Network block device, or NBD, is a protocol on Linux that the OS can use to forward a block device (usually a hard disk or partition) from one system to another. This can be accomplished by sending the block device over the network.
For instance, a hard disk drive attached to another computer may be accessed by a local machine that is part of the same network.

Connect the QCOW2 image as a network block device

sudo qemu-nbd --connect=/dev/nbd0 /var/lib/libvirt/images/miner.qcow2;
#Use QEMU Disk Network Block Device Utility

We used the above command to export the QEMU disk image (miner.qcow2) using the NBD protocol and connect it to the NBD device (/dev/nbd0).

Identify the available partitions.

Check if the device has a UUID of an LVM partition in the QCOW2 image

sudo lsblk -f /dev/nbd0;

The lsblk command will provide information about all available block devices or the ones you choose. To obtain information, the lsblk command reads the sysfs filesystem and the udev db. It then attempts to read LABELs, UUIDs, and filesystem types from the block device if the udev db is unavailable, or if lsblk was compiled without udev support. In this particular scenario, root rights are required. Sample output can be seen below:

NAME         FSTYPE      FSVER    LABEL UUID                                   FSAVAIL FSUSE% MOUNTPOINTS
nbd0                                                                                          
├─nbd0p1                                                                                      
├─nbd0p2                                                                                      
└─nbd0p3     LVM2_member LVM2 001       xniXr3-gWWj-xS0J-8TaT-EtDt-vZtR-92Z5ms                
  └─ubuntu--vg-ubuntu--lv
             ext4        1.0            be0a2dba-ac27-4dfd-9f90-60ae9196d5e6

Identify the virtual machine partitions

fdisk /dev/nbd0 -l;

fdisk is a program that is driven by dialog to create and manipulate partition tables. It can read GPT, MBR, Sun, SGI, and BSD partition tables. If no devices are specified, the operating system will use the devices listed in /proc/partitions (provided that this file already exists). Devices are always displayed in the order that they are specified on the command line or in the order that they are listed by the kernel in /proc/partitions, whichever comes first.

Mount the partition of the virtual machine

After you identify the partition that you need to mount, use the mount command to perform the action to a mounting point of your choosing.

#In this example, we assume that we want to mount nbd0p1 to /mnt/miner that we created.
sudo mkdir /mnt/miner/;
sudo mount /dev/nbd0p1 /mnt/miner/;

Upon successful execution, all the files of that partition will be available through our mounting point. If you try to mount an LVM partition, you will get the following error:

sudo mount /dev/nbd0p3 /mnt/miner/
mount: /mnt/miner: unknown filesystem type 'LVM2_member'.

In this tutorial, we do not handle this problem using this method. See below how we handled it using the guestfish tool.

Clean Up

After you are done, unmount, disconnect, and remove the NBD module if you do not plan on using it further.

#Unmount the partition
umount /mnt/miner/;
#Disconnect the image from the NBD device
qemu-nbd --disconnect /dev/nbd0;
#Unload the NBD module
rmmod nbd;

How to mount a qcow2 disk image that contains an Ubuntu LVM installation

In one case, we had an issue where we needed to mount a disk image of a VM that contained an LVN installation. The above solution did not work, as we could not access the LVM partitions properly. The volume group tools did not recognize the partitions as they were network block devices. To handle this scenario, we used guestfish.

Examining and altering the filesystems of virtual machines is possible with the help of the shell and command-line tool known as Guestfish. It uses libguestfs and makes all of the features of the guestfs API available. So, we installed guestfish straight from the repositories as follows:

sudo apt-get install guestfish;

Then, we connected to the image that contained the LVM installation as follows:

sudo guestfish --rw -a /var/lib/libvirt/images/miner.qcow2;

After connecting to the image, we executed the following:

  • run
    With run, we initiated the library and attached the disk image
  • list-filesystems
    We listed the file systems found by libguestfs
  • mount
    After identifying the partition we needed to mount, we used this command to assign it to the root path /
  • ls
    This command works as expected, we were able to list the files in various directories, etc.
  • edit
    We used edit to modify the file we needed to process
  • exit
    We used exit to terminate this session

Below is a sample example of our execution.

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: ‘help’ for help on commands
      ‘man’ to read the manual
      ‘quit’ to quit the shell

><fs> run
 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
><fs>  list-filesystems
/dev/sda1: unknown
/dev/sda2: ext4
/dev/ubuntu-vg/ubuntu-lv: ext4
><fs> mount /dev/ubuntu-vg/ubuntu-lv /
><fs> ls /home
tux
bob
><fs> edit /etc/default/grub

How to Reset Password on an Ubuntu with LVM

A few days ago, a client tasked us to recover the password of an Ubuntu server 20.04LTS. The machine owner only knew the username but had no idea about the complexity of the password. We’ve asked the client if it was OK for us to reset the password instead of recovering it (meaning that we would not even try to crack the mystery of what the previous password was and just set a new one), and thankfully, the client accepted our request.

The client set up the server using Ubuntu server edition 20.04LTS, and the disk partitions were using LVM (Logical Volume Manager). To our good luck, they were not using encrypted partitions. The procedure we followed to reset the password of that server was like so:

First of all, we shut down the server and booted it with a Live USB of an Ubuntu desktop 20.04LTS. Then we started a terminal and executed the following to get root access on the live system:

sudo su;

Then, we executed pvscan to list all physical volumes and gain some intelligence on which disk we needed to work on:

pvscan;
root@ubuntu:/home/ubuntu# pvscan
  /dev/sdc: open failed: No medium found
  PV /dev/sda3   VG ubuntu-vg       lvm2 [<3.64 TiB / 3.44 TiB free]
  Total: 1 [<3.64 TiB] / in use: 1 [<3.64 TiB] / in no VG: 0 [0   ]

Following that, we used vgscan to search for all volume groups:

vgscan;
root@ubuntu:/home/ubuntu# vgscan
  /dev/sdc: open failed: No medium found
  Found volume group "ubuntu-vg" using metadata type lvm2

From these two commands, it was clear that the disk /dev/sda3 contained an LVM partition with the logical volume group name ubuntu-vg. That logical volume group held the server’s filesystem, and it was the place we needed to access to change the user’s password.

So, we used vgchange to change the attributes of the volume group and activate it like so:

vgchange -a y;
root@ubuntu:/home/ubuntu# vgchange -a y
  /dev/sdc: open failed: No medium found
  /dev/sdc: open failed: No medium found
  1 logical volume(s) in volume group "ubuntu-vg" now active

Using lvscan, we were able to list all logical volumes in all volume groups and verify that we activated the volume group of interest successfully.

lvscan;
root@ubuntu:/home/ubuntu# lvscan
  /dev/sdc: open failed: No medium found
  ACTIVE            '/dev/ubuntu-vg/ubuntu-lv' [200.00 GiB] inherit

After these steps, we were ready to reset the password of the user finally. We continued to mount the logical volume group like any other disk on the /mnt folder:

mount /dev/ubuntu-vg/ubuntu-lv /mnt/;

Then, we used chroot to change the apparent root directory for the currently running process (and its children). This command allowed our terminal to work inside the logical volume as if we had booted the server OS itself.

chroot /mnt/;

Finally, using the passwd command, we changed the user password as so:

passwd -S bob;

To clean up, we exited the chroot environment:

exit;

Then, we unmounted the logical volume group:

umount /mnt;

And finally, we set the active flag of the volume group to no.

vgchange -a n;

After the above steps, we had safely applied all changes, so we rebooted the machine using its hard drive.


Lubuntu LVM Encrypted

While trying to setup a lubuntu GNU/Linux that would use an encrypted LVM file-system we run into several problems that should have not been there.
For example, the installer would not continue as swap memory was not encrypted and it was blocking the operation.
An other issue was that the LVM support package was not installed by default although it is needed by the installer.

Our solution requires an active internet connection so that you can install the lvm2 package.

Methodology

After booting into the live session, before we started the installation process, we opened a terminal from the main menu (Start Menu) and executed the following two commands:


sudo apt-get install lvm2 -y;

sudo swapoff --all;

These two commands disabled all swap (so that we do not get the error that swap is not encrypted) and it installed the lvm2 package that is needed by the installer to create our LVM setup.

The following video presents the full successful installation procedure by making the changes before starting the procedure.

The next video, shows our attempts to fix the installation after encountering the errors instead of fixing them beforehand.