list

5 packages can be upgraded. Run ‘apt list –upgradable’ to see them.

 in Bash / GNU/Linux  tagged apt / bash / GNU/Linux / list / packages / ubuntu / upgradable by

Updating an operating system is a critical task that ensures the system’s security, stability, and performance. In Ubuntu, updating the system involves running several commands in the terminal. In this technical post, we will explain the process of updating an Ubuntu Server installation and how to fix the issue of pending package upgrades.

The first command that needs to be executed is sudo apt update. This command updates the list of available packages from the Ubuntu repositories. The -y option instructs the system to answer “yes” to prompts, ensuring the process runs uninterrupted.

The second command is sudo apt upgrade. This command upgrades all installed packages to their latest versions. Again, the -y option is used to answer “yes” to any prompts.

The third command, sudo apt autoremove, removes any unnecessary dependencies that are no longer required by the system.

After executing these three commands, the system displayed a warning message that says that five packages can be upgraded. To see the list of upgradable packages, we needed to execute the command apt list --upgradable. This command lists all the upgradable packages along with their versions.

The following command we executed was sudo apt-get dist-upgrade. This command upgrades the system to the latest distribution release, including kernel upgrades. However, the command failed to upgrade the pending packages in this case.

To fix the issue, we executed the command sudo apt-get install --only-upgrade $PACKAGE for each pending package. This command upgrades the specified package to its latest version. The $PACKAGE variable should be replaced with the package name that needs to be upgraded.

In summary, updating an Ubuntu Server installation involves running the sudo apt update, sudo apt upgrade, and sudo apt autoremove commands in the terminal. If there are any pending package upgrades, we can use the apt list --upgradable command to see the list of upgradable packages. If the sudo apt-get dist-upgrade command fails to upgrade the pending packages, we can use the sudo apt-get install --only-upgrade $PACKAGE command to upgrade each package individually.

KVM Virtual Machines’ Backup and Restoration

 in GNU/Linux  tagged bash / domblklist / dumpxml / GNU/Linux / libvirt / list / qemu / shutdown / undefine / virsh / vm by

Introduction

According to Redhat, KVM is an open-source virtualization technology that is incorporated into Linux. KVM stands for Kernel-based Virtual Machine. To be more specific, KVM enables the transformation of Linux into a hypervisor, which makes it possible for a single physical computer to operate simultaneously several distinct virtual environments, also known as guests or virtual machines (VMs). This post will focus mainly on backing up KVM virtual machines (VMs).

Backup your KVM VM

First, log in using the sudo user and list all of the KVM virtual machines available.

virsh list --all;
# virsh help list
#  NAME
#    list - list domains
#
#  SYNOPSIS
#    list [--inactive] [--all] [--transient] [--persistent] [--with-snapshot] [--without-snapshot] [--state-running] [--state-paused] [--state-shutoff] [--state-other] [--autostart] [--no-autostart] [--with-managed-save] [--without-managed-save] [--uuid] [--name] [--table] [--managed-save] [--title]
#
#  DESCRIPTION
#    Returns list of domains.
#
#  OPTIONS
#    --all            list inactive & active domains

Next, you will need to log out of the virtual machine (VM) you intend to back up.

virsh shutdown $VM_NAME;
# virsh help shutdown
#  NAME
#    shutdown - gracefully shutdown a domain
#
#  SYNOPSIS
#    shutdown <domain> [--mode <string>]
#
#  DESCRIPTION
#    Run shutdown in the target domain.
#
#  OPTIONS
#    [--domain] <string>  domain name, id or uuid
#    --mode <string>  shutdown mode: acpi|agent|initctl|signal|paravirt

Then it would be best if you executed the following to ensure that the command was executed successfully.

virsh list --all;

We can break down the process of backing up a KVM virtual machine into two essential parts:

The definition of the domain:

Specifying the physical components that make up the VM, such as its network interfaces, physical and virtual central processing units, RAM, and disk space. You will be able to see this information by executing the following command:

virsh dumpxml $VM_NAME;
# virsh help dumpxml
#  NAME
#    dumpxml - domain information in XML
#
#  SYNOPSIS
#    dumpxml <domain> [--inactive] [--security-info] [--update-cpu] [--migratable]
#
#  DESCRIPTION
#    Output the domain information as an XML dump to stdout.
#
#  OPTIONS
#    [--domain] <string>  domain name, id or uuid
#    --inactive       show inactive defined XML
#    --security-info  include security sensitive information in XML dump
#    --update-cpu     update guest CPU according to host CPU
#    --migratable     provide XML suitable for migrations

The data file:

The path to the source file includes the location of the data file that needs to be backed up by us. It is where the virtual machine’s hard drive is stored. This section contains the internal configuration, including services, databases, etc. We can use the domain definition to determine the location of this file, or we can use the following command, which will tell you the position of the hard drive. Either way, we can find the location of this file.

virsh domblklist $VM_NAME;
# virsh help domblklist
#  NAME
#    domblklist - list all domain blocks
#
#  SYNOPSIS
#    domblklist <domain> [--inactive] [--details]
#
#  DESCRIPTION
#    Get the summary of block devices for a domain.
#
#  OPTIONS
#    [--domain] <string>  domain name, id or uuid
#    --inactive       get inactive rather than running configuration
#    --details        additionally display the type and device value

Let’s assume that the following is the given location (which is usually the default location for virtual machines created with libvirt):

/var/lib/libvirt/images/

The above command will produce output similar to the following:

Target     Source
------------------------------------------------
hda        /var/lib/libvirt/images/nba.qcow2
hdb        -

The qcow2 images are the disk files that we need to back up.

First, we need to create a place to put the backups:

mkdir -p /opt/backup/kvm/;

The following command creates a backup of the domain definition:

virsh dumpxml $VM_NAME > "/opt/backup/kvm/$VM_NAME.xml";

The following is what we employ to create a backup of the hard drives based on the feedback of the domblklist command:

cp /var/lib/libvirt/images/nba.qcow2 /opt/backup/kvm/;

Restore your KVM virtual machine

We will begin by erasing the virtual machine’s hard disk and undefining the VM so that it does not exist any longer. Using the domblklist command, we need to identify the qcow2 files to be deleted. After we collect that information and make sure that the VM is stopped using the shutdown command, we need to delete the file(s) from the hard drive:

rm /var/lib/libvirt/images/nba.qcow2;

Remove the VM definition:

Then you need to remove the existing definition before restoring a backup.

virsh undefine $VM_NAME;
# virsh help undefine
#  NAME
#    undefine - undefine a domain
#
#  SYNOPSIS
#    undefine <domain> [--managed-save] [--storage <string>] [--remove-all-storage] [--delete-snapshots] [--wipe-storage] [--snapshots-metadata] [--nvram] [--keep-nvram]
#
#  DESCRIPTION
#    Undefine an inactive domain, or convert persistent to transient.
#
#  OPTIONS
#    [--domain] <string>  domain name, id or uuid
#    --managed-save   remove domain managed state file
#    --storage <string>  remove associated storage volumes (comma separated list of targets or source paths) (see domblklist)
#    --remove-all-storage  remove all associated storage volumes (use with caution)
#    --delete-snapshots  delete snapshots associated with volume(s), requires --remove-all-storage (must be supported by storage driver)
#    --wipe-storage   wipe data on the removed volumes
#    --snapshots-metadata  remove all domain snapshot metadata, if inactive
#    --nvram          remove nvram file, if inactive
#    --keep-nvram     keep nvram file, if inactive

Restore the virtual machine (VM).

Now, to bring back the virtual machine that was removed, we will first get back the hard drive:

cp /opt/backup/kvm/nba.qcow2 /var/lib/libvirt/images/;

Then, bring back the original definition of the domain.

virsh define --file "/opt/backup/kvm/$VM_NAME.xml";
# virsh help define
#  NAME
#    define - define (but don't start) a domain from an XML file
#
#  SYNOPSIS
#    define <file> [--validate]
#
#  DESCRIPTION
#    Define a domain.
#
#  OPTIONS
#    [--file] <string>  file containing an XML domain description
#    --validate       validate the XML against the schema

If you’re moving it to a different physical host, you can check to see if the information included within the XML file needs to be updated appropriately. Check to see if the new physical host has network interfaces, for instance, and so forth.

Execute the following to check that the parameters of your virtual machine (VM) have been successfully defined:

virsh list --all;

After that, begin using the VM by:

virsh start $VM_NAME;
# virsh help start
#  NAME
#    start - start a (previously defined) inactive domain
#
#  SYNOPSIS
#    start <domain> [--console] [--paused] [--autodestroy] [--bypass-cache] [--force-boot] [--pass-fds <string>]
#
#  DESCRIPTION
#    Start a domain, either from the last managedsave
#    state, or via a fresh boot if no managedsave state
#    is present.
#
#  OPTIONS
#    [--domain] <string>  name of the inactive domain
#    --console        attach to console after creation
#    --paused         leave the guest paused after creation
#    --autodestroy    automatically destroy the guest when virsh disconnects
#    --bypass-cache   avoid file system cache when loading
#    --force-boot     force fresh boot by discarding any managed save
#    --pass-fds <string>  pass file descriptors N,M,... to the guest

Once your virtual machine (VM) is up and running, you may use SSH or other methods to log into it and check that everything was correctly restored.

Two ways to append a new argument to CMAKE_ARGS list for ExternalProject_Add

 in Programming  tagged cmake / CMAKE_ARGS / ExternalProject_Add / list / set by

Recently, we wanted to pass a new cached value to an external project in CMake via the CMAKE_ARGS variable.

CMAKE_ARGS holds various types of arguments. From which, the arguments in the form -Dname:type=value are passed to the CMake command line and cannot be changed by the user.

We found two ways to add a new argument pair to the CMAKE_ARGS of the external project.

The first method uses the set function:

set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR}${CACHE_VAR_TYPE}=\"${${CACHE_VAR}}\"")

where basically we create a new string that starts with the existing value of CMAKE_ARGS and then we append to its end the new pair.

The second method uses the list function:

list(APPEND CMAKE_ARGS "-D${CACHE_VAR}${CACHE_VAR_TYPE}=${${CACHE_VAR}}")

that treats CMAKE_ARGS as a list and it appends to its end the new pair.

To verify by hand that they were equivalent we did the following small test with success

#Copy the original value of ${CMAKE_ARGS}
set(CMAKE_ARGS_ORIGINAL ${CMAKE_ARGS})

#Define the Name, Type and Value for the new argument pairs
set(CACHE_VAR_P_NAME PRODUCTS_DIR)
set(CACHE_VAR_P_TYPE string)
set(CACHE_VAR_P_VALUE "someplace/with spaces")

set(CACHE_VAR_C_NAME CLIENTS_DIR)
set(CACHE_VAR_C_TYPE string)
set(CACHE_VAR_C_VALUE "another/place")

#Print the original value of ${CMAKE_ARGS}
MESSAGE(STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Append the two pairs to ${CMAKE_ARGS} using set
set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR_P_NAME}:${CACHE_VAR_P_TYPE}=\"${CACHE_VAR_P_VALUE}\"")
set(CMAKE_ARGS "${CMAKE_ARGS} -D${CACHE_VAR_C_NAME}:${CACHE_VAR_C_TYPE}=\"${CACHE_VAR_C_VALUE}\"")

#Print the modified value of ${CMAKE_ARGS}
MESSAGE(STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Reset ${CMAKE_ARGS} to its original value
set(CMAKE_ARGS ${CMAKE_ARGS_ORIGINAL})

#Print the original value of ${CMAKE_ARGS} for verification
MESSAGE( STATUS "CMAKE_ARGS: '" ${CMAKE_ARGS} "'")

#Append the two pairs to ${CMAKE_ARGS} using list
list(APPEND CMAKE_ARGS "-D${CACHE_VAR_P_NAME}:${CACHE_VAR_P_TYPE}=\"${CACHE_VAR_P_VALUE}\"")
list(APPEND CMAKE_ARGS "-D${CACHE_VAR_C_NAME}:${CACHE_VAR_C_TYPE}=\"${CACHE_VAR_C_VALUE}\"")

#Print the modified value of ${CMAKE_ARGS}
#Notice that here we surrounded ${CMAKE_ARGS} with quotes so that it is printed as list of delimiter separated values
#Between each element the character ';' will be added because ${CMAKE_ARGS} is surrounded with quotes
MESSAGE(STATUS "CMAKE_ARGS: '" "${CMAKE_ARGS}" "'")

Which resulted in the following output

-- CMAKE_ARGS: ''
-- CMAKE_ARGS: ' -DPRODUCTS_DIR:string="someplace/with spaces" -DCLIENTS_DIR:string="another/place"'
-- CMAKE_ARGS: ''
-- CMAKE_ARGS: '-DPRODUCTS_DIR:string="someplace/with spaces";-DCLIENTS_DIR:string="another/place"'

From the output we can see that the two options treat the variable in a different way as using set just created a huge string containing both pairs while list created a list of two elements.
Both methods seem to work properly but we chose the list method for our external project.

The CMakeLists.txt file that we used to include the external project using custom CMAKE_ARGS resulted as follows:

cmake_minimum_required (VERSION 3.2.2)
include (ExternalProject)

set (TARGET c-banana-eat-banana)
project (${TARGET} C)

set (BANANA_ROOT ${CMAKE_CURRENT_BINARY_DIR} CACHE INTERNAL "")

#Add custom CMake arguments to be passed to the CMake command line of the external project
list(APPEND CMAKE_ARGS "-DARTIFACTS_DIR:string=\"${LIBRARIES_DIR}/${TARGET}\"")

ExternalProject_Add (${TARGET}
 URL ${CMAKE_CURRENT_SOURCE_DIR}/${TARGET}.zip
 URL URL_HASH MD5=34734a678729967f426931d913326112
 CMAKE_ARGS "${CMAKE_ARGS}"
 BUILD_IN_SOURCE 1)