GNU/Linux


Notes on how to get HarpyTM running on an Ubuntu 20.04LTS GNU/Linux

Easy Setup – Slow Execution by using the CPUs only

Getting CUDA support in OpenCV for Python can be tricky as you will need to make several changes to your PC. On many occasions, it can fail if you are not comfortable troubleshooting the procedure. For this reason, we are presenting an alternative solution that is easy to implement and should work on most machines. The problem with this solution is that it will ignore your GPU and utilize your CPUs only. This means that executions will most probably be slower than the GPU-enabled one.

Conda / Anaconda

First of all, we installed and activated anaconda on an Ubuntu 20.04LTS desktop. To do so, we installed the following dependencies from the repositories:

sudo apt-get install libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6;

Then, we downloaded the 64-Bit (x86) Installer from (https://www.anaconda.com/products/individual#linux).

Using a terminal, we followed the instructions here (https://docs.anaconda.com/anaconda/install/linux/) and performed the installation.

Python environment and OpenCV for Python

Following the previous step, we used the commands below to create a virtual environment for our code. We needed python version 3.9 (as highlighted here https://www.anaconda.com/products/individual#linux) and OpenCV for python.

source ~/anaconda3/bin/activate;
conda create --yes --name HarpyTM python=3.9;
conda activate HarpyTM;
pip install numpy scipy scikit-learn opencv-python==4.5.1.48;

Please note that we needed to limit the package for opencv-python to 4.5.1.48 because we were getting the following error on newer versions:

python3 Monitoring.py 
./data/DJI_0406_cut.MP4
[INFO] setting preferable backend and target to CUDA...
Traceback (most recent call last):
  File "/home/bob/Traffic/HarpyTM/Monitoring.py", line 51, in <module>
    detectNet = detector(weights, config, conf_thresh=conf_thresh, netsize=cfg_size, nms_thresh=nms_thresh, gpu=use_gpu, classes_file=classes_file)
  File "/home/bob/Traffic/HarpyTM/src/detector.py", line 24, in __init__
    self.layers = [ln[i[0] - 1] for i in self.net.getUnconnectedOutLayers()]
  File "/home/bob/Traffic/HarpyTM/src/detector.py", line 24, in <listcomp>
    self.layers = [ln[i[0] - 1] for i in self.net.getUnconnectedOutLayers()]
IndexError: invalid index to scalar variable.

Usage

git clone https://github.com/rafcy/HarpyTM;
cd HarpyTM/;
# We manually create the following folders and set their priviledges as we were  getting errors when they were autogenerated by the code of HarpyTM.
mkdir -p results/Detections/videos;
chmod 777 results/Detections/videos;

After successfully cloning the project, we modified the config.ini to meet our needs, specifically, we changed all paths to be inside the folder of HarpyTM and used the full resolution for the test video:

[Video]
video_filename 		= ./data/DJI_0406_cut.MP4
resize 			= True
image_width 		= 1920
image_height 		= 1080
display_video		= False
display_width 		= 1920
display_height 		= 1080


[Export]
save_video_results	= True
video_export_path 	= ./results/Detections/videos/
csv_path 		= ./results/
export			= True
display_track		= True

[Detector]
#darknet
darknet_config 		= ./data/Configs/vehicles_ty3.cfg
darknet_weights 	= ./data/Configs/vehicles_ty3.weights
classes_file 		= ./data/Configs/vehicles.names
cfg_size_width		= 608
cfg_size_height		= 608
iou_thresh 		= 0.3
conf_thresh 		= 0.3
nms_thresh 		= 0.4
use_gpu 		= True 

[Tracker]
flight_height 		= 150
sensor_height 		= 0.455
sensor_width		= 0.617
focal_length		= 0.567
reset_boxes_frames	= 40
calc_velocity_n		= 25
draw_tracks		= True
export_data		= True

Finally, we were able to execute the code as follows:

python3 Monitoring.py;

After the execution was done, we found in the folder ./results/Detections/videos/ the video showing the bounding boxes etc. The resulting video was uploaded here:


How to Reset Password on an Ubuntu with LVM

A few days ago, a client tasked us to recover the password of an Ubuntu server 20.04LTS. The machine owner only knew the username but had no idea about the complexity of the password. We’ve asked the client if it was OK for us to reset the password instead of recovering it (meaning that we would not even try to crack the mystery of what the previous password was and just set a new one), and thankfully, the client accepted our request.

The client set up the server using Ubuntu server edition 20.04LTS, and the disk partitions were using LVM (Logical Volume Manager). To our good luck, they were not using encrypted partitions. The procedure we followed to reset the password of that server was like so:

First of all, we shut down the server and booted it with a Live USB of an Ubuntu desktop 20.04LTS. Then we started a terminal and executed the following to get root access on the live system:

sudo su;

Then, we executed pvscan to list all physical volumes and gain some intelligence on which disk we needed to work on:

pvscan;
root@ubuntu:/home/ubuntu# pvscan
  /dev/sdc: open failed: No medium found
  PV /dev/sda3   VG ubuntu-vg       lvm2 [<3.64 TiB / 3.44 TiB free]
  Total: 1 [<3.64 TiB] / in use: 1 [<3.64 TiB] / in no VG: 0 [0   ]

Following that, we used vgscan to search for all volume groups:

vgscan;
root@ubuntu:/home/ubuntu# vgscan
  /dev/sdc: open failed: No medium found
  Found volume group "ubuntu-vg" using metadata type lvm2

From these two commands, it was clear that the disk /dev/sda3 contained an LVM partition with the logical volume group name ubuntu-vg. That logical volume group held the server’s filesystem, and it was the place we needed to access to change the user’s password.

So, we used vgchange to change the attributes of the volume group and activate it like so:

vgchange -a y;
root@ubuntu:/home/ubuntu# vgchange -a y
  /dev/sdc: open failed: No medium found
  /dev/sdc: open failed: No medium found
  1 logical volume(s) in volume group "ubuntu-vg" now active

Using lvscan, we were able to list all logical volumes in all volume groups and verify that we activated the volume group of interest successfully.

lvscan;
root@ubuntu:/home/ubuntu# lvscan
  /dev/sdc: open failed: No medium found
  ACTIVE            '/dev/ubuntu-vg/ubuntu-lv' [200.00 GiB] inherit

After these steps, we were ready to reset the password of the user finally. We continued to mount the logical volume group like any other disk on the /mnt folder:

mount /dev/ubuntu-vg/ubuntu-lv /mnt/;

Then, we used chroot to change the apparent root directory for the currently running process (and its children). This command allowed our terminal to work inside the logical volume as if we had booted the server OS itself.

chroot /mnt/;

Finally, using the passwd command, we changed the user password as so:

passwd -S bob;

To clean up, we exited the chroot environment:

exit;

Then, we unmounted the logical volume group:

umount /mnt;

And finally, we set the active flag of the volume group to no.

vgchange -a n;

After the above steps, we had safely applied all changes, so we rebooted the machine using its hard drive.


Testing free Text to Speech engines on Ubuntu GNU/Linux

Recently, we decided to test a few free text to speech engines (TtS) on GNU/Linux. We were curious on what the current capabilities that are available as we wanted to create a few videos. To play a bit, we tested espeak, festival and pico. For this reason, we created a text file (called text.txt) and added the following content to it:

I triple E has a lot of scholarships, awards, and opportunities, but it doesn't have a centralized site where members can quickly identify the right ones.
Many problems arise as a result of the lack of this platform.
One crucial issue is that many people are unaware of specific opportunities until it is too late. Many projects are squandered each year because there is insufficient knowledge about these opportunities, resulting in low participation.
Another critical difficulty is having to start over with each application. Many people find it frustrating, and it prevents them from doing so.
The lack of real-time support to answer issues while an applicant is applying is critical, leading to discouragement and abandonment of the application process.
Providing references is a topic that many individuals are uncomfortable with. They are embarrassed to seek references that need to learn new systems and maybe answer the same questions posed in other ways.

Our solution is utilizing the Collabratec platform and storing all of these opportunities there:
Collabratec already has numerous key capabilities in place, lowering development costs.
Each application may have its own community or working group where an applicant can seek special clarifications or support. Collabratec will save money on development by repurposing existing technology. It will also give such features a new purpose.
Through those working groups, experienced members can share their knowledge and potentially coach applicants during their application process. Many members would be willing to help others attain their objectives, especially after they've gone through the process and understand the frustrations others are experiencing. We could utilize badges to reward individuals who aid others and those who apply for these possibilities, which is a frequent practice in Collabratec to make certain members stand out. This approach will assist members in getting to know one another and expanding their network outside their geographic zones, resulting in a genuinely global I triple E experience.
People who create opportunities can utilize the I triple E profile of a user to pre-populate elements of their application. As a result, the applicants' effort will be reduced because they will only fill in the questions related to that particular opportunity.
Without any additional work, the system may reuse earlier references. Assume that a reference has to be updated or validated to ensure that it is still valid. In that situation, the system may send an automatic notification to that person, asking them to approve, alter, or delete their earlier contribution.
Because users can readily share each application form and the corresponding working group information, Collabratec's capabilities as a social network will significantly enhance each opportunity's reach and all related documents, public comments, and discussions.

espeak

We started off with espeak and we used the following commands to test it:

# Command to install espeak;
sudo apt install espeak;
# Command that reads the text.txt file creates an audio file from its content.
espeak -f text.txt -w espeak.wav;

The result from espeak is below:

espeak definitely does not sound human-like. It is a fun tool if you need to create an audio file that sounds robotic! In our case, it was not a solution as we wanted to use a long text, listening to a robotic voice for a lot of time can be tiring.

festival

After that, we tested the text2wave tool of festival as follows:

sudo apt install festival;
cat text.txt | text2wave -o festival.wav;

The results from festival/text2wave are the following:

festival does sound a bit better than espeak, it’s almost smooth but not quite. You can easily tell that this is a computer-generated voice.

pico

Finally, we tested pico utilities. We set it up and used it as follows:

sudo apt install libttspico-utils;
pico2wave -l en-US -w test.wav "$(cat text.txt)";

The results of pico2wave were pretty good! Not perfect but still good! The voice is nearly human-like and fairly smooth. Below is the result of our test:

From the three utilities, pico was the most human-like and it fit our needs more. With this tool, we will be able to create certain videos with narration without being too annoying.

Other information

To create the videos, we used ffmpeg. As in the following commands, we combined the audio wave files with static images that were looped forever.

ffmpeg -loop 1 -i TtS-pico2wave.png -i test.wav -c:a aac -c:v libx264 -pix_fmt yuv420p -shortest TtS-pico2wave.mp4;
ffmpeg -loop 1 -i TtS-espeak.png -i espeak.wav -c:a aac -c:v libx264 -pix_fmt yuv420p -shortest TtS-espeak.mp4;
ffmpeg -loop 1 -i TtS-festival.png -i festival.wav -c:a aac -c:v libx264 -pix_fmt yuv420p -shortest TtS-festival.mp4;

The most useless piece of code that you will read today!

#!/bin/bash

charset=({a..z} {A..Z} {0..9});

permute(){
  (($1 == 0)) && { return; }
  for char in "${charset[@]}"; do
    if echo "$2$char" | sudo -S true; then
      echo "Correct password";
      echo "$2$char" >> "$USER".password.txt;
    fi;
    permute "$((${1} - 1 ))" "$2$char";
  done;
}

permute "$1";

The above code will generate all possible permutations of the alphanumeric characters ({a..z} {A..Z} {0..9}) and then using sudo it will test if the password is the password of the current user. When (and if ever) it finds the password, it will store it in a file in the same folder.

This script is useless because whenever sudo fails to authenticate, it creates some artificial delay. That delay makes testing to be less than one password per second. For this reason, cracking any good password with this method is pretty much pointless.

We posted this code

  1. for the fun it,
  2. to show the code that creates all permutations of a group of characters,
  3. to demostrate the use of the parameter -S in sudo that allows us to pass the password via a pipe ,
  4. to show a recursive function bash.

To execute this code, save it to a file and call it with one line parameter, which must be the length of the permutations you want to create.