qubes-os


youtube-dl does not work properly on Qubes 4.0 – Fedora 26

While working on a GNU/Linux Fedora 26 virtual machine running under the Qubes 4.0 OS, we installed youtube-dl through dnf to download some media off the net.

sudo dnf install youtube-dl;

When we tried to use it, we got the error youtube_dl.utils.RegexNotFoundError: Unable to extract Initial JS player signature function name. This issue was resolved years ago, so it led us to the conclusion that our version of youtube-dl was out of date. To test this hypothesis, we updated youtube-dl with pip.

sudo pip install --upgrade youtube_dl;

After the update was complete, we tried to use youtube-dl, this time with success!!

Full Logs:

[[email protected] Music]$ youtube-dl
bash: youtube-dl: command not found...
[[email protected] Music]$ sudo dnf install youtube-dl
Last metadata expiration check: 0:52:47 ago on Sat Nov 10 10:43:46 2018.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
youtube-dl noarch 2018.04.16-1.fc26 updates 2.6 M

Transaction Summary
================================================================================
Install 1 Package

Total download size: 2.6 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
youtube-dl-2018.04.16-1.fc26.noarch.rpm 270 kB/s | 2.6 MB 00:09 
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 258 kB/s | 2.6 MB 00:10 
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1 
Installing : youtube-dl-2018.04.16-1.fc26.noarch 1/1 
Running scriptlet: youtube-dl-2018.04.16-1.fc26.noarch 1/1 
Running as unit: run-rfddd15fff2d14d109826a90f59325e97.service
Verifying : youtube-dl-2018.04.16-1.fc26.noarch 1/1 
Notifying dom0 about installed applications

Installed:
youtube-dl.noarch 2018.04.16-1.fc26

Complete!
[[email protected] Music]$ youtube-dl https://www.youtube.com/watch?list=r6akoO34yUvK8ddtjnzL
 r6akoO34yUvK8ddtjnzL: Downloading webpage
 Downloading playlist: How to cook a banana
 playlist How to cook a banana: Downloading 18 videos
 Downloading video 1 of 18
 ir86d1hTSv1r: Downloading webpage
 ir86d1hTSv1r: Downloading video info webpage
 ir86d1hTSv1r: Extracting video information
 ir86d1hTSv1r: Downloading js player vfls4aurX
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1191, in _decrypt_signature
video_id, player_url, s
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1102, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1163, in _parse_sig_js
jscode, 'Initial JS player signature function name', group='sig')
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 808, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract Initial JS player signature function name; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
(caused by RegexNotFoundError('Unable to extract \x1b[0;34mInitial JS player signature function name\x1b[0m; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[[email protected] Music]$ sudo pip install --upgrade youtube_dl
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Collecting youtube_dl
Downloading https://files.pythonhosted.org/packages/6c/a4/c2e3fbd8b9c7ccbca3f220c4fb0914926669bf9080fb8f2f4db4811706be/youtube_dl-2018.11.7-py2.py3-none-any.whl (1.8MB)
100% |████████████████████████████████| 1.8MB 355kB/s 
Installing collected packages: youtube-dl
Successfully installed youtube-dl-2018.11.7
[[email protected] Music]$ youtube-dl https://www.youtube.com/watch?list=r6akoO34yUvK8ddtjnzL
 r6akoO34yUvK8ddtjnzL: Downloading webpage
 Downloading playlist: How to cook a banana
 playlist How to cook a banana: Downloading 18 videos
 Downloading video 1 of 18
 ir86d1hTSv1r: Downloading webpage
 ir86d1hTSv1r: Downloading video info webpage
 ir86d1hTSv1r: Downloading js player vfls4aurX
WARNING: Requested formats are incompatible for merge and will be merged into mkv.

Qubes 4.0 with Fedora 26: Setup RPM Fusion and ffmpeg

Recently we wanted to process some media on a Fedora 26 running under a Qubes OS 4.0 installation, we decided to use ffmpeg which is not part of the default repositories but it can be found in the RPM Fusion repositories. To do so, first we updated our system and enabled the RPM Fusion repositories as follows:

sudo dnf update;

sudo dnf upgrade -y;

sudo dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm;

sudo dnf config-manager --set-enabled rpmfusion-free rpmfusion-nonfree;

By default DNF on the template VM did not enable the rpmfusion repositories so we had to enable them manually with the last command above or else we would get the following error:

$ sudo dnf install ffmpeg;
Last metadata expiration check: 0:17:49 ago on Tue Oct 16 09:09:22 2018.
No match for argument: ffmpeg

Then, we updated the system once more so that the information from the new repositories would get downloaded to our system and then we performed the installation of ffmpeg. While installing ffmpeg, since it was the first time that we were using the new repositories we were asked to verify the keys that were imported. We were able to manually verify the keys from this page.

The commands used to install ffmpeg are the following:

sudo dnf update;

sudo dnf install ffmpeg;



Qubes-OS 3.2: USB printer (and other devices)

Below you will find the commands we used to enable the sys-usb VM (on an installation of Qubes 3.2 where it was not enabled by default nor was the task of handling USB devices assigned to sys-net).

On dom0 terminal emulator, we executed the following first to enable sys-usb.

sudo qubesctl top.enable qvm.sys-usb;
sudo qubesctl state.highstate;

Then we modified the configuration files for the mouse (/etc/qubes-rpc/policy/qubes.InputMouse) and keyboard (/etc/qubes-rpc/policy/qubes.InputKeyboard) so that they will automatically be granted to dom0 without prompting the used each time.

We modified the content /etc/qubes-rpc/policy/qubes.InputMouse and /etc/qubes-rpc/policy/qubes.InputKeyboard to be as below:

sys-usb dom0 allow,user=root
$anyvm $anyvm deny

Qubes-OS 3.2 GNU/ Linux: Temporarily enable second language for keyboard

Recently we started using Qubes OS 3.2, at some point we needed to enable the Greek language keyboard for a Fedora 23 based VM.

To avoid making permanent change to the VM we used the following command that enables the US and GR keyboard layouts while switching is made available by pressing the two shift buttons at the same time.

setxkbmap -layout "us,gr" -option "grp:shifts_toggle";


Qubes OS 3.2: Resize fedora-23 TemplateVM Root Image 1

Recently, we needed to increase the size of the root image for the fedora-23 TemplateVM.
We had to do this as we wanted to install in /opt a few IDEs, including android-studio which takes a lot of space when accompanied by the Android SDK.

Following the excellent guide at https://www.qubes-os.org/doc/resize-root-disk-image/, we did the following:

  1. Made sure that the fedora-23 TemplateVM and all VMs based on that template were shut down.
    This included stopping the sys-firewall and sys-net VMs as well (in this order).
  2. Then, in the VM Settings window at the Basic tab for the fedora-23 TemplateVM, we disabled the networking by selecting none on the NetVM: drop-down list and pressed OK to apply the settings changes.
  3. Following, in dom0 Terminal Emulator we run the following command:
    truncate -s 40G /var/lib/qubes/vm-templates/fedora-23/root.img;
  4. Afterwards we started the fedora-23 TemplateVM and run the following in the terminal:
    sudo resize2fs /dev/mapper/dmroot;
    Please note that if your output is Nothing to do! then most likely you forgot a VM that is based on the fedora-23 TemplateVM running. Normally your screen should print something along the lines of Resizing the filesystem...
  5. Next, we shut down the fedora-23 TemplateVM, and went to the VM Settings window to enable networking by selecting whatever we had there before (for us it was default (sys-firewall)).
  6. Finally, we could start using our VMs, their root.img were of the new extended size.

Qubes OS: Connect to Wi-Fi or Ethernet or another network

A couple of days ago we decided to give Qubes OS a go and see what it could do for a regular user. The installation was easy as it uses the same installer as Fedora, so we just created a live USB and formatted a laptop that had built in Wi-Fi.

To our surprise, we could not figure out how to change the network settings and activate an internet connection! Going to the NetworkManager (nm), even with root, would show us all fields as disabled when trying to create any new connection!

Some time passed before we realized that the NetworkManager of XFCE4 was not the way to go. After inspecting the Virtual Machines on the Qubes VM Manager, we saw that the sys-net VM was the only one that had in its hardware settings to access the Ethernet and Wi-Fi modules. So we got the hint, we needed to modify sys-net in order to connect the entire OS with its VMs to the network.

To modify the settings of the sys-net VM we needed access to the Settings Application, which was not available in the application menu. So the first thing we did, was to use the sys-net: Add more shortcuts... option under the group ServiceVM: sys-net to enable the Settings application shortcut.

In the [Dom0] Settings: sys-net window, we went to the Applications tab, on the left list we scrolled down to find the Settings option.

After selecting the Settings option, we clicked on the > button to move the Settings option to the right list.

Then we clicked on the OK button to apply the changes.

Going back to the application menu and the group ServiceVM: sys-net we could see the new option for sys-net: Settings.

Clicking on the sys-net: Settings showed us the usual settings manager for Gnome.

From there on, our job was easy, we just clicked on the Network option that gave us the window to modify all network settings. Then we selected the Wi-Fi network that we wanted to connect to, which worked without a hitch!

Finally, we had to test if the configuration was working as expected. From the application menu, under the group Domain: personal, we selected the option personal: Firefox to start the Firefox application on the personal VM.

Once Firefox started we could see that internet connection was active and everything was working as expected!