youtube-dl

youtube-dl and embedded Vimeo

 in Applications  tagged domain / referer / vimeo / youtube-dl by

Youtube-DL being an amazing tool as it is, has a way to download Vimeo videos that give the following error:

Cannot download embed-only video without embedding URL. Please call youtube-dl with the URL of the page that embeds this video.

To do so, we needed to use the --referer parameter with the value of the website that we found the embedded video on. For example:

youtube-dl -v "https://player.vimeo.com/video/622fa342f" --referer "https://example.com/courses/intro/hello/";

The --referer parameter specifies a custom referer and can be used if the video access is restricted to one domain.

(youtube-dl) tux@bob:~$ youtube-dl -v "https://player.vimeo.com/video/622fa342f"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://player.vimeo.com/video/622fa342f']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.9.7 (CPython) - Linux-5.13.0-44-generic-x86_64-with-glibc2.31
[debug] exe versions: ffmpeg present, ffprobe present
[debug] Proxy map: {}
 622fa342f: Downloading webpage
ERROR: Cannot download embed-only video without embedding URL. Please call youtube-dl with the URL of the page that embeds this video.
Traceback (most recent call last):
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/vimeo.py", line 636, in _real_extract
    webpage, urlh = self._download_webpage_handle(
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 667, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 652, in _request_webpage
    raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/vimeo.py", line 636, in _real_extract
    webpage, urlh = self._download_webpage_handle(
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 667, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 652, in _request_webpage
    raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/home/tux/anaconda3/envs/youtube-dl/lib/python3.9/site-packages/youtube_dl/extractor/vimeo.py", line 643, in _real_extract
    raise ExtractorError(
youtube_dl.utils.ExtractorError: Cannot download embed-only video without embedding URL. Please call youtube-dl with the URL of the page that embeds this video.

youtube-dl ERROR: unable to download video data: HTTP Error 403: Forbidden

 in GNU/Linux  tagged error / pip / youtube-dl by

Recently, we were getting the following error in youtube-dl :

ERROR: unable to download video data: HTTP Error 403: Forbidden

We weren’t sure what that error meant (in the case of youtube-dl as the video is public) so we updated pip and youtube-dl to be sure that our problem was not that. The commands we used were the following:

python -m pip install --upgrade pip;
pip install --upgrade youtube-dl;

After the updates, we still got the same error… After some google-fu we read that it could be a caching issue! We cleared the cache as follows:

youtube-dl --rm-cache-dir;

Guess what? After that, the problem was resolved and we were able to reuse youtube-dl.

youtube-dl does not work properly on Qubes 4.0 – Fedora 26

 in GNU/Linux  tagged dnf / fedora 26 / qubes / qubes 4.0 / qubes-os / youtube-dl by

While working on a GNU/Linux Fedora 26 virtual machine running under the Qubes 4.0 OS, we installed youtube-dl through dnf to download some media off the net.


sudo dnf install youtube-dl;

When we tried to use it, we got the error youtube_dl.utils.RegexNotFoundError: Unable to extract Initial JS player signature function name. This issue was resolved years ago, so it led us to the conclusion that our version of youtube-dl was out of date. To test this hypothesis, we updated youtube-dl with pip.


sudo pip install --upgrade youtube_dl;

After the update was complete, we tried to use youtube-dl, this time with success!!

Full Logs:

[george@local Music]$ youtube-dl
bash: youtube-dl: command not found...
[george@local Music]$ sudo dnf install youtube-dl
Last metadata expiration check: 0:52:47 ago on Sat Nov 10 10:43:46 2018.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
youtube-dl noarch 2018.04.16-1.fc26 updates 2.6 M

Transaction Summary
================================================================================
Install 1 Package

Total download size: 2.6 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
youtube-dl-2018.04.16-1.fc26.noarch.rpm 270 kB/s | 2.6 MB 00:09 
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 258 kB/s | 2.6 MB 00:10 
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1 
Installing : youtube-dl-2018.04.16-1.fc26.noarch 1/1 
Running scriptlet: youtube-dl-2018.04.16-1.fc26.noarch 1/1 
Running as unit: run-rfddd15fff2d14d109826a90f59325e97.service
Verifying : youtube-dl-2018.04.16-1.fc26.noarch 1/1 
Notifying dom0 about installed applications

Installed:
youtube-dl.noarch 2018.04.16-1.fc26

Complete!
[george@local Music]$ youtube-dl https://www.youtube.com/watch?list=r6akoO34yUvK8ddtjnzL
 r6akoO34yUvK8ddtjnzL: Downloading webpage
[download] Downloading playlist: How to cook a banana
 playlist How to cook a banana: Downloading 18 videos
[download] Downloading video 1 of 18
 ir86d1hTSv1r: Downloading webpage
 ir86d1hTSv1r: Downloading video info webpage
 ir86d1hTSv1r: Extracting video information
 ir86d1hTSv1r: Downloading js player vfls4aurX
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1191, in _decrypt_signature
video_id, player_url, s
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1102, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1163, in _parse_sig_js
jscode, 'Initial JS player signature function name', group='sig')
File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 808, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract Initial JS player signature function name; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
(caused by RegexNotFoundError('Unable to extract \x1b[0;34mInitial JS player signature function name\x1b[0m; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[george@local Music]$ sudo pip install --upgrade youtube_dl
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Collecting youtube_dl
Downloading https://files.pythonhosted.org/packages/6c/a4/c2e3fbd8b9c7ccbca3f220c4fb0914926669bf9080fb8f2f4db4811706be/youtube_dl-2018.11.7-py2.py3-none-any.whl (1.8MB)
100% |████████████████████████████████| 1.8MB 355kB/s 
Installing collected packages: youtube-dl
Successfully installed youtube-dl-2018.11.7
[george@local Music]$ youtube-dl https://www.youtube.com/watch?list=r6akoO34yUvK8ddtjnzL
 r6akoO34yUvK8ddtjnzL: Downloading webpage
[download] Downloading playlist: How to cook a banana
 playlist How to cook a banana: Downloading 18 videos
[download] Downloading video 1 of 18
 ir86d1hTSv1r: Downloading webpage
 ir86d1hTSv1r: Downloading video info webpage
 ir86d1hTSv1r: Downloading js player vfls4aurX
WARNING: Requested formats are incompatible for merge and will be merged into mkv.

Extract audio from online video

 in Applications  tagged audio / extract / video / youtube-dl by

Using the youtube-dl command line application you can download videos and directly extract the audio of the video in various formats.

youtube-dl supports a large variety of online video hosts, including:

  • youtube.com
  • 9gag.com
  • crunchyroll.com
  • dailymotion.com
  • southparkstudios.com

Note: Please be sure that you are allowed to download a video before you do that, many of these hosts do not expect that you will be downloading their videos as they do not allow that.

The following command, will download a video, convert it to an mp3 and delete the original video:

youtube-dl --extract-audio --audio-format mp3 https://www.youtube.com/watch?v=cwI-n3sI8ec

If you want to keep the original video, you just add the parameter -k or --keep-video.

The --audio-format parameter accepts other types of audio format outputs, specifically it supports

youtube-dl is a very powerful tool, advice the documentation for some of the great features it supports.