Cut a video based on start and end time using FFmpeg

You probably don’t have a keyframe at the specified second mark if you can’t cut a video at a particular moment.
Non-keyframes need all of the data beginning with the previous keyframe because they encode variations from other frames.

Using an edit list, it is possible to cut at a non-keyframe with the mp4 container without re-encoding.
In other words, if the closest keyframe before 3s is at 0s, the video will be copied starting at 0s, and FFmpeg will use an edit list to tell the player to begin playing 3 seconds in.

If you’re using the latest version of FFmpeg from git master, it’ll use an edit list when you run it with the command you give.
If this does not work for you, it is you are likely using an older version of FFmpeg or that your player does not support edit lists.
Some players can disregard the edit list and play the entire file from beginning to end, regardless of the edit list.

If you want to cut specifically at a non-keyframe and have it play at the desired point on a player that doesn’t support edit lists, or if you want to make sure the cut section isn’t in the output file (for example, if it includes sensitive information), you can do so by re-encoding so that a keyframe is present at the desired start time.
If you don’t mention copy, re-encoding is the norm.
Consider the following scenario:

ffmpeg -i input.mp4 -ss 00:00:07 -t 00:00:18 -async 1 output.mp4
  • The -t option defines a length rather than an end time.
  • The command above will encode 18 seconds of video beginning at 7 seconds.
  • Use -t 8 to start at 7 seconds and end at 15 seconds.
  • If you’re using a recent version of FFmpeg, you can also use -to instead of -t in the above command to make it end at the required time.

psftp.exe: The server’s host key is not cached in the registry. You have no guarantee that the server is the computer you think it is.

Recently, we were debugging a scheduled job running on a Microsoft SQL Server Agent. After starting the SQL Server Management Studio, we saw that the specific task was using psftp.exe to upload some data securely to a remote server. When executing the job manually, it would work as expected. On the other hand, when the job would be executed automatically, it would always fail. After review the error logs, we got the following message:

Executed as user: FSRV\SYSTEM. …s\FTP\remote-server.ppk [email protected] -batch -bc -be -b C:\putty\upload.txtThe server's host key is not cached in the registry. You  have no guarantee that the server is the computer you  think it is.  The server's rsa2 key fingerprint is:  ssh-rsa 2048 39:e4:84:b2:6f:bc:87:04:1f:21:bf:32:83:79:0b:cf  Connection abandoned.  DTSRun:  Loading…   DTSRun:  Executing…   DTSRun OnStart:  DTSStep_DTSExecuteSQLTask_1   DTSRun OnFinish:  DTSStep_DTSExecuteSQLTask_1   DTSRun OnStart:  DTSStep_DTSExecuteSQLTask_3   DTSRun OnFinish:  DTSStep_DTSExecuteSQLTask_3   DTSRun OnStart:  DTSStep_DTSActiveScriptTask_2   DTSRun OnFinish:  DTSStep_DTSActiveScriptTask_2   DTSRun OnStart:  DTSStep_DTSActiveScriptTask_3   DTSRun OnFinish:  DTSStep_DTSActiveScriptTask_3   DTSRun OnStart:  DTSStep_DTSDataPumpTask_1   DTSRun OnProgress:  DTSStep_DTSDataPumpTask_1; 34 Rows have been transformed or copied.; …  Process Exit Code 1.  The step failed.

The problem was with the account executing the scheduled job, which was different than the one that created the job. The second account, the one that was executing the scheduled jobs did not have any knowledge of the ssh-rsa key of the remote server. Because of this lack of information, psftp.exe could not verify that we were indeed trying to connect to the correct server. To fix this issue, we modified the psftp.exe execution command to match the following one:

C:\putty\psftp.exe -i C:\connections\FTP\remote-server.ppk [email protected] -batch -bc -be -b C:\putty\upload.txt -hostkey 39:e4:84:b2:6f:bc:87:04:1f:21:bf:32:83:79:0b:cf

To help any reader that is not familiar with the psftp.exe and powershell (or cmd) we will breakdown the arguments of the above command:

  • C:\putty\psftp.exe : is the exact location of the psftp.exe binary on that server
  • -i C:\connections\FTP\remote-server.ppk : PPK files are PuTTY Private Key Files developed by Putty and they serve as storage for the private keys the program generated. In this case, instead of using a combination of username and password to authenticate, the client was given a private key to use as proof of identity and authenticity.
  • [email protected] : The username and the domain or IP of the remote server.
  • -batch : Disables interactive prompts as no person will be supervising the script.
  • -bc : It displays batch commands in the same way they are run. It is useful for logging and troubleshooting.
  • -be : When running a batch file, this additional option causes psftp.exe to continue processing even if a command fails to complete successfully. An example you might want this to happen is the following: you want to delete a file and don’t care if it is already not present.
  • -b C:\putty\upload.txt : It specifies a file with batch commands. This argument helps users automate tasks by allowing them to set commands in advance.
  • -hostkey 39:e4:84:b2:6f:bc:87:04:1f:21:bf:32:83:79:0b:cf : Here, we copied the rsa-ssh key of the server that was displayed on the error and we explicitly defined it to let psftp.exe that it is trying to connect to the correct server. If you are not sure if the value you get at the errors is indeed the correct value, consult your system administrator.


Cloudflare: Error 522 – Connection timed out

Recently, we’ve been noticing weird behavior on sites that are proxied behind Cloudflare servers. While visiting a site from one internet connection, we are constantly getting a 522 error. If at that moment, we switch our internet connection on that machine and visit the same page, it works as expected.

We are sure that the servers we have access to did not run out of resources, nor does the firewall block Cloudflare IPs. All firewalls are properly configured to accept connections from the official Cloudflare IPs that were retrieved from here.

We will investigate it further, hopefully we will figure it out.


AttributeError: module ‘html5lib.treebuilders’ has no attribute ‘_base’

Recently, we were receiving the above error on a GNU/Linux Ubuntu 20.04 LTS.

AttributeError: module 'html5lib.treebuilders' has no attribute '_base'

We had installed beautifulsoup4 and html5lib using pip. To solve the issue, we had to uninstall the html5lib that was installed by pip and install it through apt.

# Remove html5lib if you have installed it with pip3:
pip3 uninstall html5lib;

pip3 install --upgrade beautifulsoup4
sudo apt-get install python3-html5lib

How to check if PyTorch is using the GPU?

The following basic code, will import PyTorch into a project and test if GPU capabilities are enabled and available.

import torch

# Should produce "True"
torch.cuda.is_available()

# Should produce the number of available devices, if you have one device it should produce the value "1"
torch.cuda.device_count()

# If there is a device and it is the first one, it should produce the "0"
torch.cuda.current_device()

# Assuming that there is at least one device, with the following two commands we will get some information on the first available device

# Should produce something similar to "<torch.cuda.device object at 0x7f12b1a298d0>"
torch.cuda.device(0)

# Should produce something similar to "'GeForce GTX 1050 Ti'"
torch.cuda.get_device_name(0)

We are using Ubuntu 20.04 LTS and the NVidia drivers were installed automatically during installation.