How to upgrade all Python packages with pip
Below is the command that we use to update all outdated packages in pip.
pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U;
update
Bind for 0.0.0.0:443 failed: port is already allocated
On a Docker installation that we have, we updated the image files for our containers using the following command:
docker images --format "{{.Repository}}:{{.Tag}}" | grep ':latest' | xargs -L1 docker pull;
Then we tried to update our container, as usual, using the docker-compose command.
export COMPOSE_HTTP_TIMEOUT=180; # We extend the timeout to ensure there is enough time for all containers to start
docker-compose up -d --remove-orphans;
Unfortunately, we got the following error:
export COMPOSE_HTTP_TIMEOUT=180; docker-compose up -d --remove-orphans; Starting entry ... Starting entry ... error ERROR: for entry Cannot start service entry: driver failed programming external connectivity on endpoint entry (d3a5d95f55c4e872801e92b1f32d9693553bd553c414a371b8ba903cb48c2bd5): Bind for 0.0.0.0:443 failed: port is already allocated ERROR: for entry Cannot start service entry: driver failed programming external connectivity on endpoint entry (d3a5d95f55c4e872801e92b1f32d9693553bd553c414a371b8ba903cb48c2bd5): Bind for 0.0.0.0:443 failed: port is already allocated ERROR: Encountered errors while bringing up the project.
We used the docker container ls command to check which container was hoarding port 443, but none was doing so. Because of this, we assumed that docker ran into a bug. The first step we took (and the last) which solved the problem was to restart the docker service as follows:
sudo service docker restart;
This command was enough to fix our problem without messing with docker further.
ERROR: for container_a UnixHTTPConnectionPool(host=’localhost’, port=None): Read timed out. (read timeout=60)
There is this docker server that we have access to, which probably due to lousy planning, we put way too many containers on it. The server does not have SSD disks, and for that reason, whenever there are too many IO operations, it becomes unresponsive. When we mass update all containers by updating the images using the following command and then issuing a fresh docker-compose, we get a lot of time-out errors.
The commands we use to update the images and recreate our containers using the new images are the following:
(Please note that these commands need to execute from the folder where the file docker-compose.yml resides)
#Update all docker images that have the 'latest' tag
docker images --format "{{.Repository}}:{{.Tag}}" | grep ':latest' | xargs -L1 docker pull;
#Rebuild all containers using the new images.
docker-compose up -d;
After executing the second command, we often get many copies of the following error:
ERROR: for container_a UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
This error indicates that the recreate command was waiting for too long for the docker daemon to respond with no success. At the end of the output, we can see that it was waiting for 60 seconds.
At the end of the output, we get the following information and advice:
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information. If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
Following the advice, we used the following command to overwrite the value of the COMPOSE_HTTP_TIMEOUT variable to a more significant number.
#Increase timeout period to 120 seconds.
export COMPOSE_HTTP_TIMEOUT=120;
#Rebuild all containers using the new images.
docker-compose up -d;
Doing so, we were able to rebuild all containers without reissuing many times the up command.
Sidenote
This server really does have a lot of containers, we had to create the file /etc/docker/daemon.json so that we would have enough network addressing space to handle all the bridges and sub-networks.
The contents of /etc/docker/daemon.json are:
{
"default-address-pools": [
{
"base": "172.80.0.0/16",
"size": 24
},
{
"base": "172.90.0.0/16",
"size": 24
}
]
}
The above configuration solved the following problem for us:
ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network
Cannot update Kali Linux due to invalid signatures 4
We have this Kali live USB for a few months and it was not used in between. Yesterday, we tried to use it and noticed that we could not perform any updates as we were getting the following error:
The following signatures were invalid: EXPKEYSIG ED444FF07D8D0BF6 Kali Linux Repository <[email protected]>
The same stick was used in the past to create a native installation on a PC that had the same issue (because it was also not used for some time).
To fix the issue we had to get a fresh copy of the archive key for Kali repositories and import it into apt.
To do so, our one line solution was the following:
wget -q -O - https://archive.kali.org/archive-key.asc | apt-key add;
To breakdown what the command is all about: we used wget, which is a non-interactive network retriever, to download the PGP public key for the Kali archives. Instead of saving the key as a file to the filesystem, we used the parameters -q and -O so that wget would be quiet and not create any output of its own and print the key on the terminal (using -O). Then we piped the key to apt-key add to add it to the system. After doing so, we were able to properly update the complete system using the following commands:
apt-get update && apt-get upgrade && apt-get dist-upgrade;
Below you will find our console logs:
root@kali:~# apt-get update && apt-get upgrade && apt-get dist-upgrade Get:1 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling InRelease [30.5 kB] Err:1 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling InRelease The following signatures were invalid: EXPKEYSIG ED444FF07D8D0BF6 Kali Linux Repository <[email protected]> Fetched 30.5 kB in 2s (10.5 kB/s) Reading package lists... Done W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling InRelease: The following signatures were invalid: EXPKEYSIG ED444FF07D8D0BF6 Kali Linux Repository <[email protected]> W: Failed to fetch http://http.kali.org/kali/dists/kali-rolling/InRelease The following signatures were invalid: EXPKEYSIG ED444FF07D8D0BF6 Kali Linux Repository <[email protected]> W: Some index files failed to download. They have been ignored, or old ones used instead. Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. root@kali:~# wget -q -O - https://archive.kali.org/archive-key.asc | apt-key add OK root@kali:~# apt-get update && apt-get upgrade && apt-get dist-upgrade Get:1 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling InRelease [30.5 kB] Get:2 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling/main amd64 Packages [16.2 MB] Get:2 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling/main amd64 Packages [16.2 MB] Get:3 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling/non-free amd64 Packages [166 kB] Get:4 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling/contrib amd64 Packages [99.0 kB] Fetched 451 kB in 1min 6s (6790 B/s) Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done
