object detection

Playing with MASK RCNN on videos .. again

 in Artificial intelligence / Deep Learning / Machine Learning / Neural Networks  tagged ai / anaconda / artificial intelligence / bash / CNN / conda / convolutional neural network / Feature Pyramid Network / GNU/Linux / keras / mask / MASK RCNN / object detection / opencv / python / RCNN / ResNet101 / TensorFlow / ubuntu by

Source code for the implementation that created this video will be uploaded soon.

A first attempt at using a pre-trained implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of an object in each frame. It’s based on Feature Pyramid Network (FPN) and a ResNet101 backbone.

Setup

Conda / Anaconda

First of all, we installed and activated anaconda on an Ubuntu 20.04LTS desktop. To do so, we installed the following dependencies from the repositories:

sudo apt-get install libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6;

Then, we downloaded the 64-Bit (x86) Installer from (https://www.anaconda.com/products/individual#linux).

Using a terminal, we followed the instructions here (https://docs.anaconda.com/anaconda/install/linux/) and performed the installation.

Python environment and OpenCV for Python

Following the previous step, we used the commands below to create a virtual environment for our code. We needed python version 3.9 (as highlighted here https://www.anaconda.com/products/individual#linux) and OpenCV for python.

source ~/anaconda3/bin/activate;
conda create --name MaskRNN python=3.9;
conda activate MaskRNN;
pip install numpy opencv-python;

Problems that we did not anticipate

When we tried to execute our code in the virtual environment:

python3 main.py --video="/home/bob/Videos/Live @ Santa Claus Village 2021-11-13 12_12.mp4";

We got the following error:

Traceback (most recent call last):
  File "/home/bob/MaskRCNN/main.py", line 6, in <module>
    from cv2 import cv2
  File "/home/bob/anaconda3/envs/MaskRNN/lib/python3.9/site-packages/cv2/__init__.py", line 180, in <module>
    bootstrap()
  File "/home/bob/anaconda3/envs/MaskRNN/lib/python3.9/site-packages/cv2/__init__.py", line 152, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/home/bob/anaconda3/envs/MaskRNN/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libGL.so.1: cannot open shared object file: No such file or directory

We realized that we were missing some additional dependencies for OpenCV as our Ubuntu installation was minimal. To fix this issue, we installed the following package from the repositories:

sudo apt-get update;
sudo apt-get install -y python3-opencv;

Installing TensorFlow 2 Object detection on Ubuntu 18.04 LTS 1

 in Bash / Face Recognition / GNU/Linux  tagged coco / object detection / protobuf / python / TensorFlow by

Following are some rough notes on Installing TensorFlow 2 Object detection on Ubuntu 18.04 LTS.
We were following this guide (https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/install.html) so we will be skipping some steps.

We had conda installed already from an older attempt so the following steps worked just fine.

conda create -n tensorflow pip python=3.8;
conda activate tensorflow;

We got an error with the following command so we used pip3 instead of pip.

pip install --ignore-installed --upgrade tensorflow==2.2.0;

Command 'pip' not found, but there are 18 similar ones.
pip3 install --ignore-installed --upgrade tensorflow==2.2.0;

Executing the above gave us another error:

Collecting tensorflow==2.2.0
Could not find a version that satisfies the requirement tensorflow==2.2.0 (from versions: 0.12.1, 1.0.0, 1.0.1, 1.1.0rc0, 1.1.0rc1, 1.1.0rc2, 1.1.0, 1.2.0rc0, 1.2.0rc1, 1.2.0rc2, 1.2.0, 1.2.1, 1.3.0rc0, 1.3.0rc1, 1.3.0rc2, 1.3.0, 1.4.0rc0, 1.4.0rc1, 1.4.0, 1.4.1, 1.5.0rc0, 1.5.0rc1, 1.5.0, 1.5.1, 1.6.0rc0, 1.6.0rc1, 1.6.0, 1.7.0rc0, 1.7.0rc1, 1.7.0, 1.7.1, 1.8.0rc0, 1.8.0rc1, 1.8.0, 1.9.0rc0, 1.9.0rc1, 1.9.0rc2, 1.9.0, 1.10.0rc0, 1.10.0rc1, 1.10.0, 1.10.1, 1.11.0rc0, 1.11.0rc1, 1.11.0rc2, 1.11.0, 1.12.0rc0, 1.12.0rc1, 1.12.0rc2, 1.12.0, 1.12.2, 1.12.3, 1.13.0rc0, 1.13.0rc1, 1.13.0rc2, 1.13.1, 1.13.2, 1.14.0rc0, 1.14.0rc1, 1.14.0, 2.0.0a0, 2.0.0b0, 2.0.0b1)
No matching distribution found for tensorflow==2.2.0

To fix it we upgraded pip using the following command.

python3 -m pip install --upgrade pip;

Then we tried again, which installed most packets but gave a new error:

pip3 install --ignore-installed --upgrade tensorflow==2.2.0;

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
launchpadlib 1.10.6 requires testresources, which is not installed.
Successfully installed absl-py-0.11.0 astunparse-1.6.3 cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 gast-0.3.3 google-auth-1.27.0 google-auth-oauthlib-0.4.2 google-pasta-0.2.0 grpcio-1.35.0 h5py-2.10.0 idna-2.10 importlib-metadata-3.4.0 keras-preprocessing-1.1.2 markdown-3.3.3 numpy-1.19.5 oauthlib-3.1.0 opt-einsum-3.3.0 protobuf-3.14.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.25.1 requests-oauthlib-1.3.0 rsa-4.7.1 scipy-1.4.1 setuptools-53.0.0 six-1.15.0 tensorboard-2.2.2 tensorboard-plugin-wit-1.8.0 tensorflow-2.2.0 tensorflow-estimator-2.2.0 termcolor-1.1.0 typing-extensions-3.7.4.3 urllib3-1.26.3 werkzeug-1.0.1 wheel-0.36.2 wrapt-1.12.1 zipp-3.4.0

To fix this error we used:

sudo apt install python3-testresources;

Then tried again the pip installation with success.

pip3 install --ignore-installed --upgrade tensorflow==2.2.0;

Successfully installed absl-py-0.11.0 astunparse-1.6.3 cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 gast-0.3.3 google-auth-1.27.0 google-auth-oauthlib-0.4.2 google-pasta-0.2.0 grpcio-1.35.0 h5py-2.10.0 idna-2.10 importlib-metadata-3.4.0 keras-preprocessing-1.1.2 markdown-3.3.3 numpy-1.19.5 oauthlib-3.1.0 opt-einsum-3.3.0 protobuf-3.14.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.25.1 requests-oauthlib-1.3.0 rsa-4.7.1 scipy-1.4.1 setuptools-53.0.0 six-1.15.0 tensorboard-2.2.2 tensorboard-plugin-wit-1.8.0 tensorflow-2.2.0 tensorflow-estimator-2.2.0 termcolor-1.1.0 typing-extensions-3.7.4.3 urllib3-1.26.3 werkzeug-1.0.1 wheel-0.36.2 wrapt-1.12.1 zipp-3.4.0

We then executed the following to test the installation:

python3 -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))";

Then we proceeded to get the TensorFlow models:

mkdir ~/TensorFlow;
cd ~/TensorFlow;
git clone https://github.com/tensorflow/models;

We then downloaded protobufs and extracted them to our home directory.
To test the installation we did the following.

export PATH="/home/bob/protoc-3.14.0-linux-x86_64:$PATH";
cd /home/bob/TensorFlow/models/research;
protoc object_detection/protos/*.proto --python_out=.;

Then we proceeded to the COCO installation:

pip3 install cython;

The above will solve the problem of:

gcc: error: pycocotools/_mask.c: No such file or directory
cd ~;
git clone https://github.com/cocodataset/cocoapi.git;
cd cocoapi/PythonAPI;
make;
cp -r pycocotools ~/TensorFlow/models/research/;

Finally we proceeded to Install the Object Detection API.

cd ~/TensorFlow/models/research/;
cp object_detection/packages/tf2/setup.py .;
python3 -m pip install .;

To test the installation we executed the following:

python3 object_detection/builders/model_builder_tf2_test.py;

We then downloaded the samples and executed the camera sample with success!!

To check against a video instead of a camera, we changed the following line from:

cap = cv2.VideoCapture(0)

to

cap = cv2.VideoCapture('/home/bob/Desktop/a2/A01_20210210164306.mp4')