Source code for the implementation that created this video will be uploaded soon.
A first attempt at using a pre-trained implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of an object in each frame. It’s based on Feature Pyramid Network (FPN) and a ResNet101 backbone.
Setup
Conda / Anaconda
First of all, we installed and activated anaconda on an Ubuntu 20.04LTS desktop. To do so, we installed the following dependencies from the repositories:
Following the previous step, we used the commands below to create a virtual environment for our code. We needed python version 3.9 (as highlighted here https://www.anaconda.com/products/individual#linux) and OpenCV for python.
1
2
3
4
source~/anaconda3/bin/activate;
conda create --name MaskRNN python=3.9;
conda activate MaskRNN;
pip installnumpy opencv-python;
Problems that we did not anticipate
When we tried to execute our code in the virtual environment:
1
python3 main.py --video="/home/bob/Videos/Live @ Santa Claus Village 2021-11-13 12_12.mp4";
We got the following error:
Traceback (most recent call last):
File "/home/bob/MaskRCNN/main.py", line 6, in <module>
from cv2 import cv2
File "/home/bob/anaconda3/envs/MaskRNN/lib/python3.9/site-packages/cv2/__init__.py", line 180, in <module>
bootstrap()
File "/home/bob/anaconda3/envs/MaskRNN/lib/python3.9/site-packages/cv2/__init__.py", line 152, in bootstrap
native_module = importlib.import_module("cv2")
File "/home/bob/anaconda3/envs/MaskRNN/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
We realized that we were missing some additional dependencies for OpenCV as our Ubuntu installation was minimal. To fix this issue, we installed the following package from the repositories:
Recently, we were working on a python 3 project that was using opencv. When we used condadefaults repository to install opencv it installed a 3.X.Y version which would produce the following error on PyCharm on Ubuntu 20.04 LTS:
pycharm cv2.error: OpenCV(3.4.2) highgui/src/window.cpp:710: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvStartWindowThread'
Following the instructions above, we installed the missing packages using sudo apt-get install libgtk2.0-dev pkg-config; with no change on the results. To fix the issue, we completely removed opencv from the stables repository and we installed version 4.5.1 from the conda-forge repository as follows:
Since there was no bug fix at the time, we had to take action into patching our application. Below are the steps you need to follow to replicate the bug fix:
Unzip the libpng for Android downloaded file. Go to [LIBPNG_FOR_ANDROID_FOLDER]/jni/ and using a text editor open file config.h. Find the values for the following 3 variables PACKAGE_STRINGPACKAGE_VERSIONVERSION and make sure that they have the correct version number. At the time this post was written, the author of the repository omitted to update those values from 1.4.1 to 1.6.23. ([download id=”2000″])
Using a terminal go into the folder that was created and execute: ./build.sh
In case you get the error ./build.sh: line 2: ndk-build: command not found then edit build.sh using a text editor and on line 2 add the full path to the file ndk-build that is in the folder of the Android NDK you unzipped before. After you do this change execute ./build.sh again.
Inside each of the folders in [LIBPNG_FOR_ANDROID_FOLDER]/obj/local/ you will find a file called libpng.a.
Copy those files while renaming them to liblibpng.a into the respective folders of the OpenCV library you are using in you project that are found here [OPENCV_FOLDER]/sdk/native/3rdparty/libs/.
Inside [LIBPNG_FOR_ANDROID_FOLDER]/obj/local/ we found the following folders: arm64-v8a, armeabi, armeabi-v7a, mips, mips64, x86, x86_64.
We did not use all of them as in [OPENCV_FOLDER]/sdk/native/3rdparty/libs/ we only had armeabi, armeabi-v7a, mips, x86.
Rebuild your applications, test them and if they are OK upload the new versions online.
Hello Google Play Developer,
We detected that your app(s) listed at the end of this email are using an unsafe version of the libpng library. Apps with vulnerabilities like this can expose users to risk of compromise and may be considered in violation of our Malicious Behavior policy.
What’s happening
Beginning September 17, 2016, Google Play will block publishing of any new apps or updates that use vulnerable versions of libpng. Your published APK version will not be affected, however any updates to the app will be blocked unless you address this vulnerability.
Action required: Migrate your app(s) to use libpng v1.0.66, v.1.2.56, v.1.4.19, v1.5.26 or higher as soon as possible and increment the version number of the upgraded APK.
Next steps
Download the latest version of libpng from the libpng website.
Check back after five hours – we’ll show a warning message if the app hasn’t been updated correctly.
The vulnerability stems from an out of bounds memory access that could potentially lead to code execution. Versions 1.0.x before 1.0.66, 1.1.x and 1.2.x before 1.2.56, 1.3.x and 1.4.x before 1.4.19, and 1.5.x before 1.5.26 are affected.
You can read more about the vulnerability in CVE-2015-8540. For other technical questions about the vulnerability, you can post to Stack Overflow and use the tag “android-security.”
While these specific issues may not affect every app that uses libpng, it’s best to stay up to date on all security patches.https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-8540
We’re here to help
If you feel we have sent this warning in error, you can contact our developer support team.
Regards,
The Google Play Team
Affected app(s) and version(s) are listed below. If you have more than 20 affected apps in your account, please check the Developer Console for a full list.
net.bytefreaks.opencvfacerecognition 1
Related post by Google Play Team:
How to fix apps containing Libpng Vulnerability
This information is intended for developers of apps that utilize any version of libpng library, that contains a security vulnerability disclosed in CVE-2015-8540. Apps with vulnerabilities like this can expose users to risk of compromise and may be considered in violation of our Malicious Behavior policy.
Please migrate your app(s) to libpng v1.0.66, v.1.2.56, v.1.4.19, v1.5.26 or higher as soon as possible and increment the version number of the upgraded APK. Beginning Sep 17, 2016, Google Play will block publishing of any new apps or updates that use vulnerable versions of libpng. Your published app version will remain unaffected, however any updates to the app will be blocked unless they address this vulnerability.
Next steps
Download the latest version of libpng from the libpng website.
Check back after five hours – we’ll show a warning message if the app hasn’t been updated correctly. Note that some processing delays are common even if your app has fixed the vulnerability.
The vulnerability stems from an out of bounds memory access that could potentially lead to code execution. Versions 1.0.x before 1.0.66, 1.1.x and 1.2.x before 1.2.56, 1.3.x and 1.4.x before 1.4.19, and 1.5.x before 1.5.26 are affected.
You can read more about the vulnerability in CVE-2015-8540.
For other technical questions, you can post to Stack Overflow and use the tags “android-security”. Note that questions about Play policy should not be posted to Stack Overflow.
While these specific issues may not affect every app that uses libpng, it’s best to stay up to date on all security patches. Apps must also comply with the Developer Distribution Agreement and Content Policy. If you feel you have received this vulnerability warning in error, contact our policy support team through the Google Play Developer Help Center.
