Tux


Automating Video Retrieval from HIKVision NVR using Python Scripts

In today’s surveillance-driven world, managing and retrieving recorded videos from Network Video Recorders (NVRs) is crucial for security professionals. This blog post will introduce a set of Python scripts that automate the process of searching for and downloading recorded videos from a HIKVision NVR. The scripts enable users to specify a date range and camera track, making it easier to access and manage video footage efficiently.

The Python Scripts:

generate.py

#!/usr/bin/env python

# This scripts calls search.py to search in the HIKVision NVR for recorded videos and the uses download.py to download those videos.
# The script loops over the camera tracks and the last 120 days.

import sys
import os
import datetime

base = datetime.datetime.today().replace(hour=0, minute=0, second=0, microsecond=0);
numdays = 120;
dateList = [base - datetime.timedelta(days=x) for x in range(numdays)];

tracks = ["101", "201", "301", "401", "501", "601", "701", "801"];

for trackID in tracks:
  for dateItem in dateList:
    os.system("python search.py " + trackID + " " + dateItem.strftime('%Y-%m-%dT%H:%M:%SZ') + " " + (dateItem + datetime.timedelta(days=1)).strftime('%Y-%m-%dT%H:%M:%SZ'));

for trackID in tracks:
  for dateItem in dateList:
    os.system("python download.py " + trackID + " " + dateItem.strftime('%Y-%m-%dT%H:%M:%SZ') + " " + (dateItem + datetime.timedelta(days=1)).strftime('%Y-%m-%dT%H:%M:%SZ'));

  • This script acts as the orchestrator, controlling the entire process.
  • It generates a list of dates, spanning the last 120 days, and a list of camera tracks to search for video recordings.
  • It then iterates through each camera track and date, calling two other Python scripts: search.py and download.py.

search.py

#!/usr/bin/env python

# This script makes an API call to the HIKVision NVR with a Track ID and a datetime range and retrieves an XML list with all videos with their download links that were recorded on that camera during that time period.

import sys
import os

trackID = sys.argv[1];
startTime = sys.argv[2];
endTime = sys.argv[3];
xmlFilename = "results/" + trackID + "." + startTime + "." + endTime + ".xml";

os.system("curl 'http://username:[email protected]/ISAPI/ContentMgmt/search' --data-raw $'<?xml version='1.0' encoding='UTF-8'?>\n<CMSearchDescription><searchID>CA77BA52-0780-0001-34B2-6120F2501D36</searchID><trackList><trackID>" + trackID + "</trackID></trackList><timeSpanList><timeSpan><startTime>" + startTime + "</startTime><endTime>" + endTime + "</endTime></timeSpan></timeSpanList><maxResults>100</maxResults><searchResultPostion>0</searchResultPostion><metadataList><metadataDescriptor>//recordType.meta.std-cgi.com</metadataDescriptor></metadataList></CMSearchDescription>' -o "+ xmlFilename);

  • This script is responsible for making an API call to the HIKVision NVR to search for recorded videos.
  • It takes three command-line arguments: track ID, start time, and end time.
  • It constructs a search request in XML format and uses curl to send the request to the NVR.
  • The search results are saved as an XML file for later processing.

download.py

#!/usr/bin/env python

# This script reads an XML file that was retrieved from the HIKVision NVR which containes videos with their download links. For each link, it appends the credentials for login and uses ffmpeg to download the video.

from xml.dom import minidom
import os
import sys

trackID = sys.argv[1];
startTime = sys.argv[2];
endTime = sys.argv[3];
xmlFilename = "results/" + trackID + "." + startTime + "." + endTime + ".xml";
dom = minidom.parse(xmlFilename)
elements = dom.getElementsByTagName('playbackURI')

i = 0
for element in elements:
    video = element.firstChild.data
    video = video.replace("rtsp://10.20.30.1", "rtsp://username:[email protected]")
    video = video.replace("\n", "")
    size = video.rsplit('=', 1)[1]
    os.system("ffmpeg -i '" + video + "' -max_muxing_queue_size " + size + "0 videos/" + trackID + "." + startTime + "." + endTime + "." + str(i+1) + ".mp4;")
    i += 1
exit

  • After the search has been performed and results stored in an XML file, this script is called to download the videos.
  • It reads the XML file and extracts the video playback URLs.
  • For each video, it appends the required credentials for login and uses ffmpeg to download the video.
  • Downloaded videos are saved with a filename indicating track ID, start time, end time, and a unique index.

Usage:

To use these scripts, you’ll need to modify the following parts:

  • Update the base variable in generate.py to set the desired starting date.
  • Adjust the tracks list in generate.py to specify the camera tracks you want to search.
  • Replace username, password, and the IP address in the curl command in search.py with your NVR’s credentials and address.
  • Ensure you have ffmpeg installed on your system for video downloading.

With these Python scripts, you can automate the process of searching for and downloading recorded videos from a HIKVision NVR. This can significantly simplify video retrieval tasks for security professionals, saving time and effort in managing surveillance footage. By customizing and expanding upon these scripts, you can further enhance your video management capabilities and streamline your security operations.


Upgrading Your HIKVision DVR Firmware using the API: A Step-by-Step Guide

Regularly updating the firmware of your HIKVision DVR (Digital Video Recorder) is crucial to ensure optimal performance and security. In this blog post, we will walk you through the process of upgrading your HIKVision DVR firmware using simple command-line tools like curl. We will also show you how to check the upgrade status to ensure a smooth and successful update.

Step 1: Preparing for the Upgrade

Before you begin, make sure you have the following information and files ready:

  • Your HIKVision DVR’s IP address (e.g., 10.20.30.1).
  • The username and password for accessing your DVR’s web interface.
  • The latest firmware file in DAV format (e.g., digicap.dav). Ensure you download this file from the official HIKVision website to guarantee its authenticity.

Step 2: Initiating the Firmware Upgrade

To start the firmware upgrade process, open your terminal or command prompt and use the curl command as follows:

curl -k --request PUT --data-binary "@digicap.dav" 'http://username:[email protected]/ISAPI/System/updateFirmware';

Explanation:

  • curl: This is a command-line tool for transferring data with URLs.
  • -k: This option tells curl to allow connections to SSL sites without certificates. It’s useful when connecting to devices with self-signed certificates.
  • --request PUT: This specifies the HTTP request method as PUT, which is used for updating the firmware.
  • --data-binary "@digicap.dav": Here, we provide the firmware file in DAV format as binary data.
  • 'http://username:[email protected]/ISAPI/System/updateFirmware': Replace username and password with your DVR’s login credentials, and 10.20.30.1 with your DVR’s IP address. This URL is where the firmware update request is sent.

Step 3: Checking the Upgrade Status

To monitor the status of the firmware upgrade and ensure everything is proceeding as expected, use the following curl command:

curl -k 'http://username:[email protected]/ISAPI/System/upgradeStatus';

Explanation:

  • curl: As before, this is the command-line tool for making URL requests.
  • -k: Again, this option allows connections to SSL sites without certificates.
  • 'http://username:[email protected]/ISAPI/System/upgradeStatus': Replace the placeholders with your DVR’s login credentials and IP address. This URL is where you can check the upgrade status.

Conclusion: Updating your HIKVision DVR’s firmware is essential for keeping it secure and running smoothly. By following these simple steps and using the curl commands provided, you can ensure that your DVR is up to date with the latest firmware. Remember to download firmware updates only from trusted sources like the official HIKVision website to avoid any security risks.

Sample Outputs

$ curl  -k  --request PUT --data-binary "@digicap.dav" 'http://username:[email protected]/ISAPI/System/updateFirmware';
<?xml version="1.0" encoding="UTF-8" ?>
<ResponseStatus version="1.0" xmlns="urn:psialliance-org">
<requestURL>/ISAPI/System/updateFirmware</requestURL>
<statusCode>7</statusCode>
<statusString>Reboot Required</statusString>
<subStatusCode>rebootRequired</subStatusCode>
</ResponseStatus>

# In another terminal as the above command blocks.
#Execute the status command.
$ curl  -k  'http://username:[email protected]/ISAPI/System/upgradeStatus';
<?xml version="1.0" encoding="UTF-8" ?>
<upgradeStatus version="1.0" xmlns="http://www.hikvision.com/ver20/XMLSchema">
<upgrading>true</upgrading>
<percent>98</percent>
</upgradeStatus>

$ curl  -k  'http://username:[email protected]/ISAPI/System/upgradeStatus';
<?xml version="1.0" encoding="UTF-8" ?>
<upgradeStatus version="1.0" xmlns="http://www.hikvision.com/ver20/XMLSchema">
<upgrading>false</upgrading>
<percent>0</percent>
</upgradeStatus>

Python: How to Connect and Use Office 365 Email

As Office 365 transitions away from Basic authentication and embraces Multi-Factor Authentication (MFA) for end-users and OAuth for other purposes, connecting Python to Office 365 email requires a slightly different approach. This blog post will explore how to connect Python to Office 365 email using the exchangelib library and OAuth2 credentials. By following these steps, you can access and interact with your Office 365 email programmatically.

Prerequisites: Before diving into the code, ensure you have the following prerequisites in place:

  1. Access to the Office 365 admin portal
  2. Basic knowledge of Python programming
  3. Required Python libraries: exchangelib

Step 1: Registering an App and Gathering Credentials: To connect Python with Office 365 email, you must register an application in the Azure Active Directory. Here’s how you can do it:

  1. Log into the Office 365 admin portal at https://admin.microsoft.com.
  2. Locate and click on the link to Azure Active Directory.
  3. Register a new app and make a note of the Directory (tenant) ID, Application (client) ID, and the secret (client secret).

Step 2: Granting App Permissions: To grant necessary permissions to the registered app, follow these steps:

  1. Navigate to the API permissions page within the Azure Active Directory.
  2. Add the full_access_as_app permission for your app.

Step 3: Verify App Permissions: To ensure the app has the required permissions, perform the following steps:

  1. Go to the Enterprise applications page in Azure Active Directory.
  2. Select your app.
  3. Continue to the Permissions page and verify that your app has the full_access_as_app permission.

Connecting Python to Office 365 Email: Now that we have the required credentials and permissions in place, let’s connect Python to Office 365 email using the exchangelib library. Here’s the code snippet to establish the connection:

import logging
from exchangelib import Account, Configuration, Identity, OAUTH2, OAuth2Credentials

logging.basicConfig(level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s')
logging.debug('Start...')

creds = OAuth2Credentials(
    client_id='4e89**********************',
    client_secret='cx67**********************',
    tenant_id='gt6**********************',
    identity=Identity(primary_smtp_address=r'[email protected]')
)

config = Configuration(server='outlook.office365.com', credentials=creds, auth_type=OAUTH2)

a = Account(
    primary_smtp_address='[email protected]',
    autodiscover=False,
    config=config
)

# Print first inbox messages in reverse order
for item in a.inbox.all().only('subject').order_by('-datetime_received')[:2]:
    print(item.subject)

a.protocol.close()

logging.debug('End...')

Make sure to replace the code’s placeholders with your credentials and email address.

Explanation of the Code:

  1. The exchangelib library is imported, and logging is set up to display any errors.
  2. OAuth2 credentials are created using the previously obtained client ID, client secret, tenant ID, and primary SMTP address.
  3. A configuration object is created with the server address, credentials, and authentication type.
  4. An Account object is initialized using the email address, disabling autodiscover, and providing the configuration.
  5. The code retrieves the two most recent inbox messages and prints their subjects.
  6. The connection is closed, and the logging is finalized.

Conclusion: By following the steps outlined in this blog post, you can easily connect Python to your Office 365 email using the exchangelib library and OAuth2 credentials. This enables you to automate email-related tasks, retrieve messages, send emails, and perform various other operations programmatically. Embracing OAuth2 and MFA adds an extra layer of security to your email communication. Enjoy leveraging the power of Python and Office 365 to streamline your workflows!


Decrypting Firefox Traffic Using Wireshark in Ubuntu GNU/Linux

Wireshark is a powerful network protocol analyzer that lets you capture and analyze real-time network traffic. By default, Wireshark does not decrypt encrypted traffic, such as HTTPS, as it is designed to maintain security and privacy. However, there are cases where decrypting network traffic can be helpful in debugging or analyzing security issues. This blog post will guide you through the steps to decrypt Firefox traffic using Wireshark in Ubuntu GNU/Linux.

Step 1: Download and Extract Firefox:

Since Ubuntu uses the snap package manager to install Firefox, which does not provide access to the file system by default, we need to download Firefox from the official website as a tar.gz archive. Open your browser and navigate to the Mozilla Firefox website (https://www.mozilla.org/en-US/firefox/new/) to download the tar.gz package suitable for your Ubuntu version.

Once the download is complete, navigate to the downloaded location and extract the tar.gz file using the following command:

tar -xvf firefox-<version>.tar.gz;

Step 2: Set up the SSLKEYLOGFILE Environment Variable:

To enable Wireshark to decrypt the SSL/TLS traffic from Firefox, we need to set up the SSLKEYLOGFILE environment variable. This variable will point to a log file where Firefox will write the session keys used for encryption. Execute the following command in the terminal:

export SSLKEYLOGFILE="/home/$USER/.ssl-key.log";

This command sets the SSLKEYLOGFILE environment variable to the specified file path, which is /home/$USER/.ssl-key.log. Feel free to change the file path and name to your preference.

Step 3: Launch Wireshark and Configure Preferences:

Open the terminal and start Wireshark by entering the following command:

wireshark;

Once Wireshark runs, go to “Edit” in the menu bar and select “Preferences” from the dropdown menu. This will open the Wireshark Preferences window.

Step 4: Configure TLS Protocol Preferences:

In the Preferences window, locate and select “Protocols” on the left-hand side. Scroll down the protocols list and find “TLS”. Click on it to expand the options.

Within the TLS section, you will find a field labeled “(Pre)-Master-Secret log filename”. Click on the folder icon next to the field and browse to select the file path for the SSLKEYLOGFILE we set earlier.

After selecting the file path, click the “OK” button to save the changes and close the Preferences window.

Step 5: Capture and Decrypt Firefox Traffic:

With the configuration set up, you can now start capturing and decrypting Firefox traffic. Keep the Wireshark application running and launch the Firefox browser you downloaded and extracted earlier.

Wireshark will capture the network traffic as you browse the web using Firefox. You should be able to see the decrypted traffic in the Wireshark capture window.

Conclusion:

Decrypting network traffic using Wireshark can be valuable for analyzing and troubleshooting network-related issues. This blog post covered the steps to decrypt Firefox traffic using Wireshark in Ubuntu GNU/Linux. By downloading Firefox directly from the website, setting up the SSLKEYLOGFILE environment variable, and configuring Wireshark preferences, you can capture and analyze unencrypted network traffic within Wireshark. Remember to use this technique responsibly and respect the privacy of others while conducting network analysis.