Security Image Processing to determine if vehicle or person is in the image


I started here on the IPCAMTALK forums with blueiris:

“This uses Sighthound’s Cloud API which uses machine learning to determine the vehicle in the image. This API is free for the first 5000 calls, which my alerts fit well within. You can sign up on their website.
The script will check the previous vehicle detected and quit if it’s the same. This is because Alert images from Blueiris may be set off by other factors and you may have a car parked in your driveway…you don’t want to be alerted every minute that there’s a White Ford F Series.”

Got that python script working with some additional information (confidence percentage). Then tied it into webcore with a URL request. Now when triggered I get an announcement or text of what pulled into my driveway.

edit: Advanced this to include processing the images to determine if there is either a vehicle, person or face detectable in the image. I keep a count of these and also pass the details of the last vehicle detected. The python script itself does checking to see if the last vehicle processed matches the current one. If so we clear it out. The webcore piston tracks the last vehicle per camera. This has dramatically increased the reliability of the cameras motion detection cutting down on rain/snow/shadows triggering events.

Also to note I have NEVER coded in python before… :slight_smile:

python script:

import base64
import http.client
import json
import os
import time
import ssl
import smtplib
import glob
import pickle
import datetime
import requests
import sys
import logging

logger = logging.getLogger('bi-sighthound')
hdlr = logging.FileHandler('bi-sighthound.log')
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')

CAM = sys.argv[1]

sighthoundapi = '{YOUR API KEY}'
homeassistanturl = "{YOUR PISTON URL}"
filelocation = "C:\\BI-Sighthound\\" + (CAM) + ".sighthound.jpg"

# Reset Variables
person = 0
face = 0
cars = 0
modelconf = 0
make = 0
model = 0
color = 0
makeconf = 0
colorconf = 0
plate = 0
licenseplate = ''

## CREATE data file if it doesn't exist so we can compare if this is a duplicate vehicle on subsequent runs
if not os.path.exists('data.pickle'):
    new_data = {'make' : "null", 'model' : "null", 'color' : "null"}
    with open('data.pickle', 'wb') as f:
     pickle.dump(new_data, f, pickle.HIGHEST_PROTOCOL)
     print("Created file...")

# Uncomment the below if you store Hi-Res Alert images, and prefer to use those. This will search the latest jpeg to use.
#list_of_files = glob.glob('C:\BlueIris\Alerts\*.jpg')
#latest_file = max(list_of_files, key=os.path.getctime)
#filelocation = latest_file

headers = {"Content-type": "application/json", "X-Access-Token": sighthoundapi}
image_data = base64.b64encode(open(filelocation, 'rb').read())
params = json.dumps({"image": image_data.decode('ascii')})
conn = http.client.HTTPSConnection("", context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))
conn.request("POST", "/v1/detections?type=person,face", params, headers)

# Parse the response and print the make and model for each vehicle found
response = conn.getresponse()
results = json.loads(

for obj in results["objects"]:
    if obj["type"] == "person":
        person = person + 1
print("Number of People Detected: %s " % (person))

for obj in results["objects"]:
    if obj["type"] == "face":
        face = face + 1
print("Number of Faces Detected: %s " % (face))

headers = {"Content-type": "application/json", "X-Access-Token": sighthoundapi}
image_data = base64.b64encode(open(filelocation, 'rb').read())
params = json.dumps({"image": image_data.decode('ascii')})
conn = http.client.HTTPSConnection("", context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))
conn.request("POST", "/v1/recognition?objectType=vehicle,licenseplate", params, headers)

## Parse the response and print the make and model for each vehicle found
## This really only gets the data from the last vehicle object. but does count them
response = conn.getresponse()
results = json.loads(
for obj in results["objects"]:
    if obj["objectType"] == "vehicle":
        make  = obj["vehicleAnnotation"]["attributes"]["system"]["make"]["name"]
        model = obj["vehicleAnnotation"]["attributes"]["system"]["model"]["name"]
        color = obj["vehicleAnnotation"]["attributes"]["system"]["color"]["name"]
        makeconf = obj["vehicleAnnotation"]["attributes"]["system"]["make"]["confidence"]
        modelconf = obj["vehicleAnnotation"]["attributes"]["system"]["model"]["confidence"]
        colorconf = obj["vehicleAnnotation"]["attributes"]["system"]["color"]["confidence"]
        cars = cars + 1
    if licenseplate in obj["vehicleAnnotation"]:
        plate = obj["vehicleAnnotation"]["licenseplate"]["attributes"]["system"]["string"]["name"]
    print("Detected: %s %s %s %s" % (color, make, model, plate))
    current_data = {
     'make' : (make),
     'model' : (model),
     'color' : (color)

## Check to see if this is the same vehicle as the last run (multiple cameras)
    with open('data.pickle', 'rb') as f:
     previous_data = pickle.load(f)

    if current_data == previous_data:
     print('Data is the same!')
     cars = 0
    with open('data.pickle', 'wb') as f:
     pickle.dump(current_data, f, pickle.HIGHEST_PROTOCOL)
print("Detected %s People %s Faces and %s Cars" % (person, face, cars))

url = homeassistanturl
payload = json.dumps({
"person" : (person),
"face" : (face),
"cars" : (cars),
"make" : (make),
"model" : (model),
"color" : (color),
"makeconf" : (makeconf),
"modelconf" : (modelconf),
"colorconf" : (colorconf),
"plate" : (plate),
"CAM" : (CAM)
headers = {
"content-type": "application/json"
response = requests.request("POST", url, data=payload, headers=headers)

## Write to log file so we can see what's being called if needed
##"%s - People:%s Faces:%s Cars:%s Make:%s Model:%s Color:%s Conf:%s - Status Code:%s  Response:%s" % (CAM, person, face, cars, make, model, color, modelconf, response.status_code, response.text))"Status Code:%s  Response:%s" % (response.status_code, response.text))

## Remove log file if older than 60 days (let's keep things cleaned up)
now = time.time()
if os.stat('bi-sighthound.log').st_mtime < (now - (60 * 86400)):


This is really awesome! Looks like the free Dev account on Sighthound is enough for API access? I’ve already got a Python script that grabs a screen grab from my Ring camera when motion is detected. Easy enough send the created file to Sighthound, I think.

Any idea how flexible the software is in terms of image angle? My cameras are probably not placed very well to get a good view of the cars.


I can tell you it’s not perfect or awesome. But it works and is pretty decent. I suspect we will see more services like this moving forward like Google’s tensorflow which is supposed to be really good if trained. I’m thinking forward and I don’t think it will be long until we are doing face recognition about who is at our front door. If you go to Sighthound’s website you can upload single images to see how it does. You can test it out with your images.


I have since developed this much further. I now have BlueIris send all alert images through sighthound to determine how many people or cars are in the image and report all of that and the cam shortname to webcore. Then I can use webcore to process and alert with much more confidence and more information (who’s home, etc).


Ok so i’ve followed over from the IPCamTalk site – bear with me… how do i find my piston url?

UPDATE: that was some serious setup that was needed but i found it!


Ok ive taken some time to review the code :slight_smile:


when i try to run the py i get an error:

E:\SCRIPTS>"E:\Program Files\Python36\python.exe" E:\SCRIPTS\ Cam202
Traceback (most recent call last):
  File "E:\SCRIPTS\", line 12, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

so to test the rest of the code i removed that and (as expected) errored with the following:

E:\SCRIPTS>"E:\Program Files\Python36\python.exe" E:\SCRIPTS\ Cam202
Created file...
Number of People Detected: 0
Number of Faces Detected: 0
Detected: black Ram pickup 0
Detected: green Jaguar XE 0
Detected: black Pontiac Sunbird 0
Detected: blue Ford Mustang 0
Detected: black Ford F Series 0
Detected 0 People 0 Faces and 5 Cars
Traceback (most recent call last):
  File "E:\SCRIPTS\", line 150, in <module>
    response = requests.request("POST", url, data=payload, headers=headers)
NameError: name 'requests' is not defined

how do i let python recognise the import needed for requests?

Update: Googled for about an hour…
pip install requests in the CMD :wink:


Did you already have a smartthings hub?


nope but i see it works with IFTTT so i was going to see if i can use this as a middle man.


Very nice, I am looking into taking this to the next level with AWS possibly. It looks like they have a commercial grade face recognition.


This has been a huge improvement to my security as a whole. Please anyone that finds/uses any other image processing sites please let us know how you are using them!


Hey, I was trying to get your python script and webcore piston to work. [using smarthings – BlueIris – paid version of Sighthound already running (activated the API for token) — multiple Echos and Google Assistants]

Python Script. (teaching myself python) (using Anaconda with Visual Studio Code)
When I try to debug the python script, I get an out of range error at “Cam = argv[1]” I am assuming because argv[1] requires the command line to be used, is the reason. Next when I duplicated the batch file, and have blueiris run it, it opens up my visual studio code, (so I am assuming that the blue iris is setup correctly, and the batch file is running properly. But I don’t think the python script is running correctly, to send over the data to the webcore.
The question about the webcore is, how do send the message for Alexa ? I just started webcore after reading your post, and already love it, but I can’t figure out how to connect my Echo to webcore, or how that works. Also, when Blueiris runs the batch file which pops open my visual studio, the webcore piston isn’t showing anything in its logs. So would that mean the python script isn’t really running, to pass the information?
Any Help is greatly appreciated.


Yes command line requires the short name of the CAM to be sent. I do this in blue iris when calling the command under Alerts/Run a command. Then in the “Parameters” box put &CAM that will put the cam shortname as an argument to the batch file which passes it to the python script.

you batch file doesn’t sound like it’s executing python correctly. It should look something like:

cd \BI-Sighthound
"C:\Program Files\Python36\python.exe" C:\BI-Sighthound\ %1

first part path to python, second path to your python script, third the argument.

Can’t help you with the alexa part…I don’t have one.


I actually dove into this myself. AWS Rekognition is the web app I used and it actually wasn’t as good as Sighthound. It wasn’t able to identify different car makes and models. Only that an “Automobile” was in the photo. It also seemed less able to pick up the presence of a human in an image. Something that this scenario is very important. It DID seem better at facial recognition if the image was very clear.

I want to try google’s cloud too, but I haven’t had a chance.


I have to give it to you, you waste no time, I wish I had the time to work on this stuff as much as I would like. So I guess the next step is trying this with FaceNet and/or tensorflow.


Thanks for the information. It solves my query.

Python Developer