1

In order to define a ROI (Region Of Interest) that might change depending on the input video, I'm using the following approach:

  • Create a blank black image

  • Subtract the background using opencv function

  • Copy the white pixels (foreground) to the blank image for 5min

Problem: Under the 2 for loops, the if statement and the instruction inside it are slowing down the process time:

if opening[i][j] == 255 or opening[i][j] == 127:
    blank_image[i][j] = [255, 255, 255]

I'm not familiar with opencv and I'm new to this stuff so I don't know if there is a better way to do it.

import numpy as np
import cv2
import sys
import time
from PIL import Image

fgbg = cv2.createBackgroundSubtractorMOG2(detectShadows = 0)
video = 'C:/Users/HP/Desktop/sentdex/testLongVideo.mp4'
cap = cv2.VideoCapture(video)

frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))      #get the frame height
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))        #get the frame width
x = 0
start = time.process_time()     #start time

while(1 and time.process_time() - start < 300):
    ret,frame = cap.read()
    fgmask = fgbg.apply(frame)      #apply the MOG2 mask on the video
    erode = cv2.erode(fgmask, None, iterations = 1)     #erosion to erase unwanted small contours
    opening = cv2.morphologyEx(erode, cv2.MORPH_OPEN, None)     #eliminate false positive on erosion
    if x < 5:       #this condition is used because if the creation of the black image starts at the same time as the video all the pixels will be white
        blank_image = np.zeros((frame_height,frame_width,3), np.uint8)          #create a black image with the same height and width of the video
        x += 1
    else:
        for i in range(len(opening)):
            for j in range(len(opening[i])):
                if [i,j] not in checked and opening[i][j] == 255 or opening[i][j] == 127:       #skip the treated pixels and check which pixels of the treated frame are white or gray (moving objects and their shadows)
                    blank_image[i][j] = [255, 255, 255]     #turn the pixels in the black image to white from the treated video by the X and Y
                    checked.append([i,j])   #treated pixels

    cv2.imwrite('backgroundMask.png', blank_image)      #saving the background treated
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break

cap.release()
cv2.destroyAllWindows()

I tried to run the loops without anything inside and it run just fine:

for i in range(len(opening)):
    for j in range(len(opening[i])):
        continue

When I add an instruction without an if statement it's pretty slow:

for i in range(len(opening)):
    for j in range(len(opening[i])):
        blank_image[i][j] = [255, 255, 255]

But when I add the if statement, even if there's no instruction in it, it's even slower:

for i in range(len(opening)):
    for j in range(len(opening[i])):
        if opening[i][j] == 255:
            continue
scrpy
  • 985
  • 6
  • 23
  • A common approach here would be downscaling your image when you're searching ROI – Dmitrii Z. Dec 23 '17 at 07:37
  • Also since your code is working - you should consider asking on https://codereview.stackexchange.com/ instead – Dmitrii Z. Dec 23 '17 at 07:39
  • 1
    You do not need to loop through the pixels like that (and you basically never should in numpy). Please edit the question to provide a *minimal* example of the slow part of your code and a suggestion would be easy to make. Also `opening` is already a numpy array (in fact you're already viewing it..)...there's no need to cast it again and assign a new variable to it. – alkasm Dec 23 '17 at 08:22
  • @DmitriiZ. Even by downscaling or small resolution video input it's still too slow, also bigger output helps me better to use it after. Thank you for your help, I'll try codereview.stackexchange.com – Anes Madani Dec 23 '17 at 09:36
  • @AlexanderReynolds Are you suggesting running the loops by incrementing i by 5 for example instead of by 1? The if statement is making it slow, I will edit the question right now and thank you for pointing out the dumb mistake that I made. – Anes Madani Dec 23 '17 at 09:41
  • @AlexanderReynolds I tried to increment by 5 instead of 1 and it's working way better, thanks ! – Anes Madani Dec 23 '17 at 10:28
  • 2
    No, that's not at all what I meant lol! But glad it has helped you out. What I mean was that you basically *never* want to do `for row in rows: for col in columns: array[row][col]` in numpy. You want to use vectorized operations. For a stupid basic example, let's say you want to change all 255 values to 127 in an image. Instead of `for row in rows: for col in columns: if img[row][col] == 255, img[row][col] === 127`, you can just do `img[img==255] = 127`. Loops in Python are slow, and even slower in numpy. Built-in numpy functions, however, run in C and Fortran so they're much faster. – alkasm Dec 23 '17 at 10:48
  • 2
    So in this specific example you could do `blank_image[(opening==255) | (opening==127)] = 255`. That will set all pixels of `blank_image` to 255 wherever `opening` is 255 or 127. This is called ['boolean' or 'masked' indexing](https://docs.scipy.org/doc/numpy-1.13.0/user/basics.indexing.html#boolean-or-mask-index-arrays). – alkasm Dec 23 '17 at 10:52
  • @AlexanderReynolds I'm sorry that I misunderstood you. Thank you very much for your help and the clear explanation, it's working smoothly ! – Anes Madani Dec 23 '17 at 23:03
  • Cool, I'll add it as an answer then. – alkasm Dec 24 '17 at 09:04

2 Answers2

0

As mentioned in the comments, you generally want to avoid patterns like

for i in range(height):
    for j in range(width):
        array[i][j] = ...

when looping over numpy arrays. Loops are slow in Python compared to loops in say C, partially because types aren't explicit or known to the interpreter before they're being used---so type checking happens on every variable inside the loop. And for other reasons too. But numpy and similar libraries are actually implemented with C and Fortran and such in the back end, so if you use built-in numpy functions, your code will speed up dramatically.

If you want to set multiple items inside an array, you can do that in a number of ways. Instead of indexing with single numbers like i and j, you can index with a sequence of indices all at once, like

>>> img
array([[0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)
>>> img[(0, 1, 2, 3), (1, 2, 3, 4)] = 1
>>> img
array([[0, 1, 0, 0, 0, 0, 0, 0],
       [0, 0, 1, 0, 0, 0, 0, 0],
       [0, 0, 0, 1, 0, 0, 0, 0],
       [0, 0, 0, 0, 1, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)

As you can see this sets (0, 1), (1, 2), ... to 1 in the img.

You can also set an entire subregion at once:

>>> img[2:6, 2:6] = 1
>>> img
array([[0, 1, 0, 0, 0, 0, 0, 0],
       [0, 0, 1, 0, 0, 0, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)

But what you're looking for is boolean or masked indexing. This replaces the pattern:

for i in range(height):
    for j in range(width):
        if array[i][j] == 1:
            array[i][j] = 2

with the simple line

array[array==1] = 2

If you break apart the sections here it's really easy to see what's going on:

>>> img
array([[0, 1, 0, 0, 0, 0, 0, 0],
       [0, 0, 1, 0, 0, 0, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)
>>> img==1
array([[False,  True, False, False, False, False, False, False],
       [False, False,  True, False, False, False, False, False],
       [False, False,  True,  True,  True,  True, False, False],
       [False, False,  True,  True,  True,  True, False, False],
       [False, False,  True,  True,  True,  True, False, False],
       [False, False,  True,  True,  True,  True, False, False],
       [False, False, False, False, False, False, False, False],
       [False, False, False, False, False, False, False, False]], dtype=bool)

So here we have an array the same shape as img but with boolean values telling which items we're interested in. This boolean array can then be used to index the original array.

>>> img[img==1]
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=uint8)

Here we see that the array img has the value 1 wherever img==1. And we can set all these values inside img to another value:

>>> img[img==1] = 2
>>> img
array([[0, 2, 0, 0, 0, 0, 0, 0],
       [0, 0, 2, 0, 0, 0, 0, 0],
       [0, 0, 2, 2, 2, 2, 0, 0],
       [0, 0, 2, 2, 2, 2, 0, 0],
       [0, 0, 2, 2, 2, 2, 0, 0],
       [0, 0, 2, 2, 2, 2, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)

In short, it looks like you just simply need

blank_image[(opening==255) | (opening==127)] = 255
alkasm
  • 22,094
  • 5
  • 78
  • 94
0

Hope this can help

If I understand you properly

for i in range(len(opening)):
    for j in range(len(opening[i])):
        if [i,j] not in checked and opening[i][j] == 255 or opening[i][j] == 127:       #skip the treated pixels and check which pixels of the treated frame are white or gray (moving objects and their shadows)
            blank_image[i][j] = [255, 255, 255]     #turn the pixels in the black image to white from the treated video by the X and Y
            checked.append([i,j])   #treated pixels

This code

Checks if opening[i][j] is either 255 or 127

if this condition is true

black_image pixel is turned white

check this solution

black_image[np.where((opening ==255).all(axis=2)) or np.where((opening ==127).all(axis=2))] = [255,255,255]

I tried similar method to change value of certain BGR values in an image using

img[np.where((img == [__b,__g,__r]).all(axis=2))] = [255, 255, 255]

You can check this OpenCV: setting all pixels of specific BGR value to another BGR value

As far as possible try using numpy in your code as it is written using C for speed.

Santhosh
  • 1,554
  • 1
  • 13
  • 19