Understanding how to deploy python code to pop up balloons

Question

I'm a newbie in programming and I need to write code to detect balloon on the fixed background using numpy and openCV in live video and to return the centre of the object [balloon].

Sorry about the ignorance of the questions.

Since I'm new, I had troubles with thinking about the logic of doing it, I don't have the resources to "teach the machine" and creating cascade XML to detect balloons so I thought about 1 possible solution : Using cv2.createBackgroundSubtractorMOG2() to detect motion with the same background and once there is some object [balloon], count all the white pixels in the live video and return the centre of it, with the right threshold amount of white pixels.

The problem is, I don't know how to get the value of the pixel from 0-255 to know if it's white or black and shows the video at the same time, I think that there is a much easier way that I couldn't find guides for it.

import numpy as np

import cv2

cap = cv2.VideoCapture(0)

fgbg = cv2.createBackgroundSubtractorMOG2()

while(1):

    ret, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    fgmask = fgbg.apply(gray)
    img_arr = np.array(fgmask)
    cv2.imshow('frame',fgmask)
    for i in fgmask:
        for j in i:
            print(fgmask)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break
cap.release()
cv2.destroyAllWindows()

I'm getting fray video on the output and lots of values that I don't know how to understand them on the output.

you use `print(fgmask)` to display values of all pixels in image. And you put it in `for`-loop so you print the same image many, many times. If you have to use `print(fgmask)` then use it only once, not in `for`-loop. — furas, Sep 16 '19 at 19:03
if you want to print in `for`-loop then better `print(j)` and you should see pixel in grayscale which you can compare with 0-255 — furas, Sep 16 '19 at 19:15
you can also compare all elements in array with `2` using `fgmask==255` or `fgmask > 10` which fives you 2D array with True/False and then you can count how many values in every row are True `sum(fgmask > 10)` and how many values in all array are True `sum(sum(fgmask > 10))`. And then you can decite that when value is bigger then ie. 1000 then there is move. — furas, Sep 16 '19 at 19:20
@furas it really helped, the problem now is that the video is super super laggy because it runs on 500*500 pixels every frame, is there any solution for this? — Snorko, Sep 16 '19 at 19:20
first: use smaller value in `waitKey()` there is no need to wait so long for key. Second: you don't need `img_arr` because `fgmask` is already array. I use old notebook with camera 640x480 and it run very fast. Maybe camera can't create images faster. — furas, Sep 16 '19 at 19:27
and don't print all pixels - `print()` always need some time to display all text so sometimes removing all `print()` can speed up program. — furas, Sep 16 '19 at 19:29

furas · Accepted Answer · 2019-09-16T20:41:16.740

0

I would use

changes = (fgmask>200).sum()

to compare all pixels with almost white value (>200) and count these pixels.

And then I can compare result with some value to treat it as move.

import numpy as np

import cv2

cap = cv2.VideoCapture(0)

fgbg = cv2.createBackgroundSubtractorMOG2()

while True:

    ret, frame = cap.read()

    if frame is None:
        break

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    fgmask = fgbg.apply(gray)

    #changes = sum(sum(fgmask>200))
    changes = (fgmask>200).sum() 
    is_moving = (changes > 10000)
    print(changes, is_moving)

    cv2.imshow('frame', fgmask)

    k = cv2.waitKey(10) & 0xff
    if k == 27:
        break

cv2.destroyAllWindows()
cap.release()

print() needs some time to display text so printing all pixels (many times in loop) can slow down program. So I skip this. I don't have to know values of all pixels.

EDIT: Using answer in how to detect region of large # of white pixels using opencv? and add code which can find white regions and draw rectangle. Program opens two window - one with grayscale fgmask and other with RGB frame and they can be hidden one behind another. You have to move one window to see another.

EDIT: I added code which use cv2.contourArea(cnt) and (x,y,w,h) = cv2.boundingRect(cnt) to create list with items (area,x,y,w,h) for all counturs and then get max(items) to get contour with the biggest area. And then it use (x + w//2, y + h//2) as center for red circle.

import numpy as np

import cv2

cap = cv2.VideoCapture(0)

fgbg = cv2.createBackgroundSubtractorMOG2()

while True:

    ret, frame = cap.read()

    if frame is None:
        break

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    fgmask = fgbg.apply(gray)

    #changes = sum(sum(fgmask>200))
    changes = (fgmask>200).sum() #
    is_moving = (changes > 10000)
    print(changes, is_moving)


    items = []

    contours, hier = cv2.findContours(fgmask, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    for cnt in contours:
        area = cv2.contourArea(cnt)
        if 200 < area:
            (x,y,w,h) = cv2.boundingRect(cnt)
            cv2.rectangle(fgmask, (x,y),(x+w,y+h),255, 2)
            cv2.rectangle(frame, (x,y),(x+w,y+h),(0,255,0), 2)
            items.append( (area, x, y, w, h) )

    if items:
        main_item = max(items)
        area, x, y, w, h = main_item
        if w > h:
            r = w//2
        else:
            r = h//2
        cv2.circle(frame, (x+w//2, y+h//2), r, (0,0,255), 2)

    cv2.imshow('fgmask', fgmask)
    cv2.imshow('frame', frame)

    k = cv2.waitKey(10) & 0xff
    if k == 27:
        break

cv2.destroyAllWindows()
cap.release()

edited Sep 16 '19 at 20:41

answered Sep 16 '19 at 19:44

furas

134,197
12
106
148

First of all, you are amazing. and this is really helped me, Second is if I have solid colour the fgmask won't recognize it because the centre doesn't look like it's changing, do you have maybe idea how to solve it? maybe I'm using the wrong general idea... I tough also to save frame by frame and to scan all the picture to look for the pixel's that over the threshold . – Snorko Sep 16 '19 at 19:48
I found on Stackoverflow question which show how to find regions with white color. I'm creating code for this. – furas Sep 16 '19 at 19:55
Thank you so much, If I wasn't newbie I would mark this answer as super useful. last super small question is, how can I get the location of the center in the major "white spot" [I want to focus a laser to this location to pop balloon]. – Snorko Sep 16 '19 at 20:14
you have `cv2.contourArea(cnt)` and `(x,y,w,h) = cv2.boundingRect(cnt)` so you can create list with items `(area,x,y,w,h)` for all counturs and then get `max(list_of_items)` to get contour with the biggest area. And then center is `(x + w/2, y + h/2)` – furas Sep 16 '19 at 20:29
I added this to last code - now it draw red circle for the biggest area. – furas Sep 16 '19 at 20:41
Yeah but bow its super laggy while taking the video, maybe always capture with the rectangle the biggest spot and to print on the output the x+w/x , y+h/2 ? or just capture as circle? – Snorko Sep 17 '19 at 06:45
code works fast on my old computer with Linux Mint and I don't know what can be the problem and what could help. Do you have the same problem when you use camera with other programs ? – furas Sep 17 '19 at 06:51
Oh no, the previous aolution was amazing and worked auper fast, the new one with the circle is laggy, its captyring the rectangles every second and once there is alot of frames that chabge, the frame is just stucked. – Snorko Sep 17 '19 at 06:53
so maybe use bigger value in `if 200 < area:` to reduce number of rectangles and then it will have less items to find `max()` for circle. Or try to replace or remove some code to see which line makes problem. You can draw rectangle instead of circlec to see if problem is circle. You can get first item `main_item = items[0]` to check if problem is `max()` – furas Sep 17 '19 at 07:12
if problem is `max()` then you can try `main_item = max(items, key=lambda x:x[0])` and it will check only first element in all items - which is `area`. – furas Sep 17 '19 at 07:15
Nvm all working amazing, thank you so much <3, You really really helped me. – Snorko Sep 17 '19 at 08:19

Understanding how to deploy python code to pop up balloons

1 Answers1