3

We're doing a project in school where we need to do basic image processing. Our goal is to use every video frame for the Raspberry Pi and do real time image processing.

We've tried to include raspistill in our python-program but so far nothing has worked. The goal of our project is to design a RC-car which follows a blue/red/whatever coloured line with help from image processing.

We thought it would be a good idea to make a python-program which does all image processing necessary, but we currently struggle with the idea of bringing recorded images into the python program. Is there a way to do this with picamera or should we try a different way?

For anyone curious, this is how our program currently looks

while True:
    #camera = picamera.PiCamera()
    #camera.capture('image1.jpg')
    img = cv2.imread('image1.jpg')
    width = img.shape[1]
    height = img.shape[0]
    height=height-1
    for x in range (0,width):
            if x>=0 and x<(width//2):
                    blue  = img.item(height,x,0)
                    green = img.item(height,x,1)
                    red   = img.item(height,x,2)
                    if red>green and red>blue:
user202729
  • 3,358
  • 3
  • 25
  • 36
anthrx
  • 161
  • 1
  • 3
  • 12
  • same problem except coloring http://stackoverflow.com/questions/26169633/capturing-video-with-raspberry-using-opencvpicamera-stream-io – emrhzc Jun 12 '16 at 12:49
  • It looks like everything you need (seen in the accepted answer) can be found in OpenCV's official tutorial? -- [OpenCV: Getting Started with Videos](https://docs.opencv.org/master/dd/d43/tutorial_py_video_display.html) Unless you're having problems with not being able to process in realtime -- in that case refer to [OpenCV-Python: How to get latest frame from the live video stream or skip old ones - Stack Overflow](https://stackoverflow.com/questions/45310718/opencv-python-how-to-get-latest-frame-from-the-live-video-stream-or-skip-old-on). – user202729 Aug 14 '21 at 00:22

2 Answers2

9

OpenCV already contains functions to process live camera data.

This OpenCV documentation provides a simple example:

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()

    # Our operations on the frame come here
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Display the resulting frame
    cv2.imshow('frame',gray)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

Of course, you do not want to show the image but all your processing can be done there.

Remember to sleep a few hundred milliseconds so the pi does not overheat that much.

Edit:

"how exactly would I go about it though. I used "img = cv2.imread('image1.jpg')" all the time. What do I need to use instead to get the "img" variable right here? What do I use? And what is ret, for? :)"

ret indicates whether the read was successful. Exit program if not.

The read frame is nothing other than your img = cv2.imread('image1.jpg') so your detection code should work exactly the same.

The only difference is that your image does not need to be saved and reopened. Also for debugging purposes you can save the recorded image, like:

import cv2, time

cap = cv2.VideoCapture(0)

ret, frame = cap.read()
if ret:
    cv2.imwrite(time.strftime("%Y%m%d-%H%M%S"), frame)

cap.release()
Simon Kirsten
  • 2,542
  • 18
  • 21
  • I got one more question, when I'm using time.sleep(0.15) for example at the end of my program, and I jump back to the beginning and try to read a frame, just as the frame changes, will it mess with my program or something? Or will it just skip the frames not used in my program in those 0.15s (if there are any)? – anthrx Jun 13 '16 at 06:47
  • I don't know what will happen but reading [the documentation](http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html) I suspect that the read call is maybe already sleeping for you and you don't actually need a separate sleep. But to be honest the docs are not that precise. I advice you to test all your code on an desktop PC with a webcam first and then deploy to the PI which is not that easy to manage. – Simon Kirsten Jun 13 '16 at 10:00
  • ok, so I just tried starting the program and now I get errors basically saying: "object has no attribute " and stuff. – anthrx Jun 16 '16 at 12:13
0

You can use picamera to acquire images.

To make it "real time", you can acquire data each X milliseconds. You need to set X depending on the power of your hardware (and the complexity of the openCV algorithm).

Here's an example (from http://picamera.readthedocs.io/en/release-1.10/api_camera.html#picamera.camera.PiCamera.capture_continuous) how to acquire 60 images per second using picamera: 

import time
import picamera
with picamera.PiCamera() as camera:
    camera.start_preview()
    try:
        for i, filename in enumerate(camera.capture_continuous('image{counter:02d}.jpg')):
            print(filename)
            time.sleep(1)
            if i == 59:
                break
    finally:
        camera.stop_preview()
Alexis Clarembeau
  • 2,776
  • 14
  • 27
  • I fail to see where your suggestion to lower the capture rate has any impact on processing ability of the PI. One wants the image processing to be as fast as possible. So capture an image, analyse, react to the results, repeat. Do this as fast as possible. If you can be faster as it is reasonable, *then* you can pause. But there is no inherent value in "conserving" time - you won't get a payback later. – deets Jun 12 '16 at 12:50