Cropping a video to a specific range of frames and region of interest in Python

Question

I have a video and I am only interested in a small section of the video (both in terms of a certain amount of frames and a small window of the video). I have been slowing chewing my way through problems, with the help of people in this thread and elsewhere. I have been pulling my hair and everyone else's hair out trying to get this to work for a few days, and I think I figured out some parts of what was wrong. If you will, please offer help if you have any.

I'm unsure also if I need to be using waitKey() or not.

I am currently trying to figure out how to make sure that the new frames are the right size, which I assume would be the size of the cropped portion of the original video. This is how I have thought of doing this so far. Do you have any thoughts or potential solutions?


# dimension for cropping out each fish
box_dim = 220 

# set parameters for locating each ZeChat unit as ROIs for cropping
y_orig = 25
x_orig = 20
wall = 20
window = 15
ro = 1
col = 1

# locate the roi so that each cell can be selected by just changing the row and column number:
y_roi = y_orig+(box_dim*(ro))+(window*int(math.ceil(ro/2)))+(wall*(int(math.ceil((ro+1)/2))-1))
x_roi = x_orig+(box_dim*(col))+(wall*col)

roi_size = (y_roi+box_dim, x_roi+box_dim)

# Initialize video writer object
out = cv2.VideoWriter("BN4 A4-F4, H4 videos/BN4 A4-F4, H4 test.avi", cv2.VideoWriter_fourcc(*'XVID'), 50, roi_size)

Anyways, this is the whole code:

# dimension for cropping out each fish
box_dim = 220 
        
# set parameters for locating each ZeChat unit as ROIs for cropping
y_orig = 25
x_orig = 20
wall = 20
window = 15
ro = 1
col = 1

# locate the roi so that each cell can be selected by just changing the row and column number:
y_roi = y_orig+(box_dim*(ro))+(window*int(math.ceil(ro/2)))+(wall*(int(math.ceil((ro+1)/2))-1))
x_roi = x_orig+(box_dim*(col))+(wall*col)

#Starting and ending frames, length of video in frames
starting_frame = 32
ending_frame = []
length = 56

def crops_frames_and_borders():
    cap = cv2.VideoCapture("BN4 A4-F4, H4.avi") 
    cap.set(cv2.CAP_PROP_POS_FRAMES, starting_frame)
    
    if (cap.isOpened() == False):
        print("Error opening the video file")
    else:
        print("Opening video BN4 A4-F4, H4.avi")
        # Get frame rate information
        fps = int(cap.get(5))
        print("Frame Rate : ",fps,"frames per second") 
        # Get frame count
        frame_count = cap.get(7)
        print("Frame count : ", frame_count)
            
    # Obtain frame size information using get() method
    frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    frame_size = (frame_width,frame_height)
    
    # Initialize video writer object
    out = cv2.VideoWriter("BN4 A4-F4, H4 videos/BN4 A4-F4, H4 test.avi", cv2.VideoWriter_fourcc(*'XVID'), 50, frame_size)         
        
#while loop that continues until the frames counted equal the desired length of the clip
    frames_length = 0
    frames_progression = 0
    
    while (frames_progression <= length):
        frames_length = frames_length + frames_progression
        frames_progression = frames_progression + 1
        while(True):
            #reads through each frome within the loop, and then writes that frame into the new video isolate the roi:
            ret, frame = cap.read()
            if ret == True:
                #determine where the frames that will be cropped for the ROI
                roi = frame[y_roi:y_roi+box_dim, x_roi:x_roi+box_dim]
                # Write the frame into the file output .avi that was read from the original video in cap.read()
                out.write(roi)    
            else:
                print("Cannot retrieve frames. Breaking.")
                break
            if (out.isOpened() == False):
                print("Error opening the video file")
        else:
            break
        print(frames_length)
    
    # With everything done, release the capture
    cap.release()
    out.release()
    cv2.destroyAllWindows()
    print("Finished writing new video")

    
crops_frames_and_borders()

As of right now, it is returning this (rate of change of the counter increases until it hits 56, which is the number of frames that I wanted the actual length to be. So there's a problem with the while loop:

Opening video BN4 A4-F4, H4.avi
Frame Rate :  50 frames per second
Frame count :  46728.0
Cannot retrieve frames. Breaking.
0
Cannot retrieve frames. Breaking.
1
Cannot retrieve frames. Breaking.
3
Cannot retrieve frames. Breaking.
6
Cannot retrieve frames. Breaking.
10
Cannot retrieve frames. Breaking.
15
Cannot retrieve frames. Breaking.
21
Cannot retrieve frames. Breaking.
28
Cannot retrieve frames. Breaking.
36
Cannot retrieve frames. Breaking.
45
Cannot retrieve frames. Breaking.
55
Cannot retrieve frames. Breaking.
66
Cannot retrieve frames. Breaking.
78
Cannot retrieve frames. Breaking.
91
Cannot retrieve frames. Breaking.
105
Cannot retrieve frames. Breaking.
120
Cannot retrieve frames. Breaking.
136
Cannot retrieve frames. Breaking.
153
Cannot retrieve frames. Breaking.
171
Cannot retrieve frames. Breaking.
190
Cannot retrieve frames. Breaking.
210
Cannot retrieve frames. Breaking.
231
Cannot retrieve frames. Breaking.
253
Cannot retrieve frames. Breaking.
276
Cannot retrieve frames. Breaking.
300
Cannot retrieve frames. Breaking.
325
Cannot retrieve frames. Breaking.
351
Cannot retrieve frames. Breaking.
378
Cannot retrieve frames. Breaking.
406
Cannot retrieve frames. Breaking.
435
Cannot retrieve frames. Breaking.
465
Cannot retrieve frames. Breaking.
496
Cannot retrieve frames. Breaking.
528
Cannot retrieve frames. Breaking.
561
Cannot retrieve frames. Breaking.
595
Cannot retrieve frames. Breaking.
630
Cannot retrieve frames. Breaking.
666
Cannot retrieve frames. Breaking.
703
Cannot retrieve frames. Breaking.
741
Cannot retrieve frames. Breaking.
780
Cannot retrieve frames. Breaking.
820
Cannot retrieve frames. Breaking.
861
Cannot retrieve frames. Breaking.
903
Cannot retrieve frames. Breaking.
946
Cannot retrieve frames. Breaking.
990
Cannot retrieve frames. Breaking.
1035
Cannot retrieve frames. Breaking.
1081
Cannot retrieve frames. Breaking.
1128
Cannot retrieve frames. Breaking.
1176
Cannot retrieve frames. Breaking.
1225
Cannot retrieve frames. Breaking.
1275
Cannot retrieve frames. Breaking.
1326
Cannot retrieve frames. Breaking.
1378
Cannot retrieve frames. Breaking.
1431
Cannot retrieve frames. Breaking.
1485
Cannot retrieve frames. Breaking.
1540
Cannot retrieve frames. Breaking.
1596
Finished writing new video

run the code. python will throw a syntax error. don't ask a question on code that you haven't run. debug your code. please review [ask] and [mre]. — Christoph Rackwitz, Jan 18 '22 at 18:24
The reading of the image is out of loop so you are getting the same frame instead of a increment in it. also please refer to https://stackoverflow.com/questions/61723675/crop-a-video-in-python for the cropping portion. — Arunbh Yashaswi, Jan 18 '22 at 18:26
Thank you, I did just edit the code and will continue to do so as I learn what I am doing wrong. @Christoph Rackwitz, I know what the code throws a syntax error because I ran it. I believe I mentioned that before I edited that part out of the code. Part of the reason why I am asking these questions is to ask for help in debugging my code. — Chris, Jan 18 '22 at 18:37
frame = 32 cap.set(cv2.CAP_PROP_POS_FRAMES, frame) this will help you set the beginning of the frame as such which you are not doing. This is only first error which i can find after reading that much. Also can you please paste the error if any. — Arunbh Yashaswi, Jan 18 '22 at 18:44
Thank you for that. Is there a way to determine which frame python will stop reading the original video at? My first thought is to have an if statement that says something like "if the frames are between [start] and [end], then cap.read(). But I'm not sure if that would work. As far as giving up the error codes, I am currently waiting for my computer to finish loading from the last time I ran it. I might just restart the kernel or move to a faster computer. I'll give any error codes as soon as I can. — Chris, Jan 18 '22 at 19:05
OK, so windows media player says "This file isn't playable. That might be because the file type is unsupported, the file extension is incorrect, or the file is corrupt. 0xc10100be" The codecs on my computer are up to date, as is the media player. Also, the properties of the media file show that the file does not have any dimensions (height, width, length, etc). — Chris, Jan 18 '22 at 21:37
it's a tiny file, right? then your code wrote nothing to the file, or the frames you did try to write have the wrong size or they aren't 3-channel uint8. ... or you break your code with a KeyboardInterrupt, and then out.release() isn't getting called — Christoph Rackwitz, Jan 18 '22 at 21:44
I have thought about the dimensions being the issue. KeyboardInterrupt wouldn't be the issue, since I've let the code run until it was finished. I am getting things set up on another computer with a little more processing power, so I won't have to wait as long after trying to run the code. I'll edit in what I am going to try to do next to make sure that the frames are the right size in the new video. Does the code that I am using to obtain the ROI make sense? Do you have any other suggestions? If so, could you write a formal answer for it? — Chris, Jan 18 '22 at 22:21

score 0 · Accepted Answer · answered Jan 19 '22 at 01:04

I got it to work! Next, I will be making it so my program will do the same thing to make about 750 more videos. Here was my solution:

# Dimension for cropping out each fish
box_dim = 220 
        
# Set parameters for locating each ZeChat unit as ROIs for cropping
y_orig = 25
x_orig = 20
wall = 20
window = 15
ro = 1
col = 1

# locate the roi so that each cell can be selected by just changing the row and column number:
y_roi = y_orig+(box_dim*(ro))+(window*int(math.ceil(ro/2)))+(wall*(int(math.ceil((ro+1)/2))-1))
x_roi = x_orig+(box_dim*(col))+(wall*col)

# Starting and ending frames, length of video in frames
starting_frame = 32
ending_frame = []
length = 56

def crops_frames_and_borders():
    cap = cv2.VideoCapture("BN4 A4-F4, H4.avi") 
    cap.set(cv2.CAP_PROP_POS_FRAMES, starting_frame)
    
    if (cap.isOpened() == False):
        print("Error opening the video file")
    else:
        print("Opening video BN4 A4-F4, H4.avi")
        # Get frame rate information
        fps = int(cap.get(5))
        print("Frame Rate : ",fps,"frames per second") 
        # Get frame count
        frame_count = cap.get(7)
        print("Frame count : ", frame_count)
            
    # Obtain frame size information using get() method
    frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    frame_size = (frame_width,frame_height)
    
    roi_size = (220, 220)
    
    # Initialize video writer object
    out = cv2.VideoWriter("BN4 A4-F4, H4 videos/BN4 A4-F4, H4 test.avi", cv2.VideoWriter_fourcc(*'XVID'), 50, roi_size) 
    
    frames_progression = 0
    # For loop that iterates until the index matches the desired length of the clip
    for i in range(length): # reads through each frome within the loop, and then writes that frame into the new video isolate the roi
        ret, frame = cap.read() 
        if ret == True:
            # Locating the ROI within the frames that will be cropped
            roi = frame[y_roi:y_roi+box_dim, x_roi:x_roi+box_dim]
            # Write the frame into the file output .avi that was read from the original video in cap.read()
            out.write(roi)    
        else:
            print("Cannot retrieve frames. Breaking.") #If a frame cannot be retrieved, this error is thrown
        if (out.isOpened() == False):
            print("Error opening the video file") # If the out video cannot be opened, this error is thrown
        else:
            frames_progression = frames_progression + 1 # Shows how far the frame writting process got. Compare this to the desired frame length
    print(frames_progression)
    
    # Release the capture
    cap.release()
    out.release()
    cv2.destroyAllWindows()
    print("Finished writing new video")

    # These were just for me to verify that the right frames were written
    cap = cv2.VideoCapture("BN4 A4-F4, H4 videos/BN4 A4-F4, H4 test.avi")
    fps = cap.get(cv2.CAP_PROP_FPS)
    frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    duration = frame_count/fps
    print('fps = ' + str(fps))
    print('number of frames = ' + str(frame_count))
    print('duration (S) = ' + str(duration))
    cap.release()
    cv2.destroyAllWindows()
    
crops_frames_and_borders()

The problems were with the two while loops, which I ditched for a single for loop, and the size of the new video like Christoph suggested. I found that simplifying the size to something more simple was enough to let the video work. Thanks to those who were helpful while responding to me.

Cropping a video to a specific range of frames and region of interest in Python

1 Answers1