Dynamically change video crop width, height, x and y using ffmpeg

Question

I am doing object detection on a video and so far I've gotten the coordinates of the objects in the video.

now I want to crop the video frame by frame given the location/coordinates of the object

My code so far:

def crop_video(input_video_path, output_video_path, coordinate_list):
    
    crop_ratio = 'crop=%s:%s:%s:%s' % (coordinate_list[0][0], coordinate_list[0][1], coordinate_list[0][2],coordinate_list[0][3])
    subprocess.run(['ffmpeg', '-i', input_video_path, '-filter:v', crop_ratio, output_video_path])

the crop_video function crops the entire video using only the first index in the coordinate_list list. How can I improve the code to change dynamically. coordinate list looks similar to this:

 coordinate_list = [[147.5, 253.5, 927, 107],
     [147.5, 253.5, 927, 107],
     [147.0, 257.5, 928, 102],
     [148.5, 258.5, 925, 104],
     [148.5, 258.5, 925, 104],
     [155.0, 258.5, 918, 103],
     [155.0, 258.5, 918, 103],]

How can I dynamically change the crop width, height, x and y using the coordinate_list. I am new to using ffmpeg

score 1 · Accepted Answer · answered May 12 '21 at 17:16

Use the sendcmd filter.

Make commands.txt:
```
0    crop w 148,
     crop h 254,
     crop x 925,
     crop y 108;

0.04 crop w 142,
     crop h 252,
     crop x 927,
     crop y 107;

0.08 crop w 147,
     crop h 258,
     crop x 928,
     crop y 102;
```
- The text does not have to be formatted exactly as above. I added line breaks for readability. You can place each timestamp on its own line if you prefer.
- sendcmd works with timestamps, not frame numbers. This example shows frames 1-3 and assumes a frame rate of 25 (1/25 = 0.04).
- Not all filters can use sendcmd (or the audio version asendcmd). See output of ffmpeg -filters. If the filter supports (a)sendcmd it will have a T preceding the filter name in the list.
- Not all filter options can be used with sendcmd. See FFmpeg Filters Documentation and look for "Commands".

Run ffmpeg:

ffmpeg -i input.mp4 -filter_complex "[0:v]sendcmd=f=commands.txt,crop" output_%03d.png

@Mario Change `output_%03d.png` to `output.mp4`. Your question did not mention what output you wanted. Because the width and height change I assumed you wanted images. MP4 has some limitations to be aware of: 1) width and height must be divisible by 2 for compatible output. 2) Each frame must have the same width and height. — llogan, May 13 '21 at 15:37

Dynamically change video crop width, height, x and y using ffmpeg

1 Answers1

Linked