1

Hello Stackoverflow community!

I dread having to ask questions, but there seems to be no efficient way to take a single input video and apply a matrix transformation/split the video into equal sized pieces, preferably 4x4=16 segments per input.

I tried using all the libraries such as ffmpeg and mencoder, but having 16 outputs can be as slow as 0.15x. The goal of my project is the split the video into 16 segments, rearrange those segments and combine back into a final video; later reversing the process in HTML5 canvas. Here is a picture to help you understand what I am talking about: the source but also the final destination after reorganizing the pieces

after the segments are exported, I will recombine the pieces in a puzzle manner (they are also inverted and rotated but unrelated)

I do not believe you can do this all in one command, so my goal is to crop into 16 mapped outputs quickly, then reassemble them in a different order. But I can do the other parts myself. Ideally there would be a way to move pixel blocks eg 100x100 and just move them around. My math is not strong enough.. I really appreciate the work you guys do!

admin@dr.com

Dylan
  • 45
  • 10
  • What did you try with ffmpeg? – Gyan Apr 21 '17 at 19:52
  • @Mulvya I tried to use the crop filter 16 times on a single input resulting in 16 outputs as I wanted, but it is 0.15x speed, very slow. – Dylan Apr 21 '17 at 20:13
  • But you want just one output, with a rearranged grid? If so, it should be faster. I'll post a command tomorrow. – Gyan Apr 21 '17 at 21:05
  • @Mulvya I do not believe it is possible to do that in one command, rearranging pixels in blocks. Or at least not in ffmpeg. Can you point me in the right direction, what is it that I am looking for? In HTML5 canvas, I am able to do what I want exactly, but recording 30-60fps HTML5 canvas at 1.0x speed is not easy or fast enough. It is wonderful to find someone who might be able to help; would be happy to buy you a coffee or something in bitcoin! – Dylan Apr 21 '17 at 21:13
  • Belongs rather to https://video.stackexchange.com – Ondra Žižka Apr 14 '20 at 23:47

1 Answers1

3

Basic template is

ffmpeg -i in.mp4
  -filter_complex
     "[0]split=16[a][b][c][d][e][f][g][h][i][j][k][l][m][n][o][p];
      [a]crop=iw/4:ih/4:0*iw/4:0*ih/4[a];
      [b]crop=iw/4:ih/4:1*iw/4:0*ih/4[b];
      [c]crop=iw/4:ih/4:2*iw/4:0*ih/4[c];
      [d]crop=iw/4:ih/4:3*iw/4:0*ih/4[d];
      [e]crop=iw/4:ih/4:0*iw/4:1*ih/4[e];
      [f]crop=iw/4:ih/4:1*iw/4:1*ih/4[f];
      [g]crop=iw/4:ih/4:2*iw/4:1*ih/4[g];
      [h]crop=iw/4:ih/4:3*iw/4:1*ih/4[h];
      [i]crop=iw/4:ih/4:0*iw/4:2*ih/4[i];
      [j]crop=iw/4:ih/4:1*iw/4:2*ih/4[j];
      [k]crop=iw/4:ih/4:2*iw/4:2*ih/4[k];
      [l]crop=iw/4:ih/4:3*iw/4:2*ih/4[l];
      [m]crop=iw/4:ih/4:0*iw/4:3*ih/4[m];
      [n]crop=iw/4:ih/4:1*iw/4:3*ih/4[n];
      [o]crop=iw/4:ih/4:2*iw/4:3*ih/4[o];
      [p]crop=iw/4:ih/4:3*iw/4:3*ih/4[p];
      [b][d][i][p]hstack=4[w];
      [n][g][j][c]hstack=4[x];
      [e][o][a][l]hstack=4[y];
      [h][f][m][k]hstack=4[z];
      [w][x][y][z]vstack=4"
      out.mp4

Not pretty, but gets the job done. Best if your video dimensions are (mod 4). If not, a scaler may be needed at the end after the vstack. The inputs specified to the hstacks are what control the shuffling of the pieces.

The rearranging part happens pretty fast - speed 7x for a 1080p input. Encoding drags that down to 0.9x. If you output to 720p, that jumps to 1.9x. For a 720p input, kept as is, speed is 2.5x. All benchmarked on a Haswell i5.

Gyan
  • 85,394
  • 9
  • 169
  • 201
  • 1
    This is what I was trying to do! You are a lifesaver! Let me know if there is anything I can do for you! – Dylan Apr 22 '17 at 16:30