0

I am stucked by a video processing feature, Specifically, upload an image and then generate a video based on various video templates.

Here are the video templates:

  1. http://video-static.biku8.com/data/video/template/3286012076458048/7437ab55-2e83-4a36-9046-5708fcddf4c1.mp4
  2. http://video-static.biku8.com/data/video/template/3274256089907264/ae8fa3f7-6c9c-45ca-810f-48db92cc14cb.mp4
  3. http://video-static.biku8.com/data/video/template/3213894231425088/bf107d439b9043a58c1ea0ba26f811db_template.mp4
  4. ...

As shown in the video templates above, I just need to upload a photo to generate a great video.

My question

  1. What is the specific idea for implementing this video?
  2. Which third-party libraries are needed? (ffmpeg, opencv)

PS: I am using dlib and opencv for face recognition. I can generate face image, but I don't know how to insert face image into the correct position of these template videos.

Zedd
  • 174
  • 1
  • 10

2 Answers2

2

I would suggest you to follow the below 3 steps

  1. Load the template video by opencv, you can access the video frame by frame
  2. Modify each frame, one by one.
  3. Save frame to video stream writer

Regarding step 2, actually, you must copy the uploaded image to the each frame by a mask (the pixel from source image would be copied to destination image if its coordinate on the mask is non-black). The mask could be defined by a list of points OR by an image. You should pre-define a mask for each frame in a file. Then load the mask for each frame and copy.

enter image description here

How to read video, save video OpenCV read-write Video

How to insert image to another image Copy non rectangular ROI

Peter Lee
  • 381
  • 3
  • 9
1

Generating videos like them are all not easy tasks. I recommend to use Adobe After Effects or other video creating software (with some scripts and actions) if you don't need to generate it by a single program or program language.

Then, I answer them below when you need to generate it by programatically.

For the first one, you should recognize faces and bones. So you should use OpenCV. ( I recommend to use tools like OpenFrameworks or TouchDesigner and so on. )

For the second one, I don't know what you exactly want, but if you want to recognize the position of the bottle dynamically, you have to use deep learning or other way to detect it. Then you may need TensorFlow or OpenCV. ( If you just want to merge layers, you can use ffmpeg etc. )

For the last one, you should split the video frame into the boxes, then you have to control. I think there are many ways to implement this. I may use OpenFrameworks, TouchDesigner, vvvv, or Processing.

I think using ffmpeg for them is not recommended. This tool is not the best for generating complicated video. But ffmpeg will do good, for example if you just merge two videos with alpha.

fumiya.f
  • 255
  • 3
  • 13
  • I am using dlib and opencv for face recognition. I can generate face image, but I don't know how to insert face image into the correct position of these template videos. Deep learning is too heavy for me. Is deep learning used here a bit overkill? – Zedd Oct 31 '19 at 01:28
  • You just mentioned that you can do these with AE. Is there a way to call AE programatically? – Zedd Oct 31 '19 at 01:32
  • I think, you should separate this question into each ones. Because it is too difficult to answer them in one combined answer. If your question is the exactly "How to insert template to the position of the face", then you will get more clear answers from other users :) – fumiya.f Oct 31 '19 at 03:12
  • Sorry, I read your PS just now. I will answer it by creating another one. – fumiya.f Oct 31 '19 at 03:14