17

What is the best way to generate the ".VTT" file and the jpg sprite attached with it for the Tooltip Thumbnails of Jwplayer (http://www.jwplayer.com/blog/building-tooltip-thumbnails-with-encodingcom/- ?

I know how to make an image sprite with php, but i dont know how to make the screenshots of each video with the time in second.. I think there must be a server tool to do all the tasks it but i cant find it.

Thanks

Tahola
  • 1,040
  • 3
  • 19
  • 38
  • 1
    I have not done this with ffmpeg before, but I have used a program called Video Thumbnails Maker by Scorp which generates the thumbs as well as the VTT files. It is here - http://www.suu-design.com/projects.html – emaxsaun Nov 18 '13 at 04:17
  • Thanks Ethan but is this for Ubuntu server to ? I mean can i do it automatically or is a manual app ? – Tahola Nov 18 '13 at 09:59
  • This is a manual app. – emaxsaun Nov 18 '13 at 14:13

4 Answers4

27

I wrote a script to do this task. Given a video file (MP4 or M4v), generate thumbnail images, compress into a sprite, and generate a VTT file compatible with JWPlayer tooltip thumbnails. All of the image manipulation uses tools from ffmpeg, ImageMagick, and optionally sips and optipng. The WebVTT generation part, I had to write.

You will have to install ffmpeg & imagemagick, at a minimum to use this.

Github code is here: https://github.com/vlanard/videoscripts (under sprites/).

The basic gist is:

  1. Create a bunch of thumbnails, e.g. every 45th second from a video

    ffmpeg -i ../archive/myvideofile.mp4 -f image2 -bt 20M -vf fps=1/45 thumbs/myvideofile/tv%03d.png

  2. Resize those thumbnails to be small, e.g. 100pixels wide

    sips --resampleWidth 100 thumbs/myvideofile/tv001.png thumbs/myvideofile/tv002.png thumbs/myvideofile/tv003.png

    OR if sips not available, use imageMagick utility:

    mogrify -geometry 100x thumbs/myvideofile/tv001.png thumbs/myvideofile/tv002.png thumbs/myvideofile/tv003.png

  3. Get the height & width dimensions of one of the thumbnails to use as the basis of our grid coordinates, using ImageMagick utility 

    identify -format "%g - %f" thumbs/myvideofile/tv001.png

    which returns output like : 100x55+0+0 - tv001.png

    from which we parse 100 and 55 as our Width & Height, and the general geometry of each thumbnail (W, H, X, Y)

  4. We then generate our single spritemap from the individual thumbnails. We determine the target grid size (e.g. 2x2, 8x8) to suit the number of thumbnails we generated for this video, as well as passing in the sprite geometry, using an ImageMagick utility

    montage thumbs/myvideofile/tv*.png -tile 2x2 -geometry 100x55+0+0 thumbs/myvideofile/myvideofile_sprite.png

  5. Optionally we can run an extra compression step here to make the sprite smaller

    optipng thumbs/myvideofile/myvideofile_sprite.png

  6. We then generate a VTT file based on the number of thumbnails we created, using the interval that we used to space out the thumbnails to label each time segment, and using the known coordinates of each consecutive image within our sprite that maps to the associated segment.

randalv
  • 900
  • 9
  • 19
  • Thanks, i finally did something similar but before taking the thumbnails i take the length of the video so i have the same amount of thumbs for all videos, i find this easier for generate the vtt. – Tahola Dec 21 '13 at 08:17
  • This is absolutely a fantastic job! I would like to make a Ruby wrapper for this as a gem if you give permission? – scaryguy Jan 28 '14 at 10:12
  • 1
    @scaryguy, please do! I checked in latest to github with several key updates. 1)using jpg instead of png, much smaller sprites, 2)added adjustment to better sync the snapshot image to the start time in vtt file, to offset time drift occurring in ffmpeg; 3)omitting first image (0seconds) from sprite because JwPlayer doesn't show it anyway; 4) configurable snapshot timing now via command line parameter – randalv Jan 29 '14 at 00:57
  • @randalv you've made a great work! Congrats! I'm pretty busy nowadays but want to do it in some way. let's see what I can do. – scaryguy Jan 29 '14 at 08:56
  • @randalv it's online :) Thanks for inspiring! http://stackoverflow.com/a/21474786/445144 – scaryguy Jan 31 '14 at 07:38
  • Thanks randalv. Very helpful. I just created a PHP adaptation here : https://github.com/F2000-FR/videoscripts-php – F2000 Dec 03 '14 at 17:58
  • After few research i finally make it, get duration, make thumbnail, join to sprites, generate vtt file :) – John Doe Nov 23 '17 at 04:09
  • That is incredible! – George Chalhoub May 22 '18 at 12:57
  • This is really cool. I am close to getting this to work but hit a snag. I get this error ... subprocess.CalledProcessError: Command '['sips', '--resampleWidth', '100']' returned non-zero exit status 4 – Sflagg Apr 11 '19 at 22:32
  • 1
    @Sflagg Try running `which sips` in your terminal to make sure you have it installed. If you see it I would also try manually running `sips --resampleWidth 100 yourfilewhatever.png` to see what that returns (obviously replacing file name with a legit one). – randalv Apr 13 '19 at 02:59
  • Figured it out. I was a using a short clip of 30 seconds and the increment was too long; reduced it to every 5th second and it worked great. Thanks – Sflagg Apr 15 '19 at 18:41
  • Generated sprites are very big even after compression. In which step should I reduce the size and how? – ufukomer Sep 23 '19 at 17:18
  • @ufokomer if optipng compression is still too large, you might want to review how many images are in your sprite (and how big they are) and play with that. you could also manually try another compression program like ImageOptim. If that doesn't help much, it's likely an issue with what's in the image itself. – randalv Sep 24 '19 at 20:55
  • @randalv I've followed your guide and created `C#` version of it, however its seems to really slow with large files. E.g. I have 2GB files, Full HD, H264 encoded and generating 1min thumbnail takes like 10-20 mins, I'm scared to try with thumbnails every 5 secs for example. :( – Expressingx Apr 22 '20 at 18:20
  • @expressingx possibly some helpful advice here? https://stackoverflow.com/questions/18534835/fastest-way-to-extract-a-specific-frame-from-a-video-php-ffmpeg-anything – randalv Apr 23 '20 at 03:20
4

I've developed a Ruby gem to easily create .VTT file and sprite of thumbnails.

Thanks for inspiring @randalv!

You can take a look at it here: https://github.com/scaryguy/jwthumbs

Usage

Instantiate your video file:

movie = Jwthumbs::Movie.new("YOUR_VIDEO.mp4")

Jwthumbs::Movie.new accepts second parameter as a options hash. You can configure several stuff at the same time you instantiate your video like this:

movie = Jwthumbs::Movie.new("YOUR_VIDEO.mp4", seconds_between: 60, sprite_name: "my_sprite_name.jpg")

or after you instentiated your video, you can use Jwthumbs::Movie file to configure things:

movie = Jwthumbs::Movie.new("YOUR_VIDEO.mp4")
movie.seconds_between = 60
movie.sprite_name = "my_sprite_name.jpg"

and then to create your thumbnails and .VTT file just run this command.

movie.create_thumbs!
scaryguy
  • 7,720
  • 3
  • 36
  • 52
  • Nice to have 2 language options now. Great contribution! – randalv Jan 31 '14 at 17:53
  • @randalv maybe you can give a back link to my repo, as I did for you :) You know, back links make it easier to find things through search engines. And don't forget to up vote this answer ;) – scaryguy Feb 01 '14 at 08:35
  • Dear down voter, rather than down voting an answer which only contains link of an 'open source' project; maybe you should try to do something for the community. – scaryguy Nov 22 '15 at 00:49
3

I know this is already a few years old but I had the same problem and found a command line tool which generates sprites pretty fast and since 1.0.6 supports WebVTT creation out of the box. The name is mt and you can check it here.

Quoting from their documentation you can use it like this:

just run mt and provide any video file as args: mt video.avi

Some of the settings can be changed through runtime flags provided directly to mt for more information just run mt --help

armatita
  • 12,825
  • 8
  • 48
  • 49
cytec
  • 59
  • 4
2

Option 1 :

You can use the encoding.com's API and tell them to export vtt file too

I recommend to read "How can I create time synced thumbnails for use in JW player?" explanation from encoding.com's Knowledge base

Option 2 :

use movie thumbnailer (mtn), this is a command line tools running on UNIX, Windows systems. But you will have to write a custom script to generate the VTT file corresponding

  • Super fast! Thanks to FFmpeg's libavcodec.
  • Command line program: canbe used on remote connections to co-location servers, or used in scripts.
  • Batch mode: recursively search directories for movie files. Run at lower priority (nice 10 on Linux, idle on Windows) by default. To run at normal priority use -n option.
  • Thumbnails are group together in one jpeg file and can be saved individually too (-I option).
  • Work fine with Unicode filenames in both Linux & Windows (might need to change the font with -f fontfile).
0x1gene
  • 3,349
  • 4
  • 29
  • 48