Upload a video, and save first frame as image JS

Question

I have an webpage where the user will drop a video file, and the page will upload the video, and generate a thumbnail based on a timestamp that the user provide.

For the moment, I am just trying to generate the thumbnail based on the FIRST frame of the video.

here is a quick exemple on my current progress :

(please use chrome as firefox will complain about the https link, and also, sorry if it autodownload an image) https://stackblitz.com/edit/rxjs-qc8iag

import { Observable, throwError } from 'rxjs'

const VIDEO = {
  imageFromFrame(videoSrc: any): Observable<any> {
    return new Observable<any>((obs) => {
      const canvas = document.createElement('canvas')
      const video = document.createElement('video')
      const context = canvas.getContext('2d')
      const source = document.createElement('source');

      source.setAttribute('src', videoSrc);

      video.appendChild(source);
      document.body.appendChild(canvas)
      document.body.appendChild(video)



      if (!context) {
        throwError(`Couldn't retrieve context 2d`)
        obs.complete()
        return
      }

      video.load()

      video.addEventListener('loadedmetadata', function () {
        console.log('loadedmetadata')
        // Set canvas dimensions same as video dimensions
        canvas.width = video.videoWidth
        canvas.height = video.videoHeight
      })

      video.addEventListener('canplay', function () {
        console.log('canplay')
        canvas.style.display = 'inline'
        context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight)

        // Convert canvas image to Base64
        const img = canvas.toDataURL("image/png")
        // Convert Base64 image to binary
        obs.next(VIDEO.dataURItoBlob(img))
        obs.complete()
      })
    })
  },

  dataURItoBlob(dataURI: string): Blob {
    // convert base64/URLEncoded data component to raw binary data held in a string
    var byteString
    if (dataURI.split(',')[0].indexOf('base64') >= 0) byteString = atob(dataURI.split(',')[1])
    else byteString = unescape(dataURI.split(',')[1])
    // separate out the mime component
    var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
    // write the bytes of the string to a typed array
    var ia = new Uint8Array(byteString.length)
    for (var i = 0; i < byteString.length; i++) {
      ia[i] = byteString.charCodeAt(i)
    }
    return new Blob([ia], { type: mimeString })
  },
}

    VIDEO.imageFromFrame('https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4?_=1').subscribe((r) => {
      var a = document.createElement('a')
      document.body.appendChild(a)
      const url = window.URL.createObjectURL(r)
      a.href = url
      a.download = 'sdf'
      a.click()
      window.URL.revokeObjectURL(url)
    })

The problem is, the image it download is empty, and do not represent the first frame of the video. but the video should have been loaded. and drawed in the canvas.

I am trying to solve it, but if someone could help me found out the issue, thanks.

FWIW, the *first frame* of most videos will be black or otherwise not very interesting - it's often better to use a frame that's about 1 second-in. — Dai, Aug 06 '20 at 02:59
1) => yes that is why we can pick different frame, but for now I just want the first (and in the demo video the first framw is not black) 2) cause it s a minimal reproducible demo on stackblitz... and it s a draft to know if it's possible. when I implement it in my app, I ll do it better ofc... — Crocsx, Aug 06 '20 at 03:02
I think the reason it's failing is because the canvas is "tainted" because the video is coming from another origin and its CORS policy disallows is. See here: https://stackoverflow.com/questions/13674835/canvas-tainted-by-cross-origin-data — Dai, Aug 06 '20 at 03:09
I got this error indeed at some point, but it's usually displayed in the console when it happens. — Crocsx, Aug 06 '20 at 03:10
I think the error is being swallowed by RxJS because you don't have a `catch` set-up for your Observable. You don't feel you should be using Observable here - a `Promise` would be fine and would also let you use native `try/catch/finally` which you can't do with `Observable`. — Dai, Aug 06 '20 at 03:14
I found the problem: the `canplay` event is firing too soon. You need to use `loadeddata` instead of `canplay`. **UPDATE** hmm, even with `loadeddata` I still needed to add a `setTimeout` to delay it by another 1000ms. — Dai, Aug 06 '20 at 03:18
https://stackblitz.com/edit/rxjs-3qfqn5 this works as long as you interact with the DOM first. which is not a problem in my app, so it's possible, will work on cleaning implementation now. Thanks for the help. — Crocsx, Aug 06 '20 at 03:18

Crocsx · Accepted Answer · 2020-08-06T08:36:04.150

for anyone looking, I made this which work correctly (need improvement but the idea is there). I use observable cause the flow of my app use observable, but you can change to promise or whatever =>

  imageFromFrame(
    videoFile: File,
    options: { frameTimeInSeconds: number; filename?: string; fileType?: string } = {
      frameTimeInSeconds: 0.1,
    }
  ): Observable<File> {
    return new Observable<any>((obs) => {
      const canvas = document.createElement('canvas')
      const video = document.createElement('video')
      const source = document.createElement('source')
      const context = canvas.getContext('2d')
      const urlRef = URL.createObjectURL(videoFile)

      video.style.display = 'none'
      canvas.style.display = 'none'

      source.setAttribute('src', urlRef)
      video.setAttribute('crossorigin', 'anonymous')

      video.appendChild(source)
      document.body.appendChild(canvas)
      document.body.appendChild(video)

      if (!context) {
        throwError(`Couldn't retrieve context 2d`)
        obs.complete()
        return
      }

      video.currentTime = options.frameTimeInSeconds
      video.load()

      video.addEventListener('loadedmetadata', function () {
        canvas.width = video.videoWidth
        canvas.height = video.videoHeight
      })

      video.addEventListener('loadeddata', function () {
        context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight)

        canvas.toBlob((blob) => {
          if (!blob) {
            return
          }
          obs.next(
            new File([blob], options.filename || FILES.getName(videoFile.name), {
              type: options.fileType || 'image/png',
            })
          )
          obs.complete()
          URL.revokeObjectURL(urlRef)

          video.remove()
          canvas.remove()
        }, options.fileType || 'image/png')
      })
    })
  },

You don't need to use Base64 at all, you can use `canvas.toBlob()`. — Dai, Aug 06 '20 at 07:51

Upload a video, and save first frame as image JS

1 Answers1