MediaPipe selfie segmentation: Getting error when trying to capture stream from canvas

Question

I'm trying to use selfie segmentation on the image from my webcam. For this, I'm using the MediaPipe lib.

Here is my code

const selfieSegmentation = new SelfieSegmentation({locateFile: (file) => {
    return `https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation/${file}`;
}});
selfieSegmentation.setOptions({
    modelSelection: 1,
});
selfieSegmentation.onResults(handleSegmentationResults);

videoElement.addEventListener('playing', function () {
    selfieSegmentation.send({image: this})
})

Here is where I get the segmentation result and draw the canvas:

const videoElement = document.getElementById('myVideo');
const canvasElement = document.getElementsByClassName('output_canvas')[0];
const controlsElement = document.getElementsByClassName('control-panel')[0];
const canvasCtx = canvasElement.getContext('2d');
const img = document.getElementById('vbackground');

function handleSegmentationResults(results) {
    canvasCtx.save();

    canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
    canvasCtx.globalCompositeOperation =  'source-in'; 
    canvasCtx.drawImage(results.image, 0, 0, canvasElement.width, canvasElement.height);

    // // Make all pixels not in the segmentation mask transparent
    canvasCtx.globalCompositeOperation = 'destination-atop';
    canvasCtx.drawImage(results.segmentationMask, 0, 0, canvasElement.width, canvasElement.height);

    canvasCtx.globalCompositeOperation = 'destination-over';
    canvasCtx.drawImage(img, 0, 0, canvasElement.width, canvasElement.height);

    canvasCtx.restore();
    //here is where I'm getting the error
    const canvasStream = canvasCtx.canvas.captureStream();
}

I want to capture the canvas stream so that I can place it on my video element. But when I try to use the method canvasCtx.canvas.captureStream(), I'm getting this error:

Uncaught DOMException: Failed to execute 'captureStream' on 'HTMLCanvasElement': Canvas is not origin-clean. at eval (eval at handleSegmentationResults (http://localhost:3000/js/host.js:1570:9), :1:18) at handleSegmentationResults (http://localhost:3000/js/host.js:1570:9) at https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation/selfie_segmentation.js:88:322

Does anyone know another way to capture the stream or what am I doing wrong? Thanks!

score 0 · Answer 1 · answered Feb 17 '23 at 12:55

0

handleSegmentationResults executes each time when result of segmentation returns something

I suppose that the first of all you need to move canvasStream outside of this function

Also, you might use canvasElement itself to capture the stream

answered Feb 17 '23 at 12:55

Dmitrii Kucher

11
3

MediaPipe selfie segmentation: Getting error when trying to capture stream from canvas

1 Answers1