Not able to use tabCapture api in the background script and content script

Question

I'm building a chrome extension for catching screen records and editing them, I'm trying to use the chrome tab capture API for getting video stream in the content script or background script and I'm getting an undefined error. I am using manifest version 3 and I set in the manifest the tab capture permission here is a sample trying to run in the content script. I also tried to send the message to the popup.js, but as soon as the popup close, the recording stops and return error :-

content-script.js
var recordButton = document.createElement("button");
    recordButton.innerHTML = "Record";
    chrome.runtime.sendMessage({message: "access_tabCapture_API"});
    
    // Add an event listener to the record button to start recording
    recordButton.addEventListener("click", function() {
      

      chrome.runtime.onMessage.addListener(function(request, sender, sendResponse) {
        if(request.message === "stream_object") {
          // Use the stream object received from the background script
          //var stream = request.stream;
          if(request.stream != undefined || request.stream != null) {
            console.log('yes, its working')
          } else {
            console.log('nothing there')
          }
         
        }
      });

background.js
chrome.runtime.onMessage.addListener(function(request, sender, sendResponse) {
    if(request.message === "access_tabCapture_API") {
      chrome.tabCapture.capture({audio: true, video: true}, function(stream) {
        // Once the tabCapture API is accessed, send the stream object back to the content script
        chrome.tabs.query({active: true, currentWindow: true}, function(tabs) {
          chrome.tabs.sendMessage(tabs[0].id, {message: "stream_object", stream: stream});
        });
      });
    }
  });

I am getting error :- Error in event handler: TypeError: Cannot read properties of undefined (reading 'capture') at chrome-......

You need to add "tabCapture" to "permissions" in manifest.json. — wOxxOm, Mar 16 '23 at 09:48
@wOxxOm, I already added the tabCapture permission in the manifest.json, but still the same error — Darwin Swartz, Mar 16 '23 at 10:25
@Yogi, I tried to send the message to the popup.js, but the popup needs to be opened, if the popup is closed,i.e, clicked on the web page, the popup got closed and the recording stops, so popup.js is not the right way of doing it — Darwin Swartz, Mar 16 '23 at 10:27
The stream object is undefined when send it from the popup.js — Darwin Swartz, Mar 16 '23 at 10:28
@wOxxOm, I tried everything, removing, saving, editing, but still in background and content script, I got the type error .capture error, if you want I could share the whole code? I am not able to figure out whats wrong with the code — Darwin Swartz, Mar 16 '23 at 10:56
@wOxxOm, After researching I found out that tabCapture is not supported in background script for manifest v3, but it does work in v2, and v2 will be depreciated this year, so is there any fix to use tabCapture in background script v3? — Darwin Swartz, Mar 16 '23 at 11:28
@Yogi, I also looked at that question, but there they try to get the audio of the current tab, and they used .mediaDevices.getUserMedia for it which is used to get the camera and microphone permission. What I want to do is to record the current tab of the user without giving the prompt to choose which screen they want to share like with desktopCapture — Darwin Swartz, Mar 16 '23 at 13:49
@wOxxOm, here is my manifest.json :- ```js { "manifest_version": 3, "name": "Once More", "version": "1.0", "description": "Description of my extension", "permissions": ["scripting", "activeTab", "tabs", "tabCapture"], "action": { "default_popup": "popup.html", "default_title": "Click Me" }, "background": { "service_worker": "background.js" }, "host_permissions": [""] } ``` — Darwin Swartz, Mar 16 '23 at 15:38
@Yogi, I tried the loom chrome extension, and once the user clicks on the extension icon, it injects the button on the content page using the content script and after clicking on it, it just records the tab normally. TabCapture is not working on the background and content-script in v3, I tried researching, but none of the solution I found that would record the tab screen — Darwin Swartz, Mar 16 '23 at 15:42
@wOxxOm, As mentioned there too, the tabCapture API does not work in background script, and I am not able to find the workaround of it. What should I use in order to capture the current tab of the user after they click on a record button inside the content script? — Darwin Swartz, Mar 16 '23 at 19:10
Open a new tab or window for popup.html (using chrome.tabs.create or chrome.windows.create in the background script) and use the API there or do it in your `action` popup. — wOxxOm, Mar 16 '23 at 19:45
@wOxxOm, The problem with capturing the tab in the popup is that it stops the capturing if the popup got closed. When clicked on the web page the popup got closed and return an undefined value for the stream. And I have seen other extension using injecting the code in the web page the then recording from there? — Darwin Swartz, Mar 17 '23 at 03:17
Add an iframe to the web page that points to an html file in your `web_accessible_resources` ([example](https://stackoverflow.com/a/68689866)) and use tabCapture inside. — wOxxOm, Mar 17 '23 at 15:31
@wOxxOm, I am trying out your method of using iframe to capture current tab. Can you please explain an general process of how to implement / how does it gonna work in a nutshell? — Darwin Swartz, Mar 21 '23 at 05:09
The content scripts adds an iframe that points to iframe.html that loads iframe.js (there are many examples for this). It will have access to all `chrome` API and it'll capture the tab. To communicate with the tab use the example I've linked above. — wOxxOm, Mar 21 '23 at 12:38
@wOxxOm, I setup a button inside the content script that adds an iframe on the document which points to an web assembly hosted HTML file name iframe.html and inside it I have a src which loads a javascript file name iframe.js, but when I call the tabCapture inside the iframe.js, it logs this error :- `Unchecked runtime.lastError: Error starting tab capture` — Darwin Swartz, Mar 21 '23 at 16:22

score 0 · Answer 1 · answered Mar 18 '23 at 23:24

Here is a sample using chrome.tabCapture.getMediaStreamId.

manifest.json

{
  "name": "chrome.tabCapture",
  "version": "1.0",
  "manifest_version": 3,
  "permissions": [
    "tabCapture"
  ],
  "content_scripts": [
    {
      "matches": [
        "<all_urls>"
      ],
      "js": [
        "matches.js"
      ]
    }
  ],
  "action": {
    "default_popup": "popup.html"
  }
}

popup.html

<html>
<body>
  <video id="video" src="" autoplay controls></video><br>
  <button id="stop">stop</button><br>
  <button id="play">play</button>
  <script src="popup.js"></script>
</body>
</html>

popup.js

let tabId;

chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
  tabId = tabs[0].id;

  chrome.tabCapture.getCapturedTabs((ci) => {
    if (!ci.some(e => e.tabId == tabId)) {
      chrome.tabCapture.getMediaStreamId({ consumerTabId: tabId }, (streamId) => {
        chrome.tabs.sendMessage(tabId, { command: "start", streamId: streamId });
      });
    }
  });
});

document.getElementById("stop").onclick = () => {
  chrome.tabs.sendMessage(tabId, { command: "stop" });
}

document.getElementById("play").onclick = () => {
  chrome.tabs.sendMessage(tabId, { command: "play" }, (url) => {
    document.getElementById("video").src = url;
  });
}

matches.js

let recorder;
let chunks;
let url;

const start = (streamId) => {
  navigator.mediaDevices.getUserMedia({
    audio: false,
    video: {
      mandatory: {
        chromeMediaSource: "tab",
        chromeMediaSourceId: streamId
      }
    }
  })
    .then((stream) => {
      recorder = new MediaRecorder(stream);
      chunks = [];
      recorder.ondataavailable = (e) => {
        chunks.push(e.data);
      };
      recorder.onstop = (e) => {
        const blob = new Blob(chunks, { type: "video/ogg; codecs=opus" });
        url = URL.createObjectURL(blob);
      }
      recorder.start();
    });
}

chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
  switch (message.command) {
    case "start":
      start(message.streamId);
      break;
    case "stop":
      recorder.stop();
      break;
    case "play":
      sendResponse(url)
      break;
  }

  return true;
});

Yes, I tried this one earlier, but the problem with this method is, when there a reload on the page, it requires a new stream id to continue recording, but because the page reloaded, it's not possible to again generate a streamid from popup.js without opening it — Darwin Swartz, Mar 19 '23 at 02:57

Mahipal Singh Khichi · Answer 2 · 2023-07-17T10:00:53.463

you can use getUserMedia to capture audio from the microphone and video from the camera. Additionally, you can use getDisplayMedia to capture audio and video from the system in a Chrome extension manifest V3. This is the process you can follow using the content script.

async function startRecording() {
try {
desktopMediaStream = await 
navigator.mediaDevices.getDisplayMedia({ 
  audio: true,
  video: false
});


microphoneStream = await navigator.mediaDevices.getUserMedia({ 
audio: true });

const audioContext = new AudioContext();
const microphoneSource = 
audioContext.createMediaStreamSource(microphoneStream);

if (desktopMediaStream.getAudioTracks().length > 0) {
  const destination = 
audioContext.createMediaStreamDestination();
  const desktopAudioSource = 
audioContext.createMediaStreamSource(desktopMediaStream);
  desktopAudioSource.connect(destination);
  microphoneSource.connect(destination);

  combinedStream = destination.stream;

  if (!MediaRecorder.isTypeSupported('audio/webm')) {
    return alert('Browser not supported');
  }

  combinedMediaRecorder = new MediaRecorder(combinedStream, {
    mimeType: 'audio/webm',
  });
}

//you can perform operations on combinedMediaRecorder
//And you need to define some of variables.

Not able to use tabCapture api in the background script and content script

2 Answers2