2

Task

I am currently trying to create a web extension for Firefox. It should be able to read images' pixels. For that purpose, I am rendering the image on an invsible canvas, and then I want to read it.

Example Code

function getImdata(reference) {
    var canvas=document.createElement("canvas");
    canvas.width=reference.naturalWidth;
    canvas.height=reference.naturalHeight;
    var context=canvas.getContext("2d");
    context.drawImage(reference,0,0);
    return context.getImageData(0,0,reference.naturalWidth,reference.naturalHeight); //Here I actually get the error...
}

Problem

However, I am getting a "Security Error" if I use "getImageData()".

Question

So I need a workaround, but couldn't find anything myself. How can I read images' pixels without getImageData() ?

EDIT

Apparently it has something to do with CORS : HTML5 Canvas getImageData and Same Origin Policy

Thanks in advance!

Luatic
  • 8,513
  • 2
  • 13
  • 34
  • You are asking the wrong question: all other methods will also be tied by this SameOrigin policy, so what you want is to workaround this, not the method to read the pixels. And this as been asked **many** times in better form than your own question. Do a simple google search with the message of your error and you'll get them. – Kaiido Feb 27 '18 at 23:32
  • @Kaiido "all other methods will also be tied by this SameOrigin policy" since he runs from an extension his script can have privileged access to cross-origin sources. –  Feb 28 '18 at 04:08
  • @K3N "*all other methods*" [to read an Image's pixels]. My comment is still correct, OP were not asking the correct question, and you'll note you didn't answered it either, since you do use `getImageData`. You are answering "*How to circumvent same-origin policy from an extension?*" Which is the question OP should have asked, and which is what my comment was asking for. – Kaiido Feb 28 '18 at 04:29
  • It wasn't meant as criticism, just that you can use getImageData (or toDataURL) from an extension since the extension has privileged status (which is OPs ultimate goal AFAICS). –  Feb 28 '18 at 04:34

1 Answers1

2

There is. Since you're running from an extension your extension will have privileged access to cross-origin sources but only if loaded via fetch() and XMLHttpRequest() from a content script (or background script) - excerpt from that link:

Content scripts get the same cross-domain privileges as the rest of the extension: so if the extension has requested cross-domain access for a domain using the permissions key in manifest.json, then its content scripts get access that domain as well.

This is accomplished by exposing more privileged XHR and fetch instances in the content script [...]

Please note that these calls when called from a content script will not set origin and referer headers which sometimes can cause problems if the cross-origin site expects these to be set - for those cases you will need to use the non-privileged content.XMLHttpRequest or content.fetch() which will bring you back to square one.

The permissions in the manifest file (or if set permissions dynamically) must also allow access to these cross-origin sites.

This means however that you will have to "reload" the image source separately via these calls. You can do this the following way by first obtaining the original URL to the image you want to load, say, from a content script:

// example loading all images in current tab
let images = document.querySelectorAll("img");
for(let image of images) loadAsBitmap(image.src); // some sub-call using the url

Then load that source via the content script's fetch():

fetch(src).then(resp => {  // load from original source
  return resp.blob();      // obtain a blob
}).then(blob => {          // convert blob, see below
  // ...
};

When the blob is obtained you can convert it to an Object-URL and set that as source for an image and be able to go around the cross-origin restriction we otherwise face. In the content script, next steps would be:

let url = URL.createObjectURL(blob);         // attach Object-URL to blob
let img = new Image();                       // create image element *
img.onload = () => {                         // attach handler
  let c = document.createElement("canvas");  // create canvas
  let ctx = c.getContext("2d");              // get context
  c.width = img.width;                       // canvas size = image
  c.height = img.height;
  ctx.drawImage(img, 0, 0);                  // draw in image
  URL.revokeObjectURL(url);                  // remove reference.
  let imageData = 
    ctx.getImageData(0,0,c.width,c.height);  // get image data
  // .. callback to a function that handles the image data
};
img.src = url;                               // start loading blob
  • Just to ensure I get you right : So the first code snippet has to be in a file - let's say - content.js. The second and third snippet will be in a file - main.js. In 2nd snippets `fetch(src)`, src has to be replaced with the "content.js". – Luatic Mar 01 '18 at 14:32
  • Unfortunately, it still doesn't work. I get Quellübergreifende (Cross-Origin) Anfrage blockiert: Die Gleiche-Quelle-Regel verbietet das Lesen der externen Ressource auf https://collector.githubapp.com/collect. (Grund: Berechtigung/Credential wird nicht unterstützt, wenn die CORS-Kopfzeile 'Access-Control-Allow-Origin' auf '*' gesetzt ist). – Luatic Mar 01 '18 at 14:37
  • All of this code must be run from a script with content-script status (see [permissions](https://developer.mozilla.org/en-US/Add-ons/WebExtensions/manifest.json/content_scripts) - you can call the script anything but it needs to be run as a content script). –  Mar 05 '18 at 06:45
  • Also make sure that the external site is allowed, see `matches` (ibid). –  Mar 05 '18 at 06:48