0

I'd like to build a chrome extension that can make requests against any web page that the user has access to, even pages that are protected by Content Security Policies, preferably in the background (without having to have the page open in the browser).

So for example, I'd like to be able to:

  • request info from a page the user may be logged into, like Gmail
  • request info from a RSS/other pages
  • request info from pages on Facebook

Is this possible? It seems like I could have the extension open a new window, and a tab for every page I want to pull info from. Is this the only way this can work? I'd prefer to have this happen behind the scenes, without having to open a window.

Brad Parks
  • 66,836
  • 64
  • 257
  • 336
  • 1
    An extension can may have access to any remote resources as long as it declares respective cross-origin permissions. But you have to keep in mind that all extension components (incl. background) are run in _**isolated world**_. This mean that your extension can scrape any web page, but _**not**_ as a user logged in. – hindmost Apr 24 '20 at 10:54
  • Thanks - great point! – Brad Parks Apr 27 '20 at 16:29

1 Answers1

1

CSP is not a problem as long as your manifest.json adds the URLs you want to process in permissions e.g. "*://*/" or "<all_urls>" will allow access to any site.

The solution, however, depends on how that page is built. If the server response contains all the info you need then you can simply make a direct request via XMLHttpRequest or fetch (more info) in the background script, parse it with DOMParser and extract the data. Otherwise you can try to run it in an iframe (you'll have to strip X-Frame-Options) or in an inactive/pinned tab and use a content script to extract the data. To access JavaScript variables of the page you'll need to add a DOM script so its code will run in page context.

wOxxOm
  • 65,848
  • 11
  • 132
  • 136
  • I found an opensource lib, called [artoo.js](http://medialab.github.io/artoo/), that does all of the above, and is a great starting point for something like this. Here's the open source [chrome extension](https://medialab.github.io/artoo/chrome/) too. – Brad Parks Apr 25 '20 at 00:38