1

I'm creating a Chrome extension that needs to store a lot of data. I've set the unlimitedStorage permission, and I'm successfully storing information using calls to chrome.storage.local.set. However, I'm unable to make calls to chrome.storage.local.get. When I make a call as simple as chrome.storage.local.get(null, () => {}), Chrome crashes.

My issue seems similar to chrome.storage.local.get limit, except that that issue was about a much smaller amount of data, and turned out to be a fluke. When I call chrome.storage.local.getBytesInUse(null, i => console.log(i)), I get a result of 176031461. (Admittedly, 180 MB is a lot more than Chrome extensions should typically use, but this extension will be running on my own machine.)

I'd like to be able to save all of this data into a JSON file, but to do that, it seems I need to bring it into memory first, and the only way to do that is through chrome.storage.local.get. I'm trying to use the method described at Chrome Extension: Local Storage, how to export to download the data, but it doesn't even get to the callback function. I'm not sure what's causing it to crash, and I don't think it's a memory limit, given that I've followed the instructions at Max memory usage of a chrome process (tab) & how do I increase it? to increase Chrome's max memory to 4 GB.

One potential solution would be to set the first parameter to something other than null and download the data in chunks, but I'd rather not do that if I don't have to.

My questions are:

  1. What is the limit to the amount of storage that can be retrieved this way?
  2. Is there any way for me to get all of that data at once without crashing?
  3. Is there a better way for me to be saving files from a Chrome extension?
Finn
  • 652
  • 7
  • 16
  • 1
    It's a bug in Chrome which I've just reported (it's hidden currently though) which limits the amount of data to 128MiB since Chrome 60. You can use HTML5 FileSystem in Chrome or IndexedDB but I don't know if the bug also affect them. – wOxxOm Nov 12 '18 at 15:49
  • @wOxxOm thanks, that's useful information. Clarification question: when you say it "limits the amount of data," you mean just for a single call to `chrome.storage.local.get`, correct? So if I saved a gigabyte of data but then downloaded it in chunks of 100 megabytes, that would work? – Finn Nov 12 '18 at 16:04
  • I didn't test it further. – wOxxOm Nov 12 '18 at 16:06
  • 1
    The bug I reported was just fixed via https://crrev.com/c/1333989. Now the limit is that of the JS engine (a few GB, I guess). Normally it'll roll out in Chrome 72. I've asked them if it'll be merged in 71 (unlikely, though). – wOxxOm Nov 14 '18 at 05:46
  • 1
    It's just been merged into Chrome 71 too, which becomes stable in December. – wOxxOm Nov 15 '18 at 06:52

0 Answers0