0

I am trying to save a string (correction: it is an object containing multiple strings of varying sizes) using chrome.storage.sync.set but am receiving the error: Error: QUOTA_BYTES_PER_ITEM quota exceeded

this is due to the limit being 8092 so I want to break the string down into multiple parts and then be able to reconstruct it when retrieving them.

my code that is giving the existing error is below.

 var obj = {};
  obj[workspaceName] = stringToSave;  
  chrome.storage.sync.set(obj, function() {
    if (chrome.runtime.lastError) {
      return customAlert("Error!: " + chrome.runtime.lastError.message);
    }
  });

Is there an existing function or code that would help me to do this?

Bob Aleena
  • 450
  • 1
  • 6
  • 16
  • 1
    You can use the standard string manipulation functions such as [slice](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/slice). You may want to compress the source first by using LZString library or a similar one. – wOxxOm May 02 '21 at 08:30
  • Thank you. I guess the part I am more curious about is an algorithm for storing/retrieving the object that needs to broken down. almost like a linked list implementation for storage – Bob Aleena May 07 '21 at 03:01

1 Answers1

3

Writing: generate an object like {foo0: chunkA, foo1: chunkB, foo2: chunkC, foo#: 3}.

function chunkedWrite(key, value) {
  return new Promise(resolve => {
    if (typeof key !== 'string') key = `${key}`;
    const str = JSON.stringify(value); // consider using LZString's compressToUTF16
    const len = chrome.storage.sync.QUOTA_BYTES_PER_ITEM - key.length - 4;
    const num = Math.ceil(str.length / len);
    const obj = {};
    obj[key + '#'] = num;
    for (let i = 0; i < num; i++) {
      obj[key + i] = str.substr(i * len, len);
    }
    chrome.storage.sync.set(obj, resolve);
  });
}

Reading is two-part: read the number of keys first, then read all the keys at once.

function chunkedRead(key) {
  return new Promise(resolve => {
    if (typeof key !== 'string') key = `${key}`;
    const keyNum = key + '#';
    chrome.storage.sync.get(keyNum, data => {
      const num = data[keyNum];
      const keys = [];
      for (let i = 0; i < num; i++) {
        keys[i] = key + i;
      }
      chrome.storage.sync.get(keys, data => {
        const chunks = [];
        for (let i = 0; i < num; i++) {
          chunks.push(data[key + i] || '');
        }
        const str = chunks.join('');
        resolve(str ? JSON.parse(str) : undefined);
      });
    });
  });
}

Deleting is similar to reading: read key#, then delete the array of keys.

function chunkedDelete(key) {
  return new Promise(resolve => {
    if (typeof key !== 'string') key = `${key}`;
    const keyNum = key + '#';
    chrome.storage.sync.get(keyNum, data => {
      const num = data[keyNum];
      const keys = [keyNum];
      for (let i = 0; i < num; i++) {
        keys.push(key + i);
      }
      chrome.storage.sync.remove(keys, resolve);
    });
  });
}
wOxxOm
  • 65,848
  • 11
  • 132
  • 136
  • Thank you! I was struggling with the retrieve and this helped! – Bob Aleena May 12 '21 at 06:30
  • I think a small tweak is needed. After the for statement, add keys[num] = keyNum; to also delete the key with the number of chunks – Bob Aleena May 20 '21 at 04:50
  • 1
    Ah, I missed the word "delete", thanks for the correction. – wOxxOm May 21 '21 at 03:30
  • Using QUOTA_BYTES_PER_ITEM as max string length will fail if the string to be stored contains characters that use more than one byte per character. To avoid this encode the string with `btoa()` after stringifying it. And decode with `atob()` before JSON.parsing it. – Kolya Mar 14 '22 at 23:01