5

I couldn't think of a good title, so hopefully that will do.

What I'm doing is creating an offline HTML5 webapp. "For certain reasons" I don't want some files to be placed in the cache manifest, but instead I want to place the content in localStorage.

All testing is done in Google Chrome for Mac/Mac OS 10.6.6

I have 4 .JSON files that are exactly 3,120,792 bytes in size total. Upon first visit (online) to the app, I am pulling down this JSON content and placing it in one string variable, say str. Before storing the JSON string, I am stripping all \n and \t , and replacing \" with ' , to minimize the string length.

Here are the string lengths (str.length in Chrome console):

original str : 3,117,764 (this is a concatenation of the contents of the 4 JSON files)
reformatted JSON: 2,732,249 (not too shabby - should save me a few bytes)

HOWEVER, now when I try to store this string in localStorage, I get an error that I have exceeded the cache size (Error: QUOTA_EXCEEDED_ERR: DOM Exception 22). BUT there is NOTHING else cached for this page (I have manually deleted all Google Chrome cache files).

Why is this happening?

What is the size of the string when stored in the cache?

How can I minimize this size to make best use of localStorage?

edited to add: I have tried saving only 1,000,000 characters of the string into localStorage (localStorage.setItem('x',str.substr(1,1000000)). When I checked the localStorage file size in Chrome's cache folder it is 2MB. It's an sqlite db. It looks like the browser is using the actual localStorage database size, not the size of the data stored inside it. Which of course sucks, but is probably standard behavior. I guess the longest string you can store in localStorage is somewhere around 2,500,000 characters?

ampersand
  • 4,264
  • 2
  • 24
  • 32
  • http://diveintohtml5.org/storage.html#limitations says you should have 5mb, and other stackoverflow questions corroberate this. how about writing a loop, store 1 char, pull it back out, store 2 chars, etc until you hit a limit? – Bruce Aldridge Apr 28 '11 at 06:07
  • Thanks, Bruce. I know about the 5mb limit, but I was baffled because I was only feeding localStorage 3mb. I didn't account for the sqlite 'overhead.'...btw, your idea is interesting, and I think I'll try it, if only to see how slow it'll be :) j/k :) – ampersand Apr 28 '11 at 06:27

3 Answers3

11

A good way to know how many charactere can be store in the localstorage is to test it thanks to that link : http://arty.name/localstorage.html.

Size dépend on Browser HTML5 implementation

My last test : Chrome 13/Safari 5/ IPAD 2 :2.600.000 chr FIREFOX 4 : 5.200.000 OPERA 11 : unlimited

ouaf
  • 111
  • 3
5

http://jsfiddle.net/brucealdridge/j6KdJ/ 2,500,000 chars works on mine ("5.0 (X11; Linux i686) AppleWebKit/534.30 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.742.9 Chrome/12.0.742.9 Safari/534.30") 3,000,000 doesn't. I was thinking it might me an encoding thing, but i've just checked, and its not

Bruce Aldridge
  • 2,907
  • 3
  • 23
  • 30
  • +1. Thanks for the handy script. I was able to get up to `2,605,000` characters written on MAC OS 10.6.6 with Chrome 11.0.696.57...eh, I guess I will have to use the cache manifest after all, as it'll store pretty much any file exactly as you serve it. Thanks, Bruce. – ampersand Apr 28 '11 at 18:15
  • 1
    Found the answer ... it was encoding .. see http://stackoverflow.com/questions/2747285/html5-localstorage-restrictions-and-limits its in the comments of the questions "Webkit-based browsers use UTF-16 for storage which haves it to 2.5MB limit" – Bruce Aldridge Apr 28 '11 at 22:26
  • ahh, that's unfortunate. Thanks for the info, Bruce! I've rewritten my app to use the cache manifest for these JSON files. – ampersand Apr 30 '11 at 00:58
-1

Weird that an answer illustrating max amount would be accepted. Shouldn't you be verifying the current size of all localStorage key/value pairs minus the max limits implemented by browsers?

Here is a jsfiddle to illustrate this usage. And for those that want a simple answer use the following bit of js...

 function _calc() { return 1024 * 1024 * 5 - unescape(encodeURIComponent(JSON.stringify(localStorage))).length; }

UPDATE: This snippit is far more accurate.

function _calc(s){var c,e=s||window.localStorage;for(var k in e){if(e.hasOwnProperty(k)){c+=e[k];}}return c?3+((c.length*16)/(8*1024)):0;}
jas-
  • 1,801
  • 1
  • 18
  • 30
  • 1
    (1) The goal wasn't to calculate how much space was *left* in localStorage, but why there was an apparent discrepancy between the theoretical *limit* and how many characters were actually able to be stored. The selected answer demonstrated that it is an encoding issue. (2) Your code assumes a 5242880 character limit, which is not valid for all browsers. – ampersand Apr 13 '12 at 10:51
  • Could you provide a list of these limits per browser? – jas- Apr 13 '12 at 13:17
  • Here's a good resource that was mentioned in another answer: http://arty.name/localstorage.html. Doesn't give exact limits, but a range. You can use that as a starting point to work towards the exact limit. – ampersand Apr 14 '12 at 19:13
  • Strange, according to the w3c spec http://www.w3.org/TR/webstorage/#disk-space they should all be using 5MB – jas- Oct 08 '12 at 12:24