2

I have data exceeding 5MB I need to store in session storage. To that end, I'm using pako to compress the data.

First, we have an Angular app that receives data from an API and adds it to a hash 'cachedLookups':

const stringifiedLookups = JSON.stringify(this.cachedLookups)
const compressedLookups = new TextDecoder().decode(pako.deflate(stringifiedLookups));
sessionStorage.setItem(this.sessionStorageKey, compressedLookups);

Then we have an AngularJS app in the same browser window that retrieves this data from the Session Storage:

const compressedLookups = localStorageService.get("cachedLookups");
const compressedLookupsUint8Array = new TextEncoder().encode(compressedLookups);
const stringifiedLookups = pako.inflate(compressedLookupsUint8Array, { to: 'string' });

When I hit pako.inflate, I get 'incorrect header check'. I've also tried inflateRaw, in which case I get 'invalid stored block lengths'. I'm using TextEncoder/Decoder here as attempting to store the Uint8Array directly into SessionStorage would force SessionStorage to exceed its quota, despite being calculated at under 5MB. I assume that issue had to do with the fact that the Storage API is all about storing key value string pairs.

georgeawg
  • 48,608
  • 13
  • 72
  • 95
Thomas Preston
  • 697
  • 1
  • 7
  • 19
  • Further information: We correctly retrieve the TextDecoded data from SessionStorage, and when we run it through TextEncoder it returns a Uint8Array identical to the original deflated data. If someone could help explain why I might possibly be getting "incorrect header check" that would be great. – Thomas Preston Jul 06 '21 at 20:32

1 Answers1

2

It seems like the zip header is causing an error when you encode/decode the zip file, because the Uint8Array returned by pako and the one returned by TextEncoder don't have the same values.

var data = "a";
var deflated = pako.deflate(data)

var textEncoded = new TextDecoder().decode(deflated)
var binary = new TextEncoder().encode(textEncoded)


// Not the same
console.log(deflated)
console.log(binary)

// ERROR
console.log(pako.inflate(binary))

if instead you use deflateRaw which doesn't add the zip header it works just fine

var data = "a";
var deflated = pako.deflateRaw(data)

var textEncoded = new TextDecoder().decode(deflated)
var binary = new TextEncoder().encode(textEncoded)

// Same content
console.log(deflated)
console.log(binary)

// SUCCESS
console.log(pako.inflateRaw(binary, {to: 'string'}))
sney2002
  • 856
  • 3
  • 6
  • I first deflateRaw my data. I then decode it and store it in SessionStorage. I then retrieve my data and encode it. Finally, I attempt to inflateRaw the encoded data. This is when I get the error "invalid block type" – Thomas Preston Jul 07 '21 at 14:51
  • @ThomasPreston in your example you are using deflate/inflate instead of deflateRaw and inflateRaw – sney2002 Jul 07 '21 at 16:55
  • 1
    Yes in the original example I was using inflate/deflate but in my current tests I was using inflateRaw/deflateRaw as I described in my previous comment. That is how I get the "invalid block type" error. – Thomas Preston Jul 08 '21 at 19:55
  • Maybe is something related to your data, Could you share a sample of your 5mb data which gives you this error? – sney2002 Jul 09 '21 at 13:25