Redux + ImmutableJS - how to garbage collect too large store?

Question

I'm using Redux with ImmutableJS. In my SPA (quite complicated administration system), users often load a lot of data into stores (thousands rows for many tables). After opening several pages and having too many data in the store, the app becomes significantly slower, because the ImmutableJS store can contain even millions entries.

How can I "delete" something from the store, so that the data don't slow down the app? I know that this would be against its main principle, but how else would you solve it?

Using a common website with for example jQuery, it would be pretty easy. With every page refresh, everything unnecessary would be garbage collected. Therefore, 2-3 thousands entries for one page would be ok, but when opening a new page, the reducer loads new data, but the old ones are still being referenced to.

And I don't want to force the user to reload the page, of course.

Answer 1

Triple-Check this is really memory bloat and not just unnecessary re-rendering/re-computation

Mutating state in Redux is almost always a really bad idea as it's a sort of prerequisite for the library. My first concern would be to triple-check that you are indeed running into memory issues due to memory bloat and not due to unnecessary re-renders or due to unnecessary computations. By this I mean, if you are keeping huge amounts of data in the same place, make sure you are not doing something that causes React to unnecessarily re-render things or sift through too much data.

You can solve that issue by judicial use of the reselect library or by using some other type of memoization that will retrieve data from your reducers in a manner that avoids unnecessary recomputation. Similarly, make sure you aren't unnecessarily re-rendering every item in your entire list just when changing a single row.

Get rid of references to previous state

To get something to be garbage compiled in JavaScript, you just make sure that nothing is referencing it any longer.

If you really need to go down this path, it's essential that you don't keep the old page's data "alive" if you will because JavaScript only garbage collects things that are no longer referenced.

Vanilla Redux is not holding on to previous states under the hood. All it's doing is keeping the current state in a let variable and then changing its value after getting the new state from dispatching the action to the root reducer (see source code).

So, I would ensure I''m not using something like redux dev tools which persists previous state.

Then in the reducer, create a new object that doesn't use the previous data in any way, but rather returns a new object with the new data:

const data = (state, action) => {
  switch (action.type) {
    case 'RETRIEVE_DATA_SUCCESS':
      return action.payload;
    default:
      return state
  }
}

I suggest reading up on Chrome's memory profiling tools here to make sure this is working correctly.

Answer 2

I would suggest instead of keeping the whole data in the store to keep a pointer of it to a memory solution (localstorage,REDIS etc). I would have use PouncDB and store the _rev revision number only in my store.

Answer 3

The previous post is wrong about Immutablejs keeping old data 'alive'...well, in the sense it's being described. Immutablejs uses structural sharing, which just means that it will only create a new node and it's children, while sharing the rest of the trie with the old data. If you look at the image below, you'll see that the green and yellow dots on the right are new the data sharing its structure with the old data.

The old data sections (the right hand blue dots) are available for garbage collection, so long as you don't keep keep a reference to it somewhere else in your app. Keeping a reference is mostly beneficial if you'd like to add a time traveling feature to your app.

Had you deep cloned each object per idiomatic Redux, then you'd have an even bigger memory problem without immutablejs' structural sharing. So without seeing some code, I doubt immutable is the issue here.

Are you by chance using toJS()? If you are, that is an expensive operation and should be avoided unless absolutely necessary. It also disconnects the immutable instance and you'll lose all the benefits of structural sharing. This is a common mistake I've seen. You can get access to the values without calling toJS() like so

const immVal = List([1,2,3]) console.log([...immVal.values()])

Lee Byron's talk (creator of immutablejs) structual sharing @23:40 and garbage collection @24:50

Answer 4

Since I saw someone recommended PouchDB, I would like you to have a trial on reduxdb, which we have used it in production for almost a year and didn't find any performance issues.