In React, what is a better place to store big data arrays?

Question

I'm building an interactive plot in React where the user can select multiple variables and add them to a single plot. Each of these "variables" represent scientific data and are stored as arrays with 100k to 3 million elements.

I don't feel very inclined to store these variables in the React state as their contents never change and I'd prefer to control when the UI needs to re-render manually instead of letting React automagically traverse these arrays trying to find out if they have changed, which renders my application unresponsive.

Is there any preferred way of storing this data and sharing it with React? Right now I just store everything in a global variable.

@stever Each array is a timeseries of observations from our sensors, obtained through our API. Each of these timeseries are numeric float-point arrays of 100k to 3M elements. The time series are fed to a plot.ly line plot to create an interactive, zoomable plot. Usually, our users will overlap 3 to 4 of these timeseries. — DanyAlejandro, Jan 25 '19 at 21:06

score 0 · Answer 1 · answered Jan 25 '19 at 20:58

0

There aren't many places you can store it though. If the data never changes, there's no downside in storing it in state, no changes means no re-renders. It sounds like there's quite a bit of it, so you can't store it in local storage due to limits. If you build a data set inside React by doing a bunch of API calls, revisit the API itself - it should return the data you care for, rarely a bunch of data segments you have to splice together yourself.

answered Jan 25 '19 at 20:58

Predrag Beocanin

1,402
3
18
25

Our users are scientists, our API serves timeseries of data from our sensors in the ocean. Our users can request specific devices and time intervals, but there's no way for them to know how much data they are asking for beforehand (the data is not predictable). Our API has a limit on the number of rows returned per request and has a "pagination system" so you can ask for the next "page" when you hit the max, which is a common occurrence. – DanyAlejandro Jan 25 '19 at 21:07
In which case, you have two entries in the state: `state = { page: 0, items: [] }`, and you update both accordingly. Pagination system like that does require a re-render, and it would be bad user experience to just preload all the data in React, and filter it there, as well as impossible, since you did say there's no way of knowing how much there is? – Predrag Beocanin Jan 25 '19 at 21:10

In React, what is a better place to store big data arrays?

1 Answers1