Importing and searching in large JSON file

Question

Context : trying to build a meteo web app using OpenWeather API and Vue.js for front-end.

The API website provides a JSON file with the list of city IDs. So, for the search functionnality, I need to import the JSON file and compare name attributes with user input. The problem is that the JSON file is large (2M+ lines unminified), so the browser freezes every time after a bit searching. How can I handle this ?

Here is the base of the search component

<template>
  <div>
    <input v-model="query" class="input" type="text" placeholder="City name" />
    <ul v-for="city in searchResults" :key="city.id">
      <li>{{ city.name }}, {{ city.country }}</li>
    </ul>
  </div>
</template>

<script>
import cities from "@/assets/data/city.list.json";

export default {
  name: "SearchBar",
  data() {
    return {
      query: null
    };
  },
  computed: {
    searchResults: function() {
      if (this.query !== null && this.query.length > 2)
        return cities.filter(city => city.name.startsWith(this.query));
      else return null;
    }
  }
};
</script>

I think you need to break your json file in many files and search the data asyncronously in all the files. Then, for each file add the result to the result list. — AlleXyS, Jul 21 '20 at 21:18
Does this answer your question? [How to avoid freezing the browser when doing long-running computations in Javascript](https://stackoverflow.com/questions/13546493/how-to-avoid-freezing-the-browser-when-doing-long-running-computations-in-javasc) — blex, Jul 21 '20 at 21:18

score 0 · Answer 1 · answered Jul 21 '20 at 21:24

0

You could try to use one of those libs :

But if your file is really too big, it could be wise to consider to do the computation on the back-end or use a webWorker to not freeze the ui thread.

answered Jul 21 '20 at 21:24

8HoLoN

1,122
5
14

But how can I stream a webpack import ? – Nas Jul 21 '20 at 22:03

score 0 · Answer 2 · answered Jul 22 '20 at 13:50

0

Have you considered pre-processing the file into multiple smaller files (maybe 1 per city)?

You will have various benefit:

each file will be small(er) and load quickly
you can then add logic to load only the file you need.

answered Jul 22 '20 at 13:50

covo

550
4
12

I searched for this, for Node there is modules for this (like JSONStream), but for front-end I have no idea of how to do this – Nas Jul 22 '20 at 14:12

Importing and searching in large JSON file

2 Answers2