0

I have a 2GB .csv file containing json in this format: {key1: value1, key2: value2, key3: value3}

I need to extract the value1 from this file.

I want to understand if there's a best to do that in MacOS. Maybe using docker to build a SQLlite or PostGres and then use somekind of json_extract_path function. I've tried using pandas but it was't good and the notebook died.

Anybody has a best practice/way to use in this case by using SQL or Python?

David Maze
  • 130,717
  • 29
  • 175
  • 215
Guilherme Barbosa
  • 13
  • 2
  • Is it really a CSV file, where the JSON is a quoted string in one of the columns, or is it just a list of JSON records? This is easy to do in plain Python; you don't need to drag in pandas or Docker or SQLite. Show us a couple of sample records, and we'll advise you. – Tim Roberts Feb 27 '21 at 02:01
  • Does this answer your question? [How do I read a large csv file with pandas?](https://stackoverflow.com/questions/25962114/how-do-i-read-a-large-csv-file-with-pandas) – bb1 Feb 27 '21 at 02:06
  • Do you know what line you are trying to read in the file? Do you know what character you want to read on that line? In general, it is very easy to find particular parts of a file and read them without reading the whole file into memory. – Bobby Ocean Feb 27 '21 at 02:32

0 Answers0