1

I have a large JSON file (2.7gb) that I would like filter for only the data I'm interested in to make the file smaller. The data consists of an array of objects. The following query works on a small subset of the JSON file, but when I try to run it on the 2.7gb file it doesn't work. How can I convert this into a stream query so that it can process the entire file?

.[] | {
    food: .foodClass,
    description: .description,
    foodNutrients: [.foodNutrients[] | { nutrients: .nutrient, amount: .amount}],
    upc: .gtinUpc,
    servingSize: .servingSize,
    servingSizeUnit: .servingSizeUnit,
    ingredients: .ingredients,
    fdcId: .fdcId,
    dataType: .dataType,
    brandOwner: .brandOwner,
    marketCountry: .marketCountry,
    brandedFootCategory: .brandedFoodCategory
}
Christophe Willemsen
  • 19,399
  • 2
  • 29
  • 36
  • For speed, use jm (https://github.com/pkoppstein/jm) or jstream (https://github.com/bcicen/jstream) – peak Nov 07 '22 at 07:11

0 Answers0