Counting multiple json inputs js

Question

I get an input like this:

input 1:

{

"name": "Ben",
"description": "Ben",
"attributes": [
    {
    "type": "Background",
    "value": "Default"
    },
    {
    "type": "Hair-color",
    "value": "Brown"
    }
]
}

input 2

{

"name": "Ice",
"description": "Ice",
"attributes": [
    {
    "type": "Background",
    "value": "Green"
    },
    {
    "type": "Hair-color",
    "value": "White"
    }
]
}

input 3

{

"name": "Itay",
"description": "Itay",
"attributes": [
    {
    "type": "Background",
    "value": "Default"
    },
    {
    "type": "Hair-color",
    "value": "Brown"
    }
]
}

What I want to do is count the amount of each type of background and each type of hair-color appearing. (These are sample examples and in reality there are more types and different values)

Let's say in these examples we have 2 objects that have a background as default then I want to have a count of that like so:

export interface TraitCount {
    value: string,
    count: number
}

export interface CountOfEachAttribute {
    trait_type: string,
    trait_count: traitCount[] | null,
    total_variations: number
}

I want the most effective code because there are other aspects to the code, in addition it will run on 5-10k queries not just three, so needs to run in good times too :D (It's similar to my other question done with python but now I need it in js also)

Atm it's something like this:

(Apart of a much bigger code so keep that in mind)

    setInitalCountOfAllAttribute( state, { payload }: PayloadAction<CountOfEachAttribute[] | null> ) {
      if (payload === null) {
        state.countOfAllAttribute = null;
      } else {
        state.countOfAllAttribute = payload;
      }
    },

    setCountOfAllAttribute(state, { payload }: PayloadAction<Attribute>) {

      if (state.countOfAllAttribute !== null) {
        state.countOfAllAttribute.map(
          (countOfEachAttribute: CountOfEachAttribute) => {

            // Find the trait type
            if (countOfEachAttribute.trait_type === payload.trait_type) {

              // initiate the trait count array to store all the trait values and add first trait value
              if (countOfEachAttribute.trait_count === null) {
                const new_trait_count = { value: payload.value, count: 1 };
                countOfEachAttribute.trait_count = [new_trait_count];
                countOfEachAttribute.total_variations++;
              } 

              // Trait array already existed. 
              else {

                // Check if value already present or not
                const checkValue = (obj: any) => obj.value === String(payload.value);
                const isPresent = countOfEachAttribute.trait_count.some(checkValue)
                const isPresent2 = countOfEachAttribute.trait_count.find((elem: any) => elem.value === String(payload.value))

                // Value matched, increase its count by one
                  if (isPresent2) {
                  countOfEachAttribute.trait_count &&
                    countOfEachAttribute.trait_count.map((trait) => {
                      if (trait.value === payload.value) {
                        trait.count++;
                      }
                    });
                } 

                // Value doesn't match, add a new entry and increase the count of variations by one
                else {
                  const new_trait_count = { value: payload.value, count: 1 };
                  countOfEachAttribute.trait_count = [
                    ...countOfEachAttribute.trait_count,
                    new_trait_count,
                  ];
                  countOfEachAttribute.total_variations++;
                }
              }
            }
          }
        );
      }
    },

@KrzysztofKrzeszewski Updated question with what has been tried, it is too slow tho — ben shalev, Oct 08 '21 at 08:07

score 1 · Answer 1 · answered Oct 08 '21 at 08:27

You can merge all arrays and use Array.reduce.

const input1 = {
  "name": "Ben",
  "description": "Ben",
  "attributes": [{
      "type": "Background",
      "value": "Default"
    },
    {
      "type": "Hair-color",
      "value": "Brown"
    }
  ]
}
const input2 = {
  "name": "Ice",
  "description": "Ice",
  "attributes": [{
      "type": "Background",
      "value": "Green"
    },
    {
      "type": "Hair-color",
      "value": "White"
    }
  ]
}
const input3 = {
  "name": "Itay",
  "description": "Itay",
  "attributes": [{
      "type": "Background",
      "value": "Default"
    },
    {
      "type": "Hair-color",
      "value": "Brown"
    }
  ]
}

const mergedInput = [input1, input2, input3];

const result = mergedInput.reduce((acc, item) => {
  
  item.attributes.forEach(attrItem => {
    const existType = acc.find(e => e.trait_type == attrItem.type);
    if (existType) {
        var existAttr = existType.trait_count.find(e => e.value == attrItem.value);
      if (existAttr) {
        existAttr.count++;
      } else {
        existType.trait_count.push({
            value: attrItem.value,
          count: 1
        });
        existType.total_variations++;
      }
    } else {
        acc.push({
        trait_type: attrItem.type,
        trait_count: [{
            value: attrItem.value,
          count: 1
        }],
        total_variations: 1
      })
    }
  });
  return acc;
}, []);

console.log(result);

Yes, we query an API and we get these attributes one by one. — ben shalev, Oct 08 '21 at 08:54
@benshalev I think you shouldn't process it one by one and set it to state. It makes your application reload one by one also. Can you move all of the queries to API into a function and merged all of them into an array? — Tuan Dao, Oct 08 '21 at 08:59
We will try merging, i will test it today and see. Merging won't cause lag (because it's 10k apis, with a bit more data than this)? — ben shalev, Oct 08 '21 at 09:30

Krzysztof Krzeszewski · Answer 2 · 2021-10-08T09:09:18.347

I suggest instead of creating an array for trait_count to make it an object so you don't have to iterate over it whenever you are adding a new attribute. In the snippet below I'm using the value of the attribute as a sort of hash that allows the access to the given property without having to call the Array.prototype.find function

const input1 = {"name":"Ben","description":"Ben","attributes":[{"type":"Background","value":"Default"},{"type":"Hair-color","value":"Brown"}]};
const input2 = {"name":"Ice","description":"Ice","attributes":[{"type":"Background","value":"Green"},{"type":"Hair-color","value":"White"}]};
const input3 = {"name":"Itay","description":"Itay","attributes":[{"type":"Background","value":"Default"},{"type":"Hair-color","value":"Brown"}]};

function countAtributes(input, totalCounts={}) {
  input.attributes.forEach((attribute) => {
    if (!totalCounts[attribute.type])
      totalCounts[attribute.type] = {trait_type: attribute.type, trait_count: {}, total_variations: 0};

    if (!totalCounts[attribute.type].trait_count[attribute.value]) {
      totalCounts[attribute.type].trait_count[attribute.value] = {value: attribute.value, count: 1};
      totalCounts[attribute.type].total_variations+=1;
    }
    else totalCounts[attribute.type].trait_count[attribute.value].count +=1;
  })
}

const totalCounts = {};
countAtributes(input1, totalCounts);
countAtributes(input2, totalCounts);
countAtributes(input3, totalCounts);

console.log(totalCounts);

It could be turned into the array afterwards with Object.values if necessary

I believe it is a much better approach to what you had before as you don't have to iterate over the tables of trait_counts. In theory it should significantly reduce the time taken. Iterating over the array and checking a condition each time is much slower than key lookup in Javascript object

How will it work if we query an API one by one to get the data (5-10k queries) — ben shalev, Oct 08 '21 at 08:55
Okay, i will test these answers and update you later this evening :) — ben shalev, Oct 08 '21 at 09:29

Counting multiple json inputs js

2 Answers2