Removing duplicate objects

Question

var foo = [ { "a" : 15 }, { "b" : 25 }, { "a" : 15 }, {"c" : 13},{"c" : 13},{"c" : 13},{ "b" : 25 }  ];

I would like to remove the duplicates and then sum up what's left. So the above would yield,

{ "a" : 15 },{ "b" : 25 }, {"c" : 13},

giving me 53 when I add what's left of the values.

I saw something similar here (Removing duplicate objects with Underscore for Javascript) but not really clear on how duplicates are removed.

I found the following solution but it did not work for me. http://bateru.com/news/2012/03/code-of-the-day-get-unique-objects-from-an-array-of-json-objects/

Any ideas? Thanks!

what if there is {"a" : 15} and {"a" : 16} will that be counted as duplicate? — trrrrrrm, Sep 03 '13 at 10:38
Seems pretty straightforward, go mess your hands with some loops! — moonwave99, Sep 03 '13 at 10:39
You might want to read this http://stackoverflow.com/questions/2218999/remove-duplicates-from-an-array-of-objects-in-javascript?rq=1 — Udo Klein, Sep 03 '13 at 10:41
No, {"a" : 15} and {"a" : 16} won't be counted as a duplicate — JustMe, Sep 03 '13 at 10:47

score 4 · Accepted Answer · edited May 23 '17 at 12:11

To address your update, here's code that keeps the first of duplicated elements:

var foo = [ { "a" : 15 }, { "b" : 25 }, { "a" : 15 }, {"c" : 13} ];

uniqs = uniqBy(foo, JSON.stringify)
values = uniqs.map(function(x) { for(var k in x) return x[k] })
sum = values.reduce(function(a, b) { return a + b })

uniqBy is defined here: https://stackoverflow.com/a/9229821/989121

Let's try this:

> var foo = [ { "a" : 15 }, { "b" : 25 }, { "a" : 15 }, {"c" : 13} ];
undefined
> fkeys = foo.map(function(x) { return Object.keys(x)[0] })
["a", "b", "a", "c"]
> sum = foo.reduce(function(s, x) {
    var key = Object.keys(x)[0];
    if(fkeys.indexOf(key) == fkeys.lastIndexOf(key)) 
        s += x[key];
    return s;
}, 0)
38

The idea is to collect the keys first and then iterate over the array, ignoring keys that occur more than once (indexOf != lastIndexOf).

If your arrays are huge (>5000 items), using a hash table instead of indexOf might be more efficient:

> fkeys = {}
Object {}
> foo.forEach(function(x) { 
    var key = Object.keys(x)[0];
    fkeys[key] = (Number(fkeys[key]) || 0) + 1;
})
undefined
> sum = foo.reduce(function(s, x) {
        var key = Object.keys(x)[0];
        if(fkeys[key] === 1) 
            s += x[key];
        return s;
    }, 0)
38

I thought this was working until I added a few more values to the object; http://jsfiddle.net/GTpgA/40/ — JustMe, Sep 03 '13 at 14:34
@Isaac Can you update the question with your new input data and the expected result? — phant0m, Sep 03 '13 at 14:51
@thg435 What am I doing wrong? The alert is not firing.. http://jsfiddle.net/9vyuY/1/ — JustMe, Sep 03 '13 at 23:04

score 0 · Answer 2 · edited May 23 '17 at 10:31

0

In the example you've posted underscore.js library is used. look at the docs here link.

uniq _.uniq(array, [isSorted], [iterator])

Alias: unique Produces a duplicate-free version of the array, using === to test object equality. If you know in advance that the array is sorted, passing true for isSorted will run a much faster algorithm. If you want to compute unique items based on a transformation, pass an iterator function.

using it may save your time.

There you'll find example relying on clean javascript: Remove Duplicates from JavaScript Array

look at thg435 answer.

edited May 23 '17 at 10:31

Community

1
1

answered Sep 03 '13 at 10:41

lexeme

2,915
10
60
125

1

The _.unq() works great but not on objects. That was one of the first solutions I tried. – JustMe Sep 03 '13 at 10:51
@Isaac No, it's not at all what you want from your description, because you want *all* occurences of the duplicate removed. – phant0m Sep 03 '13 at 10:52
@phant0m You are right, that's not what I wanted. I realized that when I added more objects. http://jsfiddle.net/GTpgA/40/ – JustMe Sep 03 '13 at 14:37

score 0 · Answer 3 · answered Sep 03 '13 at 15:14

Why dont u try this for simple calculation part..This is only u want calculation by not considering duplicate object value..Its not suitable for pure object related manipulations

var array = [{ "a" : 15 }, { "b" : 25 }, { "a" : 15 }, {"c" : 13}];
var finVal=0;
var items = {};
var keys={};
for(var i=0;i<array.length;i++){
    for(var key1 in array[i]){ 

if(items[key1]==undefined ){

    items[key1] =array[i][key1];
    var locVal = items[key1];
    finVal = finVal+locVal;
    } else {
    //items[key1] =array[i][key1];
    var locVal = items[key1];
    finVal = finVal-items[key1];

    }
}

}

console.log(finVal);

Removing duplicate objects

3 Answers3

Linked