Create array of unique objects by property

Question

I created an array of objects like so:

[
    {
        "lat": 12.123,
        "lng": 13.213,
        "city": "New York"
    },
    {
        "lat": 3.123,
        "lng": 2.213,
        "city": "New York"
    },
    {
        "lat": 1.513,
        "lng": 1.113,
        "city": "London"
    }
]

I'm trying to create a new array that filters the places to only contains objects that don't have the same city property (lat/lng duplicates are ok). Is there a built in JS or Jquery function to achieve this?

Answer 1

I'd probably use a flags object during the filtering (edit: I wouldn't anymore, see the note at the end of the answer about ES2015's Set), like this:

var flags = {};
var newPlaces = places.filter(function(entry) {
    if (flags[entry.city]) {
        return false;
    }
    flags[entry.city] = true;
    return true;
});

That uses Array#filter from ECMAScript5 (ES5), which is one of the ES5 additions that can be shimmed (search for "es5 shim" for several options).

You can do it without filter, of course, it's just a bit more verbose:

var flags = {};
var newPlaces = [];
var index;
for (index = 0; index < places.length; ++index) {
    if (!flags[entry.city]) {
        flags[entry.city] = true;
        newPlaces.push(entry);
    }
});

Both of the above assume the first object with a given city should be kept, and all other discarded.

---

Note: As user2736012 points out below, my test if (flags[entry.city]) will be true for cities with names that happen to be the same as properties that exist on Object.prototype such as toString. Very unlikely in this case, but there are four ways to avoid the possibility:

• (My usual preferred solution) Create the object without a prototype: var flags = Object.create(null);. This is a feature of ES5. Note that this cannot be shimmed for obsolete browsers like IE8 (the single-argument version of Object.create can be except when that argument's value is null).

• Use hasOwnProperty for the test, e.g. if (flags.hasOwnProperty(entry.city))

• Put a prefix on that you know doesn't exist for any Object.prototype property, such as xx:

    var key = "xx" + entry.city;
    if (flags[key]) {
        // ...
    }
    flags[key] = true;
  

• As of ES2015, you could use a Set instead:

    const flags = new Set();
    const newPlaces = places.filter(entry => {
        if (flags.has(entry.city)) {
            return false;
        }
        flags.add(entry.city);
        return true;
    });
  

Answer 2

Shortest, but not best performance (see update bellow) solution for es6 :

function unique(array, propertyName) {
   return array.filter((e, i) => array.findIndex(a => a[propertyName] === e[propertyName]) === i);
}

performance: https://jsperf.com/compare-unique-array-by-property

Answer 3

You can filter using a Set by only including elements with a property value that has not yet been added to the Set (after which it should be added to the Set). This can be accomplished in one line using the logical and operator (&&). Using this data structure has the advantage of sublinear lookup times (often O(1)).

Below is a general function to obtain a unique array of objects based on a specific property (prop) from an array of objects (arr). Note that in the case of duplicates, only the first object with the property value will be retained.

const getUniqueBy = (arr, prop) => {
  const set = new Set;
  return arr.filter(o => !set.has(o[prop]) && set.add(o[prop]));
};

Demo:

var places = [{
  lat: 12.123,
  lng: 13.213,
  city: 'New York'
}, {
  lat: 3.123,
  lng: 2.213,
  city: 'New York'
}, {
  lat: 3.123,
  lng: 4.123,
  city: 'Some City'
}];
const getUniqueBy = (arr, prop) => {
  const set = new Set;
  return arr.filter(o => !set.has(o[prop]) && set.add(o[prop]));
};
console.log(getUniqueBy(places, 'city'));

Answer 4

https://lodash.com/docs#uniqBy

https://github.com/lodash/lodash/blob/4.13.1/lodash.js#L7711

/**
 * This method is like `_.uniq` except that it accepts `iteratee` which is
 * invoked for each element in `array` to generate the criterion by which
 * uniqueness is computed. The iteratee is invoked with one argument: (value).
 *
 * @static
 * @memberOf _
 * @since 4.0.0
 * @category Array
 * @param {Array} array The array to inspect.
 * @param {Array|Function|Object|string} [iteratee=_.identity]
 *  The iteratee invoked per element.
 * @returns {Array} Returns the new duplicate free array.
 * @example
 *
 * _.uniqBy([2.1, 1.2, 2.3], Math.floor);
 * // => [2.1, 1.2]
 *
 * // The `_.property` iteratee shorthand.
 * _.uniqBy([{ 'x': 1 }, { 'x': 2 }, { 'x': 1 }], 'x');
 * // => [{ 'x': 1 }, { 'x': 2 }]
 */

Answer 5

I expanded a bit on @IgorL solution, but extended prototype and gave it a selector function instead of a property to make it a little more flexible:

Array.prototype.unique = function(selector) {
   return this.filter((e, i) => this.findIndex((a) => {
      if (selector) {
        return selector(a) === selector(e);
      }
      return a === e;
    }) === i);
};

Usage:

// with no param it uses strict equals (===) against the object
let primArr = ['one','one','two','three','one']
primArr.unique() // ['one','two','three']

let a = {foo:123}
let b = {foo:123}
let fooArr = [a,a,b]
fooArr.unique() //[a,b]

// alternatively, you can pass a selector function
fooArr.unique(item=>item.foo) //[{foo:123}] (first "unique" item returned)

Definitely NOT the most performant way to do this but as long as the selector is simple and the array isn't massive, it should work fine.

In Typescript

Array.prototype.unique = function<T>(this: T[], selector?: (item: T) => object): T[] {
   return this.filter((e, i) => this.findIndex((a) => {
      if (selector) {
        return selector(a) === selector(e);
      }
      return a === e;
    }) === i);
};

Answer 6

My suggestion :

Array.prototype.uniqueCity = function() {
    var processed = [];
    for (var i=this.length-1; i>=0; i--){
        if (processed.indexOf(this[i].city)<0) {
            processed.push(this[i].city);
        } else {
            this.splice(i, 1);
        }
    }
}

in use :

places.uniqueCity();

or

Array.prototype.uniqueObjectArray = function(field) {
    var processed = [];
    for (var i=this.length-1; i>=0; i--) {
        if (this[i].hasOwnProperty(field)) {
            if (processed.indexOf(this[i][field])<0) {
                processed.push(this[i][field]);
            } else {
                this.splice(i, 1);
            }
        }
    }
}

places.uniqueObjectArray('city');

With the above you can sort the array by any of the fields in the objects, even if they are not present for some of the objects.

or

function uniqueCity(array) {
    var processed = [];
    for (var i=array.length-1; i>=0; i--){
        if (processed.indexOf(array[i].city)<0) {
            processed.push(array[i].city);
        } else {
            array.splice(i, 1);
        }
    }
    return array;
}

places = uniqueCity(places);

Answer 7

You could use a Map so the entries with the same key property (in your case 'city') only appear once

module.exports = (array, prop) => {
   const keyValueArray = array.map(entry => [entry[prop], entry]);
   const map = new Map(keyValueArray);
   return Array.from(map.values());
};

More info about Map and array objects here

Basic example on Codepen

Answer 8

Another option:

const uniqueBy = prop => list => {
    const uniques = {}
    return list.reduce(
        (result, item) => {
            if (uniques[item[prop]]) return result
            uniques[item[prop]] = item
            return [...result, item]
        },
        [],
    )
}

const uniqueById = uniqueBy('id')

uniqueById([
    { id: 1, name: 'one' },
    { id: 2, name: 'two' },
    { id: 1, name: 'one' },
    { id: 3, name: 'three' }
])

You can paste it on your console to see it working. It should work for the scenario presented and a few others.

Answer 9

We can create the list of unique objects by any property using JavaScript Map.

For example :

var places = [{ 'lat': 12.123, 'lng': 13.213, 'city': "New York"},
                { 'lat': 3.123, 'lng': 2.213, 'city': "New York"},
                { 'lat': 43.123, 'lng': 12.213, 'city': "London"}];
                
var cityMap = new Map();
places.forEach(p=> cityMap.set(p.city, p));

console.log([...cityMap.values()]);

Execute code snippet to see the result.

Answer 10

const distinctArrayByCity= [
    ...new Map(array.map((item) => [item.city, item])).values(),
];

Answer 11

As pointed out in the comments, you could use an object as a map, which will allow you to avoid duplicates, you can then enumerate the properties of the object.

working fiddle: http://jsfiddle.net/gPRPQ/1/

var places = [];
var a = {};
a.lat = 12.123;
a.lng = 13.213;
a.city = "New York";

places.push(a);

var b = {};
b.lat = 3.123;
b.lng = 2.213;
b.city = "New York";

places.push(b);

var unique = {}

for (var i = 0; i < places.length; i++) {
    var place = places[i];
    unique[place.city] = place;
}

for (var name in unique) {
    var place = unique[name];
    console.log(place);
}

Answer 12

var places = [];
var a = {};
a.lat = 12.123;
a.lng = 13.213;
a.city = "New York";

places.push(a);

var b = {};
b.lat = 3.123;
b.lng = 2.213;
b.city = "New York";

places.push(b);

getUniqAR(places,'city'); //Return Uniq Array by property

function getUniqAR(Data,filter){
var uniar =[];
Data.forEach(function(item,ind,arr){
    var dupi=false;
    if(!uniar.length) uniar.push(item) //push first obj into uniq array 
    uniar.forEach(function(item2, ind2,arr){
    if(item2[filter] == item[filter]){  //check each obj prop of uniq array 
      dupi=true; //if values are same put duplicate is true
        }     
    })
if(!dupi){  uniar.push(item)} //if no duplicate insert to uniq

})
console.log(uniar)
return uniar;
}

Answer 13

In simple Javascript code to remove duplicate cities from places array list is

var places = [{ 'lat': 12.123, 'lng': 13.213, 'city': "New York"},
                { 'lat': 3.123, 'lng': 2.213, 'city': "New York"},
                { 'lat': 43.123, 'lng': 12.213, 'city': "London"}];
var unique = [];
var tempArr = [];
places.forEach((value, index) => {
    if (unique.indexOf(value.city) === -1) {
        unique.push(value.city);
    } else {
        tempArr.push(index);    
    }
});
tempArr.reverse();
tempArr.forEach(ele => {
    places.splice(ele, 1);
});
console.log(places);

Answer 14

Generic Typescript answer based on https://stackoverflow.com/a/18773857/49564 above:

export function isDistinct<T>(mapper: (value: T) => string): (value: T) => boolean {
  const keys: { [index: string]: boolean } = {};

  return (entry: T) => {
    const key = mapper(entry);

    if (keys[key] !== undefined) {
      return false;
    }

    return keys[key] = true;
  };
}

// Usage example:
const items = [ { id: 1 }, { id: 2 }, { id: 3 }, { id: 1 } ];
const unique = items.filter(isDistinct(i => i.id));

Answer 15

I think you want this,

NOTE: No library is required.

let array = [{ id: 1}, {id: 2}, {id: 3}];

function addUniqeObj(data) {
  let index = -1;

  for(let i = 0, i < array.length; i++) {
    if(array[i].id === data.id) {
      index = i;
    }
  }

  if(index > -1) {
    array[index] = data;
  } else {
    array.push(data)
  }

}

Answer 16

Another variation of the rafaelbiten approach:

const dedupExample = [
    {id: 1, c: 'whatever'},
    {id: 1, c: '1whatever'},
    {id: 2, c: '2whatever'},
    {id: 2, c: '2whatever'},
    {id: 3, c: '2whatever'},
]

const getUniqueBy = (prop, list) => {
    const objUniq = list.reduce((res, item) => ({ ...res, [item[prop]]: item }), {})
    return Object.keys(objUniq).map(item => objUniq[item])
}

const uniq = getUniqueBy('id', dedupExample)

console.info('info', { uniq })
   /* [
    {id: 1, c: 'whatever'},
    {id: 2, c: '2whatever'},
    {id: 3, c: '2whatever'},
  ] */

Answer 17

This thread may be old but thought I should share it. It is based on Pure JavaScript and removes Duplicate Objects based on the Properties Specified.

function removeDuplicates(originalArray, properties) {
  var newArray = [];
  var index = 0;
  var lookupObject = {};
  var totalProperties = properties.length;

  for (var i = 0; i < originalArray.length; i++) {
    var exists = false;

    for (var a = 0; a < newArray.length; a++) {
      var propsFound = 0;
      for (var b = 0; b < totalProperties; b++) {
        if (originalArray[i][properties[b]] == newArray[a][properties[b]]) {
          propsFound++;
        }
      }

      //If there is a match then break the for loop
      if (propsFound == totalProperties) {
        exists = true;
        break;
      }
    } //End of New Array

    if (!exists) {
      newArray[index] = originalArray[i];
      index++;
    }
  } //End of originalArray

  return newArray;
}

You can view the fiddle here