Lodash remove duplicates from array

Question

This is my data:

[
    {
        url: 'www.example.com/hello',
        id: "22"    
    },
    {
        url: 'www.example.com/hello',
        id: "22"    
    },
    {
        url: 'www.example.com/hello-how-are-you',
        id: "23"    
    },
    {
        url: 'www.example.com/i-like-cats',
        id: "24"    
    },
    {
        url: 'www.example.com/i-like-pie',
        id: "25"    
    }
]

With Lodash, how could I remove objects with duplicate id keys? Something with filter, map and unique, but not quite sure.

My real data set is much larger and has more keys, but the concept should be the same.

score 653 · Accepted Answer · edited Sep 06 '20 at 21:39

653

_.unique no longer works for the current version of Lodash as version 4.0.0 has this breaking change. The functionality of _.unique was splitted into _.uniq, _.sortedUniq, _.sortedUniqBy, and _.uniqBy.

You could use _.uniqBy like this:

_.uniqBy(data, function (e) {
  return e.id;
});

...or like this:

_.uniqBy(data, 'id');

Documentation: https://lodash.com/docs#uniqBy

For older versions of Lodash (< 4.0.0 ):

Assuming that the data should be uniqued by each object's id property and your data is stored in data variable, you can use the _.unique() function like this:

_.unique(data, function (e) {
  return e.id;
});

Or simply like this:

_.uniq(data, 'id');

edited Sep 06 '20 at 21:39

marsnebulasoup

2,530
2
16
37

answered Jul 31 '15 at 07:17

ntalbs

28,700
8
66
83

1

The lodash 4.0.1 docs have changed this to uniqBy. Couldn't get the above example to work for the life of me using only uniq. – Will Lovett Jan 28 '16 at 20:10
4

With lodash 4.x, uniq() accept only one parameter, which is an array. So as @WillLovett said, we can use uniqBy() to solve this problem. – ersefuril Oct 13 '16 at 08:51
1

Oh boy. Why, oh why, is it `uniq` which is pronounced _you-knee-queue_ instead of what it should be, `unique`... – Josh M. Nov 29 '18 at 15:12

score 90 · Answer 2 · answered Nov 18 '16 at 11:33

90

You could use lodash method _.uniqWith, it is available in the current version of lodash 4.17.2.

Example:

var objects = [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }, { 'x': 1, 'y': 2 }];

_.uniqWith(objects, _.isEqual);
// => [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }]

More info: https://lodash.com/docs/#uniqWith

answered Nov 18 '16 at 11:33

zdrsoft

2,417
19
10

6

` let uniqObjects = _.uniqBy(docs, function (e) { return e.x + e.y; }); ` This method will be faster for big arrays with simple objects – tagplus5 Mar 01 '17 at 14:01
6

yes but you've to pass an attribute as first argument with uniqBy, which's not convenient if you want it unique based on several attributes that you don't already know – dbrrt Mar 23 '18 at 01:59

score 29 · Answer 3 · answered Nov 06 '19 at 21:06

29

With lodash version 4+, you would remove duplicate objects either by specific property or by the entire object like so:

var users = [
  {id:1,name:'ted'},
  {id:1,name:'ted'},
  {id:1,name:'bob'},
  {id:3,name:'sara'}
];
var uniqueUsersByID = _.uniqBy(users,'id'); //removed if had duplicate id
var uniqueUsers = _.uniqWith(users, _.isEqual);//removed complete duplicates

Source:https://www.codegrepper.com/?search_term=Lodash+remove+duplicates+from+array

answered Nov 06 '19 at 21:06

Taylor Hawkes

641
7
10

1

Was looking for that `var uniqueUsers = _.uniqWith(users, _.isEqual);//removed complete duplicates` Your anwsers deserves more love – Sydney C. Mar 01 '21 at 15:44
that's exactly what i was looking for `_.uniqWith(users, _.isEqual)` for unordered object – Teebu Mar 09 '22 at 16:23

Vixson · Answer 4 · 2020-04-22T01:26:28.347

22

Or simply Use union, for simple array.

_.union([1,2,3,3], [3,5])

// [1,2,3,5]

edited Apr 22 '20 at 01:26

answered Mar 23 '19 at 19:55

Vixson

589
7
8

2

I like this answer. It's not the most readable one, but it actually works if you think about it from a mathematics perspective. `_.union([1, 2, 2, 1, 3]) => [1, 2, 3]` – Sepehr May 17 '19 at 20:24
2

clever and concise !! – Jean-Philippe Martin Nov 18 '22 at 22:52

score 19 · Answer 5 · answered Jun 09 '20 at 12:09

19

For a simple array, you have the union approach, but you can also use :

_.uniq([2, 1, 2]);

answered Jun 09 '20 at 12:09

ZettaP

719
7
11

score 10 · Answer 6 · answered May 26 '19 at 21:21

10

Simply use _.uniqBy(). It creates duplicate-free version of an array.

This is a new way and available from 4.0.0 version.

_.uniqBy(data, 'id');

or

_.uniqBy(data, obj => obj.id);

answered May 26 '19 at 21:21

Hassan Ajaz

613
8
15

score 5 · Answer 7 · answered Nov 04 '21 at 18:15

In LODASH versions lower than 4 you will find most of this function are not implemented same way. And opposite from version 4 _.uniq was modified. I personally had a project that was in transition for a few months (from V3 -> to V4).

If you run in same situation and you have a lot of functions to be updated. You can do it in stages and when you are done with transition you can come and fix it later. This is the trick i used to avoid downtime of the platform:

/* LODASH Version 3 & 4 Compatibility Mode */
if ((_.VERSION).charAt(0) <= 3){ //Detect LODASH version 3 or 4.
    //V3 or lower
    _.uniq(data, true, 'id');
} else {
    //V4 or Higher
    _.uniqBy(data, 'id');
}

Also if you look at lodash documentation for most of this cases you can find migration of _.uniq from version lower than 4 can be performed with both functions:

_.uniqBy(data, 'id') or _.unionBy(data, 'id')

Both of them will bring same result. I personally was guessing which one to pick. In the end I picked this one: _.uniqBy(data, 'id').

score 2 · Answer 8 · answered Jun 05 '18 at 11:27

2

You can also use unionBy for 4.0.0 and later, as follows: let uniques = _.unionBy(data, 'id')

answered Jun 05 '18 at 11:27

Himanshu Tanwar

378
2
18

Lodash remove duplicates from array

8 Answers8

Linked

Related