How to remove all items in smallArray from bigArray with lodash?

Question

http://codepen.io/leongaban/pen/GZgrMK?editors=0010

I'm trying to find the most efficient way to accomplish this, in my actual app, I may have over 1000 items in the bigArray and a few hundred items or even all 1000 items again in the small array.

I need to find all those items and remove them. Lodash is faster than native javascript APIs so that's what I'm trying here:

var myApp = angular.module('myApp',[]);

myApp.controller('MyController', ['$scope', function($scope) {
  $scope.greeting = 'The array below should be smaller than bigArray';
  
  $scope.bigArray = [
    { name: 'Leon'},
    { name: 'Eric'},
    { name: 'Wes'},
    { name: 'Gannon'},
    { name: 'Jordan'},
    { name: 'Chris'},
    { name: 'Paulo'},
    { name: 'Gumbo'},
    { name: 'Gilgamesh'}
  ];

  $scope.smallArray = [
    { name: 'Gannon'},
    { name: 'Gumbo'},
    { name: 'Gilgamesh'}
  ]
  
  $scope.newArray = [];
  
  $scope.removeItems = function() {
    
      _.remove($scope.bigArray, function($scope.smallArray, n) {
          return $scope.smallArray[n].name;
      });

     // $scope.newArray = _.remove($scope.bigArray, function(n) {
     //   return n.name = _.find($scope.smallArray, 'name');
     // });

     // _.each($scope.bigArray, function(value, i) {
     //   _.find($scope.smallArray, 'name');
     // });
  }

}]);

What I've tried so far without luck.

Why am I asking about a lodash solution? Because the lodash API is faster in a lot of ways than the native Javascript API.

Proof lodash beats native Javascript, these results are from the jspref link Mr.zerkms posted: http://jsperf.com/zerkms-forloop-vs-each-from-lodash

"Lodash is faster than native javascript APIs" --- that's because lodash is written in magical language by unicorns? — zerkms, Feb 29 '16 at 21:43
Lodash isn't faster than native APIs. It's just a nice layer to polyfill when necessary and give you additional functions. [It frequently uses native APIs under the hood.](https://github.com/lodash/lodash/blob/master/lodash.js#L14985) — Mike Cluck, Feb 29 '16 at 21:44
@zerkms You could have said that a little less pretentiously. Leon, zerkms means that Lodash can't be faster than JavaScript since it's built with JavaScript, but it may be better than methods you can write yourself. — Jack Guy, Feb 29 '16 at 21:44
No because, a lot of their functions skip edge cases, and other optimizations... `_.each` is faster than both `forEach` and `for` for example because of that. — Leon Gaban, Feb 29 '16 at 21:44
@LeonGaban [`each` uses a `while` loop.](https://github.com/lodash/lodash/blob/master/lodash.js#L477) A `for` loop is just syntactic sugar (with the possibility of providing *more* optimizations) over a `while` loop. See [here](http://stackoverflow.com/a/32381908/371184) for a fair comparison between lodash and a standard `for` loop. — Mike Cluck, Feb 29 '16 at 21:46
As for your actual question, are the elements ordered in any way? — Mike Cluck, Feb 29 '16 at 21:49
LoDash can't be faster than native javascript, it's just not technically possible, and a library certainly isn't faster than a `for` loop written the correct way. As for the answer, just filter the big array based on wether or not the value exists in the small array, it's not that fast, but you'll never notice the difference — adeneo, Feb 29 '16 at 21:51
Sigh, I've posted proof of this already with jspref, but I'll post it again soon. Others have commented on this as well here: http://stackoverflow.com/questions/18881487/why-is-lodash-each-faster-than-native-foreach — Leon Gaban, Feb 29 '16 at 21:57
@LeonGaban That's true for `forEach` and *only* `forEach`. A standard `for` loop without a callback will always be faster. — Mike Cluck, Feb 29 '16 at 22:00
But @LeonGaban you are being culled into believing that if it's done in lodash it has to be faster. Solution you chose can be witten to create object first and would be much faster regardless of using *"slower"* loops. Comparing an object property vs using `find()` within each iteration would be infinitely faster — charlietfl, Feb 29 '16 at 22:04
Here I found my other post with the jspref stats, in my question I added `_.each` vs `for loop` http://stackoverflow.com/questions/35368609/how-to-compare-specific-items-in-array-inside-of-basic-lodash-loop I use `_.times` just to create the dummy Array to iterate over. (http://jsperf.com/forloop-vs-each-from-lodash) — Leon Gaban, Feb 29 '16 at 22:06
@LeonGaban When I run it, the native is faster. This is especially true [when you cache the length of the array.](http://jsperf.com/forloop-vs-each-from-lodash/2) — Mike Cluck, Feb 29 '16 at 22:12
@LeonGaban http://jsperf.com/zerkms-forloop-vs-each-from-lodash --- please stop posting benchmarks that make no sense — zerkms, Feb 29 '16 at 22:13
Removing all the crap in that perf, and doing a proper comparison, would give you this -> http://jsperf.com/forloop-vs-each-from-lodash/6 — adeneo, Feb 29 '16 at 22:16
@MikeC strange for me lodash still ran faster. And @ zerms, there is nothing wrong with my benchmark. Also the link you just posted, I ran it and lodash killed native. Posting a screen in my question. — Leon Gaban, Feb 29 '16 at 22:17
@LeonGaban No need to be rude. Yes, your benchmark was messed up because you're adding in the variability of the `_.times` call as well as calls to `console.log`. Just waiting for access to the console can restrict performance. adeneo's is the most accurate. — Mike Cluck, Feb 29 '16 at 22:18
@MikeC sorry, that was not aimed at you, and I feel that Mr Zerkms has been rude to me this entire time for some reason... but yes I went too far, cleaned up my comment. — Leon Gaban, Feb 29 '16 at 22:22
@LeonGaban those images of perf test are completely irrelevant to the question. Please remember that question will live here for years — charlietfl, Feb 29 '16 at 22:25
Possible duplicate of [javascript array difference](http://stackoverflow.com/questions/1187518/javascript-array-difference) — adeneo, Feb 29 '16 at 22:29
@adeneo Not a duplicate, I'm asking how to accomplish this using Lodash. — Leon Gaban, Feb 29 '16 at 22:29
And I'm telling you there are better and faster ways, using simple native methods, like [**this**](http://stackoverflow.com/questions/1187518/javascript-array-difference#answer-4026828) — adeneo, Feb 29 '16 at 22:35
Possible duplicate of [using underscore's “difference” method on objects](http://stackoverflow.com/questions/13147278/using-underscores-difference-method-on-objects) — adeneo, Feb 29 '16 at 22:47

score 1 · Answer 1 · answered Feb 29 '16 at 21:51

1

You're looking for _.filter instead of _.remove, I think

$scope.newArray = _.filter($scope.bigArray, function(item) {
   return !_.find($scope.smallArray, {name: item.name});
});

answered Feb 29 '16 at 21:51

Jack Guy

8,346
8
55
86

2

A note: it's `O(n^2)` – zerkms Feb 29 '16 at 21:52
Sweet I think this is it! :D, I think I get it too... so return what is left over ie: what has Not been found: `!_.find` – Leon Gaban Feb 29 '16 at 21:56
I didn't like this solution, it will take too much time – Nour Sammour Feb 29 '16 at 21:59
1

@NourSammour "too much" compared to *what*? Performance is *not the absolute* property of an implementation. – zerkms Feb 29 '16 at 22:00
i thought the small array in few hundreds but look like it's less than 10 items so my wrong :) – Nour Sammour Feb 29 '16 at 22:04
@NourSammour well I choose less then 10 items for the sake of my codepen, but yes in the real app you could be attempting to remove hundreds of users from a list of thousands. – Leon Gaban Feb 29 '16 at 22:09
I'm preparing answer, you will like it ;) – Nour Sammour Feb 29 '16 at 22:09
i post my answer, what do you think? – Nour Sammour Feb 29 '16 at 22:18

score 1 · Accepted Answer · answered Feb 29 '16 at 22:18

1

it's just improvement on @Harangue answer

it will improve the performance when the second array is big

var smallObject = _.keyBy($scope.smallArray, 'name');
$scope.newArray = _.filter($scope.bigArray, function(item) {
   return !smallObject[item.name];
});

so instead of using 2 loops we can use 1 loop

answered Feb 29 '16 at 22:18

Nour Sammour

2,822
1
20
19

This also drastically increases the memory requirements. – Mike Cluck Feb 29 '16 at 22:22
@MikeC yes because I'm cloning exist object to other one so will increase memory, but if we save Ajax response directly as object (not a array) i think this approach will be better – Nour Sammour Feb 29 '16 at 22:25
Thanks +1, I need to test this out – Leon Gaban Feb 29 '16 at 22:33
So I tested it out here http://jsperf.com/lodash-used-to-diff-arrays2 looks like the same performance, but I understand how you are using an object this way, so it must be faster. – Leon Gaban Feb 29 '16 at 22:59
yes it should be faster, try to make add at least 50 item to the small array – Nour Sammour Feb 29 '16 at 23:01

How to remove all items in smallArray from bigArray with lodash?

Why am I asking about a lodash solution? Because the lodash API is faster in a lot of ways than the native Javascript API.

2 Answers2