0

Please read: I know this feels like the kind of question that's been answered ad nauseam on Stack Overflow, but I swear I can't find a good answer. Besides, marking this as "Possible duplicate" would be too meta!

I have a matrix of values in a particular order (in javascript):

[ [1,2], [1,2], [3,4], [5,6] [5,6] ]

However I recently learned that Google charts crashes when there are identical rows. I need to remove the duplicates; however, the array is fairly large and I can't afford the quadratic time of a naive implementation.

Normally, if the rows were each hashable, I'd add them to a dictionary { } to see which I'd already seen; however, javascript does not allow arrays to be hashed.

What is the best approach? I guess I could convert the array for each row into a string and use that as the key, but that feels like a pretty dirty (and potentially slow) hack. I'd really love your advice.

Community
  • 1
  • 1
user
  • 7,123
  • 7
  • 48
  • 90
  • When you say "in a particular order", does this mean you know that the array is sorted? If so, what's the sortorder? – Abraham P Feb 17 '13 at 19:06
  • Where does your array come from? Could you do this on creation? – Boris the Spider Feb 17 '13 at 19:07
  • @AbrahamP It isn't sorted (it's extracted from an XML file). It's important to keep the current order because it's not necessarily a function, so it may move backwards sometimes. – user Feb 17 '13 at 19:09
  • Upon parsing from XML create the hash using the [].toString() method you yourself mention. XML parsing is slow enough, this will not add noticeably at all, but you can create a jsperf.com test (and link to it here for posterity). Seems best to do it at THAT point, when you go through the whole set anyway. I would not be concerned about speed - any other methods will cost you MORE, because you will iterate over all of it a 2nd time. – Mörre Feb 17 '13 at 19:18
  • Could you provide more info on the data size? How many rows are there: thousands, ten thousands, millions? How long is each row in average? – georg Feb 17 '13 at 19:22

1 Answers1

0

You can remove duplicates easy with Set and JSON Stringify/Parse. So this approach will keep the order of the matrix.

const array = [[1, 2],[1, 2],[3, 4],[5, 6],[5, 6]];
const set = new Set(array.map(JSON.stringify));
const arr = [...set].map(JSON.parse);

console.log(arr);// output : [[1,2], [3,4], [5,6]]
Zhivko Zhelev
  • 321
  • 4
  • 10