-1

Lets say you have super large unsorted array 20mln elements, and now you want to remove duplicates from it. How would you do this without using second array where you push unique results in PHP. How would you unset element in array without breaking the loop? array_unique is using second array as well.

One way I was thinking is to do with 2 steps, 1st step setting value as null, 2nd step remove all elements which has null value. Is there better way to do so?

  • `array_flip` will generate a new array as well – Nick Sep 06 '20 at 00:09
  • I don't think it's going to be possible without some type of temp storage, back to the why question. – Alex Barker Sep 06 '20 at 00:11
  • Why can't you use `unset` in a loop? See for example https://stackoverflow.com/questions/1949259/how-do-you-remove-an-array-element-in-a-foreach-loop – Nick Sep 06 '20 at 00:11
  • foreach def makes a copy. You could use for with some creative index adjustment but still not sure how your going to know its a dupe. If there is a legitimate reason for this like your running out of memory and not optimization for the sake of optimizing, you could use the for loop and a second array, draining the primary as you insert into the secondary but I am still unsure of if that would even solve the issue. – Alex Barker Sep 06 '20 at 00:12
  • Look at my solution, it works with 2 `for`-loops and `splice`, so there isn't any extra array or memory needed only some time ;). – Sascha Sep 06 '20 at 00:57

2 Answers2

0

It will be quite slow, but it should only use one array.

$k = 0;
for ($i = 0; $k < count($a); $i++) {
  $found = false;
  for ($j = $i; !$found && $j < count($a); $j++) {
    if ($a[$i] == $a[$j]) {
      $found = true;
    }
  }

  if ($found) {
    unset($a[$i]);
  } else {
     $k++
  }
}
Alex Barker
  • 4,316
  • 4
  • 28
  • 47
  • This will have the same problem as my solution: You can't run it twice because the keys are not consecutively (there are missing some). – Sascha Sep 06 '20 at 00:34
  • Ah yes, you are correct. I guess you could add another index... – Alex Barker Sep 06 '20 at 00:35
  • Yes we yould with `$array = array_values($array);` but this would probably not a solution for the OP because creating anoter array. – Sascha Sep 06 '20 at 00:38
0

It will be not the fastest but you could iterate with a for-loop over the array and look for every element with a second for-loop if there exists an entry with a higher key which has the same value. If so then delete this entry with splice and decrease the index from the outer loop and go on.

Extended:

This solution now works as often as you want because I cut the double value with `splice` out, so that the indexes will be rearranged.

You can try it: http://sandbox.onlinephpfunctions.com/code/f9dbe1be31d4bfac5bf168467bfca0513812c848

for($i=0; $i<count($arr); $i++) {
    for ($j=$i+1; $j<count($arr); $j++) {
        if ($arr[$i]===$arr[$j]) {
            array_splice($arr, $i, 1);
            $j--;
            break;
        }
    }
}
Sascha
  • 4,576
  • 3
  • 13
  • 34