Compare two arrays, if values are in array, add to list, if not add values to another list. No duplicates

Question

please look at the comments in code and see if you can help me with this I am checking to see if a value from $arr1 is in $arr2. If it is, add it to a list, if it is not, add it to another list. Make sure both list/arrays do not have duplicates.

$arr1 = @(1,2,3,4,4,2,5,7,9,9,1) 
$arr2= @(5,1,2,3,6,8,1) 
$NotinList = @()
$inList = @()
$counter = 0

for ($i = 0; $i -lt $arr1.length; $i++){
    for( $j = 0; $j -lt $arr2.length; $j++ ){  
        if($arr1[$i] -ne $arr2[$j]){ #check to see if value from $arr1 is in $arr2
            for($k = 0; $k -lt $NotinList.length; $k++){ #Traverse through empty array
                write-host $arr1[$i]  
                if($NotinList[$k] -ne $arr1[$i]){                                        # *^ if empty array does not alreadycontain item from big $arr1, add it.
                    $NotinList +=  $arr1[$i]
                }
            }
        }
        else{
            $inList += $arr1[$i]

            #how would I remove duplicates from number in list since there are repeating numbers in $arr1 that are not in $arr2.

        }
}
$counter++ #keep track may use for something else??
}

You already got two answers to an almost exact same question as your [previous one](https://stackoverflow.com/questions/69638688/compare-duplicates-in-an-array-list-values-that-are-contained-and-not-contained) and both answers cover your need. — Santiago Squarzon, Oct 21 '21 at 00:18
Not even close. I do not want to perform those operations ".Add" operations I need everything to be within a certain space time complexity. I want to remove the duplicates in the most efficient way. — robertuxo, Oct 21 '21 at 00:24
I already have the solution that got the job done, but I want to do it the way up above. — robertuxo, Oct 21 '21 at 00:26
What do you mean you don’t want to do those “add” operations? What you’re doing here with += is one of the worst things one can do in regards to performance.. let alone “space time complexity” — Doug Maurer, Oct 21 '21 at 00:26
Is there another way that is more efficient? I guess using the '+' instead of '+=' — robertuxo, Oct 21 '21 at 00:28
I don't see how adding to a system.array which is actually recreating the array on each iteration (your code snipet) can be more efficient to adding values to a generic list. — Santiago Squarzon, Oct 21 '21 at 00:28
I'm unfamiliar with powershell but if you have a solution please share or any links to similar problems and I will take a look. — robertuxo, Oct 21 '21 at 00:29

score 2 · Answer 1 · answered Oct 21 '21 at 00:55

2

I would eliminate the duplicates first and then use the Where() array operator to split the array into the variables.

$arr1 = 'a','b','b','c','d','e','e','f','g'
$arr2 = 'a','b','c','g'

# not in list = d, e, f
# in list = a, b, c, g

$inlist,$notinlist = ($arr1 | Get-Unique).Where({$_ -in $arr2},'split')

Now here is what each contains

$notinlist
d
e
f

$inlist
a
b
c
g

answered Oct 21 '21 at 00:55

Doug Maurer

8,090
3
12
13

1

Nice, thats a good solution – robertuxo Oct 21 '21 at 01:07
Very clever with the split call. – Abraham Zinala Oct 21 '21 at 02:00
1

Nice, but note that since `$arr1` in the question isn't _sorted_, you cannot use `Get-Unique` (try `1, 2, 1 | Get-Unique` to see the problem) - either use sorted input or use `Select-Object -Unique` (which, unfortunately, is implemented quite inefficiently as of PowerShell 7.2 - see [GitHub issue #11221](https://github.com/PowerShell/PowerShell/issues/11221)). Depending on the size of the arrays, performing a linear lookup in the other array (`-in $arr2`) in each `.Where()` iteration could become problematic - but is definitely not a problem with the arrays as small as the sample array. – mklement0 Oct 21 '21 at 02:04
Or `([System.Collections.Generic.HashSet[string]]$arr1).where({....})` can work too – Santiago Squarzon Oct 21 '21 at 02:22
I see, sort-object -Unique works but is it better than select? – Doug Maurer Oct 21 '21 at 02:30
My question was never answered. This is not about using any .sort or get-unique pre defined methods. I want to do this at a "low level". my program up above may not be efficient but I want to do it only using like a c++ or java way of doing it if that makes sense. – robertuxo Oct 21 '21 at 13:05
Yes that makes sense, go use Java or C++ – Doug Maurer Oct 21 '21 at 13:26

score 0 · Answer 2 · answered Oct 21 '21 at 14:10

If you insist on doing it manually, I recommend using hashtables. They are extremely fast and the keys have to be unique.

If you care about the order you can use [ordered] hashtables

$arr1 = 1,2,3,4,4,2,5,7,9,9,1
$arr2= 5,1,2,3,6,8,1

$inlist = [ordered]@{}
$notinlist = [ordered]@{}

foreach($entry in $arr1){
    if($entry -in $arr2){
        if($entry -notin $inlist.keys){
            $inlist.$entry = $entry
        }
    }
    else{
        if($entry -notin $notinlist.keys){
            $notinlist.$entry = $entry
        }
    }
}

Now your unique lists are in either/both they keys or the values

$inlist.Keys

1
2
3
5

$notinlist.Values

4
7
9

Compare two arrays, if values are in array, add to list, if not add values to another list. No duplicates

2 Answers2