How do I remove duplicate values from an array in Perl?

Question

I have multiple values for each key in the hash. Here, I print the values of each key

print "$var:",join ',',@{$hash{$var}},"\n";

Output of the above print statement is:

ABC:1,2,3,1,4,2,
DEF:9,3,4,5,3,5,

Now, I want to remove duplicates in the values. The desired output is,

ABC:1,2,3,4
DEF:9,3,4,5

Can someone help me in this?

Thanks

score 3 · Answer 1 · answered Oct 16 '12 at 18:38

3

Using List::MoreUtils::uniq:

@$_ = uniq @$_ foreach values %hash;

answered Oct 16 '12 at 18:38

Axeman

29,660
2
47
102

score 3 · Accepted Answer · answered Oct 16 '12 at 19:12

3

Personally, I like how List::MoreUtils's uniq handles this. The subroutine is quite simple, and can be used as-is, or through the module.

my %seen;
my @dedupe = grep { not $seen{$_}++ } @{$hash{$var}};

The subroutine from the module looks like this:

sub uniq (@) {
    my %seen = ();
    grep { not $seen{$_}++ } @_;
}

This code preserves the initial order of the list.

answered Oct 16 '12 at 19:12

TLP

66,756
10
92
149

Hmm, do you use defined types of variables in function header all time? – gaussblurinc Oct 16 '12 at 22:16
1

If you mean do I personally use prototypes, then the answer is no. That sub is copy pasted from the module. I considered removing it, because as a general rule prototypes should not be recommended to beginners, but since it was basically a quote, I left it as it was. – TLP Oct 16 '12 at 22:26
yeah, i heard about it. but i don't understand, why people use it in CPAN modules. – gaussblurinc Oct 16 '12 at 22:42
Well, they do have a purpose. On the other hand, many people use them for no good reasons. – TLP Oct 17 '12 at 00:40

score 2 · Answer 3 · answered Oct 16 '12 at 18:35

2

That it's in a hash is irrelevant. What you're wanting to do is dedupe the values in an array.

Look at the uniq() function in List::MoreUtils module http://search.cpan.org/dist/List-MoreUtils/

You can also do it yourself like so:

my %h = map {($_,1)} @array_with_dupes;
my @deduped_array = keys %h;

This is all covered extensively in the Perl FAQ under "Data: Arrays".

answered Oct 16 '12 at 18:35

Andy Lester

91,102
13
100
152

1

The do-it-yourself solution does not preserve the initial order of the elements. – TLP Oct 16 '12 at 19:13
That wasn't specified as a requirement. I can see how you'd infer it, though. – Andy Lester Oct 16 '12 at 20:24

score 2 · Answer 4 · answered Oct 16 '12 at 18:40

2

If you are trying to remove duplicate values from an array you can use this

#!/usr/bin/perl -w
use strict;
my @array = ('a','b','c','c','e');
my %hash = map { $_ => 1 } @array;
my @out = keys %hash;
printf("%s",join(',',@out));

answered Oct 16 '12 at 18:40

Jean

21,665
24
69
119

This method does not preserve the initial order of your list. – TLP Oct 16 '12 at 19:14

How do I remove duplicate values from an array in Perl?

4 Answers4

Linked