Elixir Remove Duplicates From List

Question

Would someone be willing to provide some alternate solution to the removal of duplicate values from a List (X) using Functional Programming and Elixir Constructs?

X = [1,26,3,40,5,6,6,7] # the 6 being the duplicate

The stock solution in my mind for solving this problem, would be to iterate the list (X), and add to a new list (Y) where the key doesn't already exist.

Thank you

This is element distinctness problem, which is a widely researched problem. We know lower bounds for the problem under some configurations. It can be solved in O(nlogn) by sort + iterate or O(n) on average time + space by using a hash set. The linked question discusses this question. — amit, May 11 '15 at 11:37
@amit Correct thanks - I believe I was looking for an efficient way to solve the problem using Elixir Language Constructs and Functional Programming. — Dane Balia, May 11 '15 at 11:39
@amit, I don't think this is a duplicate. He's asking for an answer in a specific language and the question you suggested he's duplicating is a question regarding algorithmic complexity. Not exactly the same thing. — Onorio Catenacci, May 11 '15 at 11:50
In answer to your question @DaneBalia there's a built in function in Elixir to do what you're looking for. http://elixir-lang.org/docs/v1.0/elixir/Enum.html#uniq/2 Enum.uniq will remove duplicate items from a collection. — Onorio Catenacci, May 11 '15 at 11:53
@OnorioCatenacci thank you - exactly what I was looking for and no code needed ;) — Dane Balia, May 11 '15 at 11:54
Reopened it, since the focus of the question is not the algorithm, but how to do it in specific language, nevermind the algorithm behind the scenes. — amit, May 11 '15 at 12:00

Paweł Obrok · Accepted Answer · 2020-01-30T22:08:03.017

58

Enum.uniq does what you want, for example:

iex(6)> Enum.uniq([1,26,3,40,5,6,6,7])
[1, 26, 3, 40, 5, 6, 7]

In terms of how you'd implement it you could write a recursive function like so:

defmodule Test do
  def uniq(list) do
    uniq(list, MapSet.new)
  end

  defp uniq([x | rest], found) do
    if MapSet.member?(found, x) do
      uniq(rest, found)
    else
      [x | uniq(rest, MapSet.put(found, x))]
    end
  end

  defp uniq([], _) do
    []
  end
end

iex(3)> Test.uniq([1, 1, 2, 3, 4, 4, 1])
[1, 2, 3, 4]

edited Jan 30 '20 at 22:08

answered May 11 '15 at 12:08

Paweł Obrok

22,568
8
74
70

1

Good suggestion, though HashSet has been deprecated in favor of MapSet. – skwidbreth Jan 26 '20 at 22:36

score 3 · Answer 2 · edited Oct 11 '22 at 14:44

3

Yet another possible solution is to use Set when creating collection:

Hashset is now deprecated so instead of using it

#[1, 1, 2, 3, 4, 2, 1] |> Enum.into(HashSet.new) #HashSet<[2, 3, 4, 1]>

You should use Mapset

#[1, 1, 2, 3, 4, 2, 1] |> Enum.into(MapSet.new) #MapSet<[2, 3, 4, 1]>

edited Oct 11 '22 at 14:44

Jadu

489
3
10

answered Nov 27 '16 at 20:20

gayavat

18,910
11
45
55

score 2 · Answer 3 · edited Dec 01 '22 at 20:39

2

Also using MapSet

()> "3 3 2 1 1" |> String.split |> MapSet.new |> Enum.to_list
["1", "2", "3"]

edited Dec 01 '22 at 20:39

skwidbreth

7,888
11
58
105

answered Nov 03 '17 at 05:31

Andrei Sura

2,465
1
20
15

Elixir Remove Duplicates From List

3 Answers3