Java remove duplicates from List>

Question

I am attempting to remove duplicate records found in a List<List<float[]>>. I attempted to use a collection which does not allow duplicates(HashList) but I was unable to figure out how to cast this properly. To loop through all of my elements I would perform.

List<List<float[]>> tmp; 

for(int i=0; i<tmp.get(0).size();i++){
    System.out.println(java.util.Arrays.toString(tmp.get(0).get(i)));
}

I want to remove them from the list inside. So all elements found at tmp.get(0).get(Here to remove)

tmp.get(0).get(1) =[-70.89,42.12]

tmp.get(0).get(2) =[-70.89,42.12]

I would like to remove tmp.get(0).get(2)

Current implementation, which works when there is only 1 duplicate but not multiple duplicates.

for(int i=0; i<old.get(0).size();i++){
            if(i == old.get(0).size()-1){
                System.out.println("max size");
                return old;
            }
            else if(Arrays.toString(old.get(0).get(i)).equalsIgnoreCase(Arrays.toString(old.get(0).get(i+1)))){
                old.get(0).remove(i);
                i++;
            } else {
            i++;
            }

Answer 1

If I understood correctly, you are looking for Set<List<float[]>>.

Answer 2

How about:

List<List<Float[]>> outterList;
Set<Float[]> mySet = new HashSet<Float[]>();
for (List<Float[]> innerList : outterList){
    Iterator<Float[]> iterator = innerList.iterator();
    while(iterator.hasNext()){
        Float[] array = iterator.next();
        boolean added = mySet.add(array);
        if (!added)
           iterator.remove();
    }
}

To make the comparison, try converting to BigDecimal via new BigDecimal(double, MathContext)

Update: The test fails. seems to be an issue with comparing Arrays in a HashSet.

@Test
public void testArrays() {
    Set<String[]> set = new HashSet<String[]>();
    set.add(new String[] { "12.3f", "33.4f" });
    Assert.assertFalse(set.add(new String[] { "12.3f", "33.4f" }));
}

Update

So arrays work differently. Here you go:

This uses Guava's Predicate and Iterables.any(). This solution is less efficient than using a Set since it has to iterate the List each time but it does work if performance is not an issue.

private static <T> Predicate<T[]> equals(final T[] array) {
    return new Predicate<T[]>() {

        @Override
        public boolean apply(T[] arg0) {
            return Arrays.equals(array, arg0);
        }
    };
}

public static <T> List<List<T[]>> ProcessList(List<List<T[]>> old) {
    List<T[]> mySet = new ArrayList<T[]>();
    for (List<T[]> innerList : old) {
        Iterator<T[]> iterator = innerList.iterator();
        while (iterator.hasNext()) {
            T[] array = iterator.next();
            Predicate<T[]> contains = equals(array);

            if (Iterables.any(mySet, contains)) {
                iterator.remove();
            } else {
                mySet.add(array);
            }
        }
    }
    // for (int i = 0; i < old.get(0).size(); i++) {
    // System.out.println(java.util.Arrays.toString(old.get(0).get(i)));
    // }
    return old;
}

This test:

@Test
public void testListsFloat() {
    List<List<Float[]>> outter = new ArrayList();

    List<Float[]> inner1 = new ArrayList();
    inner1.add(new Float[] { 12.3f, 33.4f });
    inner1.add(new Float[] { 12.2f, 33.2f });
    inner1.add(new Float[] { 12.3f, 33.4f });

    List<Float[]> inner2 = new ArrayList();
    inner2.add(new Float[] { 12.1f, 33.1f });
    inner2.add(new Float[] { 12.2f, 33.2f });
    inner2.add(new Float[] { 12.3f, 33.4f });

    outter.add(inner1);
    outter.add(inner2);

    outter = ProcessList(outter);
    for (List<Float[]> list : outter) {
        for (Float[] array : list) {
            System.out.println(Arrays.toString(array));
        }
    }
}

resulted in this output:

[12.3, 33.4] [12.2, 33.2] [12.1, 33.1]

Answer 3

You may use LinkedHashSet to have no duplicates while filling. Though it's an implementation of Set, you can wrap it into a List if you need List methods.