Swift: How can I remove duplicates from an array of doubles?

Question

I have an array of values like [0.75, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0040000000000000001, ...] and I need to remove the duplicates. I only want to focus on the first 3 digits after the decimal point. How do I do this?

Answer 1

Your problem strongly suggests you're using the wrong type for your data. Rather than trying to fix it up at the point of uniquing, I suspect you really just want to modify your model so the issue doesn't occur.

If you want to do decimal-based math, you should use decimal-based numbers, like NSDecimalNumber. For example, considering the case where you do have doubles coming into the system, you can convert them to NSDecimalNumber with a "0.001 accuracy" this way:

let values = [0.75, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0040000000000000001]

let behavior = NSDecimalNumberHandler(roundingMode: .plain,
                                      scale: 3,
                                      raiseOnExactness: false,
                                      raiseOnOverflow: false,
                                      raiseOnUnderflow: false,
                                      raiseOnDivideByZero: false)

let decimalNumbers = values.map {
    NSDecimalNumber(value: $0).rounding(accordingToBehavior: behavior)
}

let uniqueDecimals = Set(decimalNumbers)

Once you are working with NSDecimalNumber, and applying the appropriate rounding rules, then most operations work as you expect. You can just put them into a Set to unique them. You can check for equality. You can print them, and they will behave like decimal numbers. Make your model match your meaning, and most other problems disappear.

Answer 2

You can use NumberFormatter to fix the minimum and maximum fraction digits and use a set to filter the duplicate elements:

let array = [0.75, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0040000000000000001]

let numberFormatter = NumberFormatter()
numberFormatter.numberStyle = .decimal
numberFormatter.minimumFractionDigits = 3
numberFormatter.maximumFractionDigits = 3

---

var set = Set<String>()
let orderedSet: [Double] = array.flatMap {
    guard let string = numberFormatter.string(for: $0) else { return nil }
    return set.insert(string).inserted ? $0 : nil
}

orderedSet   // [0.75, 0.005, 0.004]

If you need Strings (as suggested by @Hamish):

var set = Set<String>()
let orderedSet: [String] = array.flatMap {
    guard let string = numberFormatter.string(for: $0) else { return nil }
    return set.insert(string).inserted ? string : nil
}

orderedSet   // ["0.750", "0.005", "0.004"]

Answer 3

Your problem is a little bit complicated as your equality for Doubles is based on the first 3 digits after the decimal point. Many threads describe about removing duplicates where simple equality applies, but I cannot find one including Doubles with comparison in your question.

You usually use Set to eliminate duplicates, but Set<Double> uses strict equality which does not fulfill your requirement.

Normalizing the value may work in your case:

extension Double {
    var my_normalized: Double {
        return (self * 1000).rounded() / 1000
    }
}

print(0.0050000000000000001.my_normalized == 0.0051.my_normalized) //->true

Using this, you can write something like this:

let arr = [0.75, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0050000000000000001, 0.0040000000000000001 /*,...*/]

var valueSet: Set<Double> = []
var result: [Double] = []
arr.forEach {value in
    let normalizedValue = value.my_normalized
    if !valueSet.contains(normalizedValue) {
        valueSet.update(with: normalizedValue)
        result.append(value)
    }
}
print(result) //->[0.75, 0.0050000000000000001, 0.0040000000000000001]

If you do not mind the order of the result and it can contain normalized value, the code can be simpler:

let simpleResult = Array(Set(arr.map {$0.my_normalized}))
print(simpleResult) //->[0.75, 0.0050000000000000001, 0.0040000000000000001]