Parsing Dates takes huge amount of RAM

Question

So I have to parse potentially huge log files containing ISO8601 formatted dates. I use DateFormatter to handle date parsing. The problem is that memory grows linearly with the number of parsing done. Here is a simplified version of my code that illustrates the problem :

import Foundation

func testMem() {
    let formatter = DateFormatter()
    formatter.dateFormat = "yyyy-MM-dd' 'HH:mm:ss.SSS"
    formatter.calendar = Calendar(identifier: .iso8601)

    let dateStr = "2017-10-16 19:01:35.360"
    var after: Int = 0

    for _ in 0..<1_000_000 {
        if let t = formatter.date(from: dateStr) {
            if t > Date.distantPast {
                after += 1
            }
        }
    }
}

testMem()

RunLoop.main.run() //to let the program run

Memory usage of this program is around 200Mb. It looks like Dates are never freed. Profiling this code does not show any leaks. Because memory grows linearly means I can't really parse large files: the process will eventually get killed because it uses too much RAM.

Conceptually, I don't understand why this program consumes so much memory: only one instance of Date should be alive at any time.

Am I missing something ?

Can I somehow workaround the issue ?

This should be helpful: [Is it necessary to use autoreleasepool in a Swift program?](https://stackoverflow.com/questions/25860942/is-it-necessary-to-use-autoreleasepool-in-a-swift-program). — Martin R, Feb 20 '19 at 12:23
Thanks @MartinR ! I'll add an answer based on autoreleasepool. — Mike, Feb 20 '19 at 12:27

score 0 · Accepted Answer · answered Feb 20 '19 at 12:31

Ok so thanks to Martin R's pointer, the solution seems to involve autoreleasepool. Here is an updated version of the code which fixes the issue :

import Foundation

func testMem() {
    let formatter = DateFormatter()
    formatter.dateFormat = "yyyy-MM-dd' 'HH:mm:ss.SSS"
    formatter.calendar = Calendar(identifier: .iso8601)

    let dateStr = "2017-10-16 19:01:35.360"
    var after: Int = 0

    for _ in 0..<1_000_000 {
        autoreleasepool {
            if let t = formatter.date(from: dateStr) {
                if t > Date.distantPast {
                    after += 1
                }
            }
        }
    }
}

testMem()

RunLoop.main.run()

Creating and destroying an autorelease pool is not “free.” You may want to implement a nested loop, as Rob did here: https://stackoverflow.com/a/25880106/1187415. — Martin R, Feb 20 '19 at 12:36

Parsing Dates takes huge amount of RAM

1 Answers1