1

I have this reasonably standard ADT in Swift that I use to parse some JSON data which, by all accounts, looks very much like those data structures. The JSON file represents an array of about 155k objects that get parsed into a corresponding [LogRow].

Now, I'm perfectly aware that there is going to be some overhead compared to a plain text representation such as JSON, but what I found is that a JSON file of about 500 MB ends up taking up to 5.5 GB of my computer's memory -- in my book, that's a 10-fold overhead, unless I'm missing something.

I had a look around with Xcode's profiler, but I couldn't see any memory leak. I'm even careful of taking data and decoder into vars so I can nil them and give a nudge to the ARC, but that doesn't seem to work very well, as I assume I'm creating loads of pointers everywhere, so hardly anything gets deallocated.

I had my code checked by people more expert than me with Swift, and they all agreed the code is OK and the overhead is pretty awesome. Can anybody point me towards any docs on the memory requirements of the object model, perhaps.

EDIT For reference, here's what I see in Instruments. He slight drop at the end is when the source file is deallocated (I preallocate it entirely in a string before passing it to the JSONDecoder.

Instruments, allocations and (no) leaks.

Morpheu5
  • 2,610
  • 6
  • 39
  • 72
  • 1
    "I couldn't see any memory leak" ... Were you using "leaks" tool? That's not a very good tool for identifying problems. What you should do is focus on "allocations" tool, and identify what objects were instantiated but not released over that period of time. Then look at that and see what, if anything looks suspect. Or, better, use Xcode "debug memory graph" tool to identify anything not released that should have been. Also, make sure "zombies" is turned _off._ Also, make sure you're not doing anything extravagent during the JSON parsing (e.g. retrieving actual images, too). – Rob Sep 29 '17 at 17:34
  • 1
    What are you looking for when you ask "how this works"? Every in-memory type in Swift has some amount of memory associated with it, and it differs on a case-by-case basis... Arrays and strings have to keep track of their length and buffer capacity, for instance, which is not true of raw string data in JSON; if you have a lot of short strings, for instance, the overhead might grow as fast or faster than the data itself. Can you make your question more specific, then? – Itai Ferber Sep 29 '17 at 19:21
  • @Rob I used both leaks and allocations, the latter to see the total and current amounts of memory, the former didn't give me any indications of leaks. I'm not doing anything extravagant, I'm really just simply asking `JSONDecoder` to decode the content of a file into the data structures in that gist. Absolutely nothing else. @Itai-ferber Good point about the short strings (I wonder if I could convert them to Ruby-style symbols). I was hoping for an overview of the memory requirements of Swift's object models that could roughly explain what I'm seeing. – Morpheu5 Sep 30 '17 at 09:59
  • I wouldn't expect any leaks. On allocations, you'll want to take your analysis down a level, looking at not just what the persistent allocations totals were, but select a range of time and then see _what_ those persistent allocations consisted of. Or, perhaps easier, use the "debug memory graph" feature of Xcode, look in the left panel for objects that should have been released. See https://stackoverflow.com/questions/30992338/how-to-debug-memory-leaks-when-leaks-instrument-does-not-show-them/30993476#30993476 – Rob Sep 30 '17 at 15:09
  • By the way, when I say “I wouldn’t expect any leaks”, I should clarify. While we use “leaks” term somewhat generally, there’s actually a difference between a “leak” (something with no references, but not deallocated; this is hard to do as memory management has improved over the years) and “abandoned memory” (something that has strong references that you might not be able to access anymore, e.g. a strong reference cycle). – Rob Sep 30 '17 at 18:41
  • Thanks, I'll try the memory graph route. I think ARC in Swift is good at avoiding strong reference cycles, and I expect that, since `JSONDecoder` is in Foundation, it should be well written code by any reasonable dev standards for Swift. – Morpheu5 Oct 01 '17 at 14:28
  • I did experiment, try parse 43k records, without autoreleasePool and with, and... 110+ Mb vs 34Mb with autoreleasePool ```let data = try Data(contentsOf: url) books = try JSONDecoder().decode(Set.self, from: data)``` vs ```let decoded = try autoreleasepool { () -> Set in let data = try Data(contentsOf: url) return try JSONDecoder().decode(Set.self, from: data) } books = decoded```. books there is `Set`, also Book is `Struct` which contains only two field `code: String` and `title: String` – iTux Apr 12 '20 at 21:06

0 Answers0