How to parse huge JSON files in Swift 3?

Question

This is what I have now:

4 JSON files
~ 2 000 000 lines in each file

~ 250MB each file

if let path = Bundle.main.path(forResource: "file_1", ofType: "json") {

    let url = URL(fileURLWithPath: path)

    do {

        let data = try Data(contentsOf: url, options: Data.ReadingOptions.mappedIfSafe)

        do {

            if let results = try JSONSerialization.jsonObject(with: data, options: JSONSerialization.ReadingOptions.mutableContainers) as? Array<[AnyHashable: Any]> {
                //do something here with that
            }

        } catch {
            print("error1: \(error)")
        }

    } catch {
        print("error2: \(error)")
    }
}

But all I have is an error:

Message from debugger: Terminated due to memory issue

Is it possible at all to parse that 4 json files using iphone in Swift? I need to parse it within a few seconds.

for the error: http://stackoverflow.com/questions/19203790/is-it-possible-to-debug-terminated-due-to-memory-error you want to parse 4 json file at once? if yes, do you keep put in a directory? — Jacky Shek, Oct 06 '16 at 09:11
I do not have to keep them in RAM. All I need to do is to parse them, count the values for some "key", and display them. That is all. — Bartłomiej Semańczyk, Oct 06 '16 at 09:14
maybe [this](http://stackoverflow.com/questions/29440325/parsing-continuous-json-stream-in-ios) can help so you dont have to load the whole file — Fonix, Oct 06 '16 at 09:18
Did you find solution for this issue? If so please post it so It can help. Thank you in advance!! — iUser, Aug 22 '18 at 19:55

score 2 · Answer 1 · answered Jan 17 '17 at 17:48

2

You can try this library: https://github.com/postmates/PMJSON.

I don't try it yet, but seems a possible solution.

Sorry for my English.

answered Jan 17 '17 at 17:48

domy

41
6

Is it suitable for these large files, I don't see any mention about that in the read me. – koen Aug 20 '17 at 17:38

score 1 · Answer 2 · answered Oct 06 '16 at 09:12

1

This line

let data = try Data(contentsOf: url, options: Data.ReadingOptions.mappedIfSafe)

uses about 250MB of memory. That is a huge amount for an app. Especially if you load 4 files. You memory usage is about 1GB. So in two words - it's impossible. You need to find a better solution to your problem.

answered Oct 06 '16 at 09:12

Max Pevsner

4,098
2
18
32

7

I think that's what he's asking for, isn't it? – Vince O'Sullivan Oct 06 '16 at 09:14
2

@VinceO'Sullivan, he's asking if it's possible (_Is it possible at all to parse that 4 json files using iphone in Swift?_). The answer is **NO**. – Max Pevsner Oct 06 '16 at 09:16
The answer is yes, but he'll have to stream them. – Brill Pappin Apr 04 '18 at 19:42
1

speak with the backend guys and download in chunks – brainray Apr 19 '18 at 16:05

score 0 · Answer 3 · answered Feb 26 '20 at 08:46

Swift 5.1 Use below code for huge JSON file. It works for me.

    func readJSONFromFile(fileName: String) -> Any?
    {
        var json: Any?
        if let path = Bundle.main.path(forResource: fileName, ofType: "json") {
            do {
                let fileUrl = URL(fileURLWithPath: path)
                // Getting data from JSON file using the file URL
                let data = try Data(contentsOf: fileUrl, options: .mappedIfSafe)
                json = try? JSONSerialization.jsonObject(with: data, options: .allowFragments)
                print(json)
            } catch {
                // Handle error here
            }
        }
        return json
    }

How to parse huge JSON files in Swift 3?

3 Answers3