This is my very first question here so go easy on me!
I'm a newbie coder and I'm currently trying to loop through JSON, parse the data and backup the information to my Firebase server - using Alamofire to request the JSON information.
Swift 4, Alamofire 4.5.1, Firebase 4.2.0
The process works - but not without infinitely increasing device memory usage & up to 200% CPU usage. Through commenting out lines, I singled the memory and CPU usage down to the Firebase upload setValue line in my data pulling function - which iterates through a JSON database of unknown length (by pulling a max of 1000 rows of data at a time - hence the increasing offset values). The database that I'm pulling information from is huge, and with the increasing memory usage, the function grinds to a very slow pace.
The function detects if it's found an empty JSON (end of the results), and then either ends or parses the JSON, uploads the information to Firebase, increases the offset value by 1000 rows, and then repeats itself with the new offset value.
var offset: Int! = 0
var finished: Bool! = false
func pullCities() {
print("step 1")
let call = GET_CITIES + "&offset=\(self.offset!)&rows=1000"
let cityURL = URL(string: call)!
Alamofire.request(cityURL).authenticate(user: USERNAME, password: PASSWORD).responseJSON { response in
let result = response.result
print("step 2")
if let dict = result.value as? [Dictionary<String, Any>] {
print("step 3")
if dict.count == 0 {
self.finished = true
print("CITIES COMPLETE")
} else {
print("step 4")
for item in dict {
if let id = item["city"] as? String {
let country = item["country"] as? String
let ref = DataService.ds.Database.child("countries").child(country!).child("cities").child(id)
ref.setValue(item)
}
}
self.finished = false
print("SUCCESS CITY \(self.offset!)")
self.offset = self.offset! + 1000
}
}
if self.finished == true {
return
} else {
self.pullCities()
}
}
}
It seems to me like the data being uploaded to Firebase is being saved somewhere and not emptied once the upload completes? Although I couldn't find much information on this issue when searching through the web.
Things I've tried:
a repeat, while function (no good as I only want 1 active repetition of each loop - and still had high memory, CPU usage)
performance monitoring (Xcode call tree found that "CFString (immutable)" and "__NSArrayM" were the main reason for the soaring memory usage - both relating to the setValue line above)
memory usage graphing (very clear that memory from this function doesn't get emptied when it loops back round - no decreases in memory at all)
autoreleasepool blocks (as per suggestions, unsuccessful)
Whole Module Optimisation already enabled (as per suggestions, unsuccessful)
Any help would be greatly appreciated!
UPDATE
Pictured below is the Allocations graph after a single run of the loop (1,000 rows of data). It shows that what is likely happening is that Firebase is caching the data for every item in the result dict, but appears to only de-allocate memory as one whole chunk when every single upload has finished? Ideally, it should be de-allocating after every successful upload and not all at once. If anyone could give some advice on this I would be very grateful!
FINAL UPDATE
If anyone should come across this with the same problem, I didn't find a solution. My requirements changed so I switched the code over to nodejs which works flawlessly. HTTP requests are also very easy to code for on javascript!
