3

I need to load files in iOS, now I use the + [NSString stringWithContentsOfFile:]. The files are mostly 500kb to 5mb. I load an approx. 4mb large file and Instruments and stopwatch told me it needs 1.5 seconds to load this file. In my opinion its a bit slow, is there a way to get the string faster?

EDIT:

I try some things and notice now, the creation of the NSString is my problem it takes 97% of the time and not the real loading from disk.

Sebastian
  • 952
  • 1
  • 14
  • 41
  • What's in the "string"? – trojanfoe Mar 13 '13 at 14:31
  • its some kind of csv data – Sebastian Mar 13 '13 at 14:32
  • And are you "processing" this CSV into a better data structure after loading? – trojanfoe Mar 13 '13 at 14:34
  • yes after loading I split them up and make objects from it but this is very fast instead of the initial loading of the data – Sebastian Mar 13 '13 at 14:36
  • 1
    If the data is not needed on lauch you can load it in a background thread while running. So nobody has to wait – Mert Mar 13 '13 at 14:39
  • 1
    As each line is independent, I would suggest reading the CSV a line-at-a-time and creating your objects as you go; this would save memory at least (and might be faster). You'll probably need to use C-library functions to do this as it's hard to do using `NSString` and friends. – trojanfoe Mar 13 '13 at 14:39
  • I read the file only when its needed, because which file I need to load is determined by user interaction and I want to put too much in memory when its not needed @trojanfoe I try some kind of obj-c reader for line by line reading but this was ways slower than read the whole file did you have any links with an example for the c api? – Sebastian Mar 13 '13 at 14:40
  • 1
    Open the file using `fopen()` and then use `fgets()`. Manpage and example: http://pubs.opengroup.org/onlinepubs/007904875/functions/fgets.html – trojanfoe Mar 13 '13 at 14:46
  • 1
    See also this: http://stackoverflow.com/a/3711079/1187415 and this: http://stackoverflow.com/a/3910036/1187415 for efficently reading lines with blockwise buffering. – Martin R Mar 13 '13 at 14:49
  • Note that if you read line-by-line it's critical that you use some sort of buffered reader. (I think fgets is buffered, though I don't know how well.) – Hot Licks Mar 13 '13 at 15:25
  • 1
    Another option to consider, depending on your reference patterns, is putting all this stuff into a SQLite database. – Hot Licks Mar 13 '13 at 15:27
  • Pre-processing the data and putting it into a database is probably the best way to go here. SQLite or Core Data would be much better here, especially if you are choosing from fixed files that are included with your app (ie in the app bundle). – lnafziger Mar 13 '13 at 15:36
  • the files are downloaded but I should consider the change it when it gets real problem... and right I should proceed the data before the processing also takes a lot time (create 30000 objects) – Sebastian Mar 13 '13 at 15:38
  • If you aren't calculating totals or anything, another good option would be to only load a subset of the information (maybe the first 100 or first 1000 rows) and only load more data as they scroll down and need it. Maybe add a "Load the next 100 rows" button at the bottom or load the first 1000, display them, and load the rest in a background thread. – lnafziger Mar 13 '13 at 15:49

1 Answers1

3

If you either know the encoding or can determine it (the API you use now is basic), you can just treat it as a char buffer (in a manner which is encoding aware).

I'd begin by opening it using memory mapped data (mmap, you can also approach this using NSData). madvise can be used to hint how you will access the file.

If memory mapped I/O consumes too much memory for your use, you should drop down to incremental reads, like C I/O facilities - fopen, fread, etc.. This will typically require more I/O events than memory mapped data (can be much slower, depending on how the data is accessed).

In both cases, you would treat the string as a C string -- don't simply convert the whole file to an NSString upon opening.

Foundation has a lot of tricks, so make sure this actually improves performance for your specific use case.

If those solutions are too 'core for your use, just consider using smaller files instead (dividing existing files).

justin
  • 104,054
  • 14
  • 179
  • 226
  • the problem is in the end I need the parts itself as a NSString – Sebastian Mar 13 '13 at 15:24
  • 1
    @Sebastian yes - i saw that. you would do the 'parsing' of the string manually using the char representation, then create `NSString`s from exact sub-ranges of the chars. that could minimize I/O, especially if you need only some of the file's contents (you must understand your problem -- everything i have suggested could also *increase* I/O if used incorrectly). another possible approach is to create a mmapped representation, then pass the returned pointer to mapped memory into `-[NSString initWithBytesNoCopy:length:encoding:freeWhenDone:]`. – justin Mar 13 '13 at 15:41
  • 1
    Thanks that help me for my problem. No I have to improve the performance of the usage of the strings :) – Sebastian Mar 13 '13 at 15:49