Read specific line from CSV efficiently, C

Question

I have a large csv file in the following format:-

ID,Hash
abc,123
def,456
ghij,7890

I want to efficiently read a line corresponding to given ID and make changes to corresponding hash. I am allowed to store some information in an initial pass, but the changes need to be dynamic. What can I do?

I don't want to iterate over all lines while making changes. No assumptions can be made about size of any entry in general. It may also change. File has no order.

This seems difficult, but please provide me some code by which I can acess some part of file in constant time. I think I can figure out a heuristic. It would be best if the address can be iterated in both directions from a given point.

Does the hash have a fixed length (for example 3 like in your sample)? — Jabberwocky, Mar 20 '17 at 15:51
Actually there are some more columns. But can I store the address of each line in an array? — Meet Taraviya, Mar 20 '17 at 15:53
Does the hash have a fixed size? The answer depends heavily on this. If the size is fixed, then it's easy to update the hash directly in the file, otherwise the file must be read and rewritten which is slighly more complicated and slower. — Jabberwocky, Mar 20 '17 at 15:54
@MeetTaraviya *can I store the address of each line in an array?* How do you plan to *find* the address of each line? — Andrew Henle, Mar 20 '17 at 15:55
No, but why do you need this? I can store the size of each line in an array. Memory as not much a constraint here, dynamic nature and efficiency is a must — Meet Taraviya, Mar 20 '17 at 15:55
@MeetTaraviya please update your question and tell us more about the specifications. See my comments. — Jabberwocky, Mar 20 '17 at 16:01
what is this `dynamic nature` ? Does the file change while you're reading it? — joop, Mar 20 '17 at 16:31
@joop it is an embedded system. The program runs on forever. You can't save it when the program is complete — Meet Taraviya, Mar 20 '17 at 16:43

score -1 · Accepted Answer · edited May 23 '17 at 12:17

Michael Walz asks "Does the hash have a fixed size? The answer depends heavily on this. If the size is fixed, then it's easy to update the hash directly in the file, otherwise the file must be read and rewritten".

More general, if the records in the file have a fixed length, then you can seek to the record and replace it. If not, you have to spool the file. Assuming fixed length, you can sp[eed up a possible search process if the file has e.g. an order (is sorted) as then you can use binary search to quickly (O(log N)) find the record.

See Klas Lindbach's solution of a basic binary search at Fastest array lookup algorithm in C for embedded system?. The same idea holds for a file (an array of records, but on disk).

There is no order in file – Meet Taraviya Mar 20 '17 at 16:29 — Meet Taraviya, Mar 20 '17 at 16:29

Read specific line from CSV efficiently, C

1 Answers1