-3

I am trying to write a C program in Linux system which the main function is read a data file (csv format 200MB file) in struct array and searching condition file (few lines) then output the matching result.

The read data function takes around 1 second to run and the matching part is pretty quick. I am thinking is that possible I can pre read the data file in memory by some methods then run the searching function for many time as I want.

It maybe similar to R. Read a csv file first then do some calculate from it.

Alan Yu
  • 85
  • 10
  • 2
    Yes, it is possible. – davir Nov 30 '15 at 11:24
  • 2
    What is your question? – fuz Nov 30 '15 at 11:39
  • Unless your data file is *extremely* large, what you are suggesting is rather the default. Were you thinking of reading and parsing the entire file for each separate operation? – Jongware Nov 30 '15 at 11:41
  • 1
    What operating system? What file system? What file size (megabytes or petabytes)? What computer, and what disk? How do you read the file? How did you compile your code, and with which compiler? Show some of your code, so **edit your question** to improve it! – Basile Starynkevitch Nov 30 '15 at 11:47

1 Answers1

1

Create a tokenizer using read system call to read until you hit the comma and then update up to that part to your struct using memcpy or strncpy. After that it would be easy for searching and validation.

Marco Bonelli
  • 63,369
  • 21
  • 118
  • 128
S.I.J
  • 979
  • 1
  • 10
  • 22
  • Maybe his OS don't have any `read` *syscall* (e.g. on Windows). And maybe he could use `mmap`; see [this](http://stackoverflow.com/a/33984588/841108) – Basile Starynkevitch Nov 30 '15 at 11:50