0

I am reading a text file with more than 64k lines of integer data placed in the form of columns. What I was doing earlier is reading the file line by line using wxTextInputStream (as it was wxWidgets application) and next from the read line , I was reading each integer using wxStringTokenizer tkz(line, separator).But this method is taking 5 min for reading the file and storing data into vectors. Can you suggest and better way to reduce the time of reading the file line by line and better usage of tokenizer.

 wxFileInputStream input(filepath);
 wxTextInputStream textInput(input, wxT("\x09"), wxConvUTF8);

 while (input.IsOk() && !input.Eof())
{
    line = textInput.ReadLine(); // read a text line
    wxStringTokenizer  tkz(line, separator);
    while (tkz.HasMoreTokens())
    {
        if (lineCount < 2)
        {
            data_set.headers.push_back(string(tkz.GetNextToken().mb_str()));
            columns++;
        }
        else if (lineCount > 2) {

            tkz.GetNextToken().ToDouble(&ver);
            versionNumbers.push_back(ver);
            dataPoints++;
        }
        else tkz.GetNextToken();
    }
    lineCount++;
}
pod
  • 23
  • 7
  • 2
    Byt the way, `while (input.IsOk() && !input.Eof())` is [wrong](https://stackoverflow.com/questions/5431941/why-is-while-feof-file-always-wrong) and you should check if reading is successful between reading and processing. – MikeCAT Jul 22 '20 at 15:12
  • I think yeah as my code is working and application is running fine.My problem was to read the them faster. And any suggestion for the mistake u were pointing out – pod Jul 22 '20 at 15:15

0 Answers0