How to read a growing text file in C++?

Question

I am trying to read from a file which is growing (something similar to what tail -F does), but there must be some problems with my code:

string   log, logFile("test.log");
size_t   p = 0;

while(true)
{
    ifstream ifs(logFile.c_str());

    ifs.seekg(p);  //*1

    while(ifs.eof() == false)
    {
        getline(ifs, log);

        cout << log << endl;

        p = ifs.tellg();  //*2
    }

    nanosleep(&pause, NULL);
}

Without the lines //*1 and //*2, the log file is correctly read up to its end, but if new lines are added nothing happens.

With seekg and tellg I am trying to store the current end position of the file, so that when I reopen it I can go strait there and read what has been added.

I would like to know what is wrong in my code, and if it is really necessary to close and reopen the same file for this purpose.

Thank you.

hmjd · Answer 1 · 2012-08-01T12:38:39.200

The loop is incorrect as when eof() is encountered tellg() returns -1 and there is no check for eof() immediately after the call to getline() which there needs to be. Change loop to:

while (getline(ifs, log))
{
    cout << log << endl;
    p = ifs.tellg();
}

Additionally, as p is declared as a size_t when tellg() return -1 the value of p was being set to4294967295. This meant the seekg() was being set to beyond the end of the file. Change the type of p to std::streamoff and confirm the call to seekg() was successful:

if (ifs.seekg(p))
{
    while (getline(ifs, log))
    {
        cout << log << endl;
        p = ifs.tellg();
    }
}

if it is really necessary to close and reopen the same file for this purpose.

No, it is not necessary but you need to clear() the eof state from the stream. The following is an alternative to a corrected version of the posted code:

#include <iostream>
#include <string>
#include <fstream>

int main()
{
    std::ifstream ifs("test.log");

    if (ifs.is_open())
    {
        std::string line;
        while (true)
        {
            while (std::getline(ifs, line)) std::cout << line << "\n";
            if (!ifs.eof()) break; // Ensure end of read was EOF.
            ifs.clear();

            // You may want a sleep in here to avoid
            // being a CPU hog.
        }
    }

    return 0;
}

I noticed that if I create the ifstream object into the loop it works, but if I create it outside it does not. Is it necessary to close and reopen a file to read what has been added to it? — Pietro, Aug 01 '12 at 11:07
@Pietro, you need to clear the eof state of the `ifstream` by calling `ifs.clear()` prior to the next read attempt. I _think_ the `tellg()` and `seekg()` will be unnecessary with this approach. — hmjd, Aug 01 '12 at 11:14
@hmjd, You are the man! I had to go to hell and back to find this simple answer to a simple problem. I haven't seen anything even remotely as clear as this. — progician, Feb 07 '13 at 17:16
@hmjd: I just tried your last piece of code, and when I add new lines in the input test.log file, they are not picked. — Pietro, Nov 03 '15 at 11:19

score 3 · Accepted Answer · answered Feb 07 '21 at 05:27

Since none of these answers worked, i came up with one that does work...

#include <iostream>
#include <string>
#include <fstream>

using namespace std;

int main()
{
    string   log, logFile("test.txt");
    std::streamoff   p = 0;
    ifstream ifs(logFile.c_str());

    while(true)
    {

        ifs.seekg(p);  //*1
        while (getline(ifs, log))
        {
            cout << log << endl;
            if(ifs.tellg() == -1) p = p + log.size();
            else p = ifs.tellg();
        }
        ifs.clear();

    }
}

score 2 · Answer 3 · answered Aug 13 '15 at 00:21

This method has worked faithfully for me:

#include <string>
#include <chrono>
#include <thread>
#include <fstream>
#include <iostream>

int main(int, char* argv[])
{
    // open file passed in on command line (at end of file)
    std::ifstream ifs(argv[1], std::ios::ate);

    if(!ifs.is_open())
    {
        std::cerr << "ERROR: opening log file: " << argv[1] << '\n';
        return 1;
    }

    // remember file position
    std::ios::streampos gpos = ifs.tellg();

    std::string line;
    bool done = false;

    while(!done)
    {
        // try to read line
        if(!std::getline(ifs, line) || ifs.eof())
        {
            // if we fail, clear stream, return to beginning of line
            ifs.clear();
            ifs.seekg(gpos);

            // and wait to try again
            std::this_thread::sleep_for(std::chrono::milliseconds(100));
            continue;
        }

        // remember the position of the next line in case
        // the next read fails
        gpos = ifs.tellg();

        // process line here
        std::cout << "line: " << line << std::endl;
    }
}

score 0 · Answer 4 · answered Nov 03 '15 at 14:42

This code works for me:

struct timespec pause;
pause.tv_sec  = 1;
pause.tv_nsec = 0;

std::ifstream ifs("test.log");
std::streamoff p;

if(ifs.is_open())
{
    std::string line;

    while(true)
    {
        if(ifs.seekg(p))
        {
            while(std::getline(ifs, line))
            {
                std::cout << line << std::endl;
                p = ifs.tellg();
            }
        }

        ifs.clear();

        nanosleep(&pause, NULL);
    }
}

How to read a growing text file in C++?

4 Answers4

Linked

Related