Executing line counting bash command (wc -l) inside a c++ program

Question

I want to calculate the no of lines of data file before declaring a global array. I know in terminal if I execute wc -l filename, I get the number of lines. How do I use that command (wc -l filename) and store its result into a int type variable inside a c++ program? When I use the following code: int NT=system(" wc -l file.d"); I get NT=0.

How to get the no. of lines in file.d in NT?

Why use add a dependency against an external program for a task that you can do yourself in three lines of C++ code? — Matteo Italia, Sep 15 '16 at 19:19
No, It would be problematic for me. I need to declare an 3d global array, say A[n][3][3], where n would be the no. of lines in that data file. To declare the array globally, I need to know the value of n beforehand. — deltasata, Sep 15 '16 at 19:22
@SatadruBag ‘Beforehand’ as in before compiling? This smells like an [XY problem](http://mywiki.wooledge.org/XyProblem). — Biffen, Sep 15 '16 at 19:24
Google "how to count number of lines in c++" and use that instead. — Matt, Sep 15 '16 at 19:26
Probably [`popen`](http://stackoverflow.com/q/10938130/315052) will suffice. — jxh, Sep 15 '16 at 19:27
@SatarduBag: besides the fact that using `std::vector` having the array grow while you are filling it is a non-issue, what I'm saying is that you can count the lines (without actually parsing them) *before* allocating the array. By the way, global arrays are statically allocated anyway (you cannot decide their size at runtime depending from the output of `wc`), so I don't see what you hope to so here. — Matteo Italia, Sep 15 '16 at 19:28
Matteo Italia: yes, I forgot that even if I know no. of lines beforehand, I can not declare a global 3d array. Can I do it using a pointer? Can you help? — deltasata, Sep 15 '16 at 19:44
@SatadruBag: make a `struct` for each row (something like `struct DataPoint { int d[3][3]; }`); then for each line you read fill one instance of such structure, and push it into an `std::vector`. — Matteo Italia, Sep 15 '16 at 20:08

score 2 · Answer 1 · answered Sep 15 '16 at 20:31

What should be a fairly efficient solution, though it's not portable outside of POSIX systems (read: You'd have to rewrite to use WinAPI equivalent calls on Windows), as it avoids the need to actually construct objects, perform explicit reads for each line, etc. Aside from the mmap work (much of which can be factored out at least), it's basically a one-liner.

I don't really recommend this in general (your problem should be solved by just using std::vector or the like so your data structure can grow dynamically to match the number of lines), but I'm putting it here if you're curious.

#include <algorithm>

#include <unistd.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

const char* map_whole_file(const char *filename, size_t& map_size);

int main(...) {
    const char *filename = ...;  // From argv, constant, whatever
    size_t map_size;

    const char *data = map_whole_file(filename, map_size);

    // Number of lines is count of newline characters, +1 if final line not
    // terminated with newline
    size_t numlines = std::count(data, data+map_size, '\n') + (data[map_size-1] != '\n');

    munmap(data, map_size);
}

const char* map_whole_file(const char *filename, size_t& map_size) {
    int fd = open(filename, O_RDONLY);
    if (fd == -1)
        ...handle_error...;

    struct stat sb;
    if (fstat(fd, &sb) == -1)           /* To obtain file size */
        ...handle_error...;

    // Cast only needed because it's C++
    const char *data = (const char *)mmap(NULL, sb.st_size, PROT_READ, MAP_SHARED, fd, 0);
    if (data == MAP_FAILED)
        ...handle_error...;

    close(fd); // Don't need fd after mapping done

    // Optionally, posix_madvise(data, sb.st_size, POSIX_MADV_WILLNEED);
    // or the like will hint the OS to aggressively page in the file, so
    // count is less likely to be delayed by disk I/O

    map_size = sb.st_size;
    return data;
}

vincent · Accepted Answer · 2016-09-15T21:43:36.780

One way you can get the number of lines in a file is by using getline:

#include <fstream>
#include <iostream>
using namespace std;

int main()
{
    fstream in("file.d");
    int lines = 0;
    string s;
    while (getline(in, s))
    {
        ++lines;
    }
    cout << lines << endl;
    return 0;
}

EDIT...

A more efficient way (if you only want the count and don't want to use any of the data) is to use istream::ignore, as pointed out by Matteo Italia.

#include <fstream>
#include <iostream>
#include <limits>
using namespace std;

int main()
{
    fstream in("in.txt");
    int lines = 0;
    char endline_char = '\n';
    while (in.ignore(numeric_limits<streamsize>::max(), in.widen(endline_char)))
    {
        ++lines;
    }
    cout << lines << endl;
    return 0;
}

Either way will output the number of lines in a file called "file.d". You can compare its output to that of 'wc -l'.

This does solve the problem, but you are copying all those lines for no reason. Use `fstream::ignore()` instead. — Matteo Italia, Sep 15 '16 at 20:06

Executing line counting bash command (wc -l) inside a c++ program

2 Answers2