Piping a data file with cat to a C++ program

Question

I'm looking for some help piping a file (16-bit signed little endian integers raw data) from the command line to my program:

cat data.dat | myprogram

Then it should convert the data to 16 bit signed integers. It works for the first 12 values. The 13th value is wrong, followed by zeros.

The second problem is that the program seems to enter the while loop only once.

I am using Windows + MinGW.

My Code:

#include <iostream>
using namespace std;

#define DEFAULT_BUF_LENGTH (16 * 16384)

int main(int argc, char* argv[]) 
{
    char buf[DEFAULT_BUF_LENGTH];

    while(cin >> buf) 
    {
        int16_t* data = (int16_t*) buf; //to int

        for(int i=0;i<18;i++)
        {
            cout << data[i] << endl;
        }
    }
    return 0;
}

Output:

Thanks for your help!

possible duplicate of [Piping for input/output](http://stackoverflow.com/questions/17508626/piping-for-input-output) — Jared Burrows, Jul 19 '15 at 19:24
Check [this](http://stackoverflow.com/a/1599093/4074081) discussion. On Windows stdin is opened in text mode and certain byte indicates end of file, the solution is to reopen it in binary mode. — dewaffled, Jul 19 '15 at 20:39
You can't use `>>` it converts line endings. You have to use **unformatted input** functions like `cin.read()`. http://en.cppreference.com/w/cpp/io/basic_istream — Galik, Jul 19 '15 at 20:54

score 5 · Accepted Answer · answered Jul 19 '15 at 20:53

5

You might try to use read() instead of the >> operator, which is generally for formatted input. Also checking how much data was actually read is useful:

#include <iostream>
using namespace std;

#define DEFAULT_BUF_LENGTH (16 * 16384)

int main(int argc, char* argv[]) 
{
    char buf[DEFAULT_BUF_LENGTH];

    for(;;) 
    {
        cin.read(buf, sizeof(buf));
        int size = cin.gcount();
        if (size == 0) break;

        int16_t* data = (int16_t*) buf; //to int

        for(int i=0;i<size/sizeof(int16_t);i++)
        {
            cout << hex << data[i] << endl;
        }
    }
    return 0;
}

answered Jul 19 '15 at 20:53

Alexander Balabin

2,055
11
13

The second problem is still be there, the program only prints the first 12 values. I forgot to mention that I compile my programs with `g++ myprogram.cpp -static -o myprogram`. Perhaps this is the reason? If I compile my program without the -static flag it crashes after execution (APPCRASH: Error Module Name: libstdc++-6.dll). I found no solution for this problem, but I think it has something to do with 64bit Windows. On Linux your code works as expected. – fuxT Jul 20 '15 at 21:01
Would be useful to have your input file – Alexander Balabin Jul 20 '15 at 21:06
Even a simple helloworld.cpp program crashes if I do not set the `-static` flag. I think my MinGW is broken. My Problem is similar to that one: [http://stackoverflow.com/questions/18157464/c-helloworld-program-compiled-with-mingw-crashes-with-illegal-argument] but there is no `libstdc++.dll ` in `windows\syswow64` which I could delete... – fuxT Jul 20 '15 at 21:27
I uploaded a test file for you: http://www.filedropper.com/rawdata do you also use MinGW + 64bit Windows? – fuxT Jul 21 '15 at 17:19

nsilent22 · Answer 2 · 2015-07-19T21:03:45.773

1

Statement cin >> buf does not fill the whole buf with data. It reads only next "set" of non-whitespace characters. Change cin >> buf to read(0, buf, sizeof(buf)) > 0

If you insist on using C++ streams, change beginning of your loop to:

while (!cin.eof()) {
    cin.read(buf, sizeof(buf));
    [...]

edited Jul 19 '15 at 21:03

answered Jul 19 '15 at 20:50

nsilent22

2,763
10
14

1

I agree with the diagnosis; that is the problem. I'm not convinced the suggested remedy is the best option. There are, surely, ways to read binary data from `cin` rather than falling back on POSIX file descriptor functions like `read()`, which would require the `` header too. – Jonathan Leffler Jul 19 '15 at 20:54
@JonathanLeffler: Well, sometimes using ol' good C is simpler ;) – nsilent22 Jul 19 '15 at 21:05

Piping a data file with cat to a C++ program

2 Answers2