Copying a specific column of a file to another file in C

Question

I have two files, the first of which looks like this

125 6.678
435 9.084
234 8.874

and so on for about 2,048,000 entries, generated by my program. The second file is a file generated by gnuplot and looks a bit like:

65 321456 985
78 98374  834
54 8493   848

and so on also for about 2,048,000 entries.

Now what I need to do is to plot the second column of the first file and the two columns of the second file against each other with gnuplot in 3D. I think the first task is to put them all into the same file and I just want to write a simple c program that quickly reads both files and puts the relevant columns in one file but I'm not sure how to do it. I know how to copy the entire contents of a file and have them written to another file using a c program - for example to do this I have the following code:

#include <stdio.h>
#include <stdlib.h>

int main (int argc, const char * argv[])
{
FILE *avalanche_size_BM;
FILE *avalanche_size_BM_2000;
char ch;


avalanche_size_BM = fopen("/Users/maheensiddiqui/Documents/MSc_Proj/avalanche_size_BM.dat","r");

if (avalanche_size_BM == NULL) 
{
    printf("I couldn't open.\n");
    exit(0);
}
avalanche_size_BM_2000 = fopen("/Users/maheensiddiqui/Desktop/avalanche_size_BM_2000.dat", "w");
if (avalanche_size_BM_2000 == NULL) 
{
    printf("I couldn't open.\n");
    exit(0);
}

printf("\n success!!");

while((ch=getc(avalanche_size_BM))!=EOF)
      putc(ch,avalanche_size_BM_2000);

fclose(avalanche_size_BM);
fclose(avalanche_size_BM_2000);



return(0);
}

But how exactly do I tell it to just read the second column in the first file and the first two columns in the second file and copy them (rather than all columns) to a third file which I can use to plot my 3-D plot.

Any help will be much appreciated!

Thank you

Hi Maheen: Looks like you are using Mac OSX from the looks of it to create the file. Do you NEED to do this in C or you don't care what you use as long as you can generate your 3rd file with 2nd column from the first file and 1st two columns from the second file? — Ahmed Masud, Aug 01 '13 at 01:31
Unless there's a reason you want to write code for this then I suspect you'll be better off using a tool like `awk` to do the job for you. As to how have a look at http://stackoverflow.com/questions/5467690/how-to-merge-two-files-using-awk and http://www.hcs.harvard.edu/~dholland/computers/awk.html — torak, Aug 01 '13 at 01:34
Hi! Nope I don't need to use c but I didn't even know anything like awk existed, I'll give it a go so thanks a lot! — Maheen Siddiqui, Aug 01 '13 at 10:26

score 4 · Accepted Answer · edited Jun 20 '20 at 09:12

NON-C answer

This assumes that you just want the results and aren't really worried about how to do it in C.

To generate the file using simple command-line tools you can use paste and awk to get the results:

paste reads multiple files and combines all the lines together separated by a tab character

so if you have two files like foo.txt and bar.txt that have the following content:

foo.txt:

a1 b1 c1 
a2 b2 c2

bar.txt:

d1 e1 f1
d2 e2 f2

paste foo.txt bar.txt outputs:

a1 b1 c1    d1 e1 f1
a2 b2 c2    d2 e2 f2

You can then use awk to filter out the columns that you want.

So for instance if you want columns, 2, 4 and 5 (b1, d1, e1) you can use the following:

paste foo.txt bar.txt | awk '{ print $2 " " $4 " " $5 }'

I am going to assume that both files have IDENTICAL number of entries Start a command shell in OSX from Finder -> Utilities

at the shell-prompt you can do this:

$ cd Desktop
$ paste avalanche_size_BM.dat avalanche_size_BM_2000.dat | \
    awk '{ print $2 " " $4 " " $5 }' > avalanche_size_3d.dat

This will create a file called avalanche_size_3d.dat in the Desktop subdirectory from column 2 of the first file and first two columns of the second file (as they end up in positions 4 and 5 when pasted together).

C-answer

this is essentially like writing a small paste utility

#include <stdio.h>
#include <stdlib.h>

int main()
{
     FILE *fp1 = fopen ("file1", "r");
     FILE *fp2 = fopen("file2", "r");
     FILE *out = fopen("fileout", "w");

     while (!feof(fp1) && !feof(fp2)) {
         char buf[256], col1[256], col2[256], col3[256];
         if ( fgets(buf, sizeof(buf), fp1) == 0 )
            break;
         sscanf(buf, "%s %s", col2, col1); /* we only need col1, col2 is reused later */
         if ( fgets(buf, sizeof(buf), fp2) == 0 ) 
            break;
         sscanf(buf, "%s %s", col2, col3); 
         fprintf(out, "%s %s %s\n", col1, col2, col3);
     }
     fclose(fp1);
     fclose(fp2);
     fclose(out);
}

Using only `paste` is fine, because `gnuplot` is able itself to select specific columns: `splot "< paste file1 file2" using 2:3:4`. BTW: at the moment a `merge` command is being developed which should do this directly in `gnuplot`. — Christoph, Aug 01 '13 at 07:07
This is exactly what I was looking for thank you so much! Very nicely explained! cheers! — Maheen Siddiqui, Aug 01 '13 at 10:29
@Christoph so true, my brain wasn't even thinking about gnuplot... also, the awk step is a good example for other situations where columnar data is not extractable by target utility — Ahmed Masud, Aug 01 '13 at 14:35

jxh · Answer 2 · 2013-08-01T01:55:19.713

In C, the easiest way is to read one line from each of the two files, and print both lines out as a single line into a third file. Once you do that, you can parse which parts of each line you actually want to print, but merging both files into one is probably enough to get you to use gnuplot to plot your data.

To read in a line of input, you would use fgets(). You can use two separate buffers for each line. From the first line you get from the first file, you remove the newline at the end, then print it to the new file. Then, print the line from the second file.

char buf1[MAX_LINE];
char buf2[MAX_LINE];
FILE *infile1 = fopen(..., "r");
FILE *infile2 = fopen(..., "r");
FILE *outfile = fopen(..., "w");

while (fgets(buf1, sizeof(buf1), infile1) != 0
       && fgets(buf2, sizeof(buf2), infile2) != 0) {
    strchr(buf1, '\n')[0] = '\0';
    fprintf(outfile, "%s %s", buf1, buf2);
}

fclose(outfile);
fclose(infile1);
fclose(infile2);

After you read a line of input, you can parse the input using sscanf() to retrieve the portion of the line you are interested in printing out:

int file1_column1;
double file1_column2;
sscanf(buf1, "%d %lf", &file1_column1, &file1_column2);

int file2_column1;
int file2_column2;
int file2_column3;
sscanf(buf2, "%d %d %d", &file2_column1, &file2_column2, &file2_column3);

You can print out the values using fprintf():

fprintf(outfile, "%f %d %d\n", file1_column2, file2_column1, file2_column3);

Copying a specific column of a file to another file in C

2 Answers2

NON-C answer

C-answer

this is essentially like writing a small paste utility