3

I have columns of data arriving from standard output (in my case a call to mysql), and I would like to append at each loop the column in a file. How can I do?

Standard output:

 a1
 a2
....
 an

Saving in a file called table.dat:

table.dat:

 a1
 a2
....
 an

Then another output is produced:

Further standard output:

 b1
 b2
....
 bn

Appending to table.dat:

table.dat:

 a1   b1
 a2   b2
.... ....
 an   bn

...and so on. I can use paste, but I need three steps:

 line producing standard output > tmpfile;
 paste prevfile tmpfile > table
 mv table prevfile;

Is there a faster way, maybe by using awk?

This solution: Add a new column to the file produces an empty table.

Community
  • 1
  • 1
leonard vertighel
  • 1,058
  • 1
  • 18
  • 37
  • 2
    You can use `paste tablefile <(program)` to skip one temp file. If you have `sponge`, you can add `| sponge tablefile` to do the replacement in-place, otherwise you just need to use a temporary file and rename every iteration. – Kevin Nov 05 '13 at 19:14
  • `[mysql call] | paste table - | sponge table` is exactly what I needed, thaks! Incredible this "`sponge`" in the `moreutils` debian package: I never heard about it! – leonard vertighel Nov 05 '13 at 19:50

2 Answers2

6

You can use paste like this by reading from stdin:

paste <(command1) <(command2)

e.g.

paste <(cat f1) <(cat f2)

instead of:

paste f1 f2
anubhava
  • 761,203
  • 64
  • 569
  • 643
2

Just to clarify some details in the case where the two streams given, don't have the same number of elements. Result with paste as proposed by anubhava :

[ ~]$ cat t1.txt 
a1
a2
a3
[ ~]$ cat t2.txt 
b1
b2

[ ~]$ paste t1.txt t2.txt 
a1  b1
a2  b2
a3

Otherwise, with Bash just for fun :

[ ~]$ cat test.sh 
#!/bin/bash

f1=t1.txt
f2=t2.txt

getNumberOfRows(){
    wc -l "$1"|cut -d" " -f1
}

s1=$(getNumberOfRows "$f1")
s2=$(getNumberOfRows "$f2")
[[ $s1 -le $s2 ]] && min="$s1" || min="$s2"

i=1
while read; do
    echo "$REPLY $(sed -n ${i}p $f2)"
   (( i++ ))
   [[ $i -ge $min ]] && break
done < "$f1"

[ ~]$ ./test.sh 
a1 b1
a2 b2
[ ~]$

In this example you could see that we don't display additional lines if a file is larger than the other.

Of course you could change files by command outputs in either paste or with this script ;)

Using paste :

paste <(COMMAND1) <(COMMAND2)

Using the script : see this answer to see how to read a command output in a loop.

Community
  • 1
  • 1
Idriss Neumann
  • 3,760
  • 2
  • 23
  • 32
  • 2
    For a pure bash solution: `while read -u 4 a && read -u 5 b; do echo "$a $b"; done 4< file1 5< file2` is much more efficient than your repeated calls to `sed`. Enjoy! – gniourf_gniourf Nov 05 '13 at 22:27