how to use schell script to read element from a file, do some calculation and write back?

Question

I am just new to the script language.

Now I have a file, inside it are:

>   A1 B1 C1
>   A2 B2 C2
>   A3 B3 C3

I just want to use the shell script(bash) to read the file element by element. Then I want to do some calculation of the element A1, A2 and A3 and then write them back to a new file(or the old file). So the new file would be (supposed the calculation results are D1,D2 and D3):

 D1 B1 C1
 D2 B2 C2
 D3 B3 C3

The calculation is to convert the Unix epoch time(A's value) into human readable time(D's value) by command "date -d @(A's value)".

I try to use the awk command:

awk '{$1=`date -d @$1`}' test.txt

But it seems to have some syntax error:> The error is :

awk: {$1=`date -d @$3`}
awk:     ^ invalid char '`' in expression

Kent · Accepted Answer · 2013-01-30T20:54:46.440

2

your requirement is so unclear! what calculation did you mean?

I could show you an example, hope it helps:

kent$  cat test.txt
100 B1 C1
200 B2 C2
300 B3 C3

# in this example, the "calculation" is getting the squre of A1
kent$  awk '{$1*=$1}1' test.txt
10000 B1 C1
40000 B2 C2
90000 B3 C3

#If you want to get the result in a newfile.txt, do this:

kent$  awk '{$1*=$1}1' test.txt >newfile.txt

#then

kent$  cat newfile.txt 
10000 B1 C1
40000 B2 C2
90000 B3 C3

Edit

here I could give you an example how to invoke date in awk:

kent$  echo "1359579362 B1 C1"|awk '{"date -d @"$1|getline $1}1'                                                                                                            
Wed Jan 30 21:56:02 CET 2013 B1 C1

I guess that is what you are looking for.

good luck.

edited Jan 30 '13 at 20:54

answered Jan 30 '13 at 20:29

Kent

189,393
32
233
301

Thank you for your reply. I also just found the "awk". But I have another issue now. My calculation is "date -d @" which is used to convert the Unix epoch time into human readable time format. So I try to use awk '{$1= `date -d @$1` }' test.txt. It seems to have some syntax error: – Hao Shen Jan 30 '13 at 20:36
it is absolutely doable. but you have to write the requirement in the question, better with example input and expected output. – Kent Jan 30 '13 at 20:45
@HaoShen should be easy job for awk. edit your question to make it correct. – Kent Jan 30 '13 at 20:46
and a hint, gawk has `strftime([format [, timestamp[, utc-flag]]])` check the man page pls. – Kent Jan 30 '13 at 20:48
I have updated the question. Your way definitely works. There is only some syntax error left since I am not familiar with the scripting language. – Hao Shen Jan 30 '13 at 20:52
It works. But now I get another error "awk: (FILENAME=result.txt FNR=1020) fatal: cannot open pipe `date -d @1266407101' (Too many open files)". It seems that my file is too big.Anyway, I will accept your answer. Thanks:> – Hao Shen Jan 30 '13 at 21:09
No, it's not that your file is too big, it's that the script is opening a pipe on every line and never closing any of them. You can get round that by adding `close("date -d @"$1)` after each getline but to be honest you're wasting your time using awk for this if all you're doing is calling a shell command from it. – Ed Morton Jan 31 '13 at 01:27

score 1 · Answer 2 · answered Jan 31 '13 at 01:31

1

If you need to use UNIX date on this instead of gawks builtin time functions, then just do it all in shell:

while read -r secs rest
do
   echo "$(date -d @"$secs") $rest"
done < test.txt

Use gawk if possible though.

answered Jan 31 '13 at 01:31

Ed Morton

188,023
17
78
185

how to use schell script to read element from a file, do some calculation and write back?

2 Answers2

Linked