Add timestamp to the end of each line of a CSV file

Question

I am trying to get the first 5 lines of top command using shell script. Here is the output of my shell script.

PID,USER,PR,NI,VIRT,RES,SHR,S,%CPU,%MEM,TIME+,COMMAND
3983,arun,20,0,1874616,500356,24132,R,47.6,10.3,53:19.92,gnome-shell
11,root,20,0,0,0,0,R,4.8,0.0,0:22.51,rcuos/0
2016,root,20,0,219396,44408,9120,S,4.8,0.9,9:53.96,Xorg
12081,arun,20,0,2689552,304132,7076,S,4.8,6.3,13:36.17,java
1,root,20,0,141660,5336,2940,S,0.0,0.1,0:08.40,systemd
3983,arun,20,0,1874616,500356,24132,R,56.2,10.3,53:20.99,gnome-shell
2016,root,20,0,219396,44408,9120,R,18.8,0.9,9:54.17,Xorg
4969,arun,20,0,1287084,162948,48472,S,12.5,3.4,5:39.99,chrome
16108,arun,20,0,130020,1692,1172,R,6.2,0.0,0:00.01,top
1,root,20,0,141660,5336,2940,S,0.0,0.1,0:08.40,systemd
3983,arun,20,0,1874616,500356,24132,R,43.8,10.3,53:24.56,gnome-shell
4969,arun,20,0,1275992,147104,32424,S,37.5,3.0,5:41.10,chrome
1,root,20,0,141660,5336,2940,S,0.0,0.1,0:08.40,systemd
2,root,20,0,0,0,0,S,0.0,0.0,0:00.06,kthreadd
3,root,20,0,0,0,0,S,0.0,0.0,0:04.93,ksoftirqd/0

Here is my shell script top.sh

 #!/bin/bash
echo "started.."
top -b -n 3 | sed -n '7,1{s/^ *//;s/ *$//;s/  */,/gp;};12q' >> out.txt

while [ true ]; do
  sleep 5
  echo "inside loop.."
  top -b -n 3 | sed -n '8,12{s/^ *//;s/ *$//;s/  */,/gp;};12q' >> out.txt
done

Now , I need to add timestamp at the end of each lines . I have tried to do the same using awk command , But it didn't work for me ! I have edited my code like this -

#!/bin/bash
    echo "started.."
    t= =$(date +"%T")

    top -b -n 3 | sed -n '7,1{s/^ *//;s/ *$//;s/  */,/gp;};12q' >> out.txt
    awk -F, '{$(NF+1)="TIME";}1' OFS=, out.txt    

    while [ true ]; do
      sleep 5
      t=$(date +"%T")
      echo "inside loop.."
      top -b -n 3 | sed -n '8,12{s/^ *//;s/ *$//;s/  */,/gp;};12q' >> out.txt
      awk -F, '{$(NF+1)=$t;}1' OFS=, file
    done

`t= =$(date +"%T")` on line 3 is a syntax error. There should be a single equals sign with no whitespace on either side. You'll want to run your scripts through http://shellcheck.net/ before posting here. — tripleee, Jan 28 '16 at 06:01
The `while` syntax problem was pointed out already in [your previous question](http://stackoverflow.com/questions/34992167/how-to-capture-unix-top-command-output-to-a-csv-file#comment57712931_34992167). — tripleee, Jan 28 '16 at 06:06
@tripleee Thanks a lot for your suggestions ! I have already edited my code last time according to your corrections . Here I just copied it from from my previous question, forgot to edit the while syntax ! — Arun Ramachandran, Jan 28 '16 at 06:42

tripleee · Accepted Answer · 2016-01-28T06:40:24.100

1

Your Awk commands would run over the entire file (or an entire unrelated or unexistent file in the second instance) and print to standard output. Fixing the command to overwrite the input file would add time stamps to lines which already have them. You want to inline the datestamp addition to the command which appends new data to the file, instead of adding more datestamps to the entire file, anyway.

The sed script already performs a number of substitutions. It's simple to factor in the timestamp as well. You'll need to use double quotes instead of single in order for the shell to expand the value of the variable.

#!/bin/bash

t=$(date +"%T")   #### Syntax error fixed
top -b -n 3 |
sed -n -e '7!b;s/^ *//' -e "s/ *$/,$t/" -e 's/  */,/gp;q' >> out.txt
#### ^sed script refactored; only work on line 7, not until 12

while true; do   #### Wacky syntax fixed
  sleep 5
  t=$(date +"%T")
  top -b -n 3 |
  sed -n -e '8,12{s/^ *//' -e "s/ *$/,$t/" -e 's/  */,/gp;};12q' >> out.txt
done

edited Jan 28 '16 at 06:40

answered Jan 28 '16 at 06:05

tripleee

175,061
34
275
318

I took out the `echo` commands because they severely impede the usability of your script. If you want to check on the script's progress, run something like `tail -f out.txt` somewhere. – tripleee Jan 28 '16 at 06:32
Thanks for the solution ! – Arun Ramachandran Jan 28 '16 at 07:05
No problem. You should probably understand the `sed` script better so that you can refactor it so you don't have two slightly different instances to maintain, but I'll wait for your next question ... – tripleee Jan 28 '16 at 07:38

score 0 · Answer 2 · answered Jan 28 '16 at 05:54

0

In this line:

awk -F, '{$(NF+1)=$t;}1' OFS=, file

you are using a shell variable $t, inside single quotes, so it doesn't get expanded. Use the -v syntax to pass shell variables to awk:

awk -F, -v OFS=, -v t="$t" '{$(NF+1)=t;}1'  file

answered Jan 28 '16 at 05:54

user000001

32,226
12
81
108

Thanks for the solution ! Getting exact output in terminal , but it's not getting saved in file ! :( – Arun Ramachandran Jan 28 '16 at 06:25

Add timestamp to the end of each line of a CSV file

2 Answers2