Loop through results of sed

Question

I'm having trouble looping through the lines returned by sed that are stored in a variable.

Currently it looks a bit like this

lines=$(sed -rne '/'"$timestamplastupload"'/,/'"$timestampnow"'/ p' /var/log/test.log)

for line in "$lines"; do

        echo "This line is: $line"

done

But so far this isn't working, the loop one runs once and while the contents of the $line variable appears to be the entire $lines variable and thus only printed once not looped through line by line prefixing with "This line is: "

What am I missing to be able to loop through each line of the sed output piece by piece. (Eventually ill be uploading the matching ones line by line to an API) so I need each line separately so I can process them as required.

As requested, some output from sed which has been tided up for security

Wed Feb 28 22:33:11 2018 us=452112 xxx/1.2.3.4:55487 [xxx] Inactivity timeout (--ping-restart), restarting
Wed Feb 28 22:33:11 2018 us=452112 xxx/1.2.3.4:55487 [xxx] Inactivity timeout (--ping-restart), restarting
Wed Feb 28 22:33:11 2018 us=452112 xxx/1.2.3.4:55487 [xxx] Inactivity timeout (--ping-restart), restarting
Wed Feb 28 22:33:11 2018 us=452180 xxx/1.2.3.4:55487 SIGUSR1[soft,ping-restart] received, client-instance restarting
Wed Feb 28 22:33:11 2018 us=452180 xxx/1.2.3.4:55487 SIGUSR1[soft,ping-restart] received, client-instance restarting
Wed Feb 28 22:33:11 2018 us=452180 xxx/1.2.3.4:55487 SIGUSR1[soft,ping-restart] received, client-instance restarting
Wed Feb 28 22:33:11 2018 us=452180 xxx/1.2.3.4:55487 SIGUSR1[soft,ping-restart] received, client-instance restarting

I suggest to use a `while` loop or put `sed`'s output in an array. — Cyrus, Mar 01 '18 at 06:45
Check this post: [Looping through the content of a file in Bash](https://stackoverflow.com/a/41646525/6862601) — codeforester, Mar 01 '18 at 06:51

score 4 · Accepted Answer · answered Mar 01 '18 at 06:54

Don't use for for reading. Use process substitution instead:

while read -r line; do
  # your logic
done < <(sed -rne '/'"$timestamplastupload"'/,/'"$timestampnow"'/ p' /var/log/test.log)

See:

BashFAQ/001 - How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?

score 0 · Answer 2 · answered Mar 01 '18 at 06:31

0

The output of your sed would be useful (because of dealing with endlines), but other than that, it seems that this question provides an answer that would go along these lines:

IFS=' ' read -r -a array <<< "$lines"
for element in "${array[@]}"
do
    echo "$element"
done

answered Mar 01 '18 at 06:31

Maciek

762
6
17

I am beginning to think it might be a case of carriage returns etc but I dont know how to figure out if this is the case or not. After trying your method, that appears splitting on spaces. – LordBoBCUP Mar 01 '18 at 06:41

Loop through results of sed

2 Answers2

Related