1

I have a grep -o '100%' piped after a wget, so I'm only notified that wget completed the download, without useless information on screen. But each 100% is printed on a new line, and I don't like this flood! So, is there a way to tr the newlines into spaces, by piping tr after grep? All I got was no result, each time I tried...

2 Answers2

2

You would have gotten output eventually, but all in one go. This is due to buffering.

GNU grep has a --line-buffered to avoid output buffering. tr does input buffering with no apparent way to turn it off, so you can use awk instead:

wget -r -l 1 http://stackoverflow.com/ 2>&1 |
    grep --line-buffered -o '100%' | awk '{printf("%s ", $0);}'
100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%
that other guy
  • 116,971
  • 11
  • 170
  • 194
  • "You would have gotten output eventually, but all in one go." True, I confirmed now. I have not waited long because there were a lot of files to download, and it's very annoying to look at a screen that has no information about what is happening! –  Aug 09 '16 at 22:11
  • Are you sure that `--line-buffered` _disables_ buffering? Both `man` and `info` says otherwise: "Use line buffering on output.". By the way, using it or not gives the same result: all `100%` only at the end. –  Aug 09 '16 at 22:29
  • Are you trying the command as provided here or did you just add `--line-buffered` to your existing command? – that other guy Aug 09 '16 at 22:45
  • I followed your instructions, but only after the `wget`. So I maintained my `wget` (not recursive, input from file) and added what you suggested. Anyway, now I tested exactly what you suggested, except without the line break between `wget` and `grep`, and there was no `100%` until I aborted the process for taking too long without any output, and noticing that the download stopped at 18 items on the folder. –  Aug 10 '16 at 00:03
  • That's pretty strange and I can't reproduce it. How about if you use `stdbuf -oL wget ... `? – that other guy Aug 10 '16 at 02:22
  • Indeed, it's strange. As even with you `stdbuf`suggestion it still cannot work, I must ask: on which version of `wget` your solution worked? –  Aug 10 '16 at 05:41
0

It's buffering, see Why no output is shown when using grep twice?. Just use awk:

$ wget -r -l 1 https://stackoverflow.com/ 2>&1 | awk 'match($0,/100%/){printf "%s ", substr($0,RSTART,RLENGTH)}'
100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%

or probably more useful:

$ wget -r -l 1 https://stackoverflow.com/ 2>&1 | awk '/100%/{printf "."}'
...........
Community
  • 1
  • 1
Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • I copied and executed the commands as they were written by you. Both downloaded 18 items, gave no output, and needed to be canceled (CTRL+C) to stop executing. I think, with this repeating to more than 1 solution of more than 1 replier, that other things needs to be checked. On which version of `wget` your solution worked? –  Aug 10 '16 at 05:00
  • GNU Wget 1.16.3 built on cygwin. – Ed Morton Aug 10 '16 at 05:36
  • I was on 1.17.1, so I updated to 1.18 and tried again. Same non-results... So it's not `wget` the culprit. Maybe `awk`? Here it's being caled mawk by man and info. –  Aug 10 '16 at 21:01
  • Only suggestion I have left is to install gawk, sorry. – Ed Morton Aug 10 '16 at 22:02