3

I have the following script:

#!/bin/bash
for line in $('curl -s https://scans.io/data/rapid7/sonar.http/20141209-http.gz | zcat | head -n 1000 | dap json + select vhost + lines');
do
    echo "$line\n"
done

For which I am trying to achieve the following:

  1. Loop through a dynamically growing list of results from the scans.io curl
  2. Output the line which I then propose to pass to PHP to store and process

However I'm getting syntax error near unexpected token$'\r''` but am not a BASH expert to know what I need to tweak.

Antony
  • 3,875
  • 30
  • 32

1 Answers1

5

Use Process-Substitution, with a while-loop, see why using for-loop for command output parsing is bad. The unsetting of IFS and read with the -r flag does not allow backslashes to escape any characters and treat strings "as-is".

#!/bin/bash

while IFS= read -r line
do
    printf '%s\n' "$line"
done < <(curl -s https://scans.io/data/rapid7/sonar.http/20141209-http.gz | zcat | head -n 1000 | dap json + select vhost + lines)
Inian
  • 80,270
  • 14
  • 142
  • 161
  • Thank you for your answer. I found I could also do it but by piping the command as suggested in http://stackoverflow.com/questions/2711001 – Antony Jan 19 '17 at 12:15
  • @Antony: _Always_ use tools/suggestions available from the native shell, rather than depending on third party tools for the reasons of portability. The approach I suggested is more reliable. – Inian Jan 19 '17 at 12:17
  • With piping being the third party element? – Antony Jan 19 '17 at 12:18
  • @Antony: `pipe` is a communication medium which is a kernel built-in, I meant the usage of `xargs` – Inian Jan 19 '17 at 12:19
  • Ah sorry - was using the 2nd answer. Also will happily mark your answer but unfortunately got `dap.sh: line 2: $'\r': command not found dap.sh: line 6: syntax error near unexpected token `done'`. – Antony Jan 19 '17 at 12:20
  • 1
    @Antony: Are you using `cygwin`, if so can you do `dos2unix dap.sh` to remove Windows `CRLF` endings? and run the script again? – Inian Jan 19 '17 at 12:26