28

I need to process the shared library dependencies of a library from a bash script. The for command processes word-by-word:

for DEPENDENCY in `otool -L MyApplication | sed 1d`
do
    ...
done

What is the way to process the results line-by-line?

Jens
  • 69,818
  • 15
  • 125
  • 179
Dave Mateer
  • 17,608
  • 15
  • 96
  • 149

7 Answers7

39

You should use the read command.

otool -L MyApplication | sed 1d | \
while read i
do
  echo "line: " $i
done

See bashref for a description of the read builtin, and its options. You can also have a look at the following tutorial for examples of using read in conjunction with for.

tonio
  • 10,355
  • 2
  • 46
  • 60
  • 11
    Warning: if you need to set variables in the while loop (i.e. store what you found in otool's output), they'll vanish as soon as the loop exits because it's part of a pipeline and therefore runs in a subshell. In bash, you can get around this with a bit of rearranging: `while read i; do ... done < <(otool -L MyApplication | sed 1d)` – Gordon Davisson Jun 24 '10 at 22:37
  • 2
    Another warning: This will delete a lot of spaces and backslashes. Use `read -r i` to read lines without such modifications. – Jens May 24 '12 at 21:04
  • @Jens, no, it's the echo command eating spaces. `echo "line: $i"` will prevent that. – James Morris Jul 14 '13 at 22:27
  • @JamesMorris No, it's the read that eats leading and trailing whitespace and backslashes. Try `printf ' foo\\bar \n'|read a; echo "<$a>"` and then with `read -r`. – Jens Jul 16 '13 at 19:03
  • @Jens, read the man page, nowhere does it mention whitespace. the `-r` option simply prevents special treatment of backslashes. – James Morris Jul 17 '13 at 01:12
  • Why is it that I have exclude the first line with this answer? (sed 1d). I'm noticing if I exclude '1d' it spits back the sed help page (command fails to run). – jersey bean Aug 23 '17 at 01:11
  • @jersey-bean `otool` output consists in a first line with the name of the library followed by `:`. Then you have the dependencies, one per line.So we strip the first line since the original question was about iterating over dependencies. – tonio Aug 23 '17 at 07:50
  • 1
    @Jens, ...it's the act of clearing `IFS` in the best-practice invocation of `IFS= read -r i` that eliminates leading and trailing whitespace pruning. – Charles Duffy Oct 06 '17 at 20:01
  • @JamesMorris, jens *was* actually correct that leading and trailing whitespace gets pruned, just incorrect about what to do about it. This is covered in more detail in [BashFAQ #1](http://mywiki.wooledge.org/BashFAQ/001). – Charles Duffy Oct 06 '17 at 20:02
  • And in bash, the test demonstrating all these differences needs to be more like `printf ' foo\\bar \n' | { IFS= read -r a; echo "<$a>"; }`; without explicit grouping, the `echo` doesn't run in the same subshell that was spawned as part of the pipeline ([BashFAQ #24](http://mywiki.wooledge.org/BashFAQ/024)). – Charles Duffy Oct 06 '17 at 20:04
  • BTW, `echo "line: " $i` would be less buggy if written as `echo "line: $i"`, or even `echo line: "$i"`. It's important to quote the parameter expansion; it's not important to quote the constant string (that doesn't have any content that might cause it to be incorrectly interpreted as shell syntax). – Charles Duffy Jan 26 '23 at 16:23
  • 1
    Also, it'd be a slight efficiency improvement to replace `sed 1d | ...` with `{ read _; while IFS= read -r i; do echo "line: $i"; done; }` -- that way we don't have an extra executable in our pipeline needing to be started up and then loop over the whole data stream, but just have bash itself read the header into `_`, thus throwing it away, before diving into the loop. – Charles Duffy Jan 26 '23 at 16:25
10
otool -L MyApplication | sed 1d | while read line ;
do
  # do stuff with $line
done
Jim Lewis
  • 43,505
  • 7
  • 82
  • 96
  • Should be `while IFS= read -r line` to avoid munging leading/trailing whitespace or backslashes. See [BashFAQ #1](https://mywiki.wooledge.org/BashFAQ/001). – Charles Duffy Jan 26 '23 at 16:27
3

Try changing the Internal Field Separator to a newline. By default, bash is going to separate tokens by spaces when using a for loop. Your IFS setting will make the for loop split up the tokens based on whatever string IFS is equal to (thus splitting up the list tokens by newlines instead of by tokens by spaces).

[bash ] $ IFS="
"
[bash ] $ for DEPENDENCY in `otool -L MyApplication | sed 1d`
do
    ....
done
aoeu
  • 1,128
  • 2
  • 13
  • 22
2

You have to use shell builtin read, but be careful with lines containing spaces and tabs. I suggest locally change value of $IFS:

 otool -L MyApplication | sed 1d | \
 while IFS= read i; do
     echo "line: $i"
 done

[Edit: applied Charles' suggestion]

Jérôme Pouiller
  • 9,249
  • 5
  • 39
  • 47
  • You're currently changing `IFS` only for the `read`, but leaving it at its prior/default value for the unquoted expansion. Move `$i` in the `echo` into quotes (`echo "line: $i"`) to avoid string-splitting and glob expansion. See also [I just assigned a variable, but `echo $variable` shows something else!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) – Charles Duffy Jan 26 '23 at 16:27
1

The modern bash way:

while read i; do
    ...
done < <(otool -L MyApplication | sed 1d)

Note: the two "<"s above is not a typo, that's the correct syntax.

Ron HD
  • 161
  • 9
  • 2
    `while IFS= read -r i; do` -- if you don't clear IFS, leading and trailing whitespace is removed; if you don't use `-r`, backslashes are interpreted. – Charles Duffy Jan 26 '23 at 16:29
0

You can use awk to process things on a per-line basis. Exactly what's the best way depends on what you're trying to do though.

Daniel Egeberg
  • 8,359
  • 31
  • 44
0

you can do it awk as well. No need to use a bash for loop

otool -L MyApplication | awk 'NR>1
{
  # do your stuff here since awk process line by line.
}'
ghostdog74
  • 327,991
  • 56
  • 259
  • 343