Bash adding results to array with +=

Question

What I'm trying to do: I'm passing python file names to my bash script and trying to find all methods. I want to sort them and display them in the following fashion.

function_name file_name:line_number

Where the results are sorted by file_name.

The short version: I'm seeing some unexpected behavior from using += to add elements to my array. It looks like the last grep match from my first file and the first grep match from the second file get stuck together. Can someone help me figure out why?

The long version:

My code:

#!/bin/bash/
for i in $*
do

    IFS=','
    #result+=($(grep -n -o -P '(?<=def).*(?=\()' $i ) )
    result+=( $(grep -n -o -P '(?<=def).*(?=\()' $i | sort -t\: -k2 | tr '\n' ',' | sed 's/.$//') )
    unset IFS
done

This the results are:

34: div 39: dot 7: drop 24: minus 19: plus 3: push 15: rot 11: swap 29: times 1: one 7: three 4: two

I want:

34: div 39: dot 7: drop 24: minus 1: one 19: plus 3: push 15: rot 11: swap 29: times 7: three 4: two

After looking at this, I'm guessing the grep only performs the sort on the information passed to it and not on the entire results array. That makes me believe I need to store all the results from the grep in an array, then sort the array based on file name. This is were I am having all sorts (the puns!) of problems.

I've tried:

#!/bin/bash/

for i in $*
do

    IFS=','
    result+=($(grep -n -o -P '(?<=def).*(?=\()' $i ) )
    #result+=( $(grep -n -o -P '(?<=def).*(?=\()' $i | sort -t\: -k2 | tr '\n' ',' | sed 's/.$//') )
    unset IFS
done
#echo ${result[*]}
#for a in "${result[*]}"
#do
#       echo $a
#done
IFS=','
cheese=($(sort -t\: -k2 <<< "${result[*]}"))
unset IFS 
echo ${cheese[*]}

This is much closer, but it looks like the last grep from the first file and first grep from the second file get stuck together. Output from the echo loop:

34: div
39: dot 1: one
7: drop
24: minus  
19: plus
3: push
15: rot 
11: swap
7: three
29: times
4: two

My output from sorting was correct if the dot and one functions were stuck together:

34: div 39: dot 1: one 7: drop 24: minus 19: plus 3: push 15: rot 11: swap 7: three 29: times 4: two

Input file1:

def push(n):
    global stack
    stack = [n] + stack

def drop():
    global stack
    stack.pop(0)

def swap():
    global stack
    stack[0], stack[1] = stack[1], stack[0]

def rot():
    global stack
    stack[0], stack[2] = stack[2], stack[0]

def plus():
    global stack
    num = stack[0] + stack[1]
    stack[:2] = [num]

def minus():
    global stack
    num = stack[1] - stack[0]
    stack[:2] = [num]

def times():
    global stack
    num = stack[0] * stack[1]
    stack[:2] = [num]

def div():
    global stack
    num = stack[1] / stack[0]
    stack[:2] = num

def dot():
    global stack
    print stack.pop(0)

Input file2:

def one():
    pass

def two():
    pass

def three():
    pass

----------UPDATE----------

Here is my final solution. The response below was spot on.

 #!/bin/bash

 declare -a result
 for i in "$@"
 do
while read line; # Read the input from the grep command for the current file
do
  result+=("${line} $i")
done< <( grep -n -o -P '(?<=def).*(?=\()' ${i})
done
#set up for sorting
IFS=$'\n'
#sort the results into sorted by function name
read -r -d '' -a sorted < <(sort -t: -k2 <<<"${result[*]}" && printf '\0')
unset IFS
#for loop for printing
for a in "${sorted[@]}"; do
    #set up delimiter
    IFS=' '
    #split variable a into array printout
    read -ra printout <<< "$a"
    #get rid of ":" at the end of the number
    num=$(echo "${printout[0]}" | cut -d':' -f 1)
    #print in format
    printf "%s %s:%s\n" "${printout[1]}" "${printout[2]}" "$num"
done
unset IFS

Could you try to build a [mcve] here that isolates your specific question about the bash language and associated practices, avoiding pulling in unrelated complexity from your problem domain? — Charles Duffy, Feb 18 '18 at 17:14
BTW, the thing that stands out most about this code as a smell is the use of unquoted expansions. If you only want to append one item to an array at a time, quote the expansion: `arr+=( "$item" )` or `arr+=( "$(generate-item ...)" )`; if you want to append more than one, you're better off using `read -a` to generate a smaller array explicitly, and then append it to the larger one: `read -r -a pieces <<<"$(generate-item ...)"; arr+=( "${pieces[@]}" )` -- otherwise you get bugs like a `*` in the input being replaced with a list of filenames in the current directory. — Charles Duffy, Feb 18 '18 at 17:15
The first thing I noticed was your shebang. If you really are specifying `/bin/bash/` as your shell, and your script runs at all, then you're doing something that isn't included in your question. Best starting advice would be to take your scripts over to **http://shellcheck.net/** for sanity checking, before posting them here. Once "obvious" issues are dealt with, the remaining ones can get everyone's focus without so much distraction. — ghoti, Feb 19 '18 at 05:24

Stefan M · Accepted Answer · 2018-02-18T21:46:45.330

There were several problems with your code.

You're NOT adding the elements like "1: push" or "5: drop" to the array, you're pushing the COMPLETE output of grep -n -o -P '(?<=def).*(?=\()' file1 to it. The next output is the COMPLETE output of your grep -n -o -P '(?<=def).*(?=\()' file2, which is appended with a comma.

The usage of the IFS during adding is not necessary. Specifically, during the sorting this is counterproductive, as you still want to split on \n, but not on \ (space).

You should always quote your input to an array.

You should declare the array.

If your input may have shell globs, you should use the information from this answer.

Here is the working script (you can replace the last line with echo ${sorted[*]} to get the same string structure as before).

#!/bin/bash
declare -a result
for i in "$@"
do
  while read line; # Read the input from the grep command for the current file
    do
      result+=("${line}")
    done< <( grep -n -o -P '(?<=def).*(?=\()' ${i})
done
IFS=$'\n'
read -r -d '' -a sorted < <(sort -t: -k2 <<<"${result[*]}" && printf '\0')
unset IFS
printf "[%s]\n" "${sorted[@]}"

Incorporated the suggestions from @Charles Duffy, thanks!

Thanks for the answer! This does exactly what I was poorly attempting to do. I took the advice of @Charles Duffy and was looking into how to use 'read' as well. This showed me I still have much more to learn. Thank you again! — MJK, Feb 18 '18 at 23:37

Bash adding results to array with +=

1 Answers1