Create array from output of command that is newline delimited and contains spaces

Question

There's a lot of similar questions, but I couldn't find one that fixed my problem. It's possible I don't know what to google.

I'm trying to create a pair of arrays and then loop through both by index. The contents of the array is generated from this command:

$ cat small | grep -Po "(?<=Query: ).+$" | grep -v "use"

select count(*) from batting_small
select count(*) from batting_small_parquet
select count(*) from batting_small_parquet_stats
select yearid, count(*) from batting_small_parquet group by yearId
select yearid, count(*) from batting_small_parquet_stats group by yearId
select min(hr), max(hr) from batting_small
select min(hr), max(hr) from batting_small_parquet
select min(hr), max(hr) from batting_small_parquet_stats
...

But making an array like this

queries=( ` cat small | grep -Po "(?<=Query: ).+$" | grep -v "use" ` )

Splits by word and not by line. How can I split by line?

Looks like you analyze mysql slow log files, for that purpose there is a special routine `mysqldumpslow` in mysql package, I think it can do much what you want. mysqldumpslow --help — Yu Jiaao, Nov 08 '17 at 15:36
They are not mysql logs, but thanks for a possible alternative — bendl, Nov 08 '17 at 15:42

score 1 · Answer 1 · answered Nov 08 '17 at 15:13

1

use the mapfile bash builtin, and redirect the grep command from a process substitution:

mapfile -t queries < <( grep -Po "(?<=Query: ).+$" small | grep -v "use")

answered Nov 08 '17 at 15:13

glenn jackman

238,783
38
220
352

score 1 · Answer 2 · answered Nov 08 '17 at 15:14

You can use -z option in grep to produce a NUL terminated output and then use process substitution in a while IFS= read -d '' loop:

while IFS= read -d '' -r line; do
   echo "<<$line>>"
done < <(ggrep -zPo '(?m)(?!.*use)(?<=Query: ).+$' small)

<<select count(*) from batting_small>>
<<select count(*) from batting_small_parquet>>
<<select count(*) from batting_small_parquet_stats>>
<<select yearid, count(*) from batting_small_parquet group by yearId>>
<<select yearid, count(*) from batting_small_parquet_stats group by yearId>>
<<select min(hr), max(hr) from batting_small>>
<<select min(hr), max(hr) from batting_small_parquet>>
<<select min(hr), max(hr) from batting_small_parquet_stats>>

Change echo "<<$line>>" line to your actual code.

Also note use of (?m) to enable MULTILINE mode in regex and a negative lookahead (?!.*use) to avoid another grep -v.

Create array from output of command that is newline delimited and contains spaces

2 Answers2