Converting csv format column to comma separated list using bash

Question

I need to convert a column which is in CSV format to a comma-separated list so that I can use a for loop on the list and use each parameter.

Here what I have tried:

$ gcloud dns managed-zones list --format='csv(name)' --project 'sandbox-001'

Output:

name
dns1
dns2
dns3
dns4

I need such result: "dns1,dns2,dns3,dns4" so that I can use a for loop:

x="dns1, dns2, dns3, dns4"
for i in $x:
   print $i
done

The paste command returns me the last line:

$ gcloud dns managed-zones list --format='csv(name)' --project 'sandbox-001' |paste -sd,

,dns4

I would appreciate if someone can help me with this.

tripleee · Accepted Answer · 2022-09-07T16:13:51.717

0

The real problem is apparently that the output has DOS line feeds. See Are shell scripts sensitive to encoding and line endings? for a broader discussion, but for the immediate solution, try

tr -s '\015\012' , <file | sed 's/^[,]*,//;s/,$/\n/'

The arguments to for should just be a list of tokens anyway, no commas between them. However, a better solution altogether is to use while read instead. See also Don't read lines with for

gcloud dns managed-zones list --format='csv(name)' \
    --project 'sandbox-001' |
tr -d '\015' |
tail -n +2 |  # skip header line
while read -r i; do
    echo "$i"  # notice also quoting
done

I don't have access to gcloud but its manual page mentions several other formats which might be more suitable for your needs, though. See if the json or list format might be easier to manipulate. (CSV with a single column is not really CSV anyway, just a text file.)

edited Sep 07 '22 at 16:13

answered Sep 07 '22 at 10:27

tripleee

175,061
34
275
318

thanks for the quick reply. how can remove "name" which is in the first line of output? – Matrix Sep 07 '22 at 10:45
The `sed` solution already does that. I'll update the `while` loop similarly. – tripleee Sep 07 '22 at 10:46
nice, thanks. tail -n +2 worked . – Matrix Sep 07 '22 at 12:55

score 0 · Answer 2 · answered Sep 07 '22 at 14:07

Create an array:

arr=( $(gcloud dns managed-zones list --format='csv(name)' --project 'sandbox-001') )

Than printf it like so:

printf -v var '%s,' "${arr[@]:1}"

This will create variable $var with value 'dns1,dns2,dns3,' echo it like this to drop last comma:

echo "${var%,}"

Converting csv format column to comma separated list using bash

2 Answers2