0

I try to solve a problem in shell. Im trying to find a way to delete all newlines from each element of an array. I tried to do this with a for loop. The Strings look like this (always three numbers, separated with dots) "14.1.3\n" and I need to get rid of the newline at the end. This is what i tried to do: As a single-liner

for i in ${backup_versions[*]}; do backup_versions[$i]=echo "$i" | tr '\n' ' ' ; done

Easier to read

for i in ${backup_versions[*]}; 
do 

 backup_versions[$i]=echo "$i" | tr '\n' ' ' 

done

I think I try to reassign the element with the wrong syntax, but I tried every kind of writing i which I found or knew myself. The deletion of the newline works just fine and just the reassigning is my Problem.

Thanatos-Delta
  • 67
  • 1
  • 9
  • See https://stackoverflow.com/questions/6723426/looping-over-arrays-printing-both-index-and-value – Diego Torres Milano Aug 23 '21 at 18:25
  • On a side note, depending on how you are creating the array, it may be easier to remove the trailing delimiter from the start. `mapfile` for instance, has a dedicated `t` switch for this. – user1984 Aug 23 '21 at 18:29
  • The mapfile is approach to to fill this array but though I used the -t flag the newlines didn't disappear. I had to read in Strings from a S3 Bucket which were full of metadata. So i used awk to get just one column but still I had to get rid of the newline^^ – Thanatos-Delta Aug 23 '21 at 18:41
  • Since you're not quoting `${backup_versions[*]}`, the newlines should already be removed when you access `$i`. – Barmar Aug 23 '21 at 19:20
  • You need to use `$(...)` to assign the result of a command to a variable. – Barmar Aug 23 '21 at 19:21
  • `$i` is the array element. Why are you trying to use it as an array index? – Barmar Aug 23 '21 at 19:30
  • Maybe you could tell us how you acquire the array in the first place, because if you used `mapfile` you can use `mapfile -t` to avoid capturing newlines at end of each element. – Léa Gris Aug 23 '21 at 19:55

3 Answers3

3

If the strings are always of that form and don't contain any whitespace or wildcard characters, you can just use the shell's word-splitting to remove extraneous whitespace characters from the values.

backup_versions=(${backup_versions[*]})

If you used mapfile to create the array, you can use the -t option to prevent it from including the newline in the value in the first place.

Barmar
  • 741,623
  • 53
  • 500
  • 612
  • Ok this approach is by far the best, it got rid of the newlines AND the whitespace at the end of each element. So the Problem was not because of the input to my mapfile, somewhere I've read that at the end of each element the mapfile sets a whitespace and nextline by default. Is that correct? And the word-splitting is new for new but quite interesting just read an article. Can someone recommend any articles or other kinds of informations about htis? Thanks, really. – Thanatos-Delta Aug 24 '21 at 11:33
  • The `-t` option to `mapfile` will remove the trailing delimiter from each line. – Barmar Aug 24 '21 at 15:05
0

Use Bash's string substitution expansion ${var//old/new} to delete all newlines, and dynamically create a declaration for a new array, with elements stripped of newlines:

#!/usr/bin/env bash

backup_versions=(
  $'foo\nbar\n'
  $'\nbaz\ncux\n\n'
  $'I have spaces\n and newlines\n'
  $'It\'s a \n\n\nsingle quote and spaces\n'
  $'Quoted "foo bar"\n and newline'
)

# shellcheck disable=SC2155 # Dynamically generated declaration
declare -a no_newlines="($(
  printf '%q ' "${backup_versions[@]//$'\n'/}"
))"

# Debug print original array declaration
declare -p backup_versions

# Debug print the declaration of no_newlines
declare -p no_newlines
  • declare -a no_newlines="($(: Creates a dynamically generated declaration for the no_newlines array.
  • printf '%q ': Print each argument with quotes if necessary and add a trailing space.
  • "${backup_versions[@]//$'\n'/}": Expand each element of the backup_versions array, // replacing all $'\n' newlines by nothing to delete them.

Finally the no_newlines array will contain all entries from backup_versions, with newlines stripped-out.

Debug output match expectations:

declare -a backup_versions=([0]=$'foo\nbar\n' [1]=$'\nbaz\ncux\n\n' [2]=$'I have spaces\n and newlines\n' [3]=$'It\'s a \n\n\nsingle quote and spaces\n' [4]=$'Quoted "foo bar"\n and newline')
declare -a no_newlines=([0]="foobar" [1]="bazcux" [2]="I have spaces and newlines" [3]="It's a single quote and spaces" [4]="Quoted \"foo bar\" and newline")
Léa Gris
  • 17,497
  • 4
  • 32
  • 41
  • Does `%q` really work here? Quotes aren't parsed after expanding variables. – Barmar Aug 23 '21 at 19:37
  • @Barmar Yes `%q` works and is necessary if element contains spaces or quotes or anything that need escaping or quoting. The sub-shell call with `printf` generates a declaration. – Léa Gris Aug 23 '21 at 19:38
0

You can use a modifier when expanding the array, then save the modified contents. If the elements just have a single trailing newline, use substring removal to trim it:

backup_versions=("${backup_versions[@]%$'\n'}")

(Note: when expanding an array, you should almost always use [@] instead of [*], and put double-quotes around it to avoid weird parsing. Bash doesn't generally let you combine modifiers, but you can combo them with [@] to apply the modifier to each element as it's expanded.)

If you want to remove all newlines from the elements (in case there are multiple newlines in some elements), use a substitution (with an empty replacement string) instead:

backup_versions=("${backup_versions[@]//$'\n'/}")

(But as several comments have mentioned, it'd probably be better to look at how the array's being created, and see if it's possible to just avoid putting newlines in the array in the first place.)

Gordon Davisson
  • 118,432
  • 16
  • 123
  • 151