how to correctly use newline as delimiter for saving input into array

Question

I am working on a bash script that should backup folders which users can specify. The script is just for fun, and to get into bash. I will not use it for real backup purposes.

The idea is that users can enter specific folders they want to update. After each entry, they would hit ENTER and then type another directory into the console. After they are finished, they would press CTRL-D. The inputs would then be saved into an array USER_DIR. This array would then be checked to see if the directories exist. If they do exist, the entry will stay the same. If they don't exist the entry will be overwritten be a default value /home/$USER.

I have found an interesting idea on AskUbuntu (I am using Ubuntu):

while read line
do
    my_array=("${my_array[@]}" $line)
done
echo ${my_array[@]}

I have used that and tried to input multiple directories, but it only worked if the input was exactly one directory. Every time I used multiple directories as input, it defaulted to /home/$USER. I have put echo ${my_array[@]} on different positions in the script, and it looked as if the problem was with the delimiter, as the input was seemingly concatenated with whitespaces.

I tried to look for a way to loop through the array and change the delimiter after the input was saved. I found an interesting idea on StackOverflow:

SAVEIFS=$IFS   # Save current IFS
IFS=$'\n'      # Change IFS to new line
names=($names) # split to array $names
IFS=$SAVEIFS   # Restore IFS

I tried it out, but it also only accepts singular input. My script looks like this at the moment:

echo "Please enter absolute path of directory for backup. Default is ""/home/$USER"
echo "You can choose multiple directories. Press CTRL-D to start backup."

# save user input into array
while read line
do
    USER_DIR=("${USER_DIR[@]}" "$line")
done

SAVEIFS=$IFS   # Save current IFS
IFS=$'\n'      # Change IFS to new line
USER_DIR=($USER_DIR) # split to array $names
IFS=$SAVEIFS   # Restore IFS

# check if user input in array is valid (directory exists)
for ((i=0; i<${#USER_DIR[@]}; i++))
do
    if [[ -d "${USER_DIR[$i]}" ]]; then
        # directory exists
        USER_DIR=("${USER_DIR[@]}")
        echo "${USER_DIR[@]}" # for debugging
    else
        # directory does not exist
        USER_DIR=("/home/$USER")
        echo "${USER_DIR[@]}" # for debugging
    fi
done

# show content of array
echo "Backups of the following directories will be done:"
for i in "${USER_DIR[@]}"; do echo "$i"; done

Any help would be greatly appreciated. Thank you for your time.

Answer 1

You do not need this code snippet.

SAVEIFS=$IFS   # Save current IFS
IFS=$'\n'      # Change IFS to new line
USER_DIR=($USER_DIR) # split to array $names
IFS=$SAVEIFS   # Restore IFS

By default read command used \n as input delimiter.

Demo :

$./test.ksh 
Please enter absolute path of directory for backup. Default is /home/renegade
You can choose multiple directories. Press CTRL-D to start backup.
abc
123
def
Backups of the following directories will be done:
abc
123
def
$cat test.ksh 
echo "Please enter absolute path of directory for backup. Default is ""/home/$USER"
echo "You can choose multiple directories. Press CTRL-D to start backup."

# save user input into array
while read line
do
    USER_DIR=( "${USER_DIR[@]}" "$line" )
done

#SAVEIFS=$IFS   # Save current IFS
#IFS=$'\n'      # Change IFS to new line
#USER_DIR=($USER_DIR) # split to array $names
#IFS=$SAVEIFS   # Restore IFS

# show content of array
echo "Backups of the following directories will be done:"
for i in "${USER_DIR[@]}"; do echo "$i"; done
$

Answer 2

There are a couple of serious problems here, and I also have some more minor recommendations. The first is (probably) minor: your script doesn't start with a shebang line to tell the system what scripting language it's written in. You are using bash features (not just basic POSIX shell) so you need to start with a shebang that says to run bash, and not some generic shell. So the first line should be either #!/bin/bash or #!/usr/bin/env bash.

Next, let's look at the while read loop:

while read line
do
    USER_DIR=("${USER_DIR[@]}" "$line")
done

This works correctly as it stands. It populates the array with the user's input, one element per entry. It sounds like you used echo ${USER_DIR[@]} to check the resulting array, and got confused into thinking there was something wrong; but the loop works fine, it's the echo ${USER_DIR[@]} that's misleading. As I said in a comment, use declare -p USER_DIR instead, and bash will print a statement that (if you were to run it) would recreate the array's current contents. This turns out to be a good, generally-unambiguous, way to display the contents of an array.

I do have some minor recommendations: use lower- or mixed-case variable names (e.g. user_dir, userDir, or something like that) instead of all-caps names. There are a bunch of all-caps names with special meanings, and if you use all-caps named it's easy to accidentally use one of those and cause chaos. Also, I'd initialize the array to empty before the loop with user_dir=(), just to make sure it starts off empty. And as Jetchisel said in a comment, user_dir+=("$line") would be a simpler way to add new elements to the array. Just don't leave off the parentheses, or it'll append to the first element instead of adding a new element.

Ok, next section:

SAVEIFS=$IFS   # Save current IFS
IFS=$'\n'      # Change IFS to new line
USER_DIR=($USER_DIR) # split to array $names
IFS=$SAVEIFS   # Restore IFS

This appears to be an attempt to fix a problem that doesn't exist, that winds up creating a new problem in the process. As I said, USER_DIR already exists as a (correctly-populated) array, so $USER_DIR (without a [@] or anything) just gets its first element. That means that USER_DIR=($USER_DIR) gets just the first element, tries to split it on newlines (because that's what IFS is set to), but there aren't any so nothing happens. Then it sets USER_DIR to an array consisting of just that first element.

Essentially, it's deleting all but the first element from the array. Just get rid of this entire section.

Ok, next:

for ((i=0; i<${#USER_DIR[@]}; i++))

This'll work in bash for an array with default indexing (which is what you have), but my personal preference is to use this instead:

for i in "${!USER_DIR[@]}"

See that ! in there? That makes it get a list of the index values in the array. Works in bash, works in zsh (which uses a different numbering convention), and I don't have to try to remember which one uses which convention. It even works with associative arrays! But again this is not a big deal, just my preference.

Now, inside the loop:

    if [[ -d "${USER_DIR[$i]}" ]]; then
        # directory exists
        USER_DIR=("${USER_DIR[@]}")

This expands out the contents of the array (properly quoted and everything so it won't get messed up), and assigns it right back to the array. It does absolutely nothing! Mind you, nothing is the right thing to do here, but there are much easier ways to do nothing. I'd remove it entirely, and just use if [[ ! -d "${USER_DIR[$i]}" ]]; then followed by the "what do do if it doesn't exist" part. Speaking of which:

    else
        # directory does not exist
        USER_DIR=("/home/$USER")

This is the second serious problem. It replaces the entire array with a single entry pointing to your home directory. You want to replace only the current entry and leave the rest of the array alone, so use USER_DIR[$i]="/home/$USER". Note that there's no ${ } around the array references -- those are for when you're getting a value from an array, not when you're setting one.

Ok, one more recommendation: rather than reading all values into an array and then going back over them to fix the ones that don't exist, why not just do the existence test as you read directory paths, and then just add the right thing to the array in the first place? You can even give the user feedback as they enter names, like this:

#!/bin/bash

echo "Please enter absolute path of directory for backup. Default is ""/home/$USER"
echo "You can choose multiple directories. Press CTRL-D to start backup."

# save user input into array
user_dir=()
while read line
do
    if [[ -d "$line" ]]; then
        user_dir+=("$line")
    else
        echo "$line isn't an existing directory; we'll back up /home/$USER instead"
        user_dir+=("/home/$USER")
    fi
done

# show content of array
echo "Backups of the following directories will be done:"
for i in "${user_dir[@]}"; do echo "$i"; done