A shell script question involving rsync and symlink using cp

Question

The script under consideration has been coded as shown below:

#!/bin/bash

#recursive remove

REC_RM="rm -rf"

#recursive copy

REC_CP="cp -Rf"

#rsync

RSYNC_CMD="rsync -av --progress"

#recursive link

REC_LN="cp -as"

#the script excepts three arguments as follows:
#
# a filename such as files.txt with entries along the lines of
#
# [HEADER]
# <PATH>
#  
# the name of the specific HEADER to look for in the file, such as "FILES"
#
# a destination path such as /dst
#
# so the script should be invoked as <script> <filename> <header string> <destination path>
#
# for the sake of brevity, the segment related to argument checking has been omitted

#get the path listed under the specific header from within filename

SRC=$(grep -A1 $2 $1)
   
SRC=$(echo ${SRC} | awk '{print $2}')

#remove any preceding '/' from the path

FWD_SLSH='/'

SRC_MOD=$(echo ${SRC} | sed "s|^$FWD_SLSH\(.*\)|\1|")

#append the modified source path to the destination path

DST=${3}${SRC_MOD}

#create the destination directory path

mkdir -p ${DST}

#sync the source and destination paths

eval "$RSYNC_CMD" ${SRC} ${DST}

#recursively purge source path

eval "$REC_RM" ${SRC}

#recursively link destination path back to source path by means of copy

eval "$REC_LN" ${DST} ${SRC}

The file /src_vinod/test/data.txt and the directory /dst_vinod are created outside of the script.

Now assuming that the filename files.txt contained the following entry:

[FILES]
/src_vinod/test/

And the script were to be invoked as shown below (the name script.sh has been used):

script.sh filenames.txt FILES /dst_vinod/

After execution, I would expect the destination path to be populated as follows:

/dst_vinod/src_vinod/test/data.txt

and the source path to be populated as

/src_vinod/test/data.txt

where data.txt is a soft link to the file data.txt under the destination path

However, the result I get is as follows:

the destination path consists of

/dst_vinod/src_vinod/test/test/data.txt

and the source path consists of

/src_vinod/test/test/data.txt

where data.txt is a soft link to the file data.txt under the destination path

I am not sure why the directory test/ is replicated.

Any thoughts?

TIA

Vinod

[tag:bash] - "For shell scripts with syntax or other errors, please check them at https://shellcheck.net before posting them here.". See also [correct-bash-and-shell-script-variable-capitalization](https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization). — Ed Morton, May 03 '23 at 10:56

Jetchisel · Answer 1 · 2023-05-03T10:32:38.277

A function is what you should use instead of eval and variables.

Variables hold data. Functions hold code.

Something like:

#!/usr/bin/env bash

# Recursive remove
rec_rm(){
  rm -rf "$@"
}

# Recursive copy
rec_cp(){
  cp -Rf "$@"
}

# Rsync
rsync_cmd(){
  rsync -av --progress "$@"
}

# Recursive link
rec_ln(){
  cp -as "$@"
}

# Get the path listed under the specific header from within filename
output=$(grep -A1 "$2" "$1" 2>&1)

# Exit status of grep
status=$?

# If the exit status of `grep` failed then exit with grep's
# error message (except when empty) and exit status.
(( status > 0 )) && {
  printf '%s\n' "${output:-"$2 not_found_from $1"}" >&2
  exit "$status"
}

# Save the PATH (second line) to the variable `src`
{ read -r _ ; IFS= read -r src;  } <<< "$output"

# Just in case "$src" has no value/empty, exit the script.
[[ -n "$src" ]] || {
  printf 'No header/source path was found/provided from %s\n' "$1" >&2
  exit 1
}

# Replace double // with a single /
src=${src//\/\//\/}

# Append the modified source path to the destination path
dst=$3$src

# Replace double // with a single /
dst="${dst//\/\//\/}"

# Create the destination directory path, exit otherwise.
mkdir -vp "$dst" || exit

# Rync the source and destination paths, exit otherwise.
rsync_cmd "$src" "$dst" || exit

# Recursively purge source path, or exit.
rec_rm "$src" || exit

# Recursively link destination path back to source path by means of copy, or exit.
rec_ln "$dst" "$src" || exit

Create directories and files outside of the script.

mkdir -pv /src_vinod/test/ &&                                                                                                                                
printf 'Dummy\n' >> /src_vinod/test/data.txt
file /src_vinod/test/data.txt
mkdir -pv /dst_vinod

Output

mkdir: created directory '/src_vinod'                                                                                                                        
mkdir: created directory '/src_vinod/test/'
/src_vinod/test/data.txt: ASCII text
mkdir: created directory '/dst_vinod'

After creating the directories and dummy/empty files, executing the script with the arguments, the output is something like:

mkdir: created directory '/dst_vinod/src_vinod'
mkdir: created directory '/dst_vinod/src_vinod/test/'
sending incremental file list
./
data.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/2)

sent 109 bytes  received 38 bytes  294.00 bytes/sec
total size is 0  speedup is 0.00

Checking the files/directories:

file /src_vinod/test/data.txt

Output

/src_vinod/test/data.txt: symbolic link to /dst_vinod/src_vinod/test/data.txt

See I'm trying to put a command in a variable but the complex cases always failed!
Also upper case variables should be avoided if it is just for internal purposes of your script, to avoid overwriting/collision with env variables
- See https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
- See Correct Bash and shell script variable capitalization
A test if there is a given input is missing or some of the existing directories, also an option parser is a better approach when dealing with multiple input/arguments.
grep -A1 pattern file Grep will match MYFILE, FILE, FILED and so on, since GNU grep is being used the \b word boundary is an option as well, e.g. grep -A1 "\b$2\b" "$1" to match the exact word.

Also worth noting why you replaced the upper-case names with lower-case explaining that generally upper-case names are reserved for environment variables and shell built-ins. Good link to Bash FAQ 50. — David C. Rankin, May 03 '23 at 04:53
@Jetchisel I tried your suggested changes. I don't get the results that you got. I get the same issue of duplicate test/ directories. Technically, there is not a great deal of difference between your code and mine, except for commands replaced with functions. Please let me know if I have missed out on something. — Vinod, May 04 '23 at 04:20
@Vinod, maybe your directory structure is different, I tested your code in a vm and it produces the same result but mine is without `eval` injection and a better/useful error/exit status if and when it has encounter some errors. Like if there is no file named `files.txt` or `filenames.txt` or if there is no pattern `FILES` or if not enough permissions from the said files and so on... — Jetchisel, May 04 '23 at 05:49

score 0 · Accepted Answer · answered May 04 '23 at 20:28

How to use rsync to copy files

Note: Specifying “/” after the source directory only copies the contents of the directory. If we do not specify the “/”after the source directory, the source directory will also be copied to the destination directory.

This was the issue in my script. I fixed this issue and it started working as expected.

A shell script question involving rsync and symlink using cp

2 Answers2