1

I have read a lot about variable expansion in bash scripts this morning but I have still not been able to identify what has caused the script to stop expanding the variable in this line of code:

CURRENT_HASH=`git log --pretty=format:"%H %s" | grep ${LIBRARY_VERSION}-SNAPSHOT | head -1 | cut -d ' ' -f 1`

Here is more context, with comments showing the output of each line:


SCRIPTPATH=`pwd`

# get the application name and library name from the cloned repo in the deploy directory
DEPLOY_DIRECTORY="$(basename `pwd`)"
LIBRARY_SUFFIX="-library.txt"
cd ..
for d in */ ; do
    if [[ "$d" == *"library"* ]] ; then
        LIBRARY_NAME="${d%?}" # myAppLibrary
    fi
    if [[ "$d" != "${DEPLOY_DIRECTORY}" && "$d" != *"library"* ]] ; then
        APP_NAME="${d%?}" # myAppName
    fi
done
    
# get the latest version from the library
cd ..
LIBRARY_LATEST=$(head -n 1 $SCRIPTPATH/../${LIBRARY_NAME}/${APP_NAME}${LIBRARY_SUFFIX})
LIBRARY_VERSION=`echo $LIBRARY_LATEST | sed '$s/,//'`

echo $LIBRARY_LATEST # outputs 320,
echo $LIBRARY_VERSION # outputs 320 (no comma)

# get the hash matching the last frozen version from the app repo
cd $SCRIPTPATH/../$APP_NAME
CURRENT_HASH=`git log --pretty=format:"%H %s" | grep ${LIBRARY_VERSION}-SNAPSHOT | head -1 | cut -d ' ' -f 1`
echo "Current hash: $CURRENT_HASH" # blank

If I do the git log command from the command line I get a hash, as expected.

This block of code was working perfectly last week, but now $CURRENT_HASH is blank every time. If I manually include the ${LIBRARY_VERSION} it works fine. If I manually populate the variable, it works fine. I have also tried $LIBRARY_VERSION to no avail.

Am I missing something obvious? What would cause a variable to stop working?

Jay Blanchard
  • 34,243
  • 16
  • 77
  • 119
  • What do you get if you run the command `git log --pretty=format:"%H %s" | grep ${LIBRARY_VERSION}-SNAPSHOT | head -1 | cut -d ' ' -f 1` without capturing the output to a variable? – Code-Apprentice Feb 16 '22 at 16:45
  • See [How can I debug a Bash script?](https://stackoverflow.com/q/951336/4154375). – pjh Feb 16 '22 at 17:34
  • [Shellcheck](https://www.shellcheck.net/) finds several issues with the code in the question. – pjh Feb 16 '22 at 17:38
  • @Code-Apprentice I get a blank – Jay Blanchard Feb 16 '22 at 18:02
  • I know how to debug a bash script @pjh, and I have been debugging this one for hours. – Jay Blanchard Feb 16 '22 at 18:03
  • All of the suggestions from Shellcheck are great @pjh but it doesn't explain why this suddenly stopped working. – Jay Blanchard Feb 16 '22 at 18:11
  • @Code-Apprentice I should clarify - if I run the script without capturing to a variable I do not get a hash back. It is still blank. – Jay Blanchard Feb 16 '22 at 18:21
  • 1
    One of the downsides of pipelines is that it can be difficult to debug problems that occur within them. In this case I'd consider (temporarily, for debugging) doing the processing in sequential stages: `tmp1=$(git log --pretty=format:"%H %s"); tmp2=$(grep ${LIBRARY_VERSION}-SNAPSHOT <<<"$tmp1"); tmp3=$(head -1 <<<"$tmp2"); tmp4=$(cut -d ' ' -f 1 <<<"$tmp3"); CURRENT_HASH=$tmp4`. The `bash -x` output may then reveal at what stage the problem is happening (e.g. maybe unexpected `git log` output). – pjh Feb 16 '22 at 18:48
  • 1
    Looks like there is a carriage return there - ```++ git log '--pretty=format:%H %s' ++ grep $'0.0.320\r-SNAPSHOT' ++ head -1 ++ cut -d ' ' -f 1``` – Jay Blanchard Feb 16 '22 at 19:02
  • 1
    The carriage return fits. I guess a `-library.txt` file got converted to Windows text file line endings (CR-LF) sometime in the last week. – pjh Feb 16 '22 at 19:57

1 Answers1

2

As discussed in the comments on the question, the immediate cause of the problem appears to be that the string in $LIBRARY_VERSION has a trailing carriage return character.

A likely underlying cause of the problem is that the file from which the LIBRARY_LATEST variable value was read has recently acquired Windows text file line endings (CRLF).

Unexpected CRLF line endings are a common cause of problems described in questions relating to Bash (and similar shells). These references may be useful:

pjh
  • 6,388
  • 2
  • 16
  • 17