0

Please, consider the script below. It checks two environments to see if both have equal sets of env variables names. The problem starts on the line of second grep. If files Ds1 and Ps1 are the same, Du is not produced at all. The first grep however produces an empty Pu file.

Another strange thing. If I run the script with ./script the problem occurs, if I use bash script, both grep lines behave identically producing two empty files.

Any explanation? How can I make them run the same way regardless of invocation method? 

#!/bin/bash -e

# Script name, added to console prints 
action="\e[32m"${0}"\e[39m"

exit_code=0

echo -e ${action}": Start of script."
echo

#get var names from both files
awk '{print $1}' Ds > Ds1
awk '{print $1}' Ps > Ps1

# Get vars not in Dev
grep -Fxvf Ds1 Ps1 > Pu
# Get vars not in Prod
grep -Fxvf Ps1 Ds1 > Du

# If there are vars unique to Dev, print them
if [ -s Du ]; then
  echo -e ${action}": List of env variables unique to Dev:"
  echo

  while read p; do
    cat Ds | grep $p
  done < Du
  echo
  exit_code=1
fi

# If there are vars unique to Prod, print them
if [ -s Pu ]; then
echo -e ${action}": List of env variables unique to Prod:"

  while read line; do
     cat Ps | grep $line
  done < Pu
  exit_code=1
fi

exit ${exit_code}

Here is a part of Ds1 file if needed:

AMBURAPP_ACCESS_TOKEN
AWS_ACCESS_KEY_ID
AWS_DB_BACKUP_REGION
AWS_DB_BACKUP_S3_BUCKET_NAME
AWS_SECRET_ACCESS_KEY
AWS_USER_NAME
BUNDLE_WITHOUT
DB_ENDPOINT
DB_NAME
DB_PASSWORD
DB_PORT
DB_USER
DEVISE_SECRET_KEY
DOMAIN
INTERCOM_API_KEY
MANDRILL_SUBACCOUNT
MANDRILL_USERNAME
PARAM5

running diff Ds1 Ps1 returns nothing, i.e. diff considers both files identical AFAIK.

EDIT:

My question is about different outcome of two identical greps and NOT about ways of fast comparison of files.

ERIK_SON
  • 379
  • 5
  • 16
  • `echo -e "\e"` is a Bashism. The `while read` loop is certainly an antipattern. Does your file contain trailing whitespace and/or aberrant DOS line ending characters? – tripleee Aug 06 '16 at 10:30
  • `> Pu` already makes an empty output file, so some diff funcion will do so too. – Walter A Aug 06 '16 at 12:54
  • @tripleee How on earth my question about two greps with different output has anything to do with the question about fast comparison of files? I don't know what a Bashism is, **echo -e** was for the colorization of script name. All files are produced in the script on a Linux box so I suspect there are no "aberrant DOS line ending characters". – ERIK_SON Aug 06 '16 at 14:07
  • @WalterA Yes, Pu is an empty file. And Du is supposed to be empty but it is never produced. Why? The second grep returns absolutely nothing. – ERIK_SON Aug 06 '16 at 14:39
  • A [bashism](http://mywiki.wooledge.org/Bashism) is a construct which works in Bash but isn't portable to other Bourne-compatible shells. Your problem description invites this explanation, but I have hesitantly reopened this question. – tripleee Aug 06 '16 at 15:03
  • Why no empty file `Du` is made, is difficult to tell. Maybe you do not have a newline but only have spaces between "# Get vars not in Prod' and "grep -Fxvf Ps1 Ds1 > Du". `grep` will be considered part of the comment and your editor might wrap it to a new line. Oh, the accepted answer is already the explanation you want ! – Walter A Aug 06 '16 at 19:12

1 Answers1

1

Running the script with -e causes it to abort when a command fails, such as when grep does not produce a match.

The fix is to explicitly handle every possible error. To ignore an error, you can use something like

grep foo bar || true
tripleee
  • 175,061
  • 34
  • 275
  • 318
  • I'm pretty sure this is a duplicate of a different question than the one I originally nominated, but I can't quickly find the dupe. Pointers much appreciated. – tripleee Aug 06 '16 at 15:08
  • -e is required by my boss, not my choice. What do you think about grep working on one line then not working on the next? That is my main issue. – ERIK_SON Aug 06 '16 at 16:58
  • When `Pu` is empty, the resulting error from `grep` terminates the script. I really don't see how I could explain the workaround any better. `grep -Fxvf Ds1 Ps1 > Pu || true` and similarly elsewhere in the script where `grep` could fail to produce matches. – tripleee Aug 06 '16 at 17:03
  • Sorry, I was misunderstanding your answer. Now I see, thank you for spending so much time on my question. – ERIK_SON Aug 06 '16 at 17:09