0

I are creating a .sh using bash for validate the api sub folders versions

The objective is validate this strings into APIS_BUILD var and find all .proto files into ./proto folder to compile into protobuffer Go file

# define subfolder apis to build
APIS_BUILD=(
    prototests/v1/
    prototests2/v2
    testfolder
)
# the "testfolder" are a invalid folder

Test cases:

prototestes/v1                      # valid
prototestes/v1/cobranca             # valid
prototestes/v1/cobrnaca/faturamento # valid
outrapastacomarquivosproto/v1       # valid
prototests                          # invalid
/prototests                         # invalid

Then, I created this script for validate the APIS_BUILD string array

#!/usr/bin/env bash

# text color
RED='\033[0;31m'  # RED
BLUE='\033[0;34m' # Blue
NC='\033[0m'      # No Color

# Underline color
UCyan='\033[4;36m' # Cyan

# define subfolder apis to build
APIS_BUILD=(
    prototests/v1
    cobrancas/v1
)
DST_DIR="."       # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder

# Compile proto file
# $1 = Filename to compile
function compile() {
    protoc --go_out=$DST_DIR --proto_path=proto --go_opt=M$1=services \
        --go_opt=paths=import --go-grpc_out=. \
        $1
}

# Validate api_build's
function validateApiBuilds() {
    for t in ${APIS_BUILD[@]}; do
        IFS="/"
        read -a SUBSTR <<<"$t"
        if [ ${#SUBSTR[@]} -lt 2 ]; then
            printf "${RED}The API_BUILD value ${UCyan}\"${t}\"${RED} are declare wrong, please declare [api_folder]/[version_folder] (example: prototest/v1)${NC}\n" 1>&2
            exit 1
        fi
    done
}

validateApiBuilds

for filename in $(find $SRC_DIR -name '*.proto'); do
    [ -e "$filename" ] || continue
    echo $filename
done
  • The subfolder:

enter image description here

But I getting a strange behavior:

  • If run the .sh file with the validateApiBuilds function the return for $filename is always .
  • If run the .sh file without the validateApiBuilds function the return for $filename are getting the testservice.proto file

Pictures:

With validateApiBuilds function:

enter image description here

Without validateApiBuilds function:

enter image description here


All the variables:

# define subfolder apis to build
APIS_BUILD=(
    prototests/v1
    cobrancas/v1
)
DST_DIR="."       # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder

Bash version:

$ bash --version                                  
$ GNU bash, versão 4.4.19(1)-release (x86_64-pc-linux-gnu)

Obs.: I changed the validateApiBuilds function to use a regex validation for strings into API_BUILDS variable. But I really wanted to know the reason for this behavior.

edit 2: The make-proto.config file

# define subfolder apis to build
APIS_BUILD=(
    prototests/v1
    cobrancas/v1
)
DST_DIR="."       # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
Bruno Luiz K.
  • 168
  • 2
  • 15
  • 1
    Please minimize your script to be a [mre], the _shortest possible thing_ that can be run without changes to check whether a specific problem is present -- we don't need the code checking which variables exist, f/e, unless the bug is specifically with those checks. We don't need code that prints colorful errors unless your problem is with color; etc. – Charles Duffy Nov 20 '20 at 16:24
  • BTW, `for filename in $(find $SRC_DIR -name '*.proto'); do` is an antipattern -- see [BashPitfalls #1](http://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29) describing why, and [Using Find](https://mywiki.wooledge.org/UsingFind) describing what to do instead. – Charles Duffy Nov 20 '20 at 16:26
  • @CharlesDuffy the behavior only occurs with all code above, if I put only the function and the "for" loop, the behavior not occurs – Bruno Luiz K. Nov 20 '20 at 16:26
  • 1
    Surely not. I find it impossible to believe that it would stop happening if you took out the `if [[ -z "$APIS_BUILD" ]]; then` block, for example. So take out _as many things as you can_, while leaving in only the things you can't. If that means commenting out one line or block at a time, testing, and reverting if the problem no longer occurs, do that -- see also the "Tricks for Trimming" section of http://sscce.org/ for guidance on how to build clear, minimal reproducers. – Charles Duffy Nov 20 '20 at 16:27
  • (Also, note that `echo $filename` is itself potentially buggy -- that's [BashPitfalls #14](http://mywiki.wooledge.org/BashPitfalls#echo_.24foo), or the Stack Overflow question [I just assigned a variable, but `echo $variable` prints something else!](https://stackoverflow.com/questions/29378566), or the http://shellcheck.net/ warning [SC2086](https://github.com/koalaman/shellcheck/wiki/SC2086) -- running your code through http://shellcheck.net/ before asking questions here is a good idea in general). – Charles Duffy Nov 20 '20 at 16:31
  • ...another good place to start is using `set -x` to enable trace logging, so you can see the actual commands as they're run with variables expanded, and compare between working and broken scenarios. – Charles Duffy Nov 20 '20 at 16:33
  • The ```IFS="/"``` can possible changing the behavior of the ```find``` loop? – Bruno Luiz K. Nov 20 '20 at 16:42
  • Definitely, yes. (That's part of why other mechanisms to iterate over `find`'s output are preferred, and documented in the Using Find link above). – Charles Duffy Nov 20 '20 at 16:45
  • ...that aspect of the question is duplicative of [setting IFS for a single statement](https://stackoverflow.com/questions/33416877/setting-ifs-for-a-single-statement). – Charles Duffy Nov 20 '20 at 16:49

1 Answers1

1

Use find better

for filename in $(anything) is always an antipattern -- it splits values on characters in IFS, and then expands each result as a glob. To make find emit completely unambiguous strings, use -print0:

while IFS= read -r -d '' filename; do
    [ -e "$filename" ] || continue
    echo "$filename"
done < <(find "$SRC_DIR" -name '*.proto' -print0)

Don't change IFS unnecessarily

Change your code to make the assignment to IFS be on the same line as the read, which will make IFS only be modified for that one command.

That is to say, instead of:

IFS=/
read -a SUBSTR <<<"$t"

...you should write:

IFS=/ read -a SUBSTR <<<"$t"
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441