0

This is my script:

#!/usr/bin/env bash
set -e

set -o errexit
set -o nounset
# set -o xtrace

input="${1:-}"
table="${2:-}"

length=$(jq length $input)
step=25

for i in {0..${length}..${step}}
  do 
    low_index=$i * ${step}
    high_index=$low_index + ${step} - 1

    echo $low_index
    echo $high_index

    jq \
      --arg table "$table" \
      --arg low_index $low_index \
      --arg high_index $high_index \
      '{$table: [.[$low_index:$high_index] | {"PutRequest": {"Item": map_values({S: .})}}]}' \
      csvjson.json > dynamo_${table}_${low_index}-${high_index}.json
done

This is the context of my directory:

$ ls
csvjson.json  splitter.sh

Nevertheless, I'm getting:

$ bash ./splitter.sh csvjson.json socs
./splitter.sh: line 19: csvjson.json: command not found
John Kugelman
  • 349,597
  • 67
  • 533
  • 578
Jordi
  • 20,868
  • 39
  • 149
  • 333
  • 2
    There are numerous problems with this code; please run it through http://shellcheck.net. – chepner Mar 25 '20 at 12:37
  • 1
    You probably have a space following the backslash on line 18, which means line 19 is a new command, not a continuation of the previous command. I'm not sure how you get that far, though, with the failure to use `$(( ... ))` to set the values of `low_index` and `high_index`; you should be getting similar command-not-found errors from those lines. – chepner Mar 25 '20 at 12:39
  • 2
    `set -e` and `set -o errexit` do the same thing, and the user of either is controversial, at best. I recommend not using either one. – chepner Mar 25 '20 at 12:42
  • ...for more on that, see [the exercises in BashFAQ #105](http://mywiki.wooledge.org/BashFAQ/105#Exercises). – Charles Duffy Mar 25 '20 at 13:10
  • I'll slightly disagree with chepner. There's no controversy. Don't use `set -e` – William Pursell Mar 25 '20 at 13:11
  • BTW, running `PS4=':${BASH_SOURCE##*/}:$LINENO' bash -x yourscript` would let you know which line it's failing on, which might have helped you find the problem yourself. (I'll also second the recommendation to use http://shellcheck.net/ before asking questions here). – Charles Duffy Mar 25 '20 at 13:11
  • Also, in general, calling jq over and over in a loop means you should be designing your code differently to do that looping *inside jq*. It's a powerful language, and includes looping constructs; no reason not to use them. (Granted, this can mean you need to loop over its output to sort it into different files, but if the output is organized appropriately -- f/e, destination name before the content -- that's not hard to do, and generally a fair bit more efficient, particularly if using a tool like awk for that second stage). – Charles Duffy Mar 25 '20 at 13:12
  • 1
    I'll disagree with the chorus. `set -e` is flawed, but still quite useful. Use it if you want to. I do. Either way -- it's an unhelpful sidetrack given the many real problems in the script. – John Kugelman Mar 25 '20 at 13:17
  • @JohnKugelman, ...*you* can get away with `set -e` because you know its intricacies/gotchas/pitfalls. That's a long learning curve; it's one thing to know them yourself (and, hopefully, have your team's code reviewers also intimately familiar) and make cautious use, it's another thing to encourage someone who *doesn't* know the trouble they're getting into thereby to use the facility blindly. (chepner and I, probably amongst others, were long-time freenode #bash denizens, and have a lot of time spent helping non-experts; guiding people away from sources of problems reduces support load). – Charles Duffy Mar 25 '20 at 13:19

1 Answers1

1

Consider the line:

low_index=$i * ${step}

and assume that i=0 (which given the code in the question, it will not be, but that is clearly the intent) and step=5. When the shell parses that line, on the first round of word expansions it expands to:

low_index=0 * 5

There's no field splitting necessary, so round 2 is a no-op. The 3rd roundof expansion is pathname expansion, in which the * is expanded to all the names in the current directory, so the line expands to:

low_index=0 cvsjson.json splitter.sh 5

The shell tries to execute that line by looking for a file named cvsjson.json in PATH.

William Pursell
  • 204,365
  • 48
  • 270
  • 300
  • Would you agree with closing this as a duplicate to [Mathematical expression result assigned to a Bash variable](https://stackoverflow.com/questions/20138516/mathematical-expression-result-assigned-to-a-bash-variable), if a failure to do so correctly is the primary/only problem? – Charles Duffy Mar 25 '20 at 13:17
  • Maybe also with https://stackoverflow.com/questions/169511/how-do-i-iterate-over-a-range-of-numbers-defined-by-variables-in-bash as a second duplicate, to cover the fault in `for i in {0..$length..$step}`. – Charles Duffy Mar 25 '20 at 13:26
  • Yes, it's certainly a duplicate! – William Pursell Mar 25 '20 at 13:30