How do you set custom argument variables in bash?

Question

I'm trying to schedule a cron job involving a bash script which invokes a scrapy crawl command. For one of the scrapy arguments, I have an argument called query which is the datetime I wish to start crawling.

This is the scrapy command:

scrapy crawl TweetScraper -a query=$QUERY

The only important thing to note is that I'm trying to set the query argument to a variable defined by a Query variable defined before it.

And here is the complete code:

#!/bin/bash
QUERY=$(printf "#DeepLearning since: %(%Y-%m-%dT%H:%M:%S)T" $(($(printf "%(%s)T") - 1 * 60)))

echo $QUERY
scrapy crawl TweetScraper -a query=$QUERY

The query was based off of subtract 1 hour from date in unix shell script.

Ideally I would like to make the argument set to one hour before the current time. Is this kind of construction possible? Currently I am getting an error because scrapy is interpreting the query as two different variables.

Correct quoting, as in `-a query="$QUERY"`, will prevent string-splitting. This is something http://shellcheck.net/ will identify for you. — Charles Duffy, Jul 19 '18 at 21:06
BTW, showing the exact error message you're getting from invocation (and maybe a log from running the command after `set -x`) will go a long way. — Charles Duffy, Jul 19 '18 at 21:07
As an aside, not directly related to the question, you could also make this code a bit more efficient by making it `printf -v current_time '%(%s)T' -1; printf -v query '#DeepLearning since: %(%Y-%m-%dT%H:%M:%S)T' "$((current_time - 60))"`; that avoids the performance penalty incurred by `$( )` (but not `$(( ))`) since its operation requires `fork()`ing off a subshell. — Charles Duffy, Jul 19 '18 at 21:12
BTW, use of lower-case variable names in the above examples is intentional; see POSIX specification at http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html: *The name space of environment variable names containing lowercase letters is reserved for applications. Applications can define any environment variables with names from this name space without modifying the behavior of the standard utilities* -- keeping in mind that setting regular shell variables overwrites any like-named environment variable. — Charles Duffy, Jul 19 '18 at 21:14

How do you set custom argument variables in bash?

0 Answers0