Running Query from text file

Question

I'm trying to run a big query query from the command line, but because my query is very long I've written it in a text file. The query works from the GUI and I'm overwriting a table that already exsists

bq query --allow_large_results --replace --destination_table=me.Tbl_MyTable  '`cat query.txt`'

However, I'm getting error results:

Error in query string: Error processing job 'dev:bqjob_r_00000123456789456123_1': Encountered " "\'cat query.txt\' "" at line 1, column 1. Was expecting: EOF

Do I need to put the entire file path in the .txt filename? (this doesn't seem to make a difference)
Are there any characters I need to be careful with in the text file (e.g. "\" or quotation marks) ?
I'm using where clauses and group by clauses - is that an issue?

score 3 · Accepted Answer · answered Jan 10 '17 at 14:30

3

Instead of cat, just pipe the input from the file. The command would be:

bq query --allow_large_results --replace --destination_table=me.Tbl_MyTable < query.txt

This will send the contents of query.txt to the bq tool.

answered Jan 10 '17 at 14:30

Elliott Brossard

32,095
2
67
99

score 2 · Answer 2 · answered Jan 11 '17 at 03:27

2

Elliot is right, now if you want to cat, sed or anything, pipe it:

cat query.txt | bq query

answered Jan 11 '17 at 03:27

Felipe Hoffa

54,922
16
151
325

I tried this, but it didn't work for me, doesn't recognise cat as a function (??) – REdim.Learning Jan 11 '17 at 14:52
what OS are you using? – Felipe Hoffa Jan 11 '17 at 23:10
I'm using Windows 7 – REdim.Learning Jan 13 '17 at 07:44
Solution would involve something like http://stackoverflow.com/questions/60244/is-there-replacement-for-cat-on-windows, http://stackoverflow.com/questions/14574170/how-do-i-use-a-pipe-to-redirect-the-output-of-one-command-to-the-input-of-anothe, but I don't have Win7 to test it... – Felipe Hoffa Jan 13 '17 at 09:22

Running Query from text file

2 Answers2

Linked