0

I try to read a list of file on a remote directory then to get infos on each of them. I'm using awk for this.

My remote directory is on Google drive and I use gdrive, a command line tool, to access it. I store my access-token into a variable $tok.

My first try is to :

gdrive --access-token "$tok" list --bytes | grep 'INC' | awk -F ' ' '{gdrive --access-token "$tok" info $1}'

But I have no output. When I ask for a simple "print", I got the correct results

gdrive --access-token "$tok" list --bytes | grep 'INC' | awk -F ' ' '{print $1}'
1pZHe6YE43zmTuXnsJTMn6da_8CiBrJJH
1G9i7Y1WrRC24bJvw4idAiw55YAVsZhA5
1Vca1pMJbd5106latxTp5CNf6OnjbqkFn
1jt-n5Qw-EKFwyUCX_PNip1juoLSSv8YK
1bP9tNaOhd_Fib1sJdD_v91Vlq8FOlE-i
1NfWdK9ENdSKENexFDoS6hL1AsRqit9IZ

Then, if I perform the command "info" with one value of $1, I got a result :

    gdrive --access-token "$tok" info 1pZHe6YE43zmTuXnsJTMn6da_8CiBrJJH
Id: 1pZHe6YE43zmTuXnsJTMn6da_8CiBrJJH
Name: INC-22-06-01.bck
Path: INC-22-06-01.bck
Mime: application/octet-stream
Size: 1.0 KB
Created: 2022-06-02 16:47:14
Modified: 2022-06-02 16:47:14
Md5sum: 0f3xxxxxx
Shared: False
Parents: 0ABo477gTrFXCUk9PVA
ViewUrl: https://drive.google.com/file/d/1xxxxxxxxx
DownloadUrl: https://drive.google.com/uc?xxxxxxxx

My first conclusion is that my selection works fine and my command to get info works fine too.

So I think I have an issue using the " around the $tok variable in my awk command.

If I try to escape the ", I have an illegal statement error :

gdrive --access-token "$tok" list --bytes | grep 'INC' | awk -F ' ' '{gdrive --access-token \"$tok\" info $1}'
awk: syntax error at source line 1
 context is
    {gdrive --access-token >>>  \ <<< "$tok\" info $1}
awk: illegal statement at source line 1

Can you give me a correct syntax to perform my action ? Thanks in advance

Cyrus
  • 84,225
  • 14
  • 89
  • 153
Guzim Glorantha
  • 47
  • 7
  • Please add your desired output to your question (no comment). – Cyrus Jun 22 '22 at 07:49
  • Not an expert of awk, but if $tok is an access_token, there is a high chance it use base64 or JWT and there should be no " (a-zA-Z0-9 and _-= or +/=). Eg: they don't contains a any ". – NoDataFound Jun 22 '22 at 07:53
  • I guess `$tok` being inside single quotes, it's not replaced by its value by bash. Try to change to double quotes or to something like `| awk '{ print '$tok'}'` (not tested). – Matthieu Jun 22 '22 at 07:59
  • You can also pass variables to awk with `-v` IIRC (check man page). – Matthieu Jun 22 '22 at 08:00
  • Cyrus : You have my desired output (info on each file) – Guzim Glorantha Jun 22 '22 at 08:01
  • @NoDataFound : `gdrive --access-token "$tok" list --bytes | grep 'INC' | awk -F ' ' '{gdrive --access-token $tok info $1}' awk: illegal field $(), name "tok" input record number 1, file source line number 1` – Guzim Glorantha Jun 22 '22 at 08:02
  • [edit] your question to show the output of `gdrive --access-token "$tok" list --bytes` as that `grep 'INC'` part feels like it might be fragile (e.g. you need it to match on a specific field but it could match anywhere on the line) or could otherwise be improved on. – Ed Morton Jun 22 '22 at 11:06
  • 1
    @Matthieu no, do not do `awk '{ print '$tok'}'`, see [how-do-i-use-shell-variables-in-an-awk-script](https://stackoverflow.com/questions/19075671/how-do-i-use-shell-variables-in-an-awk-script) – Ed Morton Jun 22 '22 at 11:09
  • 1
    @EdMorton indeed, much cleaner solution (that I also hinted in the later comment), but thanks for emphasizing :) – Matthieu Jun 22 '22 at 15:03

1 Answers1

2
  • The command gdrive is not an awk command and you cannot execute it in the style of awk '{gdrive ..}'.
  • As $tok is a shell variable, it is not automatically passed to awk.

Would you please try instead (not tested):

#!/bin/bash

while read -r f1 _; do
    gdrive --access-token "$tok" info "$f1"
done < <(gdrive --access-token "$tok" list --bytes | grep 'INC')
  • read -r f1 _ reads a line from stdin, split the line on space character, then assigns variable f1 to the 1st column of the line.
  • <(gdrive ... ) feeds the output of the command to the stdin of while loop via the redirect.
tshiono
  • 21,248
  • 2
  • 14
  • 22
  • 1
    Nice answer. The `grep 'INC'` feels fragile, I wouldn't be surprised if there's a better way to do that part if the OP showed us the output of `gdrive --access-token "$tok" list --bytes`. I've [asked them to show that](https://stackoverflow.com/questions/72711483/bash-awk-how-can-i-use-to-reference-a-variable#comment128439119_72711483). – Ed Morton Jun 22 '22 at 11:05