1

I would like to log various things when calling functions in a bash script and so I am trying to have a special place to do this

function CallAndLog {
  echo "$@" > /tmp/debug
  res=$("$@" 2> /tmp/error)
  if [ $? -gt 0 ]
  then
    echo "error : $(</tmp/error)" >> /tmp/log
  fi
  echo "$res" >> /tmp/log
}

val="a test"
CallAndLog curl --data '"'"query=$val"'"' http://google.com

I get this

  • /tmp/debug

curl --data "query=a test" http://google.com

  • /tmp/error

curl: (6) Couldn't resolve host 'test"' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 967 100 959 100 8 1060 8 0:00:01 --:--:-- 0:00:01 1572

  • /tmp/log

<!DOCTYPE html>...

/tmp/debug is perfect, this is exactly what I want to execute, and if I copy and paste that it works. However the error suggests that "a" and "test" were split into 2 and curl tried "test" as the url. This is what I don't understand

I have tried LOTS of different things : $, $@, "$", "*@", various ways to put the single and double quotes, so after hours of struggling I call for your help !!! Many Thanks

Thomas
  • 8,306
  • 8
  • 53
  • 92
  • possible duplicate of [URLEncode from a bash script](http://stackoverflow.com/questions/296536/urlencode-from-a-bash-script) – glenn jackman Dec 19 '13 at 16:41
  • I don't see how you get `http://google.com` in debug – Rico Dec 19 '13 at 16:43
  • Since "http://google.com" never appears in your example, the debug/log output cannot correspond to the input you show. It's impossible to know if the rest of the input corresponds, but I am sceptical since the example should fail in a different way. The output looks like it comes from `CallAndLog curl --data '"'query=$val'"' http://google.com` – rici Dec 19 '13 at 16:58
  • @glennjackman I don't think it is a duplicate as there is something fishy with the $@ or $* and the quotes. url encode might be a way to get around there but I won't know why it was failing – Thomas Dec 19 '13 at 18:46

2 Answers2

1

You're executing correctly, but logging incorrectly. That why the log appears fine while the command fails.

Invocation should be simply 

CallAndLog curl --data "query=$val"

To log the statement, use 

printf "%q " "$@" >> /tmp/debug

Your quoting will be reformatted, but will be semantically identical (i.e. you can copy-paste the statement to run the same command the same way). This is due to the fact that quoting is shell syntax that controls various forms of splitting and expansion, and only the effects (and not the syntax) is carried into the function.

Here is an example:

function CallAndLog {
  printf "%q " "$@" > /tmp/debug
  echo >> /tmp/debug # add line feed
  res=$("$@" 2> /tmp/error)
  if [ $? -gt 0 ]
  then
    echo "error : $(</tmp/error)" >> /tmp/log
  fi
  echo "$res" >> /tmp/log
}

val="a test"
CallAndLog curl --data "query=$val" http://google.com

After running that, we can look at the output log files:

$ cat /tmp/debug
curl --data query=a\ test http://google.com 

$ cat /tmp/error
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   971  100   959  100    12   6101     76 --:--:-- --:--:-- --:--:-- 43590

$ cat /tmp/log
<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, ...
that other guy
  • 116,971
  • 11
  • 170
  • 194
  • If you do CallAndLog 'curl ..."query=$val"', $val will not be replaced by its value since it is single quoted – Thomas Dec 19 '13 at 18:41
  • I haven't tested your log statement yet but I can tell you I am not executing correctly (see /tmp/error) – Thomas Dec 19 '13 at 18:42
  • @Thomas it would be replaced by its value upon `eval`uation – that other guy Dec 19 '13 at 18:42
  • @Thomas No, I meant that your execution line in CallAndLog is correct, but your invocation of CallAndLog is wrong because you try to please the incorrect log statement rather than the correct execution line. The answer shows how to invoke it correctly. – that other guy Dec 19 '13 at 18:44
  • I have tried that but itdoesn't get replaced, it stays as $val – Thomas Dec 19 '13 at 18:52
  • @Thomas Ok, I see that the two different approaches in the answer were confusing. I deleted the last one so no one will be tempted to copy commands without reading the context. Try again now. – that other guy Dec 19 '13 at 19:00
  • @Thomas Please be careful to read the post in its entirety and use quotes exactly as shown. I posted an example showing it works as advertised. If you get a different result, please post the code you're using and the errors you get. – that other guy Dec 19 '13 at 22:45
0

As you've guessed, the problem lies in res=$("$@" 2> /tmp/error). When executing your sample curl statement the quotes are being lost, so what is actually being passed is:

curl --data query=a test http://google.com

So, how do we fix this? Well, I think the easiest way is to modify how you're calling your statement:

val="a test"
CallAndLog curl --data query=\"${val}\"

By escaping the quotes, we should now be sending in the proper syntax:

curl --data query="a test" http://google.com

This code works locally

~/tmp › cat /tmp/debug 
curl --data query="a test" http://google.com

~/tmp › cat /tmp/log 
<!DOCTYPE html> <html lang=en>   <meta charset=utf-8>   <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">   <title>Error 405 (Method Not Allowed)!!1</title>   <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}} </style>   <a href=//www.google.com/><img src=//www.google.com/images/errors/logo_sm.gif alt=Google></a>   <p><b>405.</b> <ins>That’s an error.</ins>   <p>The request method <code>POST</code> is inappropriate for the URL <code>/</code>.  <ins>That’s all we know.</ins>

Now, the error result coming back from Google is another problem altogether.

Donovan
  • 15,917
  • 4
  • 22
  • 34
  • I don't believe this analysis is correct. `"$@"` will preserve word-separation in the arguments. Furthermore, OP does *not* want to pass actual double-quotes to curl, afaik. – rici Dec 19 '13 at 16:59
  • True, but "$@" preserves word-separation only until the point that it's being sent into `$()`, that's when it loses them in this example. – Donovan Dec 19 '13 at 17:03
  • Not so. `(set -- bash -c echo\ \"\$1\" _ "a b" c; res=$("$@"); echo "$res")` – rici Dec 19 '13 at 17:12
  • It's clear from the OP that it's losing the inner quotes around `"a test"` – Donovan Dec 19 '13 at 17:14
  • what inner quotes around "a test"? `val="a test"` does not put any quotes into `$val`. (And they are not being expected by curl, either. The correct invocation is `--data query=a%20test`). You certainly don't want `--data \"query=a%20test\"`. TIAS. – rici Dec 19 '13 at 17:16
  • :sigh: you **do** want `--data query="a test"`, the _inner_ quotes are the ones around `"a test"` that are being passed in via the call to `CallAndLog`. Replacing the space with `%20` simply does the same thing. Try it locally. – Donovan Dec 19 '13 at 17:23
  • I did. Did you? Replacing the space with %20 definitely doesn't do "the same thing": you need to be aware of who is dequoting what when. Your way will actually send the double quotes to the web server, as well as a literal space. (http://google.com will reject a url with a space in it, as it should: it's your responsibility to urlencode.) – rici Dec 19 '13 at 19:02
  • works for me, even though there are still some problems if I do -- data \"q1=${val1}&q2=${val2}\" (but curl accepts multiple --data so will do) – Thomas Dec 19 '13 at 19:06
  • Any idea why my code was failing in the first place ? Copy and paste is cool but I would like to learn something from that :) – Thomas Dec 19 '13 at 19:06
  • Yes, as I said (and have been incorrectly questioned on) is that you were losing the quotes around `a test` when you passed it into the sub-shell. The escaped quotes in the call prevent that behavior. – Donovan Dec 19 '13 at 19:11
  • As others have pointed out, a space is not valid in Google's query syntax, but the code above solves the OP question. – Donovan Dec 19 '13 at 19:13
  • Right, I am not looking to send this to google.com it was just an example. – Thomas Dec 20 '13 at 06:50
  • @Donovan The result in /tmp/debug is : curl --data "query=a test" http://google.com. At what point was I losing the quotes ? I had some '"' in my call to get them – Thomas Dec 20 '13 at 06:52
  • @Thomas as I specified in my original answer; you're losing them when you pass them into the sub-shell created by `res=$("$@" 2> /tmp/error)` – Donovan Dec 20 '13 at 14:43
  • @Donovan so why were they in /tmp/debug ? (sorry for being annoying here but I want to get to the bottom of this) – Thomas Dec 20 '13 at 19:09
  • They weren't. If you look again, you'll see `"query=a test"` -- there are no quotes around `"a test"`, yes? In the example I posted in my answer, you'll see `query="a test"`. See the difference? – Donovan Dec 20 '13 at 19:10
  • if you need quotes around the entire string, you can easily add that. – Donovan Dec 20 '13 at 19:12