URL encoding a string in bash script

Question

I am writing a bash script in where I am trying to submit a post variable, however wget is treating it as multiple URLS I believe because it is not URLENCODED... here is my basic thought

MESSAGE='I am trying to post this information'
wget -O test.txt http://xxxxxxxxx.com/alert.php --post-data 'key=xxxx&message='$MESSAGE''

I am getting errors and the alert.php is not getting the post variable plus it pretty mush is saying

can't resolve I can't resolve am can't resolve trying .. and so on.

My example above is a simple kinda sudo example but I believe if I can url encode it, it would pass, I even tried php like:

MESSAGE='I am trying to post this information'
MESSAGE=$(php -r 'echo urlencode("'$MESSAGE'");')

but php errors out.. any ideas? How can i pass the variable in $MESSAGE without php executing it?

Look for my answer [here](https://stackoverflow.com/a/67448946/10534012). — Darkman, May 08 '21 at 18:19
I'm using `jq` for that, see https://stackoverflow.com/a/34407620/906265 e.g. `printf %s "k=$SOMEVAL"|jq -sRr @uri` — Ivar, Sep 21 '22 at 16:22

score 18 · Answer 1 · answered Feb 09 '17 at 08:10

18

On CentOS, no extra package needed:

python -c "import urllib;print urllib.quote(raw_input())" <<< "$message"

answered Feb 09 '17 at 08:10

Rockallite

16,437
7
54
48

2

This solution is python 2 only – Leo Gallucci Aug 15 '17 at 06:28
3

Python 3 version looks like `python3 -c "import urllib.parse; print(urllib.parse.quote(input())"`. – Jason R. Coombs Sep 11 '17 at 20:20
@JasonR.Coombs you forgot one closing brace :) – Anton K Nov 17 '17 at 16:06
2

A correct Python 3 version looks like `python3 -c "import urllib.parse; print(urllib.parse.quote(input()))"` (tested this time). – Jason R. Coombs Nov 17 '17 at 16:10
1

Python version agnostic: `python -c $'try: import urllib.request as urllib\nexcept: import urllib\nimport sys\nsys.stdout.write(urllib.quote(input()))' <<< "$message"` and adds no `\n` – Mike Mar 14 '19 at 10:54

score 9 · Accepted Answer · edited Feb 04 '22 at 04:51

9

You want $MESSAGE to be in double-quotes, so the shell won't split it into separate words, then pass it to PHP as an argument:

ENCODEDMESSAGE="$(php -r 'echo rawurlencode($argv[1]);' -- "$MESSAGE")"

edited Feb 04 '22 at 04:51

Walf

8,535
2
44
59

answered Aug 09 '12 at 03:39

Gordon Davisson

118,432
16
123
151

Murphy · Answer 3 · 2019-11-26T18:19:11.867

6

Extending Rockallite's very helpful answer for Python 3 and multiline input from a file (this time on Ubuntu, but that shouldn't matter):

cat any.txt | python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.stdin.read()))"

This will result in all lines from the file concatenated into a single URL, the newlines being replaced by %0A.

edited Nov 26 '19 at 18:19

answered May 27 '19 at 07:54

Murphy

3,827
4
21
35

2

No this doesn't work. Replace `sys.stdin.read()` with `input()` instead. Using the `read()` adds `%0A` which is file termination character. – Sudip Bhattarai Nov 22 '19 at 05:37
Using `input()` would only return the first line, but my intention was to get all lines from the file, encoded in one URL. You could easily convert `%0A` to newline again if necessary: `[...] | sed "s/%0A/\n/g"`. – Murphy Nov 22 '19 at 09:25
Works for me as posted by @Murphy (28/01/2020) – iainH Jan 28 '20 at 10:22

URL encoding a string in bash script

3 Answers3

Linked