7

Is it possible to url encode a variable within a shell script?

#!/bin/bash 

now=$(date +"%T") 

DATA=$(wget -q -O -   "http://someurl.com/x.htm?callback=webRequest&exthrs=1&extMode=&fund=1&entitlement=0&skipcache=&extendedMask=1&partnerId=2&output=json&noform=1")

wget -q -O - "http://somewhere.com?abc=$1&responseData=$DATA"

echo "-----COMPLETE----- $now   $1 $RANDOM  
"

I want to url encode the DATA variable, since its results have & in it, it messes up the params in the second wget, is there a way to url encode that DATA variable without using PHP to url encode?

Sven Kahn
  • 487
  • 1
  • 3
  • 16

2 Answers2

12

Here is one method for URL encoding the shell string DATA:

DATA=$(python -c "import urllib, sys; print urllib.quote(sys.argv[1])"  "$DATA")

Here is another:

DATA=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$DATA")
John1024
  • 109,961
  • 14
  • 137
  • 171
  • Yes perfect, they both work (y) thanks. I thought maybe there was a builtin way in shell to do it, but it's okay. Out of perl/python which one has less spend on CPU/Mem you think? Many instances of this script are to be ran concurrently, so every bit helps. – Sven Kahn Apr 20 '15 at 19:04
  • 1
    @SvenKahn On my system (Debian stable), the perl version is faster. – John1024 Apr 20 '15 at 19:13
  • Thank you, I appreciate that you looked into it. – Sven Kahn Apr 20 '15 at 19:23
  • I tried this but it converts = signs which messes up some receiving servers. – smartblonde Aug 29 '21 at 23:19
3

By the way, the way to do this in Python 3 is this:

DATA=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))" "$DATA")
Hari S. Khalsa
  • 121
  • 1
  • 4