3

I'm running a test on linux using wget/curl. I can get wget to run, but I can't do the same thing with curl.

wget version:

 wget -v -O osu.tst  -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4'  'http://ohiostate.bncollege.com/webapp/wcs/stores/servlet/TBListView?catalogId=10001&storeId=33552&termMapping=N&courseXml=<?xml version="1.0" encoding="UTF-8"?><textbookorder><courses><course dept="CHEM" num="100" sect="16030" term="S12" /></courses></textbookorder> '

curl version:

curl -v -b fftfcook -A "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4" -L  "http://ohiostate.bncollege.com/webapp/wcs/stores/servlet/TBListView?catalogId=10001&storeId=33552&termMapping=N&courseXml=<?xml version="1.0" encoding="UTF-8"?><textbookorder><courses><course dept="CHEM" num="100" sect="16030" term="S12" /></courses></textbookorder>"

curl returns a partial page of content while wget gets what the browser generates. I've tried to modify the cookies/etc.. but wget is a straight "get" so it should work in curl as a straight get as well. the user agent is set to the same...

Any pointers would be helpful.

jcollado
  • 39,419
  • 8
  • 102
  • 133
tom smith
  • 1,035
  • 7
  • 23
  • 39
  • 1
    Try single quotes around the URL, or escape the double quotes within it... or is that a typo? – Dmitri Feb 23 '12 at 08:18
  • ps.. already tried using single quotes around the url, as well as double quotes.. and escaping the inner double quotes in the url.. no difference – tom smith Feb 23 '12 at 08:23
  • curl by default does not handle http redirect while wget does. Maybe the original url returns a redirection to the right content? In this case wget will get it, but not curl (unless you use -L option) – jap1968 Feb 23 '12 at 08:48

1 Answers1

3

The problem seems to be that the GET data is not URL-encoded. Only a limited set of characters can appear in the URL or GET data without such encoding, though when entered into the browser's address bar the browser often handles this for you.

Rather than tacking the GET data onto the end of the URL, use the -d and --data-urlencode options for each value, and pass the -G option to treat these as GET data. So you'd end up with the following:

curl -v -b fftfcook -A "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4" -L  'http://ohiostate.bncollege.com/webapp/wcs/stores/servlet/TBListView' -G -d 'catalogId=10001' -d 'storeId=33552' -d 'termMapping=N' --data-urlencode 'courseXml=<?xml version="1.0" encoding="UTF-8"?><textbookorder><courses><course dept="CHEM" num="100" sect="16030" term="S12" /></courses></textbookorder>'

The various -d and --data-urlencode options will be joined together to form the GET data on the URL. Note also the changes from double to single quotes, to prevent the double quotes in the form data from causing a problem.

Dmitri
  • 9,175
  • 2
  • 27
  • 34