2

I want to capture how parameters are being sent. Usually what I do is to make a request and check on Firebug's params tab what are the parameters sent. However, when I try to do this on the following site (http://www.infraero.gov.br/voos/index_2.aspx), it doesn't work - I can't see what are the parameters in order to repeat this request using curl. How can I get it? I'm not sure but I think that cookies are being used.

EDIT

I was able to get the request content, but couldn't understand it. It seems it uses javascript to generate the proper request. How can I reproduce this request via cURL?

João Daniel
  • 8,696
  • 11
  • 41
  • 65
  • They're using post... there wouldn't be any 'parameters. Use HTTPfox or firebug's 'net' tab to view the low level http communication and you'll see exactly what's going across the wire. – Marc B Apr 22 '12 at 21:12
  • It just show the request header and not its content. How can I get the necessary information to replay this request via curl? – João Daniel Apr 22 '12 at 21:17
  • 1
    POST data goes in the body of the request, not the headers. e.g. look in HTTPFox's "Content" tab. – Marc B Apr 22 '12 at 21:19
  • Ah, now I see the request content. However I can't understand what's going on there. It seems javascript is being used. How can I reproduce this request via cURL? – João Daniel Apr 22 '12 at 21:42

2 Answers2

1

Did you see this previous question cURL post data to asp.net page ? That might answer the question right there (all I did was search "ASP.NET cURL"). And this one: Unable to load ASP.NET page using Python urllib2 talks about Python, but it approaches it in a way that should translate to cURL.

But for my $0.02, I wouldn't bother trying to untangle ASP.NET's and __VIEWSTATE and javascript. Is it an absolute requirement that you use cURL?

I think you would be better off using a client that works more like a real browser and understands javascript. That's a bit of work, but it isn't as bad as it sounds. I've done this before with http://watirwebdriver.com/ and a short Ruby script. Here's how to do it with Python and Mechanize (this is probably a bit more lightweight). http://phantomjs.org/ is another option that you script using javascript. If you Google "Scraping ASP.NET" you will see that this is a common problem.

Community
  • 1
  • 1
Eli
  • 5,500
  • 1
  • 29
  • 27
0

You didn't say how you want it done, but you can send the request with curl simply with curl -d name1=contents1&name2=contents2 [TARGETURL] etc.

Note that you probably first need to fetch the main page and extract the "__VIEWSTATE" form field and submit back that (VERY huge) contents to get your submission accepted.

Daniel Stenberg
  • 54,736
  • 17
  • 146
  • 222
  • Yes, I know how to curl a post request. I believe my problem is with that "__VIEWSTATE". I'm not used to ASP, so I have no idea what is it, or how to deal with it. Is there a 'shortcut' so I can send a request without going deep in learning it? – João Daniel Apr 26 '12 at 20:23
  • As it is filled in as a hidden form field you can assume your curl request needs to send it on. Just get the HTML page, extract the __VIEWSTATE value and pass it on in your curl request. – Daniel Stenberg Apr 27 '12 at 06:14