0

I need to write a function that downloads and stores the today's list of pre-release domains .txt file from http://www.namejet.com/pages/downloads.aspx. So as today is 8th of October you want to get the file "Monday, October 08, 2012". Tried with requests but didn't work. I'm having trouble because the file is not stored on a fixed URL but is hidden behind some Javascript.

New Folder
  • 97
  • 1
  • 2
  • 14

2 Answers2

2

This one's a little tricky as you're dealing with ASP.NET's postback system. If this is for anything other than a personal script, I'd be wary as you're effectively not only using another site's data, but reverse engineering their software as well (however, IANAL and have no idea about legalities around these issues in web systems).

What you're going to want to do is check the POST data (using Firebug, Chrome developer tools, etc) and look for the __EVENTTARGET and __VIEWSTATE attributes of the form object. You'll have to decode the __VIEWSTATE to be readable (check out http://ignatu.co.uk/ViewStateDecoder.aspx). From there, I think you should be able to figure out how to get the data you're looking for.

From Python, it's as easy as:

from urllib2 import urlopen
from urllib import urlencode

data = urlopen('url', urlencode({
    '__VIEWSTATE': 'foo',
    '__EVENTTARGET': 'bar',
})).read()
Community
  • 1
  • 1
Demian Brecht
  • 21,135
  • 5
  • 42
  • 46
  • Hello Demian. I know it's little late for follow up. I used Chrome Developer tools and found two attributes you mentioned! Now where do I proceed? How do I get the list of pre-release domains from that website? – New Folder Oct 09 '12 at 11:27
1

Actually you get text file in response to a POST request with several base64-encoded request parameters. Feel free to play with it

use Firebug or any other debug tool to see the POST content and parameters

Marat
  • 15,215
  • 2
  • 39
  • 48