Why isn't my Python program working? It uses HTTP Headers

Question

EDIT: I changed the code and it still doesn't work! I used the links from the answer to do it but it didn't work!

Why does this not work? When I run it takes a long time to run and never finishes!

import urllib
import urllib2



url = 'https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction'



values = {'inUserName' : 'USER',
          'inUserPass' : 'PASSWORD'}
data = urllib.urlencode(values)


req = urllib2.Request(url, data)

req.add_header('Host', 'www.locationary.com')
req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0')
req.add_header('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8')
req.add_header('Accept-Language', 'en-us,en;q=0.5')
req.add_header('Accept-Encoding','gzip, deflate')
req.add_header('Accept-Charset','ISO-8859-1,utf-8;q=0.7,*;q=0.7')
req.add_header('Connection','keep-alive')
req.add_header('Referer','http://www.locationary.com/')
req.add_header('Cookie','site_version=REGULAR; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; locaCountry=1033; locaState=1795; locaCity=Montreal; jforumUserId=1; PMS=1; TurnOFfTips=true; Locacookie=enable; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; PMS=1; __utmb=47547066.15.10.1324693472; __utmc=47547066; JSESSIONID=DC7F5AB08264A51FBCDB836393CB16E7; PSESSIONID=28b334905ab6305f7a7fe051e83857bc280af1a9; __utmc=47547066; __utmb=47547066.15.10.1324693472; ACTION_RESULT_CODE=ACTION_RESULT_FAIL; ACTION_ERROR_TEXT=java.lang.NullPointerException')
req.add_header('Content-Type','application/x-www-form-urlencoded')



#user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
#headers = { 'User-Agent' : user_agent }



response = urllib2.urlopen(req)
page = response.read()


print page

score 2 · Accepted Answer · edited May 23 '17 at 12:27

The remote server (the one at www.locationary.com) is waiting for the content of your HTTP post request, based on the Content-Type and Content-Length headers. Since you're never actually sending said awaited data, the remote server waits — and so does read() — until you do so.

I need to know how to send the content of my http post request.

Well, you need to actually send some data in the request. See:

Final, "working" version:

import urllib
import urllib2

url = 'https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction'

values = {'inUserName' : 'USER',
          'inUserPass' : 'PASSWORD'}

data = urllib.urlencode(values)
req = urllib2.Request(url, data)

req.add_header('Host', 'www.locationary.com')
req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0')
req.add_header('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8')
req.add_header('Accept-Language', 'en-us,en;q=0.5')
req.add_header('Accept-Charset','ISO-8859-1,utf-8;q=0.7,*;q=0.7')
req.add_header('Connection','keep-alive')
req.add_header('Referer','http://www.locationary.com/')
req.add_header('Cookie','site_version=REGULAR; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; locaCountry=1033; locaState=1795; locaCity=Montreal; jforumUserId=1; PMS=1; TurnOFfTips=true; Locacookie=enable; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; PMS=1; __utmb=47547066.15.10.1324693472; __utmc=47547066; JSESSIONID=DC7F5AB08264A51FBCDB836393CB16E7; PSESSIONID=28b334905ab6305f7a7fe051e83857bc280af1a9; __utmc=47547066; __utmb=47547066.15.10.1324693472; ACTION_RESULT_CODE=ACTION_RESULT_FAIL; ACTION_ERROR_TEXT=java.lang.NullPointerException')
req.add_header('Content-Type','application/x-www-form-urlencoded')

response = urllib2.urlopen(req)
page = response.read()

print page

Don't explicitly set the Content-Length header
Remove the req.add_header('Accept-Encoding','gzip, deflate') line, so that the response doesn't have to be decompressed (or — exercise left to the reader — ungzip it yourself)

Oh...well, I just started Python and Http so could you help me? I need to know how to send the content of my http post request. Thanks. — jacob501, Dec 24 '11 at 21:17
I got rid of the "Content-Length" header and when I ran it, it actually finished for once but gave me this character: "ﾋ" followed by a dot — jacob501, Dec 24 '11 at 21:54
instead of the html of the page. What are these weird characters? — jacob501, Dec 24 '11 at 21:55
That's because it comes back gzipped, **as you have requested by specifying `req.add_header('Accept-Encoding','gzip, deflate')`.** Remove that line. It really looks like you were just blindly trying things — you should really stop and take the time to understand exactly what each piece of code you write does. ...Anyway, there's a working version in my answer. — Matt Ball, Dec 24 '11 at 21:58
Thank you so much! One more thing...how do I open this page with the webbrowser module? Would you do "webbrowser.open(page)" or something else? — jacob501, Dec 24 '11 at 22:02
@jacob501 dude, just [lay this question to rest](http://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work), and ask another separately. This is a completely different task. — Matt Ball, Dec 24 '11 at 22:06

Why isn't my Python program working? It uses HTTP Headers

1 Answers1