0

I have the following code to add a custom user-agent to my urllib2 

  self.url_target = (self.EntryText.get()) #it gets the url from a tkinter entry widget
  self.request = urllib2.Request(self.url_target)
  self.request.addheaders = [('User-agent', 'Mozilla/5.0')]
  self.req = urllib2.urlopen(self.request)

Now a silly question. I would like to check if the user-agent is correct passed. How do I do this? By calling self.req.read() or self.req.info() I can't see the browser user agent. For example, here below what I see from google.com. How can I check if the user-agent passed is python2.7 or my custom user agent

Date: Wed, 27 Aug 2014 11:50:41 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Set-Cookie: PREF=ID=59cfdae71ba9d583:FF=0:TM=1409140241:LM=1409140241:S=22PjpHkFs493PNKQ; expires=Fri, 26-Aug-2016 11:50:41 GMT; path=/; domain=.google.it Set-Cookie: NID=67=gOPu9-bAzk01ceW2ukTfpixWIIEl2TRUHKwfDAh5AU6ee-GTXoxBMAHbK6keLCXA5oG_R-9KhPVQ6wMScI28qf-dxqBPOLi66maz_QbBkXtkTpsUcVu-Yohg5-T8w8xz; expires=Thu, 26-Feb-2015 11:50:41 GMT; path=/; domain=.google.it; HttpOnly P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info." Server: gws X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Alternate-Protocol: 80:quic Connection: close

Thank you

The One Electronic
  • 129
  • 1
  • 1
  • 9
  • [use `requests` module](http://stackoverflow.com/questions/2018026/what-are-the-differences-between-the-urllib-urllib2-and-requests-module) – Trevor Boyd Smith Nov 11 '16 at 20:06

1 Answers1

0

Call this url - http://httpbin.org/headers

The source code will have your user agent. :-)

You can embedd this in your code as you want. However for now all I want to show here in the code below is that this url will let you know your user agent,

stuff=urllib2.urlopen("http://httpbin.org/headers").read()

print stuff
{
  "headers": {
    "Accept-Encoding": "identity",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "Python-urllib/2.7",
    "X-Request-Id": "43jhc13b-3dj4-4eb5-8780-ad7cfs4790cd"
  }
}

Hope that answers your question

Md. Mohsin
  • 1,822
  • 3
  • 19
  • 34
  • Thank you @Md. Mohsin However there should be something wrong with my code because with your URL the program raises the following error: File "C:\Python27\lib\urllib2.py", line 531, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 404: NOT FOUND It's strange because normally the code works with other URLs... I must investigate – The One Electronic Sep 01 '14 at 12:15
  • Ok, I modified a bit my code to test it and it works! Moreover the headers are passed correctly. Now I must work a bit on the exceptions... Thank you @Md. Mohsin – The One Electronic Sep 01 '14 at 13:00