0

I ain't trying to be spoon-fed, but I'm trying to emulate a user-agent, and I can't seem to understand how I can with my code so far, my attempts have failed. (The reason I'm trying to emulate a user-agent, is because I'm being blocked by a(n) 503 Error)

import urllib
import urllib.request
from bs4 import BeautifulSoup

url = "https://website.com/random-junk"
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page, "html.parser")

print(soup.title)

I'm new to Python, so I've been looking into Documents, and other sources that've recommended to me, so if I'm using deprecated methods, or I should do something another way, please tell me! Thanks in advance!

IndieGuts
  • 13
  • 5
  • Check [this](http://stackoverflow.com/questions/24226781/changing-user-agent-in-python-3-for-urrlib-request-urlopen). – Sangbok Lee Mar 09 '17 at 06:23
  • 2
    Possible duplicate of [Changing User Agent in Python 3 for urrlib.request.urlopen](http://stackoverflow.com/questions/24226781/changing-user-agent-in-python-3-for-urrlib-request-urlopen) – Sangbok Lee Mar 09 '17 at 06:23
  • Yes, I've seen other posts similar to what I'm asking.. and I may need to be spoon-fed a little, if anyone could possibly do that for me :) However, I'm using BeautifulSoup, I don't understand how I can use the same thing, but with Soup. – IndieGuts Mar 09 '17 at 07:00

1 Answers1

0
import urllib
import urllib.request
from bs4 import BeautifulSoup

url = "https://website.com/random-junk"
req = urllib.request.Request(url)
req.add_header('User-agent', 'Mozilla/5.0') #set user-agent header
r = urllib.request.urlopen(req)
soup = BeautifulSoup(r, "html.parser")
print(soup.title)
Arun
  • 136
  • 1
  • 10
  • Thank you, sorry for you having to spoon-feed me, I'm a complete moron in general, and a complete noob to Python. – IndieGuts Mar 10 '17 at 08:32