python requests - 403 forbidden

Question

I have this

payload = {'from':'me', 'lang':lang, 'url':csv_url}
headers = {
   'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11'
}

api_url = 'http://dev.mypage.com/app/import/'
sent = requests.get(api_url , params=payload, headers=headers)

I just keep getting 403. I am looking up to this requests docs

what am I doing wrong?

UPDATE:

The url only accepts loggedin users. how can I login there with requests?

"You first need to confirm if you have encountered a "No directory browsing" problem. You can see this if the URL ends in a slash '/' rather than the name of a specific Web page (e.g. .htm or .html). If this is your problem, then you have no option but to access individual Web pages for that Web site directly." via http://www.checkupdown.com/status/E403.html. Also do you have access to that page. Could it be a CORS issue? — Craicerjack, Jan 15 '15 at 15:15
The problem doesn't seem to be with the way you are using the `requests` package, but probably the API. — dursk, Jan 15 '15 at 15:20
@Craicerjack you are totally right :) this is a page which needs login. how can I overcome this now? — doniyor, Jan 15 '15 at 15:27
Two different answers here - http://stackoverflow.com/questions/11892729/how-to-log-in-to-a-website-using-pythons-requests-module http://stackoverflow.com/questions/16994044/using-python-requests-library-to-login-to-website — Craicerjack, Jan 15 '15 at 15:29

score 1 · Answer 1 · answered Jan 15 '15 at 15:44

1

This is how it's usually done using a Session object:

# start new session to persist data between requests
session = requests.Session()

# log in session
response = session.post(
    'http://dev.mypage.com/login/',
    data={'user':'username', 'password':'12345'}
)

# make sure log in was successful
if not 200 <= response.status_code < 300:
    raise Exception("Error while logging in, code: %d" % response. status_code)

# ... use session object to make logged-in requests, your example:
api_url = 'http://dev.mypage.com/app/import/'
sent = session.get(api_url , params=payload, headers=headers)

You should obviously adapt this to your usage scenario.

The reason a session is needed is that the HTTP protocol does not have the concept of a session, so sessions are implemented over HTTP.

answered Jan 15 '15 at 15:44

Reut Sharabani

30,449
6
70
88

Are you using the session object to make the second request? – Reut Sharabani Jan 15 '15 at 15:56
yes, i am using the session object. i am even using inside ``with`` – doniyor Jan 15 '15 at 15:57
It's possible the website doesn't let you log in but returns a 200 on log in failure. See if that's the case. – Reut Sharabani Jan 15 '15 at 15:58
I did ``print session.cookies._cookies`` and it is empty. does it say something to you? – doniyor Jan 15 '15 at 16:06
Well, only that you're not logged in... Try using the browser (something like firebug can work) to see what happens when you do this manually. – Reut Sharabani Jan 15 '15 at 16:08
i can login with the given credentials. only thru api itsnot working – doniyor Jan 15 '15 at 16:09
You need to make sure you're doing the same things done as a browser logs in. Use firebug's network monitoring tool to see what requests are sent, and what responses are received. – Reut Sharabani Jan 15 '15 at 16:10
i printed the headers of login request, it has ``python-requests CPython`` as user-agent. this is the reasion. but why it is overriding it? – doniyor Jan 15 '15 at 16:12
Well, I can't set up a environment to replicate this now, but it's **highly unlikely** the problem is with requests. – Reut Sharabani Jan 15 '15 at 16:14
i asked another question about this case – doniyor Jan 15 '15 at 16:24

python requests - 403 forbidden

1 Answers1