Basic Auth urllib2 with HTTPPasswordMgrWithDefaultRealm and POST data

Question

I have this cURL call that works perfectly:

curl -H 'X-Requested-With: SO demo' -d 'parameter=value' https://username:password@api.domain.com/api/work/

My conversion does not work.

import urllib2
# Create a password manager.
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add the username and password.
top_level_url = 'https://api.server.com'
password_mgr.add_password(None, top_level_url, 'username', 'password')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
# Create "opener" (OpenerDirector instance).
opener = urllib2.build_opener(handler)
# Install the opener so all calls to urllib2.urlopen use our opener.
urllib2.install_opener(opener)
# Create request.
headers = {'X-Requested-With':'SO demo.'}
uri = 'https://api.domain.com/api/work/'
data='parameter=value'
req = urllib2.Request(uri,data,headers)
# Make request to fetch url.
result = urllib2.urlopen(req)
urllib2.HTTPError: HTTP Error 401: Unauthorized

Here's what I don't get. The same server has a separate API which similar code does work on, where the only thing that has changed is the parameter and uri. Note the cURL call works on both API calls.

Second API cURL call (that works):

curl -H 'X-Requested-With: SO demo' -d 'parameter=value' https://username:password@api.domain.com/api2/call.php

Equivalent code that works below:

import urllib2
# Create a password manager.
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add the username and password.
top_level_url = 'https://api.server.com'
password_mgr.add_password(None, top_level_url, 'username', 'password')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
# Create "opener" (OpenerDirector instance).
opener = urllib2.build_opener(handler)
# Install the opener.
# Now all calls to urllib2.urlopen use our opener.
urllib2.install_opener(opener)
# Create request.
headers = {'X-Requested-With':'SO demo.'}
uri = 'https://api.server.com/api2/call.php'
data='parameter=value'
req = urllib2.Request(uri,data,headers)
# Make request to fetch url.
result = urllib2.urlopen(req)
# Read results.
result.read()

Why does urllib2 work when the uri ends with a '.php', but not work when the uri ends with a '/'?

score 2 · Answer 1 · answered Jan 28 '12 at 03:29

2

In the first request you are setting:

uri = 'https://api.domain.com/api/work/'

But if you were to do it the same as the second run, you probably meant to write it as:

uri = 'https://api.server.com/api/work/'

answered Jan 28 '12 at 03:29

Spike

5,040
5
32
47

The second Python run was a comparison to the second curl run against the same uri. – paragbaxi Jan 30 '12 at 17:55
I guess I'm confused then why sometimes you're sometimes calling api.domain.com and sometimes calling api.server.com. – Spike Jan 30 '12 at 21:31
The server has two API versions with different content for each version. – paragbaxi Jan 31 '12 at 23:10

score 1 · Answer 2 · edited May 23 '17 at 11:48

1

From Python urllib2 Basic Auth Problem

The problem [is] that the Python libraries, per HTTP-Standard, first send an unauthenticated request, and then only if it's answered with a 401 retry, are the correct credentials sent. If the ... servers don't do "totally standard authentication" then the libraries won't work.

This particular API does not respond with a 401 retry on the first attempt, it responds with an XML response containing the message that credentials were not sent.

edited May 23 '17 at 11:48

Community

1
1

answered Mar 02 '12 at 20:32

paragbaxi

3,965
8
44
58

Be careful when posting copy and paste boilerplate/verbatim answers to multiple questions, these tend to be flagged as "spammy" by the community. If you're doing this then it usually means the questions are duplicates so flag them as such instead: http://stackoverflow.com/questions/9495279 – Kev Mar 03 '12 at 00:01
Thanks Kev. It turns out the solution was the same but I was only seeing different symptoms of the problem. Should it still be a dup? – paragbaxi Mar 04 '12 at 22:08

Basic Auth urllib2 with HTTPPasswordMgrWithDefaultRealm and POST data

2 Answers2