Log in to website using Python Requests module

Question

I'm trying to use the requests module to log in to a website. I'm not sure what to reference in the html form to post the username and password. Here is the form I'm trying to use to post with to log in:

<div class="login-box contents" id="login">
                        <!--<div class="login-instruction">
                            <label class="fl-label"> Enter your information below to login. </label>
                        </div>-->
                        <div class="login-username">
                            <label for="username" class="fl-label">Username: </label>
                            <div class="clearboth"></div>


                            <input id="proxyUsername" name="proxyUsername" class="required" tabindex="1" maxLength="100" type="text" value="" onChange="remove_Error()" autocomplete="off"/>

                        </div>
                        <div class="float-right">
                            <b><input type="checkbox" id="proxyRememberUser" name="proxyRememberUser" tabindex="-1" value="checked">&nbsp;Remember Username</input></b>
                        </div>
                        <br/>
                        <div class="login-password">
                            <label for="password" class="fl-label">Password: </label>
                            <div class="clearboth"></div>

                            <input id="proxyPassword" name="proxyPassword" class="required" tabindex="2" maxLength="50" type="password" value="" onChange="remove_Error()" autocomplete="off" />
                        </div>

I'm trying to figure out where/how in the form I tell it to put the username and password. So in the code below, the keys for the payload_login variable are not correct:

import requests


username = raw_input('Please enter your username: ')
password = raw_input('Please enter your password: ')

payload_login = {
    'Username': username,
    'Password': password
}

with requests.Session() as s:
    con = s.post('somewebsite.com', \
       data=payload_login)

Answer 1

As @hlt have commented, you must name field the same, as they are named in the form.

Also server may validate "Remember Username" checkbox, so it is better to include it in your request.

payload_login = {
    'proxyUsername': username,
    'proxyPassword': password,
    'proxyRememberUser': true
}

If this does not work for you, it means what site send auth-data is different way. For example, some JS-script may add hidden data in request, or encode some fields.

To find it out, you need to search this HTTP-request in your Browser's Developter Panel or in a external HTTP-sniffer (like Fiddler).

Answer 2

You will need to add the username and password as authentication header to the request. You can find more details here: http://docs.python-requests.org/en/latest/user/advanced/

You could simply use s.auth = (username, password). Thats the easiest way to implement it. But if you want to add it into the header yourself, you will first have to build the header. The authorization header contains the username and password which need to be b64encoded. For example:

[In python3]

from base64 import b64encode
import requests

username = input('Please enter your username: ')
password = input('Please enter your password: ')

authHandler = '{0}:{1}'.format(username, password).encode()
authHeader = {'Authorization' : 'Basic {0}'.format(b64encode(authHandler).decode("ascii"))}
with requests.Session() as s:
    con = s.post('somewebsite.com', headers=authHeader)

[In python2.7]

from base64 import b64encode
import requests

username = raw_input('Please enter your username: ')
password = raw_input('Please enter your password: ')

authHandler = '{0}:{1}'.format(username, password)
authHeader = {'Authorization' : 'Basic {0}'.format(b64encode(authHandler))}
with requests.Session() as s:
    con = s.post('somewebsite.com', headers=authHeader)