How to use urllib3 to POST x-www-form-urlencoded data

Question

I am trying to use urllib3 in Python to POST x-www-form-urlencoded data to ServiceNow API. The usual curl command would look like this

curl -d "grant_type=password&client_id=<client_ID>&client_secret=<client_Secret>&username=<username>&password=<password>" https://host.service-now.com/oauth_token.do

So far, I have tried the following:

import urllib3
import urllib.parse
http = urllib3.PoolManager()
data = {"grant_type": "password", "client_id": "<client_ID>", "client_secret": "<client_Secret>", "username": "<username>", "password": "<password>"}
data = urllib.parse.urlencode(data)

headers = {'Content-Type': 'application/x-www-form-urlencoded'}

accesTokenCreate = http.request('POST', "https://host.service-now.com/outh_token.do", headers = headers, fields= data)
print(accesTokenCreate.data)

However, it does not generate the result similar to curl command and gives errors like below:

Traceback (most recent call last):
  File "/VisualStudio/Python/ServiceNow.py", line 18, in <module>
    accesTokenCreate = http.request('POST', "https://visierdev.service-now.com/outh_token.do", headers = headers, fields= data)
  File "/usr/local/homebrew/lib/python3.7/site-packages/urllib3/request.py", line 80, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "/usr/local/homebrew/lib/python3.7/site-packages/urllib3/request.py", line 157, in request_encode_body
    fields, boundary=multipart_boundary
  File "/usr/local/homebrew/lib/python3.7/site-packages/urllib3/filepost.py", line 78, in encode_multipart_formdata
    for field in iter_field_objects(fields):
  File "/usr/local/homebrew/lib/python3.7/site-packages/urllib3/filepost.py", line 42, in iter_field_objects
    yield RequestField.from_tuples(*field)
TypeError: from_tuples() missing 1 required positional argument: 'value'

Could someone please help me understand how to properly use urllib3 to post such data to the ServiceNow API?

score 2 · Answer 1 · answered Dec 10 '20 at 01:16

According to the urlllib3 documentation, you are not using the request() method properly. Specifically, the fields parameter in your code is not a "parameter of key/value strings AND key/filetuple". It's not suppose to be a URL-encoded string.

To fix your code, simply change the request call's fields parameter to body as in:

accesTokenCreate = http.request(
  'POST', "https://host.service-now.com/outh_token.do", 
  headers=headers, body=data)

Better yet, you can use the request_encode_body() function and pass in the fields directly without urlencode-ing it and let that function call urllib.parse.urlencode() for you (per the same documentation).

score 0 · Answer 2 · answered Aug 15 '23 at 23:51

A shorter way to send a POST application/x-www-form-urlencoded request using urllib3 is to use the request_encode_body method with encode_multipart=False, as documented here:

import json

import urllib3

http = urllib3.PoolManager()

r = http.request_encode_body(
    'POST', 'http://httpbin.org/post', encode_multipart=False, fields={'hello': 'world'}
)
json_data = json.loads(r.data)
print(json_data['form'])  # {'hello': 'world'}
print(json_data['headers'])  # {'Content-Type': 'application/x-www-form-urlencoded'}

How to use urllib3 to POST x-www-form-urlencoded data

2 Answers2