Converting cURL POST into urllib2 without Requests

Question

I currently have a cURL command that works in my environment. I need to convert this into a python 2.7 urllib equivalent and need some assistance

cURL command in question:

curl -k -v -XPOST -H "Authorization: Bearer $Token" -F file=@${local_filename} https://interesting_url.com/

I need the ability to support both a Header (-H) and a form/file (-F) and have yet to be successful.

This post includes a discussion on headers, but I have not been able to get the '-F' equivalent working

If you can use Requests, you can use https://curl.trillworks.com/ to translate the cURL command — Nick T, Sep 13 '17 at 20:31
Requests works just fine with 2.7, and you should be able to install it by [downloading the repo](https://github.com/requests/requests/releases) and running `python setup.py install` — Nick T, Sep 13 '17 at 20:48
Sorry - I should have been more clear. The devices where this is running from need to run "pristine" python and can't install extras unfortunately — Numpty, Sep 13 '17 at 20:57

Oleh Rybalchenko · Accepted Answer · 2017-09-13T20:58:33.173

With python3 you can do it with requests:

import requests

headers = {'Authorization': 'Bearer $Token'}

files = [('file', open('${local_filename}', 'rb'))]

requests.post('https://interesting_url.com/', headers=headers, files=files, verify=False)

Uploading files with urllib2 is quite a complicated task (example). So I suggest you requests.

Long answer without using pip and third-party packages

You can implement custom class MultiPartForm and then use it to encode files:

import itertools
import mimetools
import mimetypes
from cStringIO import StringIO
import urllib
import urllib2

class MultiPartForm(object):
    """Accumulate the data to be used when posting a form."""

    def __init__(self):
        self.form_fields = []
        self.files = []
        self.boundary = mimetools.choose_boundary()
        return

    def get_content_type(self):
        return 'multipart/form-data; boundary=%s' % self.boundary

    def add_field(self, name, value):
        """Add a simple field to the form data."""
        self.form_fields.append((name, value))
        return

    def add_file(self, fieldname, filename, fileHandle, mimetype=None):
        """Add a file to be uploaded."""
        body = fileHandle.read()
        if mimetype is None:
            mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
        self.files.append((fieldname, filename, mimetype, body))
        return

    def __str__(self):
        """Return a string representing the form data, including attached files."""
        # Build a list of lists, each containing "lines" of the
        # request.  Each part is separated by a boundary string.
        # Once the list is built, return a string where each
        # line is separated by '\r\n'.  
        parts = []
        part_boundary = '--' + self.boundary

        # Add the form fields
        parts.extend(
            [ part_boundary,
              'Content-Disposition: form-data; name="%s"' % name,
              '',
              value,
            ]
            for name, value in self.form_fields
            )

        # Add the files to upload
        parts.extend(
            [ part_boundary,
              'Content-Disposition: file; name="%s"; filename="%s"' % \
                 (field_name, filename),
              'Content-Type: %s' % content_type,
              '',
              body,
            ]
            for field_name, filename, content_type, body in self.files
            )

        # Flatten the list and add closing boundary marker,
        # then return CR+LF separated data
        flattened = list(itertools.chain(*parts))
        flattened.append('--' + self.boundary + '--')
        flattened.append('')
        return '\r\n'.join(flattened)

with open(`local_file.txt`) as f:
    form = MultiPartForm()
    form.add_file('file', `local_file`, 
                  fileHandle=f)

    # Build the request
    request = urllib2.Request('https://interesting_url.com/')
    request.add_header('Authorization', 'Bearer $Token')
    body = str(form)
    request.add_header('Content-type', form.get_content_type())
    request.add_header('Content-length', len(body))
    request.add_data(body)

    print
    print 'OUTGOING DATA:'
    print request.get_data()

    print
    print 'SERVER RESPONSE:'
    print urllib2.urlopen(request).read()

The source

@Numpty, see the updated answer with complicated version of the request without installing additional packages — Oleh Rybalchenko, Sep 13 '17 at 21:00
Thanks - might take me a day to adapt this and test. You weren't kidding about complexity :| — Numpty, Sep 13 '17 at 22:39

score 0 · Answer 2 · answered Sep 13 '17 at 21:06

Try looking into the requests library. You can install it using pip install requests after that just search up some example code online. Depending on your application it will vary. The requests library allows you to do a lot of things. There is a lot of documentation. I highly recommend that you check it out.

Converting cURL POST into urllib2 without Requests

2 Answers2

Long answer without using pip and third-party packages