send both string and buffer using httplib with python to the server

Question

How can I POST parameters with a file object to a URL using httplib in python web services.

Am using the following scripts:

import httplib
import urllib
params = urllib.urlencode({"@str1":"string1", "@str2":"string2", "@file":"/local/path/to/file/in/client/machine", "@action":"action.php" })
headers = {"Content-type":"application/pdf , text/*" }
conn = httplib.HTTPConnection("192.168.2.17")
conn.request("POST", "/SomeName/action.php", params, headers)
response = conn.getresponse()
print response.status, response.reason
data = response.read()
data
conn.close()

And I have the following output:

200
OK
<html>.....some html code </html>

I wrote some php code for save those string and the file in DB My problem is that, Am only getting the file path as a sting but not my file. May be I have to send the file object like,

file_obj = open("filename.txt","r")
conn.request("POST", "/SomeName/action.php", file_obj, headers)

But I want to send both strings and file. Any suggestions to solve this?

EDIT I change my code as follows: When i send a pdf file, by directly using httplib, to my server the file saves as BIN file.

def document_management_service(self,str_loc_file_path,*args):
    locfile = open(str_loc_file_path,'rb').read()
    host = "some.hostname.com"
    selector = "/api/?task=create"
    fields = [('docName','INVOICE'),('docNumber','DOC/3'),('cusName','name'),('cusNumber','C124'),('category','INVOICE'),('data','TIJO MRS,SOME/DATA/CONTENT,Blahblah,2584.00,blahblah'),('action','create')]
    files = [('strfile','File.pdf',locfile)]
    response = self.post_multipart(host, selector, fields, files)
    print response
    pass

def post_multipart(self,host, selector, fields, files):
    content_type, body = self.encode_multipart_formdata(fields, files)
    h = httplib.HTTP(host)
    h.set_debuglevel(1)
    h.putrequest('POST', selector)
    h.putheader('content-type', content_type)
    h.putheader('content-length', str(len(body)))
    h.putheader('Host', host)
    h.endheaders()
    h.send(body)
    errcode, errmsg, headers= h.getreply()
    return h.file.read()

def encode_multipart_formdata(self, fields, files):
    LIMIT = '----------lImIt_of_THE_fIle_eW_$'
    CRLF = '\r\n'
    L = []
    for (key, value) in fields:
        L.append('--' + LIMIT)
        L.append('Content-Disposition: form-data; name="%s"' % key)
        L.append('')
        L.append(value)
    for (key, filename, value) in files:
        L.append('--' + LIMIT)
        L.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
        L.append('Content-Type: %s' % self.get_content_type(filename))
        L.append('')
        L.append(value)
    L.append('--' + LIMIT + '--')
    L.append('')
    body = CRLF.join(L)
    content_type = 'multipart/form-data; boundary=%s' % LIMIT
    return content_type, body

def get_content_type(self, filename):
    return mimetypes.guess_type(filename)[0] or 'application/octet-stream'

I have debug the request which shows as:

[('content-length', '4191'), ('accept-ranges', 'bytes'), ('server', 'Apache/2.2.12 (Ubuntu)'), ('last-modified', 'Tue, 23 Oct 2012 04:46:36 GMT'), ('etag', 'W/"567dd-105f-4ccb2a7a9a500"'), ('date', 'Tue, 23 Oct 2012 04:46:36 GMT'), ('content-type', 'application/pdf')]
multipart/form-data; boundary=----------lImIt_of_THE_fIle_eW_$

And I didn't try requests,Coz I would like to solve this with httplib(without any external lib)

score 2 · Accepted Answer · edited May 23 '17 at 11:56

2

To post parameters and a file in a body you could use multipart/form-data content type:

#!/usr/bin/env python
import requests # $ pip install requests

file = 'file content as a file object or string'
r = requests.post('http://example.com/SomeName/action.php',
                  files={'file': ('filename.txt', file)},
                  data={'str1': 'string1', 'str2': 'string2'})
print(r.text) # response

requests.post sends to the server something like-this:

POST /SomeName/action.php HTTP/1.1
Host: example.com
Content-Length: 449
Content-Type: multipart/form-data; boundary=f27f8ef67cac403aaaf433f83742bd64
Accept-Encoding: identity, deflate, compress, gzip
Accept: */*

--f27f8ef67cac403aaaf433f83742bd64
Content-Disposition: form-data; name="str2"
Content-Type: text/plain

string2
--f27f8ef67cac403aaaf433f83742bd64
Content-Disposition: form-data; name="str1"
Content-Type: text/plain

string1
--f27f8ef67cac403aaaf433f83742bd64
Content-Disposition: form-data; name="file"; filename="filename.txt"
Content-Type: text/plain

file content as a file object or string
--f27f8ef67cac403aaaf433f83742bd64--

To reproduce it with httplib see POST form-data with Python example.

A simpler solution if your parameters do not contain much data is to pass them in the url query part and leave the body to contain only the file:

#!/usr/bin/env python
import urllib
import requests # $ pip install requests

params = {'str1': 'string1', 'str2': 'string2', 'filename': 'filename.txt'}
file = 'file content as a file object or string, etc'    
url = 'http://example.com/SomeName/action.php?' + urllib.urlencode(params)
r = requests.post(url, data=file, headers={'Content-Type': 'text/plain'})
print(r.text) # response

It corresponds to the following HTTP request:

POST /SomeName/action.php?str2=string2&str1=string1&filename=filename.txt HTTP/1.1
Host: example.com
Content-Length: 39
Content-Type: text/plain
Accept-Encoding: identity, deflate, compress, gzip
Accept: */*

file content as a file object or string

It should be easier to translate to httplib if you need it.

edited May 23 '17 at 11:56

Community

1
1

answered Oct 20 '12 at 07:52

jfs

399,953
195
994
1,670

# sebastian. I reproduced the code with above link (for httplib[link]http://stackoverflow.com/questions/680305/using-multipartposthandler-to-post-form-data-with-python/681182#681182). But when am sending a PDF file it reaches as a BIN file at server side. – SKT Oct 22 '12 at 11:41
I have used the recipe to post multipart using httplib directly [link] http://stackoverflow.com/questions/680305/using-multipartposthandler-to-post-form-data-with-python/681182#681182. But that script showed a Not Found response for my request. Then i add a line **h.putheader('Host', host)** just before the **h.endheaders()** statement, Now It works for me. But my file it saves as a BIN file at server side – SKT Oct 22 '12 at 12:02
@SKT: just modify [`get_content_type(filename)` function](http://stackoverflow.com/a/681182/4279) to return whatever you like. – jfs Oct 22 '12 at 20:58
.. in particular `application/pdf` – jfs Oct 22 '12 at 21:01
@SKT: [update your question](http://stackoverflow.com/posts/12895981/edit) or [create a new one](http://stackoverflow.com/questions/ask) to include a minimal complete code example that reproduces your problem and the description of what you expect to happen, what happens instead. What happens if you use a browser as a client (with a real html form). How does the content that the browser sends differ from your client? Have you tried the `requests`-based client? – jfs Oct 23 '12 at 04:38
@SKT: you should unaccept the answer if it doesn't answer your question. The purpose of trying requests-based solution (and browser-based one) is to test that the problem is indeed in your httplib client and not something else – jfs Oct 23 '12 at 05:25
@ J.F. Sebastian, Actually It didn't solve my problem, but those were helpful, Thats why I accepted as my answer. – SKT Oct 23 '12 at 05:44
@ J.F. Sebastian ,I have solved my problem. It doesnt work because of me!!! :-( (Coz f An invalid input). Thanks for the help. – SKT Oct 24 '12 at 08:23

score 0 · Answer 2 · answered Oct 24 '12 at 08:39

The following code can also solve the problem with transfering file with other meta data using httplib (with out any external libraries):

def document_management_service_success(self,str_loc_file_path,*args):
    locfile = open(str_loc_file_path,'rb').read()
    str_loc_file = locfile.split('#end_pymotw_header')
    initial_data = str_loc_file[0]
    encoded_data = ''.join("{0:08b}".format(ord(c)) for c in initial_data)
    params = urllib.urlencode({'docName':'INVOICE', 'docNumber':'RF/2', 'cusName':'Tijo john', 'cusNumber':'C124', 'category':'INVOICE', 'data':encoded_data})
    headers = {"Accept": "Text/*","Content-Disposition":"form-data" ,"Content-Type": "application/x-www-form-urlencoded, application/pdf, form-data"}
    conn = httplib.HTTPConnection("efiling.nucoreindia.com")
    conn.connect()
    conn.set_debuglevel(1)
    conn.request("POST", "/api/?task=create", params, headers)
    response = conn.getresponse()
    print "Server Response status is"+str(response.status)+"and Reason is,"+str(response.reason)
    print response.getheaders()
    print response.read()
    pass

send both string and buffer using httplib with python to the server

2 Answers2