0

After searching for how to POST large files using python, I came across this and I wrote a program based on that, but when I run it I get the following error message:

  Traceback (most recent call last):
  File "Test3.py", line 187, in <module>
    main()
  File "Test3.py", line 184, in main
    do_upload(options, args)
  File "Test3.py", line 48, in do_upload
    response = urllib2.urlopen(request)
  File "C:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
  File "C:\Python27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python27\lib\urllib2.py", line 438, in error
    return self._call_chain(*args)
  File "C:\Python27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 521, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: Internal Server Error

and here is my code, to run the program I use --upload "path_to_file" [space] "filename". I'm new to python programming so most of this still confuses me.

def do_upload(options, args):
    url = 'http://127.0.0.1/test_server/upload'

    path = args[0]
   # print path
    filename = args[1]
    if not os.access(args[0], os.F_OK):
                print "Directory/file Doesn't exist"
                exit(1)

    os.chdir(path)
    f = open(filename, 'rb')
    mmapped_file_as_string = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)


    request = urllib2.Request(url, mmapped_file_as_string)
    contenttype = mimetypes.guess_type(filename)[0] 
    request.add_header(contenttype, 'application/octet-stream')
    response = urllib2.urlopen(request)


    #close everything
    mmapped_file_as_string.close()
    f.close()

UPDATE

I have changed the code from above and now I'm getting some socket error.

Updated code

def do_upload(options, args):

    host = 'http://localhost:80'
    selector = '/test_server/upload'
    url = 'http://localhost:80/test_server/upload'

    if len(args) == 2:
        print "len of args = 2"
        files = "File is " + str(args[1])
        print files
        path = args[0]
        print "Path is " + str(args[0])



    content_type, body = encode_multipart_formdata(files)
    h = httplib.HTTP(host)
    h.putrequest('POST', selector)
    h.putheader('content-type', content_type)
    h.putheader('content-length', str(len(body)))
    h.endheaders()
    h.send(body)
    errcode, errmsg, headers = h.getreply()
    return h.file.read()

    f = open(files, 'rb')
    mmapped_file_as_string = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
    request = urllib2.Request(url, mmapped_file_as_string)
    request.add_header('Content-Type', content_type)
    response = urllib2.urlopen(request)

    mmapped_file_as_string.close()
    f.close()


def encode_multipart_formdata(files):

    BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
    CRLF = '\r\n'
    L = []

    for (filename) in files:
        L.append('--' + BOUNDARY)
        L.append('Content-Disposition: form-data;  filename="%s"' % (filename))
        L.append('Content-Type: %s' % get_content_type(filename))
        L.append('')
        #L.append(value)
    L.append('--' + BOUNDARY + '--')
    L.append('')
    body = CRLF.join(L)
    content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
    return content_type, body

def get_content_type(filename):
    return mimetypes.guess_type(filename)[0] or 'application/octet-stream'

Error message

Traceback (most recent call last):
  File "Test3.py", line 208, in <module>
    main()
  File "Test3.py", line 205, in main
    do_upload(options, args)
  File "Test3.py", line 41, in do_upload
    h.endheaders()
  File "C:\Python27\lib\httplib.py", line 951, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 811, in _send_output
    self.send(msg)
  File "C:\Python27\lib\httplib.py", line 773, in send
    self.connect()
  File "C:\Python27\lib\httplib.py", line 754, in connect
    self.timeout, self.source_address)
  File "C:\Python27\lib\socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno 11004] getaddrinfo failed
Community
  • 1
  • 1
cyberbemon
  • 3,000
  • 11
  • 37
  • 62

1 Answers1

4

You are setting a non-existing header instead of the Content-Type header:

request.add_header(contenttype, 'application/octet-stream')

Change that to:

request.add_header('Content-Type', contenttype)

instead.

Your biggest problem however is that you are not uploading a multipart POST, but rather just the file itself as the POST body, while your server is expecting only a multipart upload.

Take a look at this SO answer for a very simple way to generate a proper multipart POST body: https://stackoverflow.com/a/681182/100297. Note that you'll have to adjust your Content-Type header accordingly.

Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • That Didn't fix it , it changed the first 3 lines from the error messaged and replace them with these `File "Test3.py", line 178, in main() File "Test3.py", line 175, in main do_upload(options, args) File "Test3.py", line 39, in do_upload response = urllib2.urlopen(request)` – cyberbemon May 29 '12 at 16:03
  • Thanks, Looking at the code I'm a little confused as to what `def post_multipart(host, selector, fields, files):` the selector in that function means – cyberbemon May 30 '12 at 09:36
  • It's the path portion of the URL; `/test_server/upload` in your case. – Martijn Pieters May 30 '12 at 09:41
  • So Host = http://127.0.0.1 Selector = /test_server/upload Field = Myfile files = Path to file+filename ? – cyberbemon May 30 '12 at 09:54
  • Your error menas there is no such host (`'http://localhost:80'`). Read the [documentation for HTTP](http://docs.python.org/library/httplib.html) and you'll see you need to provide a bare hostname, so `'localhost'`. – Martijn Pieters May 30 '12 at 17:08
  • Thank you , that fixed the error , but the files are not getting uploaded after entering the url + file name. No error messages this time – cyberbemon May 31 '12 at 08:33
  • I'm sorry, I do not know why; you are dealing with advanced subjects here requiring a full understanding of the HTTP protocol, python and debugging of client-server interactions, and we are stretching the SO format to the limits here. – Martijn Pieters May 31 '12 at 08:54
  • No probs , thanks for the help so far :) really appreciate it ! – cyberbemon May 31 '12 at 09:08