1

This question is related to this one.
Please read the problem that Chris describes there. I'll narrow it down: there's a CURL error 26 if a filename is utf-8-encoded and contains characters that are not in the range of those supported by non-unicode programs.
Let me explain myself:

local_filename = filename.encode("utf-8")
self.curl.setopt(self.curl.HTTPPOST, [(field, (self.curl.FORM_FILE, local_filename, self.curl.FORM_FILENAME, local_filename))])

I have windows 7 with Russian set as the language for non-unicode programs. If I don't encode filename to utf-8 (and pass filename, not local_filename to pycurl(, everything goes flawlessly if the filename contains either English or Russian chars. But if there is, say, an à, — it throws an error 26. If I pass local_filename (so encoded to UTF-8), even Russian chars aren't allowed.
Could you help, please? Thanks!

Community
  • 1
  • 1
Andre Polykanine
  • 3,291
  • 18
  • 28
  • Please expand your code snippet to a self-contained test. It's not clear what is `filename`, I suppose it must be unicode; what is `field`, what other options are set... – Dima Tisnek Jun 13 '13 at 12:31

2 Answers2

2

This is easy to answer, harder to fix:

pycurl uses libcurl for formposting. libcurl uses plain fopen() to open files for posting. Therefore you need to tell libcurl the exact file name that it should open and read from your local file system.

dda
  • 6,030
  • 2
  • 25
  • 34
Daniel Stenberg
  • 54,736
  • 17
  • 146
  • 222
  • Hej @Daniel, thanks for answer! I saw that you are Libcurl's developer. So maybe could you fix this so that it would accept utf-8 names? If not, maybe is there a way to override this? Thanks! – Andre Polykanine Jun 08 '13 at 09:13
  • 1. I answer the question. If you want to discuss libcurl details, this is not the place 2. If you have additional quetions, file a second question or take it to a libcurl mailing list. – Daniel Stenberg Jun 08 '13 at 20:19
2

Decompose this problem into 2 components:

  1. tell pycurl which file to open to read file data
  2. send filename in correct encoding to the server

These may or may not be same encodings.

For 1, use sys.getfilesystemencoding() to convert unicode filename (which you use throughout python code correctly) to a string that pycurl/libcurl can open correctly with fopen(). Use strace (linux) or equivalent windows osx to verify correct file path is being opened by pycurl.

If that totally fails you can always feed file data stream from Python via pycurl.READFUNCTION.

For 2, learn how filename is transmitted during file upload, example. I don't have a good link, all I know it's not trivial, e.g. when it comes to very long file names.

I hacked up your code snippet, I have this, it works against nc -vl 5050 at least.

#!/usr/bin/python
import pycurl
c = pycurl.Curl()

filename = u"example-\N{EURO SIGN}.mp3"
with open(filename, "wb") as f:
    f.write("\0\xfffoobar\x07\xff" * 9)

local_filename = filename.encode("utf-8")
c.setopt(pycurl.HTTPPOST, [("xxx", (pycurl.FORM_FILE, local_filename, pycurl.FORM_FILENAME, local_filename))])
c.setopt(pycurl.URL, "http://localhost:5050/")
c.setopt(pycurl.HTTPHEADER, ["Expect:"])
c.perform()

My test doesn't cover the case where encoding is different between OS and HTTP.

Should be enough to get your started though, shouldn't it?

Community
  • 1
  • 1
Dima Tisnek
  • 11,241
  • 4
  • 68
  • 120
  • thanks for your answer! I've managed to do step 1, but with step 2 no dice with whatever encoding... it always says error 26. Any ideas, please? – Andre Polykanine Jun 11 '13 at 22:01
  • `CURLE_READ_ERROR (26) There was a problem reading a local file or an error returned by the read callback.` it does not look like you solved component 1. let me try to reproduce. – Dima Tisnek Jun 13 '13 at 12:18