2

I'm redirecting the stdio of a server written in Python to a file:

python server.py &> file

The input is transmitted via a client.py which uses the XMLRPC library. If I transmit UTF-8 input, I get a UnicodeEncodeError before I can do anything.

The curiosity here is: If I don't redirect the stdoutput of the server.py, I don't get an error.

Locale is set to en_US.utf8, bash correctly displays unicode, the client encodes the text. I have not the slightest idea what is going on.

Laughingman
  • 265
  • 1
  • 4
  • 11
  • If you post some of the relevant code of `server.py`, it might be easier to help. – brc Oct 13 '11 at 14:26
  • There is a similar question with a good solution: http://stackoverflow.com/questions/492483/setting-the-correct-encoding-when-piping-stdout-in-python – Ander2 Sep 20 '12 at 06:33

1 Answers1

0

The Python code is irrelevant and I won't modify the SimpleXMLRPCServer module. It is Bash related:

The file created by redirection is us-ascii and only becomes utf-8 once a unicode character is inserted which won't work in this case since it is processed by the XMLRPC module first and hence the UnicodeDecodeError occurs.

I tried to create the file for redirection first but even with iconv -f us-ascii -t utf-8 the file stays us-ascii if there's no unicode sequence inside the file.

The idea was to create a basic "quiet mode" without modifying the Python code which didn't work, therefore I created an OptionParser which maps stdout to codecs.open("silent.log", "w", encoding = "utf-8"). This works well.

Laughingman
  • 265
  • 1
  • 4
  • 11