1

As many hopefully can relate, this encoding problem is driving me mental. I would really appreciate some light on this!

End goal is to be able to run the same script.py from both terminal and cron, and from cron with > stdout.txt. And needless to say, I'm having serious encoding troubles.

My script.py runs fine from terminal thus: python script.py
It throws an error, however, when run from terminal thus: python script.py > stdout.txt
It throws same error when run in cron, either way.

I have a python script, entered in crontab -e as root.

This is my script.py header:

#!/usr/bin/python
# -*- coding: utf-8 -*-

This is my cron entry:

* * * * * python /home/ubuntu/parrot/script.py > /home/ubuntu/parrot/stdout.txt

This is my stdout.txt (the relevant part):

Unexpected error!  (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeError('ascii', 'blabla some weird text n\xc3\xa5r end', 54, 55, 'ordinal not in range(128)'))

This is my env from terminal (the relevant part):

LANG=en_US.UTF-8

This is my env from cron (the relevant part):

LANG=en_US.UTF-8

This is the (first) line in script.py throwing the error:

print 'Posting @%s: %s' % (statusObj.user.screen_name.encode('ascii', 'replace'), statusObj.text.encode('utf-8', 'replace'))

Edit: sys.getdefaultencoding() returns ascii

Any help is greatly appreciated!

knutole
  • 1,709
  • 2
  • 22
  • 41
  • What does `sys.getdefaultencoding()` return? – John La Rooy Aug 07 '12 at 00:34
  • 1
    why do i have a feeling that's bad?.. – knutole Aug 07 '12 at 00:49
  • 1
    If this isn't the same problem as in http://stackoverflow.com/questions/2224130/unicodeencodeerror-when-redirecting-stdout it is close. – msw Aug 07 '12 at 01:26
  • possible duplicate of [Setting the correct encoding when piping stdout in python](http://stackoverflow.com/questions/492483/setting-the-correct-encoding-when-piping-stdout-in-python) – Mechanical snail May 31 '13 at 19:17
  • Possible duplicate of [UnicodeEncodeError only when running as a cron job](http://stackoverflow.com/questions/9932406/unicodeencodeerror-only-when-running-as-a-cron-job) – Ariel M. May 20 '16 at 01:01

1 Answers1

1

If you have control over the statusObj you should check the relevant code where data is being parsed into the object and try to get the input as clean as possible.

You want to make sure your string is decoded to unicode before you try to encode it.

If not you can try:

# try to get the string into unicode
screen_name = unicode(statusObj.user.screen_name) 
post = unicode(statusObj.text) # probably an error here?
output_str = u"Posting @{name}: {post}".format(name=screen_name, post=post)
print output_str.encode("utf8", "replace") # encode the unicode string on
monkut
  • 42,176
  • 24
  • 124
  • 155
  • but my code already works from CLI. so it must be something with the encoding environment of cron?? i will try tho. thank you! – knutole Aug 07 '12 at 01:14
  • i just added sys.stdout = codecs.getwriter('utf8')(sys.stdout) to the top and deleted the .encode'ings altogether. problem solved! thanks a lot all! =) – knutole Aug 07 '12 at 04:37