3

I'm calling a Python script from another program (Stata), which has a command for calling the operating system (shell/winexec). This, in turn, opens a system window where you can see the Python script running. Sometimes the script runs just fine and sometimes it abruptly quits before finishing. I have no idea why. I never have this problem when I run the script in IDLE. I'm using Windows and Python 3.4.

I've tried catching the error by logging the Python script, but no error ever gets recorded there. Everything seems to be working normally until the program just quits (and not always at the same point). Also, I'm calling 7 scripts and only two of them ever have this sudden quitting problem (again, they always run fine through IDLE). I tried changing the order in which I call the scripts and using shell instead of winexec and vise-versa. At this point, I don't even know what to try. Is it the operating system? Python? Stata? Any ideas would be appreciated!

RandomCat
  • 187
  • 2
  • 11
  • 1
    Is the Python program exiting because of an exception? By the way, _what_ Python program? – Davis Herring Oct 21 '17 at 17:57
  • No, I don't see any exception in the log file. Not sure what you mean by "what Python program"; if it's about what the script is doing, it's scraping stuff. Again, runs just fine when I use IDLE. – RandomCat Oct 21 '17 at 18:14
  • Put a `try/except` around most of the logic in the Python script that catches _any_ exception`s that might be raised during its execution (and prints a meaningful error message before exiting). – martineau Oct 21 '17 at 18:15
  • martineau, trying that now. Of course, now it's not breaking down...will have to try several times, probably. – RandomCat Oct 21 '17 at 18:46
  • 1
    @RandomCat: I meant that we can’t help very well without seeing any code. – Davis Herring Oct 21 '17 at 19:21
  • Davis, I didn't post any code because it's pretty long (I have no idea which line it's breaking down at, but it seems to be random) and because the code runs just fine on IDLE. Martineau, your suggestion was excellent. I was able to log the error: it's "'charmap' codec can't encode character '\u2013' in position 448: character maps to ". That explains why it sometimes quit and sometimes didn't - the problematic characters weren't always there. I'm guessing running "chcp 65001" or something similar prior to starting the script should do the trick? – RandomCat Oct 22 '17 at 13:02
  • @martineau, I posted an answer based on your solution, but I'm happy to give you credit for it if you want - just post your answer and I'll accept it! – RandomCat Oct 24 '17 at 19:33
  • Random: That's OK, the credit you gave me in the answer is good enough—thanks for offering though. – martineau Oct 24 '17 at 20:13

1 Answers1

4

Couldn't have figured this out without Martineau! Posting answer in case anyone has a similar (seemingly) mysterious error.

First, I logged the error, as Martineau suggested:

import logging

logging.basicConfig(filename='C:/MyLog.log',level=logging.DEBUG)

try:
    [all my code]

except Exception as e:
    logging.info(e)

This recorded the error: "charmap' codec can't encode character '\u2013' in position 448: character maps to undefined". For some reason that's beyond my pay grade, IDLE and python.exe (in Python 3.4) don't handle encoding in the same way. That was causing the problem (and only when special characters were appearing, which is why it didn't break down every time).

I looked for the solution to the encoding problem and found it here. Basically, I upgraded from Python 3.4 to Python 3.6, and the encoding problem disappeared.

RandomCat
  • 187
  • 2
  • 11