Python not closing file descriptors

Question

I am developing a long-running python script which makes many connections to different serial ports. The script crashes a few hours into its execution citing "Too many open files".

I have tracked the issue to the serial module where the .close() method does not seem to reduce the number of file descriptors python is using. I am checking this using lsof | grep python | wc. Using Debian 7.2 & Python 2.7.3

The example below slowly uses up more and more file descriptors until it hits the limit. Why is this and how can I avoid it??

#!/usr/bin/env python
import serial                   #Used to communicate with pressure controller
import logging
import time
from time import gmtime, strftime
logging.basicConfig(filename="open_files_test.log")

# Write unusual + significant events to logfile + stdout
def log( message ):
    time = strftime("%Y-%m-%d %H:%M:%S", gmtime())
    logging.warning( time + " " + message )
    print( message )

for i in range(2000):
    for n in range(1, 12):
        try:
            port_name = "/dev/tty" + str(n+20)
            com = serial.Serial(port_name,9600,serial.EIGHTBITS,serial.PARITY_NONE,serial.STOPBITS_ONE,0.0,False,False,5.0,False,None)
            com.open()
            com.flushInput()
            com.flushOutput()
            log("Opened port: " + port_name)
        except serial.SerialException:
            com = None
            log("could not open serial port: " + port_name)

        com.close()
        log("Closed port: " + port_name)
        time.sleep(1)

log("Finished Program")

Thanks

Call the GC after you're done with the file(s) http://stackoverflow.com/questions/4599980/python-close-file-descriptor-question — BinaryThor, Nov 26 '13 at 15:06
@BinaryThor: He does that. After the try..except thingy, there's `com.close()`. — jazzpi, Nov 26 '13 at 15:15
@Shiftee, do you encounter any errors which are getting logged? — glglgl, Nov 26 '13 at 15:37
@glglgl No. My main program raises an IOError but I can't find anything in the logfiles — Shiftee, Nov 26 '13 at 15:48

score 5 · Accepted Answer · answered Nov 26 '13 at 15:52

It seems that extra com.open is causing the issue. According to the docs serial.Serial returns it open, so you don't need to open it once more. In Linux (all POSIXes as far as I know) open just increments the counter and close just decrements it. There is a deleted answer here by @wheaties advising using with which I would recommend as well:

with serial.Serial(port_name, 9600, ...) as com:
    com.flushInput()
    com.flushOutput()
    ...

score 2 · Answer 2 · answered Nov 26 '13 at 15:23

2

Use finally to ensure the port is closed.

try:
    ...
except serial.SerialException:
    ...
finally:
    com.close()

answered Nov 26 '13 at 15:23

tim

3,191
2
15
17

The `com.close` call is done *after* the try..except, so it is executed anyways. – jazzpi Nov 26 '13 at 15:28
@jazzpi Not on an exception other than `SerialException`... and, after `com = None`, it fails anyways. – glglgl Nov 26 '13 at 15:33
@tim This is, in general, good style, but probably won't help here as `close()` is called anyways. – glglgl Nov 26 '13 at 15:36
@glglgl: Yeah, but that would fail with another error than "Too many files opened". – jazzpi Nov 26 '13 at 15:38

Python not closing file descriptors

2 Answers2