0

I had an issue with os.path.exists() when working with UNCs and network paths in general.

Some servers tend to die in weird fashion that instead of returning error they hang for 130 seconds and then return False (I guess samba has some weird fetish timeout that I was unable find and configure).

So my question is: How to timeout such (atomic) operations?

I just need it to finish in under 2 seconds. I've tried using threading and mutable object for value returning like this:

import time
import threading
import os.path

class ValueContainer(object):
    ''' Generic imutable type
    '''

    def __init__(self, value=None):
        self.value = value


def timed_out_function(target, timeout, *args, **kwargs):
    ''' Times out function execution
    '''

    val = ValueContainer()

    # Inner function that passes result to mutable type
    def inner(retval, *args, **kwargs):
        retval.value = target(*args, **kwargs)

    # Create thread with this function
    t = threading.Thread(target=inner, args=(val, ) + args, kwargs=kwargs)
    t.daemon = True
    t.start()

    # Just wait for it
    t.join(timeout)

    if t.is_alive():
        raise Exception('Timeout')

    return val.value


print(timed_out_function(os.path.exists, 2, 'python_non_blocking_io.py'))
print(timed_out_function(os.path.exists, 2, 'Nope nope nope'))
timed_out_function(time.sleep, 2, 5)

# True
# False
# Traceback (most recent call last):
#   File "D:\tmp\python_non_blocking_io.py", line 39, in <module>
#     timed_out_function(time.sleep, 2, 5)
#   File "D:\tmp\python_non_blocking_io.py", line 32, in timed_out_function
#     raise Exception('Timeout')
# Exception: Timeout

But I'm not sure whether that won't create too many parallel IO requests (there's continuous stream of requests one after another each 2 seconds handing for 130), threads or some similar issue.

Do you have any experience with this kind of workarounds?

Community
  • 1
  • 1
Vyktor
  • 20,559
  • 6
  • 64
  • 96
  • Are you dealing with lots of servers, or just a few? If Windows 7 is not fully patched, it does sometimes glitch if you are attempting to connect to a lot of unresponsive servers simultaneously. Multiple simultaneous requests to a few unresponsive servers should be OK. Unfortunately I don't think there is any way to abort the attempt to connect; even if your process exits, Windows continues to attempt the connection. – Harry Johnston Oct 02 '14 at 21:58
  • @HarryJohnston it's just one server with 10 parallel processes doing the same... Based on calculation `130/2*10` it can create probably up to 650 connections (and pending IO requests) that doesn't seem that much. – Vyktor Oct 03 '14 at 07:40
  • I'm not absolutely certain, but I believe that the second connection attempt to the same server will wait for the first one to succeed or fail rather than trying to establish an independent connection. Of course you could also do that yourself. – Harry Johnston Oct 03 '14 at 22:23

0 Answers0