0

I'm trying to apply a timeout to one of my methods which normally works fine. I've tried this code on simple printing function and everything worked fine but now I'm getting error. Could you tell me where is the problem?

The method self.check_flights should be running max __TIMEOUT__ seconds.

The code below is in a method which is in a class some_class

            try:
                p = multiprocessing.Process(target=self.check_flights, args=(destination, start_date , end_date_2))
                p.start()
                p.join(__TIMEOUT__)
                if p.is_alive():
                    print 'TIMEOUT'
                    p.terminate()
                    p.join()

            except Exception as e:
                raise e

>ERROR: pickle.PicklingError: Can't pickle <class __main__.some_class at 0x02DD10A0>: it's not the same object as __main__.some_class

EDIT: This timeout solution is from THIS SO answer

EDIT 2: To elaborate it more, I'm attaching another lines of the error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 880, in load_eof
    raise EOFError
EOFError
Community
  • 1
  • 1
Milano
  • 18,048
  • 37
  • 153
  • 353

1 Answers1

0

This is actually some multiprocess problem while multiprocessing tries pickling some object to effectively paralleling.

Why do you use so complicate way of timout, if you have direct access to your method, thus you can include a timeout inside of that.

As your only time sensitive method is the load_whole_page method, I would modify the sections of your code as follows simply leaving out the multiprocess base elements:

def load_whole_page(self,destination,start_date,end_date):
    deb()
    .
    .
    .   

    header = wait.until(EC.visibility_of_element_located((By.TAG_NAME, 'header')))
    footer = wait.until(EC.visibility_of_element_located((By.TAG_NAME, 'footer')))


    results = []
    # setting time threshold
    threshold = 10
    t = time()
    while True:
        # cheking if we reach the time we don't want to exceed.
        if (t - time()) >= threshold:
            with open('log.txt', 'a') as f:
                f.write(str(e))
                f.write(traceback.format_exc())
                print 'TIMEOUT ERROR'
            break

        wait_2.until(wait_for_more_than_n_elements((By.CSS_SELECTOR, "div.flightbox"), len(results)))
Geeocode
  • 5,705
  • 3
  • 20
  • 34
  • 1. I'm new in this and I don't know how to do that. – Milano Jul 28 '15 at 10:28
  • 2. I don't know either, It was advised in the question I've posted in my question. Could you give me an advise how to improve the code? – Milano Jul 28 '15 at 10:28
  • How would you incude a timeout inside the function? I though that this is a simpler way because I have to call some function when I want to open second thread. – Milano Jul 28 '15 at 10:31
  • I could check whether time exceed in every loop in check_flights method but I'm afraid that method load_whole_page inside check_flights could exceed time too. – Milano Jul 28 '15 at 10:35
  • First we try to remove all `join()` after the `terminate()` and check what happens. Please provide some the datafile too to check the code on my side if you can – Geeocode Jul 28 '15 at 10:36
  • There are many files and libraries (inculing selenium a my library) which are used so I would have to send you all off these files but I can't do that via SO. Or what you mean by "datafile"? – Milano Jul 28 '15 at 10:41
  • I've tried to put join() away but still the same error is being raised. – Milano Jul 28 '15 at 10:41