0

Though I googled and got a lot of result, but they are not what I want. My main code as below

def main:
    start = datetime.now()
    browser = webdriver.PhantomJS()
    download()
    browser.quit()
    showTime()

def download:
    for imageSecond in imageSeconds:
        urlServer = imageSecond.get("src")
        pathLocal = formatPath(downloadLocationPath, ntpath.basename(urlServer))

        if not os.path.isfile(pathLocal):
             ts.append(createNewDownloadThread(browser, urlServer, pathLocal))
        else:
            logger.info('Downloaded: {}'.format(urlServer + " -> " + pathLocal))
            showTime()

    for t in ts:
        t.join()

def showTime:
    end = datetime.now()
    runtime = end - start
    logger.info('Sta Time: {}'.format(start))
    logger.info('End Time: {}'.format(end))
    logger.info('Run Time: {}'.format(runtime))
    sys.exit(0)

I got output as below

2017-02-27 09:42:12,817 - INFO - MainThread - Downloaded: https://secure-api.userlocal.jp
2017-02-27 09:42:12,833 - INFO - MainThread - Sta Time: 2017-02-27 09:41:43.895126
2017-02-27 09:42:12,833 - INFO - MainThread - End Time: 2017-02-27 09:42:12.833492
2017-02-27 09:42:12,833 - INFO - MainThread - Run Time: 0:00:28.938366
2017-02-27 09:42:12,849 - INFO - Thread-323 - Download: https://secure-api.userlocal.jp
2017-02-27 09:42:12,849 - INFO - Thread-324 - Download: https://secure-api.userlocal.jp

But what I want to output as below, what can I do?

2017-02-27 09:42:12,817 - INFO - MainThread - Downloaded: https://secure-api.userlocal.jp
2017-02-27 09:42:12,849 - INFO - Thread-323 - Download: https://secure-api.userlocal.jp
2017-02-27 09:42:12,849 - INFO - Thread-324 - Download: https://secure-api.userlocal.jp
2017-02-27 09:42:12,833 - INFO - MainThread - Sta Time: 2017-02-27 09:41:43.895126
2017-02-27 09:42:12,833 - INFO - MainThread - End Time: 2017-02-27 09:42:12.833492
2017-02-27 09:42:12,833 - INFO - MainThread - Run Time: 0:00:28.938366
mikezang
  • 2,291
  • 7
  • 32
  • 56
  • This may help: http://stackoverflow.com/questions/15085348/what-is-the-use-of-join-in-python-threading – EvanL00 Feb 27 '17 at 02:17

1 Answers1

2

There is missing code in your sample, I guess you are calling newthread.start() in your createNewDownloadThread() method above, aren't you?

You might know the usual way of working is by calling thread.start() and thread.join(), so it will block until the thread has finished.

I'd say it might work better by doing this in your for loop:

   for t in ts:
      t.start()
      t.join()
Alberto
  • 687
  • 6
  • 21
  • I start it in createNewDownloadThread, your code is not what I need because threads no in parallel. Then I found a strange result, the result above was used with PhantomJS, if I use ChromeDriver, the result is what I need, I am not sure why... – mikezang Feb 27 '17 at 01:37
  • Ah!, probably because PhantomJS is single threaded and it's not working properly in your multithreading code. – Alberto Feb 27 '17 at 01:40
  • Really? I aslo found PhantomJS is very slow on Mac, So PhantomJS is not good for my case? May I have to use ChromeDriver? – mikezang Feb 27 '17 at 01:44
  • I'm thinking another problem you might have is the fact that you are starting the thread in a different context, as you are starting it into another function, but joining it in the caller one. Could you try to start() the thread after .append to your list? You could try also using .setDaemon(True) before the .append() and .start(), and see if that makes any difference? – Alberto Feb 27 '17 at 01:50
  • I start() after append but same result. But setDaemon(True) show what I need! Can you explain for me about setDaemon(True)? – mikezang Feb 27 '17 at 02:10
  • This may help: http://stackoverflow.com/questions/15085348/what-is-the-use-of-join-in-python-threading – EvanL00 Feb 27 '17 at 02:14
  • @mikezang you have a good explanation of .setDaemon(True) -now .daemon(True)- in this other thread: http://stackoverflow.com/questions/190010/daemon-threads-explanation. – Alberto Feb 27 '17 at 02:17