1

I'm new to coding and to python. I tried to create a code that will distribute files over the network, problem is, it iterates through a list I made, and distributes one after another. Only when the first location gets the files, just then it will start with the second distribution.

This is the code I wrote:

from shutil import copytree

source = r"O:\versions"

Eilat = r"\\eilat-zrv\c$\version"
BeerSheba = r"\\beer-sheba-zrv\c$\version"
Ashdod = r"\\ashdod-zrv\c$\version"
Rehovot = r"\\rehovot-zrv\c$\version"
Rishon = r"\\rishon-zrv\c$\version"
TelAviv = r"\\dizingof-zrv\c$\version"
Netanya = r"\\netanya-zrv\c$\version"
RamatIshay = r"\\ramaty-zrv\c$\version"
Haifa = r"\\haifa-zrv\c$\version"



DestList = [Eilat, BeerSheba, Ashdod, Rehovot, Rishon, TelAviv, Netanya, RamatIshay, Haifa]

def Copy_funct(i):
    for i in DestList:
         copytree(source, i)

Copy_funct(DestList)

How can I make it distribute simultaneously??

user3091216
  • 85
  • 2
  • 3
  • 13
  • Rephrasing: "how do I spawn threads/sub-processes in Python". Would that be a fair interpretation? – Floris Dec 11 '13 at 13:15
  • Some useful insights at http://stackoverflow.com/questions/6286235/multiple-threads-in-python – Floris Dec 11 '13 at 13:17

1 Answers1

2

Using threads it would look sth like this:

import threading

def Copy_funct(DestList):
  threads = [ threading.Thread(target=lambda dest=dest: copytree(source, dest))
              for dest in DestList ]
  for thread in threads:
    thread.start()
  for thread in threads:
    thread.join()

EDIT:

A bit of explanation:

Threading is a way of splitting your program into several threads of action which all are pursued in parallel by the computer; if one thread has to wait for something (typically I/O), another thread can continue its own task. Doing stuff in a parallel fashion opens up a large Box of Pandora of possible bugs you probably never have heard of if you haven't done concurrency before, and I can't go into detail here. But I think your usecase is simple enough to not hit such problems.

The first statement in my function builds a list of threads (a "thread" is represented by an object in Python). These threads are told upon creation what their task will be (using the parameter target). The argument given is a lambda closure; this way I can easily pass information (dest) into the closure. Basically, what I pass to parameter target is a function that, when called, will do that copytree() call with the correct parameters for one of your tasks.

The second statement is a loop over all threads and just starts each. Before that they are just unstarted threads, like post-it notes with ToDo reminders but without an action taking place. With start() I start each thread; that starting takes virtually no time, the call to start them will return immediately, so I can start all threads practically at the same moment. I have no direct control over when they actually start doing their thing; the first may have already begun its work when the last is being started.

The third statement also is a loop over all threads and uses join() to wait for their finishing. It doesn't matter which threads finishes first or last, I will first wait for the first thread, then for the second, etc. If the second actually finishes earlier than the first, this is no problem, it will just be a finished thread which hasn't been joined yet. It will be joined (and then properly cleaned up) by the join in the second iteration of the second loop.

I hope this explains it enough to be useful for you. Feel free to suggest further improvements on this answer if still vital things are unclear.

Alfe
  • 56,346
  • 20
  • 107
  • 159
  • Wow. I could not have figured that out myself. Pretty sure this would solve OPs problem. – Floris Dec 11 '13 at 14:09
  • Hi,Tried this code over Python IDLE. doesn't seem to work. give me an invalid syntax error. – user3091216 Dec 11 '13 at 14:30
  • Maybe someone can understand how I actually use threading. cause I dont have a clue. this will definetly help! – user3091216 Dec 11 '13 at 14:43
  • can you paste your syntax error code into pastebin or so, so i can have a look at it? – Alfe Dec 11 '13 at 18:53
  • I'm using IDLE so it just write "Invalid Syntax" – user3091216 Dec 12 '13 at 07:51
  • I've seen the pasted code in the "answer" below and found two problems, one is my fault: There was a closing parenthesis missing after `copytree(source, dest)`, so it has to be `copytree(source, dest))` instead. I fixed that in my post above. The other problem is your indentation in the two `for` loops. Please compare that with my post. – Alfe Dec 12 '13 at 08:57
  • I added a part explaining the mechanism behind the threading in brief. But I encourage you to seek further advice on that topic otherwise you are bound to run into the typical concurrency problems sooner or later. – Alfe Dec 12 '13 at 09:52