0

I want to pass each element of an array to a process by multiprocessing in python. How can I do it? For example, I have a[i] as an array and want to send a[0] to core 1, a[1] to core 2,.... Is there any way to do it in python?

Ehsan
  • 45
  • 6
  • 2
    What you're looking for is subprocess, pool, and map. Look at the very first example in the documentation: https://docs.python.org/3/library/multiprocessing.html – Roy2012 Jun 19 '20 at 11:58
  • @Roy2012 My problem is that I have 3 array with large size and I don't want to pass whole of arrays to cores because it will be time consumer. For example, I want to send a[0], b[0] and T[0] to one core, a[1], b[1] and T[1] to another core, .... This way that you said, send whole of array to all cores. – Ehsan Jun 19 '20 at 12:51

1 Answers1

0

CPython (the standard python implementation) has something called the GIL (Global Interpreter Lock) ,as Gabriel Grant write in this answer Python threads all executing on a single core each time a request to make a new thread is made, the interpreter actually calls into the operating system’s libraries and kernel to generate a new thread. so you can use multi-threading to use multitask at the same time but you can't force wich core will be executed

Belhadjer Samir
  • 1,461
  • 7
  • 15
  • 1
    how is this relevant to [process based parallelism](https://docs.python.org/3/library/multiprocessing.html) that the OP seems to be talking about? – Sam Mason Jun 19 '20 at 12:31
  • @Belhadjer Samir My problem is that I have 3 array with large size and I don't want to pass whole of arrays to cores because it will be time consumer. For example, I want to send a[0], b[0] and T[0] to one core, a[1], b[1] and T[1] to another core, .... – Ehsan Jun 19 '20 at 12:49
  • Threads uniquely run in the same unique memory heap. Whereas Processes run in separate memory heaps. – Belhadjer Samir Jun 19 '20 at 12:53
  • @Belhadjer Samir So can't I do it in multiprocessing? I have to do it in multi-threading? – Ehsan Jun 19 '20 at 13:00
  • @Ehsan i suggest to you to use numpy it is very fast than simpe array and you can use multithreading if you need it check this answer https://stackoverflow.com/questions/16617973/why-isnt-numpy-mean-multithreaded – Belhadjer Samir Jun 19 '20 at 13:10