70

What is the difference between multiprocessor programming and multicore programming?

Preferably show examples in python how to write a small program for multiprocessor programming & multicore programming

blackraven
  • 5,284
  • 7
  • 19
  • 45
gath
  • 2,286
  • 5
  • 19
  • 11

7 Answers7

97

There is no such thing as "multiprocessor" or "multicore" programming. The distinction between "multiprocessor" and "multicore" computers is probably not relevant to you as an application programmer; it has to do with subtleties of how the cores share access to memory.

In order to take advantage of a multicore (or multiprocessor) computer, you need a program written in such a way that it can be run in parallel, and a runtime that will allow the program to actually be executed in parallel on multiple cores (and operating system, although any operating system you can run on your PC will do this). This is really parallel programming, although there are different approaches to parallel programming. The ones that are relevant to Python are multiprocessing and multithreading.

In languages like C, C++, Java, and C#, you can write parallel programs by executing multiple threads. The global interpreter lock in the CPython and PyPy runtimes preclude this option; but only for those runtimes. (In my personal opinion, multithreading is dangerous and tricky and it is generally a good thing that Python encourages you not to consider it as a way to get a performance advantage.)

If you want to write a parallel program which can run on multiple cores in Python, you have a few different options:

  • Write a multithreaded program using the threading module and run it in the IronPython or Jython runtime.
  • Use the processing module, (now included in Python 2.6 as the multiprocessing module), to run your code in multiple processes at once.
  • Use the subprocess module to run multiple python interpreters and communicate between them.
  • Use Twisted and Ampoule. This has the advantage of not just running your code across different processes, but (if you don't share access to things like files) potentially across different computers as well.

No matter which of these options you choose, you will need to understand how to split the work that your program is doing up into chunks that make sense to separate. Since I'm not sure what kind of programs you are thinking of writing, it would be difficult to provide a useful example.

dano
  • 91,354
  • 19
  • 222
  • 219
Glyph
  • 31,152
  • 11
  • 87
  • 129
  • 3
    *There is no such thing as "multiprocessor" or "multicore" programming* - sure there is. *Multicore* programming is writing applications that take advantage of multiple cores. – johndodo Jan 26 '12 at 09:12
  • 1
    As opposed to applications that take advantage of multiple processors? – Glyph Jan 26 '12 at 09:41
  • 9
    Those are multiprocessor applications... Not sure if I got the point through. I wanted to say this: most people understand what terms "multicore programming" and "multiprocessor programming" mean. So the OP's question could be translated to "Is it possible to write programs that run on multiple cores/processors in python?" The terms you say are not valid - well, they are. That's all I wanted to add. – johndodo Jan 26 '12 at 12:05
  • 2
    Sorry but I have to disagree with your thread opinion. Using threads well does require a little thought and an appropriate design but we're supposed to be programmers who know how to think and solve problems. Yes it's possible to use them badly, but the same applies to almost everything we do. Why ignore a tool just because it requires thought to use? Multiple processes just aren't the same. There's overhead to set up/tear down, wasted memory, communication overhead and good luck if you want to use an object that can't be pickled. Oh that's right, just redesign your app around the limitations – Basic Jan 07 '14 at 17:31
  • 7
    It's not that threading requires *thought* to use; of course all programming requires thought. The problem with threads is that they require *constant awareness of their presence in every single line of code*. By contrast, if you're programming with message passing, you can forget all about external systems until your function is done and it's time to receive the next message. Shared-mutable-state multithreaded programming produces constant, crippling anxiety for the programmer or a constant stream of impossible-to-reproduce for the user. – Glyph Jan 08 '14 at 21:22
  • _it is generally a good thing that Python encourages you not to consider multithreading as a way to get a performance advantage_. What about load sharing? Would this not increase performance? It is my opinion that if there is processing power in the machine you should use it and not wait "in case other apps may need it". It is much better to split things up in smaller jobs to use the capacity. This will save money in terms of saving on hardware. If python cannot handle this it might be preferred to build something in another language which can be used to run single thread python tasks. – patrik Oct 04 '17 at 20:42
  • Yet another way to leverage multiple cores, processors, or even a cluster of multiple different machines (or virtual machines) is using an orchestrator/scheduler framework like Kubernetes, Spark, Beam, Flink, etc.. These can run many processes across many machines in parallel to do large scale tasks like Analytics, ML, HPC, etc.. – Mark Pevec May 26 '23 at 23:12
24

As mentioned in another post Python 2.6 has the multiprocessing module, which can take advantage of multiple cores/processors (it gets around GIL by starting multiple processes transparently). It offers some primitives similar to the threading module. You'll find some (simple) examples of usage in the documentation pages.

rslite
  • 81,705
  • 4
  • 44
  • 47
  • These kind of answers are surprising since processes do not share their address spaces, which leads to a substantially different programming model. – Alex Kreimer May 24 '14 at 17:42
  • There is a small and clearly explained how-to here https://pythonprogramming.net/threading-tutorial-python/ – Chris McCowan Sep 26 '19 at 15:29
5

You can actually write programs which will use multiple processors. You cannot do it with threads because of the GIL lock, but you can do it with different process. Either:

  • use the subprocess module, and divide your code to execute a process per processor
  • have a look at parallelpython module
  • if you use python > 2.6 have a look at the multiprocess module.
Mapad
  • 8,407
  • 5
  • 41
  • 40
  • 1
    Threads and python will be split up on multiple cores, but some of those cores (all but one, unless you do some magic in C) will just be waiting for the GIL. This is why before Python 3.2, CPU-bound threads perform better on single core than multicore machines. – Chad Jul 02 '11 at 20:26
  • 1
    Some modules are implemented in C and will release the GIL. The zlib module (also used by the gzip module) is one of these. You can use multiple cores to decompress or compress data using multiple threads in python. Other examples exist in the standard library (re regular expression library), and some other third-party libraries, like net-snmp among other. – Will Pierce Dec 04 '13 at 01:45
  • @WillPierce That reads like "It can be done, and very well, but only if you don't use Python". I can write a C module for use in almost any language. That's a plus for C but just highlights the failings of Python. – Basic Jan 07 '14 at 17:34
3

If I understand things correctly, Python has something called the GIL (Global Interpreter Lock) that effectively makes it impossible to take advantage of multicores when doing multiple threads in Python.

See eg Guido van Rossum's blog entry on the topic. As far as I know, among the "mainstream" languages only C/C++ and Java have effective support for multicores.

lindelof
  • 34,556
  • 31
  • 99
  • 140
2

You can read about multithreading in python, and threading in general

Multithreading in Python: http://www.devshed.com/c/a/Python/Basic-Threading-in-Python/

milot
  • 1,060
  • 2
  • 8
  • 17
1

The main difference is how you organize and distribute data. Multicore typically has higher bandwidths between the different cores in a cpu, and multiprocessor needs to involve the bus between the cpus more.

Python 2.6 has gotten multiprocess (process, as in program running) and more synchronization and communication objects for multithreaded programming.

Lasse V. Karlsen
  • 380,855
  • 102
  • 628
  • 825
0

If you don't have Python 2.6 (which you don't if you're using Ubuntu Edgy or Intrepid for example), you can use the Google code backported version of multiprocessing. It is part of PyPI, which means you can easily install it using EasyInstall (which is part of the python-setuptools package in Ubuntu).

Gert
  • 363
  • 2
  • 7