4

When it comes to parallel programming, I have only been exposed to multithreading but recently I have been presented with mpi I tried to make sense of what the difference is through searching online but was to no avail. I think they are both libraries for the sake of parallelizing your code correct? If so how are they different?or are they even comparable like an apple and an orange. Can someone clarify?

1 Answers1

5

Multithreading is strictly on the same computer, same program and same memory space. MPI is basically multiprocessing, which is different programs that communicate together. MPI has the advantage that you can parallelize its programs and run it on cluster systems, so, different computers. Multithreading is considered much simpler. C++ doesn't have a standard implementation of multiprocessing, which is why MPI libraries are used for that purpose.

More information here.

The Quantum Physicist
  • 24,987
  • 19
  • 103
  • 189
  • Got you ! that makes sense another question if you're running a parallel program on a single machine would you need to install an implementation? I came across MPICH2 online ? –  Aug 08 '17 at 18:58
  • @JohnKang I don't understand your question. What is "parallel" exactly? Your main question makes a distinction between two kinds of parallel. – The Quantum Physicist Aug 08 '17 at 19:00
  • @JohnKang It depends on how you link your compiled program. If you link statically, then no. If you link with a shared library, then yes. – The Quantum Physicist Aug 08 '17 at 19:22
  • Thank you so much ! @The Quantum Physicist –  Aug 08 '17 at 19:34
  • I've always felt that MPI, being process orientated, gets it wrong. If processes that are communicating using MPI are themselves multithreaded (e.g. using OpenMP), then changing the deployment of those threads amongst processes is a major, major rewrite. Using something like ZMQ is quite often a better bet; You can use it between threads, and also between processes, making the deployment of threads across the system a relatively trivial problem. Though it's not quite in a same areana as OpenMPI; ZeroMQ support for Infiniband seems a little shakey. – bazza Aug 08 '17 at 21:25
  • @bazza OpenMPI is the very old way of doing things. Back then in the 90s, good RPC systems didn't exist. I don't think OpenMPI is necessary anymore. However, the scientific community (computational physics, for example) is still very used to it. – The Quantum Physicist Aug 09 '17 at 07:49
  • @TheQuantumPhysicist, yes, MPI is old but still pretty common in supercomputing last time I looked. It can do some unique things (e.g. transpose matricies) that other transports don't come close to achieving. A lot of super computers are designed around MPI as a transport; it has been implemented on the K Computer in the Riken Institue, Kobe, Japan, a machine with a particularly impressive inter-node network called Tofu. For the right application it's very good. Old just means that it was a good idea, so it's stuck around. – bazza Aug 09 '17 at 18:20