58

Correct me if I'm wrong, but I'm surprised this hasn't been asked before on here ...

blank
  • 17,852
  • 20
  • 105
  • 159

10 Answers10

124

It's a pretty simple difference. In a shared memory model, multiple workers all operate on the same data. This opens up a lot of the concurrency issues that are common in parallel programming.

Message passing systems make workers communicate through a messaging system. Messages keep everyone seperated, so that workers cannot modify each other's data.

By analogy, lets say we are working with a team on a project together. In one model, we are all crowded around a table, with all of our papers and data layed out. We can only communicate by changing things on the table. We have to be careful not to all try to operate on the same piece of data at once, or it will get confusing and things will get mixed up.

In a message passing model, we all sit at our desks, with our own set of papers. When we want to, we can pass a paper to someone else as a "message", and that worker can now do what they want with it. We only ever have access to whatever we have in front of us, so we never have to worry that someone is going to reach over and change one of the numbers while we are in the middle of summing them up.

Ok, silly analogy!

Chris Pitman
  • 12,990
  • 3
  • 41
  • 56
  • 21
    Silly analogy, but +1 for creativity. :) – Sasha Chedygov Dec 05 '09 at 20:16
  • 1
    @SashaChedygov Would either of you or Chris care to explain why that is a silly analogy? – Donbhupi Dec 13 '15 at 12:17
  • @Donbhupi I just meant it's silly because it's not a realistic real-world scenario. It works really well as an analogy and maps well to the concepts, it just created a funny image in my head. That's all. :) – Sasha Chedygov Dec 15 '15 at 01:54
  • 2
    @SashaChedygov ah ok... I think Chris may have meant it in a modest way and I actually found the analogy pretty excellent and innovative, so just wanted to make sure I wasn't missing out on any particular detail. thanks for the response. – Donbhupi Dec 15 '15 at 17:24
20
  1. In shared memory model, memory is shared by cooperating processes, which can exchange information by reading and writing data but in message passing communication takes place by means of messages exchanged between the cooperating processes.
  2. Shared memory helps run processes concurrently but message passing cannot.
  3. Message passing facility has two operations: send (message) and receive (message). The process of which has fixed or variable size.
  4. Message passing is useful for exchanging smaller amounts of data, because no conflicts need be avoided. Message passing is also easier to implement than is shared memory for interprocess communication.
  5. In shared-memory systems, system calls are required only to establish shared-memory regions. Once shared memory is established, all accesses are treated as routine memory accesses, and no assistance from the kernel is required.

Faster

Shared memory allows maximum speed and convenience of communication, as it can be done at memory speeds when within a computer. Shared memory is faster than message passing, as message-passing systems are typically implemented using system calls and thus require the more time-consuming task of kernel intervention.

Community
  • 1
  • 1
Farah Nazifa
  • 909
  • 8
  • 14
  • 6
    on point 2), I do think concurrency can be achieved in an message-passing model, by broadcasting messages to other workers that can start working simultaneously. Clearly it needs other orchestration methods to join the output. I agree this might be faster on a shared-memory model, but the message-passing model do support concurrent process execution – Juan Zamora Mar 16 '17 at 17:30
  • 4
    Now, we can not say that shared memory is faster than message passing because of system calls. From the Operating System Concepts book's(Abraham Silberschatz) "Recent research on systems with several processing cores indicates that message passing provides better performance than shared memory on such systems. Shared memory suffers from cache coherency issues, which arise because shared data migrate among the several caches. As the number of processing cores on systems increases, it is possible that we will see message passing as the preferred mechanism for IPC. :) – serkan kucukbay Jan 04 '18 at 14:37
  • 2
    #2 and #5 are wrong (#5 might be formally correct, but in fact, you need to sync on data, which in turn requires either polling or kernel sync). – No-Bugs Hare Mar 26 '18 at 05:07
12

Message passing models (Erlang, for example) do not have any shared state; all synchronization and communication is done by exchanging messages. Shared memory models communicate by reading/writing to shared memory blocks, which are protected by semaphores or similar.

JesperE
  • 63,317
  • 21
  • 138
  • 197
4

message passing are good method to justify data but it has slow response time for faster communication.But in shared memory model data are extracted from one memory and a working group can do different work over same data

ravindra pratap ssiet
  • 49
  • 1
  • 1
4

Though you are asking for the differences between message-passing model and shared memory model and have already got good answers concerning about their performances, ways of exchanging information, and concurrency issues, I would like to point out that:

There can be no fundamental differences between them regarding their computability (under certain conditions).

You can simulate a shared memory over the underlying message-passing system. This makes it possible to view the shared memory model as a higher-level language for designing algorithms in asynchronous distributed message-passing systems.

In particular, this paper ABD@JACM'95 shows that

Any wait-free algorithm based on atomic, single-writer (and multi-writer) multi-reader registers can be automatically emulated in message-passing systems, provided that at least a majority of the processors are not faulty and remain connected. The overhead introduced by these emulations is polynomial in the number of processors in the system.

hengxin
  • 1,867
  • 2
  • 21
  • 42
4

The answers already given are informative, but most mention the idea that shared memory is faster than message passing, which is actually a pretty naïve statement. In any real system doing something useful, shared memory access needs locking mechanisms to control access from seperate threads, which almost always ends up SLOWER than implementing the same system using message passing.

John Vincent
  • 681
  • 6
  • 4
2

The throughput of the message passing system may be too low for some applications that require fast responses time, but if you need higher speed or real time processing, then you can use a shared memory system.

Yisela
  • 6,909
  • 5
  • 28
  • 51
juned ahmed
  • 29
  • 1
0

To differentiate between Message passing and shared memory consider five things:

  1. communication:- In the case of message passing, communication is dependent on Programmer like to which process to which it will communicate. but in the case of shared memory communication is done automatically.
  2. data distribution:-m.p(Manually) s.m(Automatic)
  3. H/W support:- m.p(simple) s.m(Extensive because it have to be intelligent to detect the interprocess automatically)
  4. correctness:-m.p(difficult) s.m(less difficult)
  5. performance:-m.p(difficult) s.m(very difficult)`
Artemis
  • 2,553
  • 7
  • 21
  • 36
Sarthak Dalabehera
  • 67
  • 1
  • 5
  • 2
    Please edit your answer to make it easier to read. Putting it into a table format would make it much easier to understand. – Tim Morton Apr 14 '18 at 21:12
0

Shared memory S/m require to communication process to share some variable . The processes as expected to exchange information through the use of these shared variables . In a shared memory scheme the responsibility for providing communication rests with the application programmers. The operating system only nNeeds to provide shared memory.

Shah
  • 1
0

Did a test in a course at DTU calculating PI, and MPI(Message Passing) was generally better than pthreads or OpenMP(Share Memory) on our HPC.

Kim Sandberg
  • 95
  • 9
  • 1
    Hi Kim, please read up on [writing answers](https://stackoverflow.com/help/how-to-answer) before answering your next question! Happy coding :) – Diggy. May 14 '20 at 12:02