4

I was just going through programming structure studies; In particular I was studying concurrent programs. I came across few articles where distinction between concurrent programs, parallel programs and distributed programs seems confounding.

My understanding is as follows:

concurrent programs : performs several tasks at the same time or gives a notion of doing so

parallel programs : algorithms for solving such problems allow some related tasks to be executed at the same time

distributed programs: Has more to do with available resources than inherent parallelism in the corresponding algorithm. More than one computing resource is available to solve the problem.

Could someone please point me to correct and detailed explanation of these terms?

1 Answers1

2

These terms are indeed related and sometimes overlapping: Parallel programming is a subset of concurrent programming that aims to use several concurrently executing entities (threads, processes) in order to speedup some computation. Distributed computing is a very wide topic that, as you said, focuses on using resources to solve a problem. These resources can be on the same machine, or can be geographically dispersed or heterogeneous. It is hard to draw a clear line between parallel and distributed computing. Parallel computing may or may not be distributed, depending on the resources used, while distributed computing may or may not be used to solve a problem in parallel.

In my opinion, the subtle distinction between these terms comes from the abstraction layers at which they are considered. When the main focus is the interaction of separate computational entities over some shared data, then we talk about concurrent programming. When the key aspect is speeding up a computation by using several computational units, we have parallel computing. Finally, if the focus is on the way resources are managed to solve some problem, then we are talking about distributed computing.

Tudor
  • 61,523
  • 12
  • 102
  • 142