Take garbage collection. When it's time to collect garbage in other
languages, the entire system has to stop while the garbage collector
runs. This approach is perfectly fine if your computer program is
supposed to run once, write some output, and then quit. But in
long-running applications, such as desktop, mobile, or server
programs, this strategy results in occasionally frozen UIs and slow
response times. Erlang programs, on the other hand, can have thousands
of independent heaps which are garbage-collected separately; in this
way, the performance penalty of garbage collection is spread out over
time, and so a long-running application will not mysteriously stop
responding from time to time while the garbage collector runs.
Evan Miller, creator of the popular Chicago Boss framework.
So I believe erlang garbage collects concurrently, that is, the various heaps are garbage collected independently of one another. Whether there is any parallelization depends on whether your node is running on multiple cores or not, but if so then the garbage collection is done in parallel, yes.