Questions tagged [pmap]
161 questions
42
votes
4 answers
Trying to locate a leak! What does anon mean for pmap?
I'm trying to locate where my memory has gone for a java process running in linux. Someone suggested I use pmap -x to see exactly what the memory is doing.
The output is really long but basically a good portion of it is a repeat of…
erotsppa
- 14,248
- 33
- 123
- 181
25
votes
2 answers
understanding pmap output
I was trying to see memory map of a process on Linux x86-64 using pmap -x command. I got confused looking at the output of the pmap. Particularly for the entries for mapping dynamic libraries. There are multiple entries for them (actually 4 for all…
Arka
- 955
- 2
- 12
- 21
22
votes
4 answers
How many threads does Clojure's pmap function spawn for URL-fetching operations?
The documentation on the pmap function leaves me wondering how efficient it would be for something like fetching a collection of XML feeds over the web. I have no idea how many concurrent fetch operations pmap would spawn and what the maximum would…
dan
- 43,914
- 47
- 153
- 254
14
votes
3 answers
'Shared Object Memory' vs 'Heap Memory' - Java
What is difference between 'Shared Object Memory' and 'Heap Memory' in Java. Is it like 'Shared Object Memory' is superset of 'Heap Memory'?
The source of this question is documentation of jmap. It provides different options to Print 'Shared Object…
Sandeep Jindal
- 14,510
- 18
- 83
- 121
14
votes
1 answer
Are the pmap's RSS and htop's RES the same?
I run the following simple program
#include
#include
int
main() {
malloc(1024*1024*32);
getchar();
return 0;
}
htop gives this
VIRT RES SHR
36684 312 240
pmap -x gives this
Address Kbytes RSS …
MKo
- 4,768
- 8
- 32
- 35
10
votes
2 answers
Difference between parallel map and parallel for-loop
when I read the Julia document of multi-core parallel computing, I noticed there are both parallel map pmap and for-loop @distributed for.
From the documentation, "Julia's pmap is designed for the case where each function call does a large amount of…
Tianjing Zhao
- 111
- 4
10
votes
3 answers
Java process memory usage (jcmd vs pmap)
I have a java application running on Java 8 inside a docker container. The process starts a Jetty 9 server and a web application is being deployed. The following JVM options are passed: -Xms768m -Xmx768m.
Recently I noticed that the process consumes…
Konrad
- 1,605
- 3
- 24
- 45
9
votes
1 answer
Where do these java native memory allocated from?
JDK version is hotspot 8u_45
I researched native memory of my java process. The native memory even consumes more space than heap. However there are many native memory blocks which confuses me. The result of pmap -x for example:
00007f8128000000 …
chenatu
- 827
- 2
- 10
- 22
8
votes
1 answer
Julia: use of pmap with Arrays vs SharedArrays
I have been working in Julia for a few months now and I am interested in writing some of my code in parallel. I am working on a problem where I use 1 model to generate data for several different receivers (the data for each receiver is a vector).…
Landon
- 828
- 8
- 23
8
votes
3 answers
clojure pmap vs map
I tested the clojure function map and pmap as show below, in cojure REPL. It make me confused: why the parallel pmap is slower than map?
user=> (def lg (range 1 10000000))
user=> (time (def rs (doall (pmap #(* % %) lg))))
"Elapsed time:…
catchyoulater
- 89
- 2
- 4
7
votes
1 answer
map + pmap, cannot find variables
I am trying to collate results from a simulation study using dplyr and purrr. My results are saved as a list of data frames with the results from several different classification algorithms, and I'm trying to use purrr and dplyr to summarize these…
Melissa Key
- 4,476
- 12
- 21
7
votes
1 answer
Julia, run function multiple times, save results in array
I am building a microsimulation model in Julia. I have built the structure of my function and it runs great for for 1 "person". I'd like to write the script to run 100000+ people through the model and save the results in one location.
Eventually…
MJH
- 398
- 2
- 14
6
votes
1 answer
Test jax.pmap before deploying on multi-device hardware
My question is fairly simple:
I am coding on a single-device small laptop and I am using jax.pmap because my code will run on multiple TPUs. I would like to "fake" having multiple devices to test my code and try different things.
Is there any way to…
Valentin Macé
- 1,150
- 1
- 10
- 25
6
votes
1 answer
JAX pmap with multi-core CPU
What is the correct method for using multiple CPU cores with jax.pmap?
The following example creates an environment variable for SPMD on CPU core backends, tests that JAX recognises the devices, and attempts a device lock.
import…
DavidJ
- 326
- 2
- 10
6
votes
1 answer
Julia pmap performance
I am trying to port some of my R code to Julia;
Basically I have rewritten the following R code in Julia:
library(parallel)
eps_1<-rnorm(1000000)
eps_2<-rnorm(1000000)
large_matrix<-ifelse(cbind(eps_1,eps_2)>0,1,0)
matrix_to_compare =…
Vitalijs
- 938
- 7
- 18