2

I’ve been recently playing with the MPI wrappers for Julia (MPI.jl) – yesterday I built a dumb test problem in both C and Julia and was comparing performance, and Julia is pretty close to C which is impressive!

Julia/C parallel comparison

I thought I might be able to close the gap even further by using the PackageCompiler.jl package to speed up the initial loading cost in Julia, but I can’t get it to play nicely. If I run:

julia --sysimage myImage.so --project myProject.jl

it works fine (where myImage.so is a system image I generated with PackageCompiler.jl), but if I run:

mpiexec -n np julia --sysimage myImage.so --project myProject.jl

I get a signal 6 abort. This is my first experience with both the MPI.jl and PackageCompiler.jl packages, so any tips/insights are much appreciated!

kirklong
  • 203
  • 1
  • 2
  • 9
  • 1
    update based on a suggestion in the Julia Slack I tried wrapping the Julia command in a bash script for `mpiexec` to call, but this has actually led me to believe the problem is with the sysimage as the bash script core dumps even if run *without* MPI... returning the following error: ```julia: /buildworker/worker/package_linux64/build/src/processor.cpp:465: std::vector<{anonymous}::TargetData > {anonymous}::deserialize_target_data(const uint8_t*) [with long unsigned int n = 9; uint8_t = unsigned char]: Assertion `nfeature == n' failed.``` – kirklong Nov 15 '20 at 02:37
  • 1
    second update: I thought maybe it was a weird environment issue but upon comparing the environments when run through bash script vs. just in terminal the only difference was that one had an environment variable that pointed to the link to Julia in my `/usr/bin/local` whereas the other pointed straight to the executable instead of the link so it's sadly not that simple :( – kirklong Nov 15 '20 at 02:59

1 Answers1

0

This does work! The problem was that I recently updated Julia and had made my script executable with #!/usr/bin/env julia. When I compiled the system image that ended up using Julia 1.5.x while the symlink in /usr/bin still pointed to the old Julia (1.4.x).

So if you want to run your Julia project in Parallel with a custom image you can indeed just write:

mpiexec -n np julia --sysimage myImage.so --project myProject.jl

and it will run blazing fast! (assuming your OS isn't confused about which Julia to use)

kirklong
  • 203
  • 1
  • 2
  • 9