6

I'm trying to run my program using torque scheduler using mpi run. Though in my pbs file I load all the library by 

export LD_LIBRARY_PATH=/path/to/library

yet it gives error i.e. 

error while loading shared libraries: libarmadillo.so.3: 
cannot open shared object file: No such file or directory.

I guess error lies in variable LD_LIBRARY_PATH not set in all the nodes. How would I make it work?

arbitUser1401
  • 575
  • 2
  • 8
  • 25

2 Answers2

18

LD_LIBRARY_PATH is not exported automatically to MPI processes, spawned by mpirun. You should use

mpirun -x LD_LIBRARY_PATH ...

to push the value of LD_LIBRARY_PATH. Also make sure that the specified path exists on all nodes in the cluster and that libarmadillo.so.3 is available everywhere.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • 2
    If Open MPI was compiled with native Torque/TM support, the LD_LIBRARY_PATH (and the rest of the environment) should be propagated out to all the nodes in the allocation automatically. If Open MPI wasn't built with the native Torque support, then I'm guessing you're using rsh/ssh to launch, in which case the "-x LD_LIBRARY_PATH" clause is necessary. I'd actually advise re-building OMPI with Torque/TM support; it's better than rsh/ssh support for multiple reasons. – Jeff Squyres Jul 09 '12 at 14:56
  • THANK YOU. I WAS DRIVING MAD WITH THIS SHIT.... Exporting the LD_LIBRARY_PATH is supposed to let child processes use those variables, but it turns out it doesn't. – Greg K. Mar 22 '16 at 22:56
  • @GregK. exporting only works for processes launched on the same node where `mpiexec` runs as those are child processes of the latter. Processes launched on remote nodes are children of the respective ORTE shepherd daemon and don't directly inherit the environment of `mpiexec` but only the environment that gets forwarded to and additionally set by the daemon. (I might be wrong; Jeff Squyres knows better) – Hristo Iliev Mar 22 '16 at 23:29
3

On some systems, your environment isn't always propagated via mpirun. You should set all those variables in your .bashrc file.

eduffy
  • 39,140
  • 13
  • 95
  • 92