3

I want to change a little bit my workflow for running NetLogo on a HPC using Slurm. For context, I run around 360 simulations in parallel, each one can take from 5 to 7 days (I know, not efficient) and they write some outputs at the end of X ticks. I post-process all these outputs after with Python.

I currently use JobArrays, but since the cluster I'm using has nodes with only 48 cores I have to create 9 BehaviorSpaces by hand (I know, I could use XML files).

I have been reading and it seems that in order to run multi-nodes experiments I have to be careful. A few alternatives I have been thinking:

  1. pynetlogo + mpi4pi
  1. pynetlogo + ipyparallel, it would be similar to this example https://pynetlogo.readthedocs.io/en/latest/_docs/SALib_ipyparallel.html

I have no experience with MPI or ipyparallel, but I have experience with Python, NetLogo and a little with Slurm.

I considered netlogo-headless but as far as I know that works with BehaviorSpaces and that is exactly what I want to avoid.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
  • Not an answer to your question, but based on the description of your models, it sounds like you might be affected by the "NetLogo headless still runs plots" memory usage issue: https://github.com/NetLogo/NetLogo/issues/2120 That issue says BehaviorSpace, but the same would be true for any run of NetLogo through the command line without a UI. – Jasper Jul 10 '23 at 15:37
  • What is your *question*? – desertnaut Aug 12 '23 at 21:38

0 Answers0