9

I created some slurm scripts and then tried to execute them with sbatch. But the output file is updated not frequently (once a minute maybe).

Is there a way to change the output buffering latency in sbatch? I know stdbuf is used in such situations but I could not make it work with sbatch.

ahemya
  • 101
  • 1
  • 2
  • see http://stackoverflow.com/questions/25170763/how-to-change-how-frequently-slurm-updates-the-output-file-stdout/25189364#25189364 – Carles Fenoy May 20 '15 at 23:41
  • I am aware of that question and answer. but it looks like the buffering of output in sbatch is not related to stdbuf. It looks like sbatch command has some internal buffering. – ahemya May 21 '15 at 18:38
  • Sbatch just runs the shell stated on the first line of the script. Usually bash, so there is no buffering by Slurm itself. What exactly are you trying to run? – Carles Fenoy May 21 '15 at 21:53
  • 4
    if you use srun within sbatch you could try to use the -u option (u=unbuffered – PlagTag Jun 01 '15 at 13:36

1 Answers1

4

The issue is certainly with buffering. If you are trying to run python code, add flush=True in print command like print(...,flush=True).

sohaib kiani
  • 49
  • 2
  • 1
    A better alternative is `python -u` or something like `sys.stdout.reconfigure(line_buffering=True, write_through=True)`. – heiner Jul 11 '23 at 20:38