1

I would like to use a bash script and qsub to run 30-40 python programs at once. Each python program reads and searches through the same set of files (~400 total) for a set of sequences. Is there a problem where multiple python programs could be trying to read from the same file? If so, what are the consequences?

Patrickc01
  • 145
  • 1
  • 1
  • 6

1 Answers1

2

There are no problems with multiple jobs reading from the same files that are imposed by Torque. (I think that statement is likely to be true for any resource manager / scheduler.)

The main issue I can imagine would be your file system's performance and whether or not it can keep up with potentially concurrent accesses to those files.

dbeer
  • 6,963
  • 3
  • 31
  • 47