0

How would I write a script that loops through all of my subjects and creates a new script per subject? The goal is to create a script that runs a program called FreeSurfer per subject on a supercomputer. The supercomputer queue restricts how long each script/job will take, so I will have each job run 1 subject. Ultimately I would like to automate the job submitting process since I cannot submit all the jobs at the same time. In my subjects folder I have three subjects: 3123, 3315, and 3412.

I am familiar with MATLAB scripting, so I was envisioning something like this

for i=1:length(subjects)
    nano subjects(i).sh
        <contents of FreeSurfer script>
        input: /subjects(i)/scan_name.nii
        output: /output/subjects(i)/<FreeSurfer output folders>
end

I know I mixed aspects of MATLAB and linux but hopefully it's relatively clear what the goal is. Please let me know if there is a better method.

Here is an example of the FreeSurfer script for a given subject

#!/bin/bash
#PBS -l walltime=25:00:00
#PBS -q long

export FREESURFER_HOME=/gpfs/software/freesurfer/6.0.0/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh

export SUBJECTS_DIR=/gpfs/projects/Group/ppmi/freesurfer/subjects/

recon-all -i /gpfs/projects/Group/ppmi/all_anat/3105/Baseline/*.nii -s 
$SUBJECTS_DIR/freesurfer/subjects/3105 -autorecon-all

The -i option gives the input and the -s option gives the output.

rwales
  • 1
  • 1
  • 1
    What do you mean by "subjects"? Can you show an example of what you're trying to do (the manual process)? Then we can try to automate it. – melpomene Aug 21 '18 at 18:58
  • @melpomene I updated the question with the sample script – rwales Aug 21 '18 at 19:13
  • 2
    This sounds like an [XY problem](http://xyproblem.info/). Rather than creating scripts on the fly, how about running the same script with different parameters? – John Kugelman Aug 21 '18 at 19:13
  • @JohnKugelman do you mean editing the script to change the subject number each time you run the subject? If so, that's what I've been doing, but it's not efficient. – rwales Aug 21 '18 at 19:17
  • 1
    What do you mean, *the* sample script? Wasn't this about multiple scripts? And you still haven't explained what a subject is. – melpomene Aug 21 '18 at 19:32
  • @melpomene sorry for the confusion. In this case a subject is a participant in this experiment that has an MRI brain scan in the format of a .nii file. Freesurfer runs a complex analysis on each subject's brain scan. The goal is to have this Freesurfer script run every single subject, as in keep the entire script the same but just change the subject number. I was wondering if there's a way I can automate creating a new script per subject. Does that help? Thanks for your interest and sorry if it's still not clear! – rwales Aug 21 '18 at 19:44
  • 1
    Why don't you just make the subject number a parameter to the script, instead of having to create a separate script for each subject? – Barmar Aug 21 '18 at 19:48

1 Answers1

0

change your script to accept the subject as an argument, so that you have only one generic script.

#!/bin/bash
#PBS -l walltime=25:00:00
#PBS -q long

subject="$1"

export FREESURFER_HOME=/gpfs/software/freesurfer/6.0.0/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh

export SUBJECTS_DIR=/gpfs/projects/Group/ppmi/freesurfer/subjects/

recon-all -i /gpfs/projects/Group/ppmi/all_anat/"$subject"/Baseline/*.nii -s 
$SUBJECTS_DIR/freesurfer/subjects/"$subject" -autorecon-all

and you can call it for all your subjects

for s in 3123 3315 3412; 
do
   ./yourscriptnamehere.sh "$s"
done

add error handling as desired.

karakfa
  • 66,216
  • 7
  • 41
  • 56
  • I believe the `#PBS` comments are directives to a job scheduler -- the OP's issue may be that they don't know how to pass arguments through that scheduler. – Charles Duffy Aug 21 '18 at 20:08
  • I think they only pass the input/output directories, not sure the scheduler values need to be parametrized as well. – karakfa Aug 21 '18 at 20:10
  • I agree that the scheduler values don't need to be parameterized, but the point is that the directory names may need to be passed *through* `qsub`, and it's not evident that the OP knows that this is possible or how to do it. [parameter for shell scripts that is started with qsub](https://stackoverflow.com/questions/3504081/parameter-for-shell-scripts-that-is-started-with-qsub) may be pertinent. – Charles Duffy Aug 21 '18 at 20:11