Modules not loading when executed via a script (Bash)

Question

I'm writing a bash script to perform some commands on files. In order to do that, I need to load the modules which contain these commands. Now I can do these from the command line just fine, but I wanted to implement a check in a script to see if those modules are loaded, and if they are not, to load them. Here is what I have done so far (I am just 3 days into learning bash, so excuse any newbie errors):

#A list with all the necessary modules
declare -a modules=("CDO/1.9.7-gompi-2019a" "ncview/2.1.7-gompi-2019a")

echo "Running script"
echo ""

#Check if modules are loaded. If not, load them
for mod in "${modules[@]}"
do
    module is-loaded $mod

    if [ $? = 1 ]
    then
        echo "Loading ${mod}"
        module load $mod
    else
        echo "${mod} is already loaded"
    fi
done

echo ""
echo "Finished running script"

It enters the if statement, meaning the module is not loaded, and then says "Loading CDO...", but when I try to use commands from this module, it does not work because the module just doesn't seem to be executed. But the "module is-loaded" executes just fine. What am I doing wrong?

[diyon@login2 scripts]$ ./script.sh
Running script

Loading CDO/1.9.7-gompi-2019a
Loading ncview/2.1.7-gompi-2019a

Finished running script
[diyon@login2 scripts]$ cdo
-bash: cdo: command not found

The script logic seems fine. It might be more helpful to post the question about why it's not working on [unix.se]. — Barmar, Oct 28 '21 at 21:07
Basically the command is not found. I added some console log to show you — Diyon335, Oct 28 '21 at 21:18
Out of curiosity: I've never seen a `module` command in all my time of using Linux, nor does `is-loaded` mean anything to me - where does that command come from, what kind of modules are these? — tink, Oct 28 '21 at 21:42
You can read about it here: https://lmod.readthedocs.io/en/latest/ . I am just starting out with Linux so I am not sure how to explain what this command does. But these modules are used for climate data analysis — Diyon335, Oct 28 '21 at 21:54
But if you type `module load CDO/1.9.7-gompi-2019a` by hand it works? I can't think of a good reason why it would work differently. — Barmar, Oct 28 '21 at 22:55

score 2 · Accepted Answer · answered Oct 29 '21 at 05:02

The clue is in the full name: “Environment Modules”

These modules (not to be confused with kernel modules or any other modules), are a tool used to configure the current environment (for development environments, administrators etc.).

I had not heard of these until today. According to these slides they use module files containing a declarative environment description written in Tcl. When you load them, they are converted to shell syntax, and evaled in the current environment.

———————-

The reason your script did not work is that a shell script executes in a new process, and can not change the environment of its parent process - the shell you launched it from. You configured the environment of the script (where the module dependant commands would have worked), but that environment is gone when the script exits.

The solution is probably to configure your environment modules in a module file (man modulefile or https://linux.die.net/man/4/modulefile), or whatever is recommended by the “Modules” documentation.

Or, you can implement your script as a shell function. Shell functions execute in the current shell environment, so your modules will still be loaded at the end of it.

If you run type module, you can see that module itself is a shell function, which is why it’s able to modify the current environment.

If you append the following function to the file ~/.bashrc and restart the shell, you can run load_modules to execute it.

I think quoting (variables etc) is unnecessary in this situation, but I added it anyway.

load_modules ()
{
    # A list with all the necessary modules
    local -a modules=('CDO/1.9.7-gompi-2019a'
                      'ncview/2.1.7-gompi-2019a')
    local mod

    echo "Running function"
    echo

    # Check if modules are loaded. If not, load them
    for mod in "${modules[@]}"
    do
        if module is-loaded "$mod"
        then
            echo "$mod is already loaded"
        else
            echo "Loading $mod…"
            module load "$mod"
        fi
    done

    echo
    echo "Finished running the function"
}

I've lost SSH access currently and I am working on resolving this with my university. Once I get it back I will try this and let you know — Diyon335, Oct 31 '21 at 19:12

score 0 · Answer 2 · answered Oct 31 '21 at 08:27

The module command, from the Environment Modules software update the current environment session.

As explained by @dan, when executed as ./script.sh, your script is run in a subshell that ends at the end of the script execution. So when run this way the script updates the shell environment of the subshell not the parent shell.

If you source your script in your current shell session with source script.sh you will perform the module load commands in the current shell (not in a subshell session) so you will end with the current shell environment updated.

Modules not loading when executed via a script (Bash)

2 Answers2