Pass a function/method to a python script

Question

I have defined this method/function in a google colab cell

[5]:def lstm_model2(input_features, timesteps, regularizers=regularizers, batch_size=10):

       model = Sequential()
       model.add(LSTM(15, batch_input_shape=(batch_size, timesteps, 
          input_features), stateful=True,
                   recurrent_regularizer=regularizers))
       model.add(Dense(1))
       model.compile(loss='mae', optimizer='adam')

       return  model

I want to pass this method to a script I am executing in the next cell using argparse.

[6]:!python statefulv2.py --model=lstm_model2

I tried an approach similar to type argument in argparse like defining a identically named abstract method inside the statefulv2.py script so that argparse.add_argument('--model', help='LSTM model', type=lstm_model2, required=True) can be written inside statefulv2.py But this raises an invalid argument error.

Is there a neat way to pass methods as arguments in argparse?

The reason for keeping the method outside is to edit it for trying differeny functions since GoogleColab does not provide separate editor for changing model in a separate file.

If you are trying to call a function based on argparse https://stackoverflow.com/questions/27529610/call-function-based-on-argparse then see the related answer there. — ayman, Aug 05 '19 at 19:24
It's probably easier to just write your function into a module file that you load based on a keyword argument. — Linuxios, Aug 05 '19 at 19:31
I mentioned at the end of my post how that is a problem on google colab. I will have to upload the file everytime i modify it since colab doesn't have editors — CS101, Aug 05 '19 at 20:05
On a different note, is there no way to pass a method as an argument during a script execution? — CS101, Aug 05 '19 at 20:06
What get's passed through the command line and an argparse (or similar parser) is just a string, a name. Not an object or function. Typically when you want a new function in a script, you import it - via another `.py` file. — hpaulj, Aug 05 '19 at 20:09
When trying ideas like this, it's a good idea to check `sys.argv` to see what strings get passed. And if the parser does run, print its `args` to be sure of what it has seen and done. — hpaulj, Aug 05 '19 at 20:33

Vedang Waradpande · Answer 1 · 2019-08-07T05:16:51.027

It's best that you don't try to pass arguments like that. Stick to the basic types. An alternative would be to store the different models in a file like models.py, such as:

def lstm_model1(...):
    # Definition of model1

def lstm_model2(...):
    # Definition of model2

and so on.

In statefulv2.py you can have:

import models
import argparse

parser = ...
parser.add_argument('-m', "--model", help='LSTM model', 
                      type=str, choices=['model1', 'model2'], required=True)

model_dict = {'model1': models.lstm_model1(...),
              'model2': models.lstm_model2(...)}

args = parser.parse_args()

my_model = model_dict[args.model]

EDIT: If saving model to file is not allowed.

In case you absolutely have to find a workaround, you can save the code in a buffer file which can be read into statefulv2.py as an argument of type open. Here is a demo:

In your Colab cell, you write the function code in a string:

def save_code_to_file():
    func_string = """
def lstm_model3():
    print ("Hey!")
"""
    with open("buffer", "w") as f:
        f.write(func_string)

In statefulv2.py, you can have:

import argparse

parser = argparse.ArgumentParser()

parser.add_argument('-m', "--model", help='LSTM model', type=open, required=True)

args = parser.parse_args()

lines = args.model.readlines()

model_def = ""
for line in lines:
    model_def += line

exec(model_def)

lstm_model3()

Output (this is in iPython, change it accordingly for Colab):

In [25]: %run statefulv2.py -m buffer                                                  
Hey!

Yes, this works but the issue is that my local workstation does not have resources for running larger models. And while colab does that, it does not let us edit files on the go. In this case I will have to reupload models.py everytime — CS101, Aug 05 '19 at 20:04
There are two workarounds I can think of: Running ```statefulv2.py``` in a Colab cell and writing the function to a temporary string and then reading as an ```open``` type in ```statefulv2.py```. I will edit my answer. — Vedang Waradpande, Aug 07 '19 at 05:06

Pass a function/method to a python script

1 Answers1