2

I am testing how different variables affect a model using a parameter sweep. I am trying to read in parameters from a text file and pass them to a function where the model is calculated and answer returned and output.

For example, if I have an input file with the parameter name as the first row and successive rows as the parameter combinations I would like to compute:

input.txt 
param1, param2, param3, ..., paramm
0,0,1
0,0,2
0,1,1
0,1,2
0,2,1
0,2,2
1,0,1
1,0,2
1,1,1
1,1,2
1,2,1
1,2,2

I have a function that computes my model. To keep it simple, I will say the model is the sum of all my parameters:

def model(param1,param2,param3...,paramm):
    param1=0
    param2=0
    param3=0
    param4=0
    param5=0
    .
    .
    .
    paramn=0

    answer=parma1+param2+param3+param4+param5

    return answer

where setting the parameters=0 is a way of initializing them (I now know it does not work for param1,param2 or param3 because it overwrites what was written to them). I include param4 and param5 to signify that there might be other parameters that are there that I do not vary but take some default value.

How do I read in the file and run all parameter combinations? Should I just create some tuple called params and adjust the values each time after reading in the file or is there a better way?


EDIT1:

I have edited the question to add a higher number of paramemeters that are known in the text file and constants. Would I be better off creating a parameter object and editing the values of the object for each pass to the function.

Please realize that though the function is a sum in this example, it will be much more complicated in the real program, likely calling other functions with only a subset of the parameters. For example

def model1(param1):
    a=1
    b=2
    c=3

    return a*param1**2+b*param1+c 

def model(param1,param2,param3...,paramm):
    param1=0
    param2=0
    param3=0
    param4=0
    param5=0
    .
    .
    .
    paramn=0

    answer=model1(param1)+param2+param3+param4+param5
js16
  • 43
  • 5

2 Answers2

0

Use with open and make model much easier to solve the problem, use the below as the full code:

def model(param1,param2,param3):
    return param1 + param2 + param3
with open('input.txt','r') as f:
    next(f)
    for i in f:
        print(model(*i.split(',')))
iz_
  • 15,923
  • 3
  • 25
  • 40
U13-Forward
  • 69,221
  • 14
  • 89
  • 114
  • `f.readlines()[1:]` is unoptimal, you can use `islice` or simply call `next(f)` followed by `for i in f` – iz_ Jan 23 '19 at 00:33
  • It reads the whole file into memory, instead of getting lines when needed. – iz_ Jan 23 '19 at 00:34
  • @Tomothy32 Thanks, i would do that but kinda got confused on what you said, lol :-) – U13-Forward Jan 23 '19 at 00:39
  • For future reference, you can check out https://stackoverflow.com/questions/4796764/read-file-from-line-2-or-skip-header-row – iz_ Jan 23 '19 at 00:39
  • Isn't the first line in his file "input.txt", though? In which case, wouldn't `model()` fail as it tries to get three input values from that `split`? – Bill M. Jan 23 '19 at 00:43
  • @BillM.WHy would it? – U13-Forward Jan 23 '19 at 00:44
  • @U9-Forward, beats me, but that's what the user says is being used. (Quote: "I have an input file with the parameter name as the first row and successive rows as the parameter combinations") – Bill M. Jan 23 '19 at 00:56
  • 1
    @BillM. See my edit on U9's answer. `next(f)` skips a line. – iz_ Jan 23 '19 at 01:38
0

There are several ways you could do this. If that first line in the file weren't there, then you could just treat the file as a .csv file (the top row of "param1,param2,param3" is the header, and the subsequent rows are comma-separated values for those three columns). Python already has readily-available modules that will read and write data to and from CSV files. You can read about it here.

In your particular case though, your file begins with a single line of data, followed by CSV headers on the second line, and then the third row and onward has CSV dat. If you're ultimately trying to read each row of numerical data and pass it to a function, then you could do this:

def find_sum(values):
    return sum(values)

with open(r"C:\tmp\myfile.txt", 'r') as f:
    data = f.readlines()

dataname = data[0].strip()            # The first row of the file
row_names = data[1].strip().split(',')  # The second row of the file

for line in data[2:]:
    values = [ float(i) for i in line.strip().split(',') ]
    print "My function on", values, "returns", find_sum(values)

This prints out:

My function on [0.0, 0.0, 1.0] returns 1.0
My function on [0.0, 0.0, 2.0] returns 2.0
My function on [0.0, 1.0, 1.0] returns 2.0
My function on [0.0, 1.0, 2.0] returns 3.0
My function on [0.0, 2.0, 1.0] returns 3.0
My function on [0.0, 2.0, 2.0] returns 4.0
My function on [1.0, 0.0, 1.0] returns 2.0
My function on [1.0, 0.0, 2.0] returns 3.0
My function on [1.0, 1.0, 1.0] returns 3.0
My function on [1.0, 1.0, 2.0] returns 4.0
My function on [1.0, 2.0, 1.0] returns 4.0
My function on [1.0, 2.0, 2.0] returns 5.0

Taking a closer look at the command [ float(i) for i in line.strip().split(',') ], what this does is take a line of numbers from the file, uses .strip() to chop off the newline character at the end, splits up the content on the comma, and converts the contents from strings to floats. You could use integers instead of floats, but I don't know if your real data might include non-numeric values. I also made variables out of the first two lines of the file, if you need to use them.

EDIT: If, as the commenter explained, readlines() won't do the job because you have zillions of lines, then here's a reworking, albeit one that skips lines until it finds one that starts with a digit which it presumes is a row of data:

reached_numbers = False
with open(r"C:\tmp\myfile.txt", 'r') as f:
    for line in (f):
        if reached_numbers:
            values = [ float(i) for i in line.strip().split(',') ]
            print "My function on", values, "returns", find_sum(values)
        else:
            reached_numbers = line[0].isdigit()
Bill M.
  • 1,388
  • 1
  • 8
  • 16