how to dynamically create a function with changing number of parameter pairs

Question

So I've been attempting to create a function during runtime, which should dynamically adds pairs of parameters. To give an idea of what I'm looking for, here is what I've made so far :

def smart_func(terms):
    params = []
    for n in range(terms):
        params.append((2*n*np.pi, 2*n*np.pi))

    def func(t, freq, offset, *params):
        result = 0
        for (a,b) in zip(params):
            result += np.sin(a*freq*t) + np.cos(b*freq*t)
        return result
    return func

I know this does not work but should give some idea of what I'm attempting to do. I've looked at this question but still have been unable to come up with a solution.

To give a little bit more explanation, i then need to pass this newly created function into this

from scipy.optimize import curve_fit
    f_vars, f_cov = curve_fit(smart_func(terms=3), time_in_hours, full_fit_flux, p0=p0)

Which will allow me to easily determine the fewest amount of parameters needed to adequetly fit my data.

This is a hard coded function that I've used successfully. If smart_func had a 3 passed to it, it would return this function.

    def func(t, freq, offset, a0, b0, a1, b1, a2, b2):
        return b0 + a0 \
            + a1*np.sin(2.*np.pi*freq*t) \
            + b1*np.cos(2.*np.pi*freq*t) \
            + a2*np.sin(4*np.pi*freq*t) \
            + b2*np.cos(4*np.pi*freq*t) \
            + offset

This is what it would be if smart_func had a 2 passed to it

    def func(t, freq, offset, a0, b0, a1, b1):
        return b0 + a0 \
            + a1*np.sin(2.*np.pi*freq*t) \
            + b1*np.cos(2.*np.pi*freq*t) \
            + offset

What I want is something where an additional a and b term are added depending on how many terms are specified.

I don't understand, why you would do this at runtime at all? Why not just define a single function that takes terms, t, freq, and offset? — Nir Friedman, Jul 20 '17 at 21:33
It isn't clear to me what you want... `params` is a list of numbers, AFAIKT — juanpa.arrivillaga, Jul 20 '17 at 21:33
`def my_function(*params):` defines the function to take any number of arguments and they will be collected into a single sequence of values named `params`—which you can then process like any other variable-length container. Your `def func` already has that at then end, so it's unclear what you're asking. — martineau, Jul 20 '17 at 21:34
because I'm giving this newly created function to another curve_fit, a fitting function. I need to have all the parameters defined in the function. — Giovanni, Jul 20 '17 at 21:37
They follow a pattern, basically starting a1 b1, a2 b2, and so forth. But I don't really care what they are named, as long as a1 is associated with b1 — Giovanni, Jul 20 '17 at 21:45
I might be alittle slow sry, but I'm still not getting how params is used. do you use a different set of params (ie tuples (a1,b1)...) every time func is called? Or is your data list dependent on terms? — Xingzhou Liu, Jul 20 '17 at 21:51
I need to check how well the function is fitted by a certain number of parameters, increasing the number of parameters used until I am satisfied with the fit of the curve to my data. — Giovanni, Jul 20 '17 at 21:55
Ok, so the process looks like smart_func(x) -> fit using different values of t, frequency, and offset, for x points (a,b) -> evaluate -> then smart_func(x+increment) -> fit using different values of t, frequency, and offset, and x+increment points (a,b) then evaluate If so that's the assumptions I made doing the code in answer below. — Xingzhou Liu, Jul 20 '17 at 22:04
Can you please [edit] your question and show exactly what the dynamic function would look like for an example number of parameters? It's unclear how the code in it should be written to handle different numbers of individually named arguments (unlike what you have in it at this time). — martineau, Jul 20 '17 at 22:06
updated code below. is there a way you can pass in a list of [ (ai,bi) ... ] pairs, or if not two lists? [a1, a2 ....] , [b1, b2, .... ]. func(t, freq,offset, a1, b1 ... ) is just a really awkward format. — Xingzhou Liu, Jul 20 '17 at 22:39
curve_fit() seems to require a function of the form that I have hardcoded sadly. — Giovanni, Jul 20 '17 at 22:51

martineau · Accepted Answer · 2017-07-21T17:42:28.440

The following shows how to dynamically create the desired function. Note that I simplified the dynamic function's code to minimize redundant calculations.

from textwrap import dedent

def test(num_terms):

    def smart_func(num_terms):  # nested to mimic OP's usage
        template = dedent('''
            def func(t, freq, offset, a0, b0, {params}):
                ang = 2.*np.pi*freq*t
                sin_ang = np.sin(ang)
                cos_ang = np.cos(ang)
                return (a0 + b0
            {terms}
                        + offset)
            ''')
        indent = ' ' * 12
        params, terms = [], []
        for i in range(1, num_terms):
            params.append('a{i}, b{i}'.format(i=i))
            terms.append((indent + '+ a{i}*sin_ang\n' +
                          indent + '+ b{i}*cos_ang').format(i=i))

        src_code = template.format(params=', '.join(params), terms='    \n'.join(terms))

        print('Dynamically created function of {} terms:'.format(num_terms))
        print(src_code)

        exec(src_code, globals(), locals())  # compile into function object
#        exec src_code in globals(), locals()  # older Python 2 syntax

        return locals()['func']  # return compiled function

    return smart_func(num_terms)  # return result of calling nested function

print(test(3))

Output:

Dynamically created function of 3 terms:

def func(t, freq, offset, a0, b0, a1, b1, a2, b2):
    ang = 2.*np.pi*freq*t
    sin_ang = np.sin(ang)
    cos_ang = np.cos(ang)
    return (a0 + b0
            + a1*sin_ang
            + b1*cos_ang
            + a2*sin_ang
            + b2*cos_ang
            + offset)

<function func at 0x0232A228>

hmm i get the error SyntaxError: unqualified exec is not allowed in function 'smart_func' because it is a nested function. I got this earlier trying to use exec — Giovanni, Jul 20 '17 at 23:02
It's a Python version issue, I'm using 3.6.2 and you must be using 2.x. Try using `exec(func_def, globals(), locals())` and if if that doesn't work try `exec func_def in globals(), locals()`. — martineau, Jul 21 '17 at 01:26
Thank you! With a few tweaks I got it to do exactly what I was looking for with the exact results that the hard coded function gave me. — Giovanni, Jul 21 '17 at 17:31
Giovanni: Good to hear. Note I made a few relatively minor changes, primarily to improve the readability of the code. — martineau, Jul 21 '17 at 17:46

Xingzhou Liu · Answer 2 · 2017-07-20T22:51:38.957

1

try:

def smart_func(terms):
        params = []
        for n in range(terms):
            params.append(2*n*np.pi)

        # def func(t, freq, offset, *args ) will
        # overwrites the original params list within func
        #

        def func(t, freq, offset, *args):
            an = []
            bn = []

            for i in range(len(args)):
                if i%2==0 :
                   an.append(args[i])
                else:
                   bn.append(args[i])
            result = 0

            pairs = zip(an,bn)

            for (q,ab) in zip(params, pairs):
                #q is 2 * n * pi  
                ai, bi = ab
                result += ai * np.sin(q*freq*t) + bi * np.cos(q*freq*t)
            return result
        return func

Where q are terms of the sequence 2 * i * pi for i in range(terms) , and pairs (ai, bi) are coefficients for sin(q * freq * t) + cos(q * freq * t).

edited Jul 20 '17 at 22:51

answered Jul 20 '17 at 21:38

Xingzhou Liu

1,507
8
12

for the first run through i%0 throws a divide by zero error – Giovanni Jul 20 '17 at 22:50
sry, i meant i%2, not i%0. changed – Xingzhou Liu Jul 20 '17 at 22:51
This is close. However args* contains t, freq, offset, and then the 5 additional args. – Giovanni Jul 20 '17 at 22:57
so t , freq, and offset come before *args, so they don't affect one another. as for 5 arguments, I count 3 - t, freq, and offset. what are other 2? – Xingzhou Liu Jul 20 '17 at 23:02
actually I just ran it without the guess parameters and noticed that it's completely empty. The parameters as curve_fit sees them are non existent. – Giovanni Jul 20 '17 at 23:07
change func to def func(*args): print(args) to see what its actually giving you. – Xingzhou Liu Jul 20 '17 at 23:08
I do suspect that they are giving you an array/vector somewhere. evaluating a function with an indefinite and large number of arguments causes memory and performance issues as n gets large. – Xingzhou Liu Jul 20 '17 at 23:17
I get t = [...] freq = 1.0 offset = 1.0 args = () – Giovanni Jul 20 '17 at 23:24
so the docs seem to suggest its passing time_in_hours - the entire ndarray , as the first argument. Is time_in_hours a N * 1 or N * M, where M is terms, for each set of terms ? gotta vectorize func. – Xingzhou Liu Jul 20 '17 at 23:44
I forgot how to do it in numpy, but believe that you'd need to make args an N * terms matrix (since coding all the coefficients in is too time consuming), since t is a vector. I don't remember the syntax, but it needs to be expressed as a matrix multiply. results should be a length N vector, not a scalar. – Xingzhou Liu Jul 21 '17 at 00:07

score 1 · Answer 3 · answered Jul 20 '17 at 21:42

What you want is a partial function. Using partial functions you can pass a specific value of params to func and it will return a new function with a constant value of param. Check out the funcy library.

from funcy import rpartial

def func(t, freq, offset, *params):
    result = 0
    for (a,b) in zip(params):
        result += np.sin(a*freq*t) + np.cos(b*freq*t)
    return result

params = (2 * np.pi, 2 * np.pi)
partial_func = rpartial(func, params)

#call partial_func like below
result = partial_func(t0, freq0, offset0)

how to dynamically create a function with changing number of parameter pairs

3 Answers3