Python chain several functions into one

Question

I have several string processing functions like:

def func1(s):
    return re.sub(r'\s', "", s)

def func2(s):
    return f"[{s}]"
...

I want to combine them into one pipeline function: my_pipeline(), so that I can use it as an argument, for example:

class Record:
    def __init__(self, s):
        self.name = s
    
    def apply_func(self, func):
        return func(self.name)

rec = Record(" hell o")
output = rec.apply_func(my_pipeline)
# output = "[hello]"

The goal is to use my_pipeline as an argument, otherwise I need to call these functions one by one.

Thank you.

Mechanic Pig · Accepted Answer · 2022-06-27T13:26:43.643

You can write a simple factory function or class to build a pipeline function:

>>> def pipeline(*functions):
...     def _pipeline(arg):
...         result = arg
...         for func in functions:
...             result = func(result)
...         return result
...     return _pipeline
...
>>> rec = Record(" hell o")
>>> rec.apply_func(pipeline(func1, func2))
'[hello]'

This is a more refined version written with reference to this using functools.reduce:

>>> from functools import reduce
>>> def pipeline(*functions):
...     return lambda initial: reduce(lambda arg, func: func(arg), functions, initial)

I didn't test it, but according to my intuition, each loop will call the function one more time at the python level, so the performance may not be as good as the loop implementation.

score 1 · Answer 2 · answered Jun 04 '22 at 10:05

1

You can just create a function which calls these functions:

def my_pipeline(s):
    return func1(func2(s))

answered Jun 04 '22 at 10:05

quamrana

37,849
12
53
71

Thank you! This solution works for this specific case, but a solution to combine various number of different functions would be more suitable. – MonkandMonkey Jun 05 '22 at 01:35

jonsca · Answer 3 · 2022-06-04T10:47:42.647

Using a list of functions (so you can assemble these elsewhere):

def func1(s):
    return re.sub(r'\s', "", s)

def func2(s):
    return f"[{s}]"

def func3(s):
    return s + 'tada '

def callfuncs(s, pipeline):
    f0 = s
    pipeline.reverse()
    for f in pipeline:
        f0 = f(f0)        
    return f0

class Record:
    def __init__(self, s):
        self.name = s

    def apply_func(self, pipeline):
        return callfuncs(s.name, pipeline)
        
# calling order func1(func2(func3(s)))
my_pipeline = [func1, func2, func3]
rec = Record(" hell o")
output = rec.apply_func(my_pipeline)

Python chain several functions into one

3 Answers3

Linked