What is scikit-learn FunctionTransformer used for?

Question

From the documentation I already read that:

A FunctionTransformer forwards its X (and optionally y) arguments to a user-defined function or function object and returns the result of this function. This is useful for stateless transformations such as taking the log of frequencies, doing custom scaling, etc.

However, I don't understand what use this function has. Could anybody explain the purpose of this function?

One use-case i could think of: add some *stateless transformation* like scaling to a non-simple pipeline (e.g. dimension-reduction -> classifier within cross-validation). This abstract function should deliver the interface needed for this pipeline (fit, transform and co.) without any need to write this for yourself. — sascha, Jul 19 '16 at 19:37

bogatron · Accepted Answer · 2022-04-04T00:38:16.650

10

In addition to simply wrapping a given user-defined function, the FunctionTransformer provides some standard methods of other sklearn estimators (e.g., fit and transform). The benefit of this is that you can introduce arbitrary, stateless transforms into an sklearn Pipeline, which combines multiple processing stages. This makes executing a processing pipeline easier because you can simply pass your data (X) to the fit and transform methods of the Pipeline object without having to explicitly apply each stage of the pipeline individually.

Here is an example copied directly from the sklearn documentation (located here):

def all_but_first_column(X):
    return X[:, 1:]

def drop_first_component(X, y):
    """
    Create a pipeline with PCA and the column selector and use it to
    transform the dataset.
    """
    pipeline = make_pipeline(
        PCA(), FunctionTransformer(all_but_first_column),
    )
    X_train, X_test, y_train, y_test = train_test_split(X, y)
    pipeline.fit(X_train, y_train)
    return pipeline.transform(X_test), y_test

Note that the first principal component wasn't explicitly removed from the data. The pipeline automatically chains the transformations together when pipeline.transform is called.

edited Apr 04 '22 at 00:38

answered Jul 21 '16 at 13:48

bogatron

18,639
6
53
47

Thanks for the help. Could you provide an example?. – tumbleweed Jul 21 '16 at 18:27
@ml_student See the example I added to my answer. – bogatron Jul 21 '16 at 20:14
Thanks for the help! – tumbleweed Jul 22 '16 at 01:33
The code example has been relocated or removed. Here's a versioned link to [Using FunctionTransformer to select columns](https://scikit-learn.org/0.23/auto_examples/preprocessing/plot_function_transformer.html). – Daniel Himmelstein Feb 24 '21 at 00:23
Beware, you need to define `inverse_transform` – Kermit Jul 05 '22 at 16:59

Kermit · Answer 2 · 2022-04-04T21:36:11.670

Custom Function Use Case

Let's say you have image arrays with a known value range between 0-255 that you want to scale down between 0-1, but you don't want to use StandardScaler because not all images would have values of 0 and 255 in them. In simpler terms. No one scored a 100% on the test, but you still want to scale between 0-100.

from sklearn.preprocessing import FunctionTransformer
import numpy as np


data = np.array([[100, 2], [240, 80], [139, 10], [10, 150]])


def div255(X): return X/255 #encode
def mult255(X): return X*255 #decode
scaler = FunctionTransformer(div255, inverse_func=mult255)


# --- encode ---
mutated = scaler.fit_transform(data)
"""
array([[0.39215686, 0.00784314],
       [0.94117647, 0.31372549],
       [0.54509804, 0.03921569],
       [0.03921569, 0.58823529]])
"""

# --- decode ---
scaler.inverse_transform(mutated)
"""
array([[100.,   2.],
       [240.,  80.],
       [139.,  10.],
       [ 10., 150.]])
"""

Pro Tip

Make sure you define these custom functions in a place where they can be referenced by the rest of your program (e.g. helper functions). Especially for when it comes time to inverse_transform your predictions and/or encode new samples!

score 2 · Answer 3 · edited May 23 '17 at 12:24

2

Here is a nice example. It really is what it says: given X input, it applies your function to X and returns the result. The most important part of it is its statelessness. Here and here you can find what statelessness is and here you can read a discussion about its advantages.

edited May 23 '17 at 12:24

Community

1
1

answered Jul 20 '16 at 18:53

lte__

7,175
25
74
131

Please add examples to your answers. The link is now broken. – Jakob Nov 07 '22 at 16:25
I wrote this 5 years ago. I have no idea what was under that link. – lte__ Nov 08 '22 at 13:09

score -1 · Answer 4 · answered Jul 21 '16 at 12:23

X =  [[5,6,7],
      [8,9,10],
      [1,2,3]]    

def exampleFunctionTransformer(X):
        return  X**2

    def exampleofFunctionTransfor():
        fx = FunctionTransformer(exampleFunctionTransformer)
        Y1= fx.transform(X)
        print(Y1)
        return Y1

    Z = exampleofFunctionTransfor()

    print(Z)


o/p 

    [[ 25  36  49]
     [ 64  81 100]
     [  1   4   9]]

refer this : http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.FunctionTransformer.html#sklearn.preprocessing.FunctionTransformer

What is scikit-learn FunctionTransformer used for?

4 Answers4

Custom Function Use Case

Pro Tip