Python Scikit学习变压器管道产生的结果不同于单独运行

当我尝试使用管道组合两个变压器时，第二个变压器（log）似乎未应用

我试图简化日志转换器以执行简单的加法，但同样的问题仍然存在

import pandas as pd
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.base import BaseEstimator, TransformerMixin

class Impute(BaseEstimator, TransformerMixin):
    def __init__(self, columns=None, value='mean'):
        """
        columns: A list of columns to apply the imputation to.
        value: 
            - "mean": Fills in missing values with mean of training data
            - number: Fills in values with that number
            - dictionary: Fills in values where dictionary keys are column names
        """
        self.columns = columns
        self.value = value

    def fit(self, X, y=None):
        if self.columns is None:
            self.columns = X.columns
        if isinstance(self.value, str):
            if self.value == "mean":
                self.value = X[self.columns].mean()
            elif self.value == 'median':
                self.value = X[self.columns].median()
        return self

    def transform(self, X):
        X[self.columns] = X[self.columns].fillna(self.value)
        return X

class Log(BaseEstimator, TransformerMixin):
    def __init__(self, columns=None, offset_value=0):
        """
        offset_value: a value to specify to handle invalid outputs such as log(0) or log(negative values)
        """
        self.columns = columns
        self.offset_value = offset_value

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        X_new = X.copy()
        X_new[self.columns] = np.log(X_new[self.columns] + self.offset_value)
        return X_new

###########################
temp = pd.DataFrame([[590,3,None, "2018-01-01"],[0,2,3, "2018-01-01"],
                     [590,2,4, "2019-01-01"], [None ,None,4, "2018-01-01"], 
                     [850 ,None,4, "2018-01-01"]], columns=["credit_score", "n_cats", "premium", "fix_date"])

print(temp)

impute = Impute(columns=["credit_score", "n_cats", "premium"], value="mean")
impute.fit(temp)
temp = impute.transform(temp)

log = Log(columns=["credit_score", "n_cats", "premium"], offset_value=1)
log.fit(temp)
temp = log.transform(temp)
temp


###########################
temp = pd.DataFrame([[590,3,None, "2018-01-01"],[0,2,3, "2018-01-01"],
                     [590,2,4, "2019-01-01"], [None ,None,4, "2018-01-01"], 
                     [850 ,None,4, "2018-01-01"]], columns=["credit_score", "n_cats", "premium", "fix_date"])

print(temp)

impute = Impute(columns=["credit_score", "n_cats", "premium"], value="mean")
log = Log(columns=["credit_score", "n_cats", "premium"], offset_value=1)

steps = [("impute", impute),
         ("log", log)
        ]

pipe = Pipeline(steps)

pipe.fit(temp)
pipe.transform(temp)
temp

单独使用变压器时，显示：

    credit_score    n_cats  premium fix_date
0   6.381816    1.386294    1.558145    2018-01-01
1   0.000000    1.098612    1.386294    2018-01-01
2   6.381816    1.098612    1.609438    2019-01-01
3   6.231465    1.203973    1.609438    2018-01-01
4   6.746412    1.203973    1.609438    2018-01-01

当我尝试使用管道时，它显示

    credit_score    n_cats  premium fix_date
0   590.0   3.000000    3.75    2018-01-01
1   0.0 2.000000    3.00    2018-01-01
2   590.0   2.000000    4.00    2019-01-01
3   507.5   2.333333    4.00    2018-01-01
4   850.0   2.333333    4.00    2018-01-01

---

问题在于

transform

方法在

Impute

和

Log

类中的实现不同。在

Impute

中，您就地修改

X

（无复制），然后返回它。但是，在

日志中

首先复制

X

，对该副本应用修改，然后返回副本

快速修复方法是查看返回值以获得正确答案：

pipe = Pipeline(steps)

pipe.fit(temp)
new_df = pipe.transform(temp)

---

一般来说，更好的做法是根本不修改原始的

数据帧

X

，而只将修改应用于其副本。这样，

transform

方法总是返回一个全新的

DataFrame

，而原始的

DataFrame

保持不变。

需要其他帮助吗？