Python 3.x 如何获得逻辑回归的正确答案？

我在二进制分类问题上没有得到期望的结果

问题在于使用二元分类将乳腺癌标记为： -良性，或 -恶性的

它没有给出期望的输出

首先，有一个函数加载数据集，该数据集返回形状的测试和训练数据：

x_train is of shape: (30, 381),
y_train is of shape: (1, 381),
x_test is of shape:  (30, 188),
y_test is of shape:  (1, 188).

然后是一个逻辑回归分类器类，用于预测输出

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

def load_dataset():
    cancer_data = load_breast_cancer()
    x_train, x_test, y_train, y_test = train_test_split(cancer_data.data, cancer_data.target, test_size=0.33)
    x_train = x_train.T
    x_test = x_test.T
    y_train = y_train.reshape(1, (len(y_train)))
    y_test = y_test.reshape(1, (len(y_test)))
    m = x_train.shape[1]
    return x_train, x_test, y_train, y_test, m

class Neural_Network():
    def __init__(self):
        np.random.seed(1)
        self.weights = np.random.rand(30, 1) * 0.01
        self.bias = np.zeros(shape=(1, 1))

    def sigmoid(self, x):
        return 1 / (1 + np.exp(-x))

    def train(self, x_train, y_train, iterations, m, learning_rate=0.5):

        for i in range(iterations):
            z = np.dot(self.weights.T, x_train) + self.bias
            a = self.sigmoid(z)

            cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))

            if (i % 500 == 0):
                print("Cost after iteration %i: %f" % (i, cost))

            dw = (1 / m) * np.dot(x_train, (a - y_train).T)
            db = (1 / m) * np.sum(a - y_train)

            self.weights = self.weights - learning_rate * dw
            self.bias = self.bias - learning_rate * db

    def predict(self, inputs):
        m = inputs.shape[1]
        y_predicted = np.zeros((1, m))
        z = np.dot(self.weights.T, inputs) + self.bias
        a = self.sigmoid(z)
        for i in range(a.shape[1]):
            y_predicted[0, i] = 1 if a[0, i] > 0.5 else 0
        return y_predicted

if __name__ == "__main__":
    '''
    step-1 : Loading data set
                 x_train is of shape: (30, 381)
                 y_train is of shape: (1, 381)
                 x_test is of shape:  (30, 188)
                 y_test is of shape:  (1, 188)
    '''

    x_train, x_test, y_train, y_test, m = load_dataset()

    neuralNet = Neural_Network()

    '''
       step-2 : Train the network
    '''

    neuralNet.train(x_train, y_train,10000,m)


    y_predicted = neuralNet.predict(x_test)

    print("Accuracy on test data: ")
    print(accuracy_score(y_test, y_predicted)*100)

给出此输出的程序：

    C:\Python36\python.exe C:/Users/LENOVO/PycharmProjects/MarkDmo001/Numpy.py
Cost after iteration 0: 5.263853
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:25: RuntimeWarning: overflow encountered in exp
  return 1 / (1 + np.exp(-x))
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:33: RuntimeWarning: divide by zero encountered in log
  cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:33: RuntimeWarning: invalid value encountered in multiply
  cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
Cost after iteration 500: nan
Cost after iteration 1000: nan
Cost after iteration 1500: nan
Cost after iteration 2000: nan
Cost after iteration 2500: nan
Cost after iteration 3000: nan
Cost after iteration 3500: nan
Cost after iteration 4000: nan
Cost after iteration 4500: nan
Cost after iteration 5000: nan
Cost after iteration 5500: nan
Cost after iteration 6000: nan
Cost after iteration 6500: nan
Cost after iteration 7000: nan
Cost after iteration 7500: nan
Cost after iteration 8000: nan
Cost after iteration 8500: nan
Cost after iteration 9000: nan
Cost after iteration 9500: nan

Accuracy: 
0.0

---

问题是梯度爆炸。您需要将输入标准化为

[0，1]

如果查看训练数据中的功能3和功能23，您将看到大于3000的值。在这些值与初始权重相乘后，它们仍然位于范围

[0,30]

内。因此，在第一次迭代中，

z

向量只包含值高达50左右的正数。因此，

a

向量（乙状结肠的输出）如下所示：

[0.9994797 0.99853904 0.99358676 0.99999973 0.98392862 0.99983016 0.99818802 ...]

所以在第一步中，你的模型总是以很高的置信度预测1。但这并不总是正确的，而且模型输出导致大梯度的概率很高，当您查看

dw

的最高值时可以看到。就我而言

• dw[3]

  388

• dw[23]

  was 571

其他值位于

[0,55]

中。因此，您可以清楚地看到这些特性中的大量输入是如何导致梯度爆炸的。因为梯度下降现在会朝相反的方向迈出一大步，所以下一步的权重不是

[0,0.01]

，而是

[-285,0.002]

，这只会让事情变得更糟。在下一次迭代中，

z

包含的值约为-100万，这会导致sigmoid函数溢出

解决方案

将您的输入标准化为

[0，1]

在

[-0.01，0.01]

中使用权重，以便它们大致相互抵消。否则，您在

z

中的值仍然与您拥有的特征数成线性比例

对于输入的规范化，您可以使用sklearn：

---

.T

s是因为sklearn希望训练输入具有形状

（num_样本，num_特征）

，而你的

x_训练和
x_测试具有形状
（num_特征，num_样本）
，我按照你说的做了，但我没有得到输出，请告诉我如何应用解决方案2，我确实尝试了不同的权重，但仍然得到了相同的输出。我复制了上面的代码，只是用我答案中的代码块替换了第60到71行。然后，我得到了0.06的损失和97.8%的准确率。顺便说一下，为了评估准确性，您需要调用
print（准确性评分（y\u测试[0]，y\u预测[0]）*100）
，因为
y\u测试
和
y\u预测
是二维数组。
x_train, x_test, y_train, y_test, m = load_dataset()

scaler = MinMaxScaler()
x_train_normalized = scaler.fit_transform(x_train.T).T

neuralNet = Neural_Network()

'''
   step-2 : Train the network
'''

neuralNet.train(x_train_normalized, y_train,10000,m)

# Use the same transformation on the test inputs as on the training inputs
x_test_normalized = scaler.transform(x_test.T).T
y_predicted = neuralNet.predict(x_test_normalized)