Python Keras模型只能预测一个类 总结

我有一个二进制分类任务。毕竟，准确率似乎是81%，但该模型只预测了一个类

问题：我知道模特学的东西不多；但是为什么它只预测一个类呢？我不确定在数据生成过程中是否出错。有什么不对劲吗

信息：

• 我有一个大数据（~2GB）。因此，我使用Keras的

  fit_generator

  方法训练了一个模型；它不适合内存，将来数据可能会变得更大

• 我正在生成器函数中执行预处理和重采样步骤

• 数据不平衡。我尝试进行上采样（参见下面的代码）

• 在训练过程中，准确度没有增加；损失几乎没有减少

• 这是我在测试集上运行

  classification\u report

  时得到的结果（我对测试集执行完全相同的预处理步骤）：

代码：
数据生成器方法：

def data_generotto(path: str, batchsize: int):
    while True:
        chunks = pd.read_csv(os.path.join(
            path, "shuffled.csv"), sep=';', chunksize=batchsize)

        for i, chunk in enumerate(chunks):
            X, y = preprocess.preprocess(chunk)

            # X = np.array(X).reshape(X.shape[0], 1, X.shape[1])

            yield (X, y)

# sorry for messy code
def balance_train_data(data, fold_count=3):
    """Balance the data using sklearn.utils resample to max sentiment count."""
    balanced_data = pd.DataFrame()
    data_dict = dict(data['label'].value_counts())

    for label in data_dict.keys():
        df = data[data.label == label]
        samples_count = int(
            (max(data_dict.values()) - data_dict[label])/fold_count)
        df_up = resample(df, replace=True,
                         n_samples=samples_count, random_state=42)

        print("Resampled {} tweets: {} + {} = {}".format(label,
                                                         len(df), len(df_up), len(df)+len(df_up)))
        balanced_data = pd.concat([balanced_data, df, df_up])
    return shuffle(balanced_data, random_state=42)

    model.fit_generator(data_generotto(
        "./complete_csv", BS), steps_per_epoch=TRAIN_SIZE // BS, epochs=EPOCHS, callbacks=[es])

    save_model(model, f"./models/{model_folder_name}/MODEL.h5")

重采样（在其他预处理步骤中）：

def data_generotto(path: str, batchsize: int):
    while True:
        chunks = pd.read_csv(os.path.join(
            path, "shuffled.csv"), sep=';', chunksize=batchsize)

        for i, chunk in enumerate(chunks):
            X, y = preprocess.preprocess(chunk)

            # X = np.array(X).reshape(X.shape[0], 1, X.shape[1])

            yield (X, y)

# sorry for messy code
def balance_train_data(data, fold_count=3):
    """Balance the data using sklearn.utils resample to max sentiment count."""
    balanced_data = pd.DataFrame()
    data_dict = dict(data['label'].value_counts())

    for label in data_dict.keys():
        df = data[data.label == label]
        samples_count = int(
            (max(data_dict.values()) - data_dict[label])/fold_count)
        df_up = resample(df, replace=True,
                         n_samples=samples_count, random_state=42)

        print("Resampled {} tweets: {} + {} = {}".format(label,
                                                         len(df), len(df_up), len(df)+len(df_up)))
        balanced_data = pd.concat([balanced_data, df, df_up])
    return shuffle(balanced_data, random_state=42)

    model.fit_generator(data_generotto(
        "./complete_csv", BS), steps_per_epoch=TRAIN_SIZE // BS, epochs=EPOCHS, callbacks=[es])

    save_model(model, f"./models/{model_folder_name}/MODEL.h5")

一个非常简单的ANN模型：

def create_model(model_folder_name):
    global folder_name
    folder_name = model_folder_name
    model = Sequential()

    model.add(Dense(8, activation='relu', input_dim=4))
    model.add(Dense(4, kernel_initializer='uniform', activation='relu'))
    model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))

    optimizer = optimizers.Adam(lr=0.0001)

    model.compile(optimizer=optimizer, loss="binary_crossentropy",
                  metrics=['accuracy'])
    model.summary(print_fn=myprint)

    return model, optimizer.get_config(), "ann"

安装发电机

功能：

def data_generotto(path: str, batchsize: int):
    while True:
        chunks = pd.read_csv(os.path.join(
            path, "shuffled.csv"), sep=';', chunksize=batchsize)

        for i, chunk in enumerate(chunks):
            X, y = preprocess.preprocess(chunk)

            # X = np.array(X).reshape(X.shape[0], 1, X.shape[1])

            yield (X, y)

# sorry for messy code
def balance_train_data(data, fold_count=3):
    """Balance the data using sklearn.utils resample to max sentiment count."""
    balanced_data = pd.DataFrame()
    data_dict = dict(data['label'].value_counts())

    for label in data_dict.keys():
        df = data[data.label == label]
        samples_count = int(
            (max(data_dict.values()) - data_dict[label])/fold_count)
        df_up = resample(df, replace=True,
                         n_samples=samples_count, random_state=42)

        print("Resampled {} tweets: {} + {} = {}".format(label,
                                                         len(df), len(df_up), len(df)+len(df_up)))
        balanced_data = pd.concat([balanced_data, df, df_up])
    return shuffle(balanced_data, random_state=42)

    model.fit_generator(data_generotto(
        "./complete_csv", BS), steps_per_epoch=TRAIN_SIZE // BS, epochs=EPOCHS, callbacks=[es])

    save_model(model, f"./models/{model_folder_name}/MODEL.h5")

---

您可以尝试使用

imblearn

并使用不同的采样方法

---

此外，为了处理不平衡问题，您可以尝试在拟合模型时使用具有适当权重的

class_weight

。

类不平衡是仅预测一个类的原因。这不是一个真正的编程问题。你不认为重新采样应该解决这个问题吗？你建议降采样吗？你可以尝试重采样，更好的方法可能是在keras

fit

@SreeramTP中使用

class\u weight

，我已经应用了重采样，就像我上面写的代码一样。你认为这有什么错误吗？我还将尝试使用

class\u weight

并通知您，谢谢您可能可以尝试

imblearn

不同的采样方法，看看哪种方法有效