Python 如何确保培训数据中至少包含n个课程中的2个

我目前正在训练一名CNN记者。我使用的指标之一是AUC。我注意到的一个问题是，有时候我的生成器只会从一个类中选择示例（我在这个项目中有3个类）。所以，如果我的批量大小是20，它有时会从第一类中随机选择20个示例，用于1个历元。如果发生这种情况，我会得到一个错误，指出AUC不能仅用一个类计算，然后培训结束

有没有一种方法可以在生成器中设置一个条件，使您或多或少地声明至少需要n个类中的2个？无需使用tf.metrics.auc

多谢各位

# load training data
def load_train_data_batch_generator(batch_size=32, rows_in=48, cols_in=48, zs_in=32, 
                                    channels_in=2, num_classes=3, 
                                    dir_dict=dir_dict):

    # dir_in_train = main_dir + '/test_CT_PET_combo'

    # required when using hyperopt
    batch_size = int(batch_size)
    # if not: TypeError: 'float' object cannot be interpreted as an integer

    fnames = os.listdir(dir_dict['dir_in_train_combo'])

    y_train = np.zeros((batch_size, num_classes))
    x_train = np.zeros((batch_size, rows_in, cols_in, zs_in, channels_in))

    while True:
        count = 0
        for fname in np.random.choice(fnames, batch_size, replace=False):

            data_label = scipy.io.loadmat(os.path.join(dir_dict['dir_out_train'], fname))['output']

            # changing one hot encoding to integer
            integer_label = np.argmax(data_label[0], axis=0)
            y_train[count,:] = data_label

            # Loading train ct w/ c and pet/ct combo 
            train_combo = scipy.io.loadmat(os.path.join(dir_dict['dir_in_train_combo'], fname))[fname]
            x_train[count,:,:,:,:] = train_combo

            count += 1

        yield(x_train, y_train)

每个请求：度量和错误代码 公制代码

def sk_auroc(y_true, y_pred):
    import tensorflow as tf
    from sklearn.metrics import roc_auc_score
    return tf.py_func(roc_auc_score, (y_true, y_pred), tf.double)


Epoch 1/200
 57/205 [=======>......................] - ETA: 11s - loss: 1.2858 - acc: 0.3632 - sk_auroc: 0.4581 - auc: 0.5380ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.
Traceback (most recent call last):

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 158, in __call__
    ret = func(*args)

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 277, in roc_auc_score
    sample_weight=sample_weight)

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/sklearn/metrics/base.py", line 118, in _average_binary_score
    sample_weight=score_weight)

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 268, in _binary_roc_auc_score
    raise ValueError("Only one class present in y_true. ROC AUC score "

ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.


     [[Node: metrics_1/sk_auroc/PyFunc = PyFunc[Tin=[DT_FLOAT, DT_FLOAT], Tout=[DT_DOUBLE], token="pyfunc_24", _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_predictions_target_1_0_1, predictions_1/Softmax/_857)]]
Traceback (most recent call last):
  File "<ipython-input-48-34101247f335>", line 8, in optimize_cnn
    model, results = train_model(space)
  File "<ipython-input-47-254bd056a344>", line 40, in train_model
    validation_steps=round(len(os.listdir(dir_out_val))/space['batch_size'])
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/keras/engine/training_generator.py", line 217, in fit_generator
    class_weight=class_weight)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1217, in train_on_batch
    outputs = self.train_function(ins)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2715, in __call__
    return self._call(inputs)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2675, in _call
    fetched = self._callable_fn(*array_vals)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1454, in __call__
    self._session._session, self._handle, args, status, None)
  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.
Traceback (most recent call last):

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 158, in __call__
    ret = func(*args)

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 277, in roc_auc_score
    sample_weight=sample_weight)

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/sklearn/metrics/base.py", line 118, in _average_binary_score
    sample_weight=score_weight)

  File "/home/mikedoho/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 268, in _binary_roc_auc_score
    raise ValueError("Only one class present in y_true. ROC AUC score "

ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.


     [[Node: metrics_1/sk_auroc/PyFunc = PyFunc[Tin=[DT_FLOAT, DT_FLOAT], Tout=[DT_DOUBLE], token="pyfunc_24", _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_predictions_target_1_0_1, predictions_1/Softmax/_857)]]

看起来tf.metrics.auc太平滑了，可能有什么问题需要我以后再研究

---

您可以在tensorflow中使用

tf.metrics.auc

，而不是在sklearn中使用

sklearn.metrics.roc\u auc\u score

。例如：

import tensorflow as tf
label = tf.Variable([1,0,0,0,1])
pred = tf.Variable([0.8,1,0.6,0.23,0.78])
auc,op = tf.metrics.auc(label,pred)

with tf.Session()as sess:
    init = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
    sess.run(init)
    for i in range(3):
        auc_value, op_value = sess.run([auc,op])
        print(auc_value)
0.0
0.6666667
0.66666657

---

您不会有任何问题。

为什么您会为一批有多少个类而困扰？如果这是一个样本平衡问题，也许你需要使用样本取样和其他方法。哇，我完全忘记了这样一句话：如果批次只有1个类，我会出错，因为它无法计算AUC。但你是对的，虽然这更多的是一个样本平衡问题，这将与更多数据进行斗争，但目前我正在解决上述问题。你能给出你的度量auc代码和错误吗？添加到上面。再次感谢您的帮助谢谢您的帮助，但我已经知道tf.metrics.auc，并与sklearn一起运行。不幸的是，当我看数据时，它提供的东西似乎不正确。我将包括在我原来的帖子另一张照片。我再次希望解决我的问题是关于数据生成器的问题。再次感谢你的帮助

import tensorflow as tf
label = tf.Variable([1,0,0,0,1])
pred = tf.Variable([0.8,1,0.6,0.23,0.78])
auc,op = tf.metrics.auc(label,pred)

with tf.Session()as sess:
    init = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
    sess.run(init)
    for i in range(3):
        auc_value, op_value = sess.run([auc,op])
        print(auc_value)
0.0
0.6666667
0.66666657