Python 请提供单个阵列或阵列列表作为模型输入
我使用countvector获取注释中每个单词的向量,并将其用作神经网络的输入数据。然而,它总是有问题。代码和错误如下所示:Python 请提供单个阵列或阵列列表作为模型输入,python,arrays,tensorflow,machine-learning,keras,Python,Arrays,Tensorflow,Machine Learning,Keras,我使用countvector获取注释中每个单词的向量,并将其用作神经网络的输入数据。然而,它总是有问题。代码和错误如下所示: train_X = vectorizer.transform(train_dataframe['comment']) valid_X = vectorizer.transform(valid_dataframe['comment']) test_X = vectorizer.transform(test_dataframe['comment']) print (train
train_X = vectorizer.transform(train_dataframe['comment'])
valid_X = vectorizer.transform(valid_dataframe['comment'])
test_X = vectorizer.transform(test_dataframe['comment'])
print (train_X.shape)
print (valid_X.shape)
print (test_X.shape)
train_Y = train_dataframe['label'].to_numpy()
valid_Y = valid_dataframe['label'].to_numpy()
train_inputs=train_X
train_targets=train_Y
validation_inputs=valid_X
validation_targets=valid_Y
# Set the input and output sizes
input_size = 31124
output_size = 1
# Use same hidden layer size for both hidden layers. Not a necessity.
hidden_layer_size = 50
# define how the model will look like
model = tf.keras.Sequential([
# tf.keras.layers.Dense is basically implementing: output = activation(dot(input, weight) + bias)
# it takes several arguments, but the most important ones for us are the hidden_layer_size and the activation function
tf.keras.layers.Dense(hidden_layer_size, activation='relu'), # 1st hidden layer
tf.keras.layers.Dense(hidden_layer_size, activation='relu'), # 2nd hidden layer
# the final layer is no different, we just make sure to activate it with softmax
tf.keras.layers.Dense(output_size, activation='sigmoid') # output layer
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
### Training
# That's where we train the model we have built.
# set the batch size
batch_size = 100
# set a maximum number of training epochs
max_epochs = 100
# fit the model
# note that this time the train, validation and test data are not iterable
model.fit(train_inputs, # train inputs
train_targets, # train targets
batch_size=batch_size, # batch size
epochs=max_epochs, # epochs that we will train for (assuming early stopping doesn't kick in)
validation_data=(validation_inputs, validation_targets), # validation data
verbose = 2 # making sure we get enough information about the training process
)
test_loss, test_accuracy = model.evaluate(test_inputs, test_targets)
print('\nTest loss: {0:.2f}. Test accuracy: {1:.2f}%'.format(test_loss, test_accuracy*100.))
错误是:
请提供单个数组或数组列表作为模型输入。你通过了:x=(011404)1
(0, 4453) 2
(0, 6653) 1
(0, 8151) 1
(0, 11070) 1
(0, 14557) 1
(1, 817) 1
(1, 1134) 1
(1, 1813) 1
(1, 1827) 1
(1, 2151) 1
(1, 4505) 1
(1, 4647) 1
(1, 8244) 2
(1, 8296) 1
(1, 8332) 1
(1, 9109) 1
(1, 9611) 1
(1, 10080) 1
(1, 10791) 1
(1, 11821) 1
(1, 12714) 1
(1, 12760) 1
(1, 13665) 1
(1, 14349) 1
: :
(42423, 16238) 1
(42423, 17253) 1
(42423, 18627) 1
(42423, 19322) 1
(42423, 19811) 1
(42423, 21232) 1
(42423, 23128) 1
(42423, 25572) 1
(42423, 25681) 1
(42423, 27132) 1
(42423, 27568) 2
(42423, 27580) 1
(42423, 27933) 1
(42423, 30921) 2
(42424, 932) 1
(42424, 4078) 1
(42424, 10791) 1
(42424, 10835) 1
(42424, 27628) 1
(42424, 27933) 1
(42424, 30220) 1
(42425, 1813) 1
(42425, 13868) 1
(42425, 27580) 1
(42425, 28749) 1
train\u inputs
是一个类型为scipy.sparse.csr.csr\u matrix
的稀疏矩阵,是调用sklearn.feature\u extraction.text.countvectorier.transform
的结果,如本文所述:您可以尝试将稀疏矩阵转换为密集矩阵,并将其用作培训的输入:
model.fit(train_inputs.toarray().astype(float), ...)
不过,这种方法可能会导致大型数据集出现内存问题。如果您需要更复杂的方法,您可以在此处找到有关如何正确使用KERA处理稀疏矩阵的更多信息: