Machine learning Keras文本分类器错误:层顺序_1的输入0与层不兼容

Machine learning Keras文本分类器错误:层顺序_1的输入0与层不兼容,machine-learning,keras,text-classification,sequential,Machine Learning,Keras,Text Classification,Sequential,我对Keras有些陌生,所以对于这个可能是新手的问题,我深表歉意,但网上似乎没有任何东西能为我指明正确的方向 我正在建立一个简单的模型,将文本分类为7个情绪箱中的一个。一切似乎都进行得相当顺利,对我来说很有意义,但我一定是在某个时候把这件事弄错了,因为它最终失败了。我认为它在最后吐出的数组可能会映射到mood的值标签,但不管我使用什么作为示例文本,它都会吐出相同的分数数组 有人有什么想法或者可以在路上帮我吗? 错误如下: ValueError: Input 0 of layer sequ

我对Keras有些陌生,所以对于这个可能是新手的问题,我深表歉意,但网上似乎没有任何东西能为我指明正确的方向

我正在建立一个简单的模型,将文本分类为7个情绪箱中的一个。一切似乎都进行得相当顺利,对我来说很有意义,但我一定是在某个时候把这件事弄错了,因为它最终失败了。我认为它在最后吐出的数组可能会映射到mood的值标签,但不管我使用什么作为示例文本,它都会吐出相同的分数数组

有人有什么想法或者可以在路上帮我吗? 错误如下:

    ValueError: Input 0 of layer sequential_1 is incompatible with the layer: expected axis -1 of input shape to have value 6835 but received input with shape [None, 57]
import numpy as np
import pandas as pd
import re
import pickle
import os

import keras
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential, load_model
from keras.layers import Dense, Embedding, LSTM, Bidirectional, Flatten, Dropout, Activation
from keras.callbacks import EarlyStopping as ES
from keras.utils.np_utils import to_categorical

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder


# Read in data
emo = pd.read_csv('dev_sent_emo.csv', dtype = 'str')

# Split train and test data
X_train, X_test, Y_train, Y_test = train_test_split(emo.Utterance, emo.Emotion, test_size=0.3, random_state = 0)

# tokenize and build model framework
totallength = 0
for i in range(0, len(emo)):
    length = len(emo.at[i, 'Emotion'])
    totallength = totallength+length

print(totallength)

# Tokenize
max_words = totallength
tokenizer = Tokenizer(num_words=max_words)
num_classes = emo.Emotion.nunique()

# Convert X text to array
X_train = tokenizer.texts_to_sequences(X_train)
X_train = tokenizer.sequences_to_matrix(X_train, mode='binary')
X_test = tokenizer.texts_to_sequences(X_test)
X_test = tokenizer.sequences_to_matrix(X_test, mode='binary')

# Label outcomes
le = LabelEncoder()
le.fit(Y_train)
Y_train = le.transform(Y_train)
le.fit(Y_test)
Y_test = le.transform(Y_test)
#list(le.inverse_transform(Y_test))

# Y to array matrix
Y_train = keras.utils.to_categorical(Y_train, num_classes)
Y_test = keras.utils.to_categorical(Y_test, num_classes)

#Build model
model = Sequential()
model.add(Dense(300, input_shape=(1, max_words)))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
batch_size = 32
epochs = 20

history = model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_split=0.1)

# OOB scoring
score = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1)
print('Test accuracy:', score[1])

# Test the model in-house
sample_string = "I am so angry at this, I hate it so much - how dare you!?"
sample_x = tokenizer.texts_to_sequences([sample_string])
sample_x = pad_sequences(sample_x, maxlen=len(sample_string))
prediction = model.predict(sample_x)
至于代码块,如下所示:

    ValueError: Input 0 of layer sequential_1 is incompatible with the layer: expected axis -1 of input shape to have value 6835 but received input with shape [None, 57]
import numpy as np
import pandas as pd
import re
import pickle
import os

import keras
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential, load_model
from keras.layers import Dense, Embedding, LSTM, Bidirectional, Flatten, Dropout, Activation
from keras.callbacks import EarlyStopping as ES
from keras.utils.np_utils import to_categorical

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder


# Read in data
emo = pd.read_csv('dev_sent_emo.csv', dtype = 'str')

# Split train and test data
X_train, X_test, Y_train, Y_test = train_test_split(emo.Utterance, emo.Emotion, test_size=0.3, random_state = 0)

# tokenize and build model framework
totallength = 0
for i in range(0, len(emo)):
    length = len(emo.at[i, 'Emotion'])
    totallength = totallength+length

print(totallength)

# Tokenize
max_words = totallength
tokenizer = Tokenizer(num_words=max_words)
num_classes = emo.Emotion.nunique()

# Convert X text to array
X_train = tokenizer.texts_to_sequences(X_train)
X_train = tokenizer.sequences_to_matrix(X_train, mode='binary')
X_test = tokenizer.texts_to_sequences(X_test)
X_test = tokenizer.sequences_to_matrix(X_test, mode='binary')

# Label outcomes
le = LabelEncoder()
le.fit(Y_train)
Y_train = le.transform(Y_train)
le.fit(Y_test)
Y_test = le.transform(Y_test)
#list(le.inverse_transform(Y_test))

# Y to array matrix
Y_train = keras.utils.to_categorical(Y_train, num_classes)
Y_test = keras.utils.to_categorical(Y_test, num_classes)

#Build model
model = Sequential()
model.add(Dense(300, input_shape=(1, max_words)))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
batch_size = 32
epochs = 20

history = model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_split=0.1)

# OOB scoring
score = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1)
print('Test accuracy:', score[1])

# Test the model in-house
sample_string = "I am so angry at this, I hate it so much - how dare you!?"
sample_x = tokenizer.texts_to_sequences([sample_string])
sample_x = pad_sequences(sample_x, maxlen=len(sample_string))
prediction = model.predict(sample_x)

非常感谢你们的时间和建议

您的模型中似乎存在尺寸/形状问题。尝试打印模型的summary(),这将为您提供所有层的大小和形状。你能将结果粘贴到你的问题中吗?还要检查你的输入测试数据数组是否与火车数据具有相同的形状。我认为这确实是形状的问题-当我更仔细地检查我所做的事情时,似乎运行标记器的文本\u到\u序列,然后再运行序列\u到\u矩阵是一个错误。它不会将字符串数组转换为可在Keras ML模型中使用的数组,因为它会输出一个只有零的数组。回过头来看,我发现文本序列会产生一个空列表。我在网上找到的文档在现阶段并不是非常有用。事实上,这是你需要自己习惯的东西,也许在尝试编码之前,用最小尺寸矩阵将模型写在纸上,将有助于您了解如何为这个特定的应用程序构建合适的神经网络体系结构。您的模型中似乎存在维度/形状问题。尝试打印模型的summary(),这将为您提供所有层的大小和形状。你能将结果粘贴到你的问题中吗?还要检查你的输入测试数据数组是否与火车数据具有相同的形状。我认为这确实是形状的问题-当我更仔细地检查我所做的事情时,似乎运行标记器的文本\u到\u序列,然后再运行序列\u到\u矩阵是一个错误。它不会将字符串数组转换为可在Keras ML模型中使用的数组,因为它会输出一个只有零的数组。回过头来看,我发现文本序列会产生一个空列表。我在网上找到的文档在这一阶段并不是非常有用。事实上,这是你需要自己习惯的东西,也许在尝试编码之前,用最小尺寸矩阵将模型写在纸上,这将帮助你理解如何为这个特定的应用构建一个合适的神经网络体系结构