Python可以';t将fit_生成器应用于多输入keras模型
我有以下模型-这是LSTM+CNN,有3个输入。 我使用fit_generator构建了这个生成器函数来训练模型(基于此:): 我得到了一个错误:Python可以';t将fit_生成器应用于多输入keras模型,python,keras,deep-learning,neural-network,lstm,Python,Keras,Deep Learning,Neural Network,Lstm,我有以下模型-这是LSTM+CNN,有3个输入。 我使用fit_generator构建了这个生成器函数来训练模型(基于此:): 我得到了一个错误: --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-38-66
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-38-669153f703e6> in <module>()
net.fit_generator(generator=training_generator,
---> validation_data=validation_generator,)
#use_multiprocessing=True)#, workers=6)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
96 dtype = dtypes.as_dtype(dtype).as_datatype_enum
97 ctx.ensure_initialized()
---> 98 return ops.EagerTensor(value, ctx.device_name, dtype)
99
100
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray).
或者X[i,]=[X_id]
而不是X[i,]=X_id
但都不管用
你知道怎么解决这个问题吗
已编辑:添加时:
astype(np.float32)
和tf.转换为张量(X)
我得到一个错误:
ValueError回溯(最近一次调用上次)
在()
在解决问题之前,让我们首先总结一下您正在使用的数据集。根据您的描述,我创建了一个类似您的示例
DataFrame
import pandas as pd
dataset_size = 500
train_idx,val_idx = train_test_split(range(dataset_size),test_size=0.2,)
# create an example DataFrame that I assume will be resemble yours
example_df = pd.DataFrame({'vids':np.random.randint(0,10000,dataset_size)})
# create feature columns
for ind in range(14): example_df['feature_%i' % ind] = np.random.rand(dataset_size)
# each cell contains a list
example_df['text'] = np.random.randint(dataset_size)
example_df['text'] = example_df['text'].astype('object')
for ind in range(dataset_size):example_df.at[ind,'text'] = np.random.rand(768).tolist()
# create the label column
example_df['label'] = np.random.randint(low=0,high=5,size=dataset_size)
# extract information from the dataframe, and create data generators
all_vids = example_df['vids'].values
feature_columns = ['feature_%i' % ind for ind in range(14)]
all_features = example_df[feature_columns].values
all_text = example_df['text'].values
all_labels = example_df['label'].values
如您所见,列text
是一列列表,其中每个列表包含768项。列labels
包含示例的标签,无论您使用一种热编码还是其他类型的编码,只要其形状与整个神经网络模型的输出层的形状匹配即可。列vids
是一列seed
s,用于动态生成随机图像
解决问题(基于上述数据集) 对于方法
\uuu getitem\uuu
,可以使用此语法返回{'feature':features,'text':text,'vid':vid},y
,而不是堆叠三个输入数组
为了解释这一点,让我们首先构建一个与您相似的玩具模型
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input,Dense,Flatten,Add
def features_part(x):
y = Dense(14)(x)
y = Dense(10,activation='linear')(y)
return y
def text_part(x):
y = Dense(768)(x)
y = Dense(10,activation='linear')(y)
return y
def vid_part(x):
y = Flatten()(x)
y = Dense(10,activation='linear')(y)
return y
input_features = Input(shape=(14,),name='feature')
input_text = Input(shape=(768,),name='text')
input_vid = Input(shape=(3,244,244,),name='vid')
feature_block = features_part(input_features)
text_block = text_part(input_text)
vid_block = vid_part(input_vid)
added = Add()([feature_block,text_block,vid_block])
# you have five classes at the end of the day
pred = Dense(1)(added)
# build model
model = Model(inputs=[input_features,input_text,input_vid],outputs=pred)
model.compile(loss='mae',optimizer='adam',metrics=['mae'])
这个模型最重要的一点是,我指定了三个输入层的名称
input_features = Input(shape=(14,),name='feature')
input_text = Input(shape=(768,),name='text')
input_vid = Input(shape=(3,244,244,),name='vid')
对于此模型,可以构造一个生成器,如
# provide a seed for generating a random image
def fn2img(seed):
np.random.seed(seed)
# fake an image with three channels
return np.random.randint(low=0,high=255,size=(3,244,244))
class MultiInputDataGenerator(keras.utils.Sequence):
def __init__(self,
all_inds,labels,
features,text,vid,
shuffle=True):
self.batch_size = 8
self.labels = labels
self.all_inds = all_inds
self.shuffle = shuffle
self.on_epoch_end()
self.features = features
self.text = text
self.vid = vid
def __len__(self):
return int(np.floor(len(self.all_inds) / self.batch_size))
def __getitem__(self,index):
indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]
batch_indices = [self.all_inds[k] for k in indexes]
features,text,vid,y = self.__data_generation(batch_indices)
return {'feature':features,'text':text,'vid':vid},y
def on_epoch_end(self):
self.indexes = np.arange(len(self.all_inds))
if self.shuffle == True:
np.random.shuffle(self.indexes)
def __data_generation(self,batch_indices):
# Generate data
features = self.features[batch_indices,:]
# note that you need to stack the slice in order to reshape it to (num_samples,768)
text = np.stack(self.text[batch_indices])
# since batch_size is not a super large number, you can stack here
vid = np.stack([fn2img(seed) for seed in self.vid[batch_indices]])
y = self.labels[batch_indices]
return features,text,vid,y
如您所见,\uuu getitem\uuu
方法返回一个字典{'feature':features,'text':text,'vid':vid},y
。字典的键与三个输入层的名称匹配。此外,随机图像是动态生成的
为了确保一切正常,您可以运行下面的脚本
import numpy as np
import pandas as pd
from tensorflow import keras
from sklearn.model_selection import train_test_split
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input,Dense,Flatten,Add
# provide a seed for generating a random image
def fn2img(seed):
np.random.seed(seed)
# fake an image with three channels
return np.random.randint(low=0,high=255,size=(3,244,244))
class MultiInputDataGenerator(keras.utils.Sequence):
def __init__(self,
all_inds,labels,
features,text,vid,
shuffle=True):
self.batch_size = 8
self.labels = labels
self.all_inds = all_inds
self.shuffle = shuffle
self.on_epoch_end()
self.features = features
self.text = text
self.vid = vid
def __len__(self):
return int(np.floor(len(self.all_inds) / self.batch_size))
def __getitem__(self,index):
indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]
batch_indices = [self.all_inds[k] for k in indexes]
features,text,vid,y = self.__data_generation(batch_indices)
return {'feature':features,'text':text,'vid':vid},y
def on_epoch_end(self):
self.indexes = np.arange(len(self.all_inds))
if self.shuffle == True:
np.random.shuffle(self.indexes)
def __data_generation(self,batch_indices):
# Generate data
features = self.features[batch_indices,:]
# note that you need to stack the slice in order to reshape it to (num_samples,768)
text = np.stack(self.text[batch_indices])
# since batch_size is not a super large number, you can stack here
vid = np.stack([fn2img(seed) for seed in self.vid[batch_indices]])
y = self.labels[batch_indices]
return features,text,vid,y
# fake a dataset
dataset_size = 500
train_idx,val_idx = train_test_split(range(dataset_size),test_size=0.2,)
# create an example DataFrame that I assume will be resemble yours
example_df = pd.DataFrame({'vids':np.random.randint(0,10000,dataset_size)})
# create feature columns
for ind in range(14): example_df['feature_%i' % ind] = np.random.rand(dataset_size)
# each cell contains a list
example_df['text'] = np.random.randint(dataset_size)
example_df['text'] = example_df['text'].astype('object')
for ind in range(dataset_size):example_df.at[ind,'text'] = np.random.rand(768).tolist()
# create the label column
example_df['label'] = np.random.randint(low=0,high=5,size=dataset_size)
# extract information from the dataframe, and create data generators
all_vids = example_df['vids'].values
feature_columns = ['feature_%i' % ind for ind in range(14)]
all_features = example_df[feature_columns].values
all_text = example_df['text'].values
all_labels = example_df['label'].values
training_generator = MultiInputDataGenerator(train_idx,all_labels,all_features,all_text,all_vids)
# create model
def features_part(x):
y = Dense(14)(x)
y = Dense(10,activation='linear')(y)
return y
def text_part(x):
y = Dense(768)(x)
y = Dense(10,activation='linear')(y)
return y
def vid_part(x):
y = Flatten()(x)
y = Dense(10,activation='linear')(y)
return y
input_features = Input(shape=(14,),name='feature')
input_text = Input(shape=(768,),name='text')
input_vid = Input(shape=(3,244,244,),name='vid')
feature_block = features_part(input_features)
text_block = text_part(input_text)
vid_block = vid_part(input_vid)
added = Add()([feature_block,text_block,vid_block])
# you have five classes at the end of the day
pred = Dense(1)(added)
# build model
model = Model(inputs=[input_features,input_text,input_vid],outputs=pred)
model.compile(loss='mae',optimizer='adam',metrics=['mae'])
model.fit_generator(generator=training_generator,epochs=10)
print(model.history.history)
在解决问题之前,让我们首先总结一下您正在使用的数据集。根据您的描述,我创建了一个类似您的示例
DataFrame
import pandas as pd
dataset_size = 500
train_idx,val_idx = train_test_split(range(dataset_size),test_size=0.2,)
# create an example DataFrame that I assume will be resemble yours
example_df = pd.DataFrame({'vids':np.random.randint(0,10000,dataset_size)})
# create feature columns
for ind in range(14): example_df['feature_%i' % ind] = np.random.rand(dataset_size)
# each cell contains a list
example_df['text'] = np.random.randint(dataset_size)
example_df['text'] = example_df['text'].astype('object')
for ind in range(dataset_size):example_df.at[ind,'text'] = np.random.rand(768).tolist()
# create the label column
example_df['label'] = np.random.randint(low=0,high=5,size=dataset_size)
# extract information from the dataframe, and create data generators
all_vids = example_df['vids'].values
feature_columns = ['feature_%i' % ind for ind in range(14)]
all_features = example_df[feature_columns].values
all_text = example_df['text'].values
all_labels = example_df['label'].values
如您所见,列text
是一列列表,其中每个列表包含768项。列labels
包含示例的标签,无论您使用一种热编码还是其他类型的编码,只要其形状与整个神经网络模型的输出层的形状匹配即可。列vids
是一列seed
s,用于动态生成随机图像
解决问题(基于上述数据集) 对于方法
\uuu getitem\uuu
,可以使用此语法返回{'feature':features,'text':text,'vid':vid},y
,而不是堆叠三个输入数组
为了解释这一点,让我们首先构建一个与您相似的玩具模型
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input,Dense,Flatten,Add
def features_part(x):
y = Dense(14)(x)
y = Dense(10,activation='linear')(y)
return y
def text_part(x):
y = Dense(768)(x)
y = Dense(10,activation='linear')(y)
return y
def vid_part(x):
y = Flatten()(x)
y = Dense(10,activation='linear')(y)
return y
input_features = Input(shape=(14,),name='feature')
input_text = Input(shape=(768,),name='text')
input_vid = Input(shape=(3,244,244,),name='vid')
feature_block = features_part(input_features)
text_block = text_part(input_text)
vid_block = vid_part(input_vid)
added = Add()([feature_block,text_block,vid_block])
# you have five classes at the end of the day
pred = Dense(1)(added)
# build model
model = Model(inputs=[input_features,input_text,input_vid],outputs=pred)
model.compile(loss='mae',optimizer='adam',metrics=['mae'])
这个模型最重要的一点是,我指定了三个输入层的名称
input_features = Input(shape=(14,),name='feature')
input_text = Input(shape=(768,),name='text')
input_vid = Input(shape=(3,244,244,),name='vid')
对于此模型,可以构造一个生成器,如
# provide a seed for generating a random image
def fn2img(seed):
np.random.seed(seed)
# fake an image with three channels
return np.random.randint(low=0,high=255,size=(3,244,244))
class MultiInputDataGenerator(keras.utils.Sequence):
def __init__(self,
all_inds,labels,
features,text,vid,
shuffle=True):
self.batch_size = 8
self.labels = labels
self.all_inds = all_inds
self.shuffle = shuffle
self.on_epoch_end()
self.features = features
self.text = text
self.vid = vid
def __len__(self):
return int(np.floor(len(self.all_inds) / self.batch_size))
def __getitem__(self,index):
indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]
batch_indices = [self.all_inds[k] for k in indexes]
features,text,vid,y = self.__data_generation(batch_indices)
return {'feature':features,'text':text,'vid':vid},y
def on_epoch_end(self):
self.indexes = np.arange(len(self.all_inds))
if self.shuffle == True:
np.random.shuffle(self.indexes)
def __data_generation(self,batch_indices):
# Generate data
features = self.features[batch_indices,:]
# note that you need to stack the slice in order to reshape it to (num_samples,768)
text = np.stack(self.text[batch_indices])
# since batch_size is not a super large number, you can stack here
vid = np.stack([fn2img(seed) for seed in self.vid[batch_indices]])
y = self.labels[batch_indices]
return features,text,vid,y
如您所见,\uuu getitem\uuu
方法返回一个字典{'feature':features,'text':text,'vid':vid},y
。字典的键与三个输入层的名称匹配。此外,随机图像是动态生成的
为了确保一切正常,您可以运行下面的脚本
import numpy as np
import pandas as pd
from tensorflow import keras
from sklearn.model_selection import train_test_split
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input,Dense,Flatten,Add
# provide a seed for generating a random image
def fn2img(seed):
np.random.seed(seed)
# fake an image with three channels
return np.random.randint(low=0,high=255,size=(3,244,244))
class MultiInputDataGenerator(keras.utils.Sequence):
def __init__(self,
all_inds,labels,
features,text,vid,
shuffle=True):
self.batch_size = 8
self.labels = labels
self.all_inds = all_inds
self.shuffle = shuffle
self.on_epoch_end()
self.features = features
self.text = text
self.vid = vid
def __len__(self):
return int(np.floor(len(self.all_inds) / self.batch_size))
def __getitem__(self,index):
indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]
batch_indices = [self.all_inds[k] for k in indexes]
features,text,vid,y = self.__data_generation(batch_indices)
return {'feature':features,'text':text,'vid':vid},y
def on_epoch_end(self):
self.indexes = np.arange(len(self.all_inds))
if self.shuffle == True:
np.random.shuffle(self.indexes)
def __data_generation(self,batch_indices):
# Generate data
features = self.features[batch_indices,:]
# note that you need to stack the slice in order to reshape it to (num_samples,768)
text = np.stack(self.text[batch_indices])
# since batch_size is not a super large number, you can stack here
vid = np.stack([fn2img(seed) for seed in self.vid[batch_indices]])
y = self.labels[batch_indices]
return features,text,vid,y
# fake a dataset
dataset_size = 500
train_idx,val_idx = train_test_split(range(dataset_size),test_size=0.2,)
# create an example DataFrame that I assume will be resemble yours
example_df = pd.DataFrame({'vids':np.random.randint(0,10000,dataset_size)})
# create feature columns
for ind in range(14): example_df['feature_%i' % ind] = np.random.rand(dataset_size)
# each cell contains a list
example_df['text'] = np.random.randint(dataset_size)
example_df['text'] = example_df['text'].astype('object')
for ind in range(dataset_size):example_df.at[ind,'text'] = np.random.rand(768).tolist()
# create the label column
example_df['label'] = np.random.randint(low=0,high=5,size=dataset_size)
# extract information from the dataframe, and create data generators
all_vids = example_df['vids'].values
feature_columns = ['feature_%i' % ind for ind in range(14)]
all_features = example_df[feature_columns].values
all_text = example_df['text'].values
all_labels = example_df['label'].values
training_generator = MultiInputDataGenerator(train_idx,all_labels,all_features,all_text,all_vids)
# create model
def features_part(x):
y = Dense(14)(x)
y = Dense(10,activation='linear')(y)
return y
def text_part(x):
y = Dense(768)(x)
y = Dense(10,activation='linear')(y)
return y
def vid_part(x):
y = Flatten()(x)
y = Dense(10,activation='linear')(y)
return y
input_features = Input(shape=(14,),name='feature')
input_text = Input(shape=(768,),name='text')
input_vid = Input(shape=(3,244,244,),name='vid')
feature_block = features_part(input_features)
text_block = text_part(input_text)
vid_block = vid_part(input_vid)
added = Add()([feature_block,text_block,vid_block])
# you have five classes at the end of the day
pred = Dense(1)(added)
# build model
model = Model(inputs=[input_features,input_text,input_vid],outputs=pred)
model.compile(loss='mae',optimizer='adam',metrics=['mae'])
model.fit_generator(generator=training_generator,epochs=10)
print(model.history.history)
numpy数组的数据类型是什么?@yudhiesh类型是object(请参见以下行:X=np.empty((self.batch\u size,1,3),dtype=object)y=np.empty((self.batch\u size,dtype=object))您能试试
astype(np.float32)吗
而不是object?@yudhiesh在这种情况下,我得到的错误是:ValueError:使用序列设置数组元素
。对于以下行:X[i,]=X\u id
看起来您正试图从一个形状不像多维数组的列表中创建一个数组。在声明X
和Y
时,您是否设置了dtype=np.float32
?numpy数组的数据类型是什么?@yudhiesh类型是object(请参见以下行:X=np.empty((self.batch\u size,1,3),dtype=object)Y=np.empty((self.batch\u size,dtype=object))您可以尝试astype(np.float32)
而不是object?@yudhiesh在这种情况下,我得到的错误是:ValueError:使用序列设置数组元素
。对于以下行:X[i,]=X\u id
看起来您正试图从一个形状不像多维数组的列表中创建一个数组。在声明X
和Y
时,您是否设置了dtype=np.float32
?谢谢!我得到错误:ValueError:无法将NumPy数组转换为张量(不支持的对象类型int)。
或ValueError:无法将NumPy数组转换为张量(不支持的对象类型NumPy.ndarray)。
在模型中。fit_生成器(生成器=training_生成器,epochs=10)
。当我在\u data\u generation
中打印标签、特征、文本、视频的形状时,我得到((8,),(8,14),(8,),(8,224,224,3))。这些形状是正确的,对吗?@okuoub不客气,我不知道你为什么会出现这样的错误,你在我的帖子中运行脚本时遇到错误了吗?形状不是真正的问题,只要它们与输入层的形状相匹配。@meTchaikovsky Iur代码运行得很好,我只是稍微编辑了一下以适合我的数据,所以我添加了(当df是一个数据帧时):all_features=df[\u FEATS\u COLS]。value all_text=df[\u text\u COL]。value#x_text是1X768 all\u vid=df[\u FILENAME\u COL]
和在数据生成中:features=self.features[batch\u index,:]text=self.text[batch\u index]vids=vid\u idx.apply(lambda x:fn2img(x))vid=np.concatenate([x代表vids中的x])y=self标签[batch\u indic]