Python 3.x 在迁移学习中尝试生成数据时出现tensorflow图错误
我试图在预训练的初始模型上使用迁移学习,因此我创建了一个用于从模型中提取特征的类:Python 3.x 在迁移学习中尝试生成数据时出现tensorflow图错误,python-3.x,tensorflow,keras,Python 3.x,Tensorflow,Keras,我试图在预训练的初始模型上使用迁移学习,因此我创建了一个用于从模型中提取特征的类: from prototype import Dataset, VideoStreamHandler import numpy import random from keras.applications.inception_v3 import preprocess_input from keras.preprocessing import image from scipy.misc import imresize
from prototype import Dataset, VideoStreamHandler
import numpy
import random
from keras.applications.inception_v3 import preprocess_input
from keras.preprocessing import image
from scipy.misc import imresize
import time
class Extractor(Dataset.Dataset):
"""
"""
def __init__(self, path_to_data, seq_len, base_model, image_shape=(299, 299, 3)):
super().__init__(path_to_data, seq_len, input_shape=image_shape)
self._extractor = base_model
def extract_features(self, batch_size):
"""
passes the data through the base model to get the feature map to later train on
:return: feature map
"""
class_one_hot = self.one_hot_encode() # get the one hot for the classes
data = self.clean_data(self.get_data(), self._input_shape[0])
print("Processing {} videos".format(len(self.get_data())))
transfer_maps, labels = [], []
rand = random.SystemRandom()
while True:
for _ in range(batch_size):
row = rand.choice(data)
sequence = self.get_frames(row[0])
if len(sequence) > self._input_shape[0]:
sequence = self.rescale_frame_list(sequence, self._input_shape[0])
print("{} video processing is complete".format(row[0].split('\\')[-1]))
features = []
for frame in sequence:
frame_arr = image.img_to_array(frame) # turn image to numpy array
frame_arr = numpy.expand_dims(frame_arr, axis=0)
frame_arr = preprocess_input(frame_arr)
features.append(self._extractor.predict(frame_arr))
transfer_maps.append(features)
labels.append(class_one_hot[row[1]])
yield numpy.array(transfer_maps), numpy.array(labels)
def get_frames(self, pth):
"""
:type: string
:param pth: path to the specific file from which we take the frames
:return: the frames in the file
"""
f_queue = VideoStreamHandler.VideoStream(pth) # This object opens a thread that reads frames with opencv
# capture independently from the frame processing to prevent i/o delay and speed up processing
f_queue.start()
time.sleep(1.0) # wait a moment so the thread could start reading frames
sequence = []
while f_queue.isnt_empty():
frame = f_queue.read()
# resize is used to keep all frames from all videos the same size
frame = imresize(frame, (self._input_shape[1], self._input_shape[2]))
sequence.append(frame)
f_queue.close() # close the thread
return sequence
然后,我尝试使用keras的fit_生成器来训练一个新模型:
my_model.fit_generator(generator=train_gen, epochs=10, steps_per_epoch=steps_per_epoch, verbose=1, workers=4)
但是,我得到了这个错误:
大宗报价
回溯(最近一次呼叫最后一次):
文件“C:/Users/Aviad Lazar/Desktop/project/prototype/transfer_learning.py”,第41行,在
main()
文件“C:/Users/Aviad Lazar/Desktop/project/prototype/transfer_learning.py”,第34行,主视图
my_model.fit_generator(generator=train_gen,epochs=10,steps_per_epoch=steps_per_epoch,verbose=1,workers=4)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\legacy\interfaces.py”,第91行,在包装器中
返回函数(*args,**kwargs)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\models.py”,第1315行,在fit\u generator中
初始_历元=初始_历元)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\legacy\interfaces.py”,第91行,在包装器中
返回函数(*args,**kwargs)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\engine\training.py”,第2194行,在fit_generator中
发电机输出=下一个(输出发电机)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\utils\data\u utils.py”,第793行,在get中
六、重放(值。类,值,值。回溯)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\six.py”,第693行,在reraise中
增值
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\utils\data\u utils.py”,第658行,在数据生成器任务中
发电机输出=下一个(自发电机)
文件“C:\Users\Aviad Lazar\Desktop\project\prototype\FeatureExtractor.py”,第48行,在extract\u features中
features.append(self.\u extractor.predict(frame\u arr))
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\engine\training.py”,第1832行,在predict中
self.\u make\u predict\u函数()
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\engine\training.py”,第1031行,在“生成预测”函数中
**kwargs)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\backend\tensorflow\u backend.py”,第2506行,在函数中
返回功能(输入、输出、更新=更新,**kwargs)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\keras\backend\tensorflow\u backend.py”,第2449行,位于init
具有tf.control_依赖项(自输出):
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\tensorflow\python\framework\ops.py”,第4863行,在控件依赖项中
返回get_default_graph()。控制依赖项(控制输入)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\tensorflow\python\framework\ops.py”,第4481行,在控制依赖项中
c=自成图元素(c)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\tensorflow\python\framework\ops.py”,第3478行,位于as\u graph\u元素中
返回self.\u as\u graph\u element\u locked(对象、允许张量、允许操作)
文件“C:\Users\Aviad Lazar\Desktop\project\venv\lib\site packages\tensorflow\python\framework\ops.py”,第3557行,位于图元素中
raise VALUERROR(“张量%s不是此图的元素。”%obj)
ValueError:Tensor Tensor(“全局平均值”2D\u 1/Mean:0),shape=(?,2048),dtype=float32)不是此图的元素
您确定这是fit_generator的问题,而不是模型的问题吗?
my_model.fit()
可以正常工作而不引发错误吗?是的,我几天前切换到fit,它一直工作正常。