Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/tensorflow/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
如何在tf.keras中进行错误处理?在Tensorflow Keras中排除坏数据集而不使程序崩溃_Tensorflow_Machine Learning_Keras_Computer Vision_Tensorflow2.0 - Fatal编程技术网

如何在tf.keras中进行错误处理?在Tensorflow Keras中排除坏数据集而不使程序崩溃

如何在tf.keras中进行错误处理?在Tensorflow Keras中排除坏数据集而不使程序崩溃,tensorflow,machine-learning,keras,computer-vision,tensorflow2.0,Tensorflow,Machine Learning,Keras,Computer Vision,Tensorflow2.0,我试图用大量数据运行一个keras模型。某个地方似乎有一个损坏的文件,我不确定具体在哪里。如果试图处理坏文件,keras似乎会崩溃。我在下面附上了准确的错误 我正在努力使程序不会在每次运行这个模型时崩溃,而不必查找所有的坏数据点。我想自动将其从模型中排除。但是,因为它是在keras model.fit函数中运行的,所以我不确定如何在它周围设置try-catch循环 我已经附上了具体的错误消息,但我想知道我是否可以只是把一个尝试捕捉周围的图像解码器。例如,我能做一个快速的测试吗 try: ###d

我试图用大量数据运行一个keras模型。某个地方似乎有一个损坏的文件,我不确定具体在哪里。如果试图处理坏文件,keras似乎会崩溃。我在下面附上了准确的错误

我正在努力使程序不会在每次运行这个模型时崩溃,而不必查找所有的坏数据点。我想自动将其从模型中排除。但是,因为它是在keras model.fit函数中运行的,所以我不确定如何在它周围设置try-catch循环

我已经附上了具体的错误消息,但我想知道我是否可以只是把一个尝试捕捉周围的图像解码器。例如,我能做一个快速的测试吗

try:
###decoder part here###
except:
    continue
不把我的代码弄乱?请告知如何处理此问题。 感谢您抽出时间,下面是具体的错误消息

b_concat (Concatenate)          (None, 1, 62, 768)   0           concatenate_2[0][0]              
                                                                 concatenate_3[0][0]              
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 62, 768)      0           b_concat[0][0]                   
__________________________________________________________________________________________________
label (InputLayer)              [(None, None)]       0                                            
__________________________________________________________________________________________________
dense2 (Dense)                  (None, 62, 67)       51523       reshape_1[0][0]                  
__________________________________________________________________________________________________
ctc_loss (CTCLayer)             (None, 62, 67)       0           label[0][0]                      
                                                                 dense2[0][0]                     
==================================================================================================
Total params: 4,296,131
Trainable params: 4,289,763
Non-trainable params: 6,368
__________________________________________________________________________________________________
None
Epoch 1/50
WARNING:tensorflow:Model was constructed with shape (None, 32, 512, 1) for input Tensor("image:0", shape=(None, 32, 512, 1), dtype=float32), but it was called on an input with incompatible shape (None, 512, 32, 1).
WARNING:tensorflow:Model was constructed with shape (None, 32, 512, 1) for input Tensor("image:0", shape=(None, 32, 512, 1), dtype=float32), but it was called on an input with incompatible shape (None, 512, 32, 1).
  214/50944 [..............................] - ETA: 143:30:19 - loss: 26.43842020-11-06 10:51:17.056847: E tensorflow/core/lib/jpeg/jpeg_mem.cc:323] Premature end of JPEG data. Stopped at line 0/31
  217/50944 [..............................] - ETA: 143:29:56 - loss: 26.3900Traceback (most recent call last):
  File "keras_tutorial_main.py", line 235, in <module>
    callbacks=callbacks,
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1098, in fit
    tmp_logs = train_function(iterator)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 807, in _call
    return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError:  Invalid JPEG data or crop window, data size 1024
     [[{{node DecodePng}}]]
     [[IteratorGetNext]] [Op:__inference_train_function_12954]

Function call stack:
train_function

b_concat(Concatenate)(无,1,62,768)0 Concatenate_2[0][0]
连接_3[0][0]
__________________________________________________________________________________________________
重塑1(重塑)(无,62768)0 b_concat[0][0]
__________________________________________________________________________________________________
标签(InputLayer)[(无,无)]0
__________________________________________________________________________________________________
dense2(致密)(无,62,67)51523重塑_1[0][0]
__________________________________________________________________________________________________
ctc_损失(CTCLayer)(无、62、67)0标签[0][0]
dense2[0][0]
==================================================================================================
总参数:4296131
可培训参数:4289763
不可培训参数:6368
__________________________________________________________________________________________________
没有一个
纪元1/50
警告:tensorflow:为输入张量(“图像:0”,shape=(None,32,512,1),dtype=float32)使用形状(None,32,512,1)构造模型,但在形状不兼容的输入(None,512,32,1)上调用了该模型。
警告:tensorflow:为输入张量(“图像:0”,shape=(None,32,512,1),dtype=float32)使用形状(None,32,512,1)构造模型,但在形状不兼容的输入(None,512,32,1)上调用了该模型。
214/50944[预计到达时间:143:30:19-丢失:26.43842020-11-06 10:51:17.056847:E tensorflow/core/lib/jpeg/jpeg_mem.cc:323]jpeg数据过早结束。在第0/31行停止
217/50944[……]-预计到达时间:143:29:56-损失:26.3900回拨(最近一次呼叫最后一次):
文件“keras_tutorial_main.py”,第235行,在
回调=回调,
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/keras/engine/training.py”,第108行,在方法包装中
返回方法(self、*args、**kwargs)
文件“/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py”,第1098行,适合
tmp_logs=训练函数(迭代器)
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/eager/def_function.py”,第780行,在调用中__
结果=自身调用(*args,**kwds)
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/eager/def_function.py”,第807行,在调用中
返回self._无状态_fn(*args,**kwds)35; pylint:disable=不可调用
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/eager/function.py”,第2829行,在__
返回图形\函数。\过滤\调用(args,kwargs)\ pylint:disable=受保护的访问
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/eager/function.py”,第1848行,在
取消管理器=取消管理器)
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/eager/function.py”,第1924行,在调用平面中
ctx,args,取消管理器=取消管理器)
调用中的第550行文件“/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py”
ctx=ctx)
文件“/usr/local/lib/python3.6/dist packages/tensorflow/python/eager/execute.py”,第60行,快速执行
输入、属性、数量(输出)
tensorflow.python.framework.errors\u impl.InvalidArgumentError:无效的JPEG数据或裁剪窗口,数据大小1024
[{{node DecodePng}}]]
[[IteratorGetNext]][Op:[推理\训练\函数\ U12954]
函数调用堆栈:
列车功能

如果您遇到XY问题,您应该尝试查找哪些文件已损坏以将其删除,而不是尝试解决该问题。如果使用tensorflow一个接一个地加载每个文件,并注意哪些文件出错,那么这应该很容易