Python 使用_生成器中的tensorflow.data.Dataset.data时,参数输入无效
我正在尝试生成自己的图像数据集,以便在单个GPU上使用Tensorflow数据集API来测量推理性能Python 使用_生成器中的tensorflow.data.Dataset.data时,参数输入无效,python,tensorflow,Python,Tensorflow,我正在尝试生成自己的图像数据集,以便在单个GPU上使用Tensorflow数据集API来测量推理性能 resolutions = [ (2048, 1080) ] def generate_image(size, channels): image_value = random.random() image_shape = [1, size[1], size[0], channels] return tf.constant( value=image
resolutions = [
(2048, 1080)
]
def generate_image(size, channels):
image_value = random.random()
image_shape = [1, size[1], size[0], channels]
return tf.constant(
value=image_value,
shape=image_shape,
dtype=tf.float32)
def generate_single_input(size):
source = generate_image(size, 3)
target = generate_image(size, 3)
return source, target
def input_generator_fn():
for res in resolutions:
for i in range(10):
yield generate_single_input(res)
def benchmark():
...
ds = tf.data.Dataset.from_generator(
generator=input_generator_fn,
output_types=(tf.float32, tf.float32),
output_shapes=(tf.TensorShape([1, 1080, 2048, 3]),
tf.TensorShape([1, 1080, 2048, 3])))
iterator = ds.make_one_shot_iterator()
next_record = iterator.get_next()
inputs = next_record[0]
outputs = next_record[1]
predictions = {
'input_images': inputs
'output_images': outputs
}
session = tf.Session()
with session:
tf.global_variables_initializer()
for res in resolutions:
for i in range(10):
session.run(predictions)
.....
但我在跑步后观察到以下异常情况:
2018-04-06 13:38:44.050448: W tensorflow/core/framework/op_kernel.cc:1198] Invalid argument: ValueError: setting an array element with a sequence.
2018-04-06 13:38:44.050581: W tensorflow/core/framework/op_kernel.cc:1198] Invalid argument: ValueError: setting an array element with a sequence.
[[Node: PyFunc = PyFunc[Tin=[DT_INT64], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_1"](arg0)]]
Traceback (most recent call last):
File "tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
return fn(*args)
File "tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
status, run_metadata)
File "tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: ValueError: setting an array element with a sequence.
[[Node: PyFunc = PyFunc[Tin=[DT_INT64], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_1"](arg0)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[1,1080,2048,3], [1,1080,2048,3]], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](OneShotIterator)]]
你弄明白了吗 我遇到了完全相同类型的问题,我的问题是生成器和输入到输出形状中的内容之间的尺寸不匹配
另外,看看你的代码,我相信你必须输入有效的数据,比如numpy数组,而不是TensorFlow常量。简而言之,原因是from_生成器可以展平numpy数组,但不能展平张量 下面是一个较短的代码,它将再现错误:
import tensorflow as tf
import numpy as np
print(tf.__version__)
def g():
img = tf.random_uniform([3])
# img = np.random.rand(3)
# img = tf.convert_to_tensor(img)
yield img
dataset = tf.data.Dataset.from_generator(g, tf.float64, tf.TensorShape([3]))
iterator = dataset.make_one_shot_iterator()
next_iterator = iterator.get_next()
sess = tf.Session()
sess.run(next_iterator)
版本1.14中的错误消息非常有用。(由于版本不同,代码的具体行会发生变化,但我已经检查了1.12和1.13,我使用的原因是相同的。)
当生成的元素是张量时,from_generator会将其展平为输出\u类型
。convert函数不起作用
要解决这个问题,只要在生成器生成张量时不要使用来自生成器的。您可以使用来自张量的或来自张量切片的
img = tf.random_uniform([3])
dataset = tf.data.Dataset.from_tensors(img).repeat()
iterator = dataset.make_initializable_iterator()
next_iterator = iterator.get_next()
sess = tf.Session()
sess.run(iterator.initializer)
sess.run(next_iterator)
使用此时遇到相同的问题。从_generator(),原因我不知道。有人有可能解决这个问题吗?
img = tf.random_uniform([3])
dataset = tf.data.Dataset.from_tensors(img).repeat()
iterator = dataset.make_initializable_iterator()
next_iterator = iterator.get_next()
sess = tf.Session()
sess.run(iterator.initializer)
sess.run(next_iterator)