Python 为图像-文本配对数据构造TfRecord
我一直坚持让tfrecords为图像-文本对数据工作 下面是从图像特征的numpy数组和文本文件创建tfrecord的代码Python 为图像-文本配对数据构造TfRecord,python,tensorflow,tensorflow-datasets,tfrecord,Python,Tensorflow,Tensorflow Datasets,Tfrecord,我一直坚持让tfrecords为图像-文本对数据工作 下面是从图像特征的numpy数组和文本文件创建tfrecord的代码 def npy_to_tfrecords(numpy_array, text_file, output_file): f = open(text_file) # write records to a tfrecords file writer = tf.python_io.TFRecordWriter(output_file)
def npy_to_tfrecords(numpy_array, text_file, output_file):
f = open(text_file)
# write records to a tfrecords file
writer = tf.python_io.TFRecordWriter(output_file)
# Loop through all the features you want to write
for X, line in zip(numpy_array, f) :
#let say X is of np.array([[...][...]])
#let say y is of np.array[[0/1]]
txt = "{}".format(line[:-1])
txt = txt.encode()
# Feature contains a map of string to feature proto objects
feature = {}
feature['x'] = tf.train.Feature(float_list=tf.train.FloatList(value=X.flatten()))
feature['y'] = tf.train.Feature(bytes_list=tf.train.BytesList(value=[txt]))
# Construct the Example proto object
example = tf.train.Example(features=tf.train.Features(feature=feature))
# Serialize the example to a string
serialized = example.SerializeToString()
# write the serialized objec to the disk
writer.write(serialized)
writer.close()
在此之后,我无法创建数据集:
def load_data_tfr():
train = tf.data.TFRecordDataset("train.tfrecord")
# example proto decode
def _parse_function1(example_proto):
keys_to_features = {'x': tf.FixedLenFeature(2048, tf.float32),
'y': tf.VarLenFeature(tf.string) }
parsed_features = tf.parse_single_example(example_proto, keys_to_features)
return {"x": parsed_features['x'], "y": parsed_features['y']} # ['x'], parsed_features['y']
# Parse the record into tensors.
train = train.map(_parse_function1)
return train
我坚持。获取错误:
train_data = load_data_tfr()
random.shuffle(train_data)
有什么帮助吗?谢谢。MapDataset没有长度 所以,把这两行放在代码的最上面
import tensorflow as tf
tf.enable_eager_execution()
然后试试看
iterator = train_data.make_one_shot_iterator()
image, label = iterator.get_next()
当然,我假设您的tfrecord部分没有任何错误
根据Tensorflow教程,图像以字节格式保存,而不是np数组
iterator = train_data.make_one_shot_iterator()
image, label = iterator.get_next()