Python 如何使用tensorflow队列处理多标签数据?

Python 如何使用tensorflow队列处理多标签数据?,python,tensorflow,queue,multilabel-classification,Python,Tensorflow,Queue,Multilabel Classification,我有一个很大的图像数据库,里面有相应的标签。我想用CNN对其进行分类,但我的问题是关于使用Tensorflow的输入管道方法。 由于数据库太大,我不得不使用队列 我的数据格式是一个文本文件,如下所示: my_data.txt: filename1, label1, label2, label3 filename2, label4, label5 filename3, label2, label6, label7, label8 ... 我将其解析为在python列表中使用,如下所示: my_d

我有一个很大的图像数据库,里面有相应的标签。我想用CNN对其进行分类,但我的问题是关于使用Tensorflow的输入管道方法。 由于数据库太大,我不得不使用队列

我的数据格式是一个文本文件,如下所示:

my_data.txt:

filename1, label1, label2, label3
filename2, label4, label5
filename3, label2, label6, label7, label8
...
我将其解析为在python列表中使用,如下所示:
my_data=[[filename1,filename3,filename3,…],[label1,label2,label3],[label4,label5],[label2,label6,label7,label8],…]

我想把它放在一个队列中,并将一个filename元素与其对应的标签一起出列

我尝试的是:

import tensorflow as tf

queue = tf.train.slice_input_producer(my_data, capacity=100,  shuffle=True)
filename = queue[0]
labels = queue[1:]
file_content = tf.read_file(filename)
record = tf.image.decode_png(file_content, channels=3)
image = preprocess(record) # to put shape at a regular value
data_batch = tf.train.batch([image, labels])

init = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())

with tf.Session() as sess :
    sess.run(init)
    coord =tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord, sess=sess)
    png_images, labels= sess.run(data_batch)




Traceback (most recent call last):
  File "test_queue.py", line 19, in <module>
    shuffle=True)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/input.py", line 298, in slice_input_producer
    tensor_list = ops.convert_n_to_tensor_or_indexed_slices(tensor_list)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 925, in convert_n_to_tensor_or_indexed_slices
    values=values, dtype=dtype, name=name, as_ref=False)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 896, in internal_convert_n_to_tensor_or_indexed_slices
    value, dtype=dtype, name=n, as_ref=as_ref))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 857, in internal_convert_to_tensor_or_indexed_slices
    as_ref=as_ref)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 702, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 111, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 100, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_util.py", line 379, in make_tensor_proto
    _GetDenseDimensions(values)))
ValueError: Argument must be a dense tensor: [['label1', 'label2', 'label3'], ['label4', 'label5'], ['label2', 'label6', 'label7', 'label8']] - got shape [3], but wanted [3, 3].
将tensorflow导入为tf
queue=tf.train.slice\u input\u producer(我的数据,容量=100,随机播放=True)
文件名=队列[0]
标签=队列[1:]
file\u content=tf.read\u文件(文件名)
record=tf.image.decode\u png(文件内容,频道=3)
图像=预处理(记录)#将形状设置为常规值
data\U batch=tf.train.batch([图像,标签])
init=tf.group(tf.global\u variables\u initializer(),tf.local\u variables\u initializer())
使用tf.Session()作为sess:
sess.run(初始化)
coord=tf.train.Coordinator()
线程=tf.train.start\u queue\u运行者(coord=coord,sess=sess)
png\u图像,标签=sess.run(数据\u批处理)
回溯(最近一次呼叫最后一次):
文件“test_queue.py”,第19行,在
洗牌=真)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/training/input.py”,第298行,在slice\u input\u producer中
张量列表=运算。将张量列表转换为张量切片或索引切片(张量列表)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py”,第925行,在convert\n\u-to\u-tensor\u或\u-indexed\u切片中
values=values,dtype=dtype,name=name,as_ref=False)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/framework/ops.py”,第896行,在内部\u convert\n\u到\u tensor\u或\u索引\u切片中
值,dtype=dtype,name=n,as_ref=as_ref))
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/framework/ops.py”,第857行,在内部\u convert\u to \u tensor\u或\u索引\u切片中
as_ref=as_ref)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py”,第702行,在内部\u-convert\u-to\u-tensor中
ret=conversion\u func(值,dtype=dtype,name=name,as\u ref=as\u ref)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant\u op.py”,第111行,在常量张量转换函数中
返回常量(v,dtype=dtype,name=name)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py”,第100行,常量
tensor_util.make_tensor_proto(值,dtype=dtype,shape=shape,verify_shape=verify_shape))
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/framework/tensor\u util.py”,第379行,在make\u tensor\u proto中
_GetDenseDimensions(值)))
ValueError:参数必须是稠密张量:[[label1]、[label2]、[label3']、[label4]、[label5']、[label2]、[label6]、[label7]、[label8']]-获得形状[3],但需要[3,3]。
如果a在每个文件名元素中放置相同数量的标签,它就会工作,那么我该怎么做呢