Python 2.7 Tensorflow：从TFRecord文件读取特征长度与写入特征长度不同_Python 2.7_Tensorflow_Deep Learning

Python 2.7 Tensorflow：从TFRecord文件读取特征长度与写入特征长度不同

python-2.7 tensorflow deep-learning

Python 2.7 Tensorflow：从TFRecord文件读取特征长度与写入特征长度不同,python-2.7,tensorflow,deep-learning,Python 2.7,Tensorflow,Deep Learning,我的源代码如下，我可以成功地将图像数据转换为tf记录，但我无法正确解析从tf记录读取的示例，我真的很困惑 # -*- coding: utf-8 -*- import xml.etree.ElementTree as ET import os import tensorflow as tf import matplotlib.pyplot as plt import numpy as np import math import sys import shutil from CNN import

我的源代码如下，我可以成功地将图像数据转换为tf记录，但我无法正确解析从tf记录读取的示例，我真的很困惑

# -*- coding: utf-8 -*-
import xml.etree.ElementTree as ET
import os
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import math
import sys
import shutil
from  CNN import Alex_inference
from PIL import Image


# used for tf.train.Example Protocal Buffer
def _int64_feature(value):
    return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

# string feature
def _bytes_feature(value):
    return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

def _float_feature(value):
    return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))

def create_multi_TFRecordFile():
    output_filename = "./test-tf-recorder"
    tfrecord_writer = tf.python_io.TFRecordWriter(output_filename)

    image_file_path = "/media/VOCdevkit/VOC2012/JPEGImages/2007_000027.jpg"
    #image_raw_data = tf.read_file(image_file_path) 
    #change to below
    image = Image.open(image_file_path)

    #print image,image.size
    test = plt.imread(image_file_path)
    print test


    label_num = 1

    # Example Protocol Buffer.
    # I think we should provide tf.tain.Example with python data instead of tf tensors.
    example = tf.train.Example(features=tf.train.Features(feature={
        'image_raw_data': _bytes_feature(image.tobytes()), #in python 3,maybe need encoding='utf8' in bytes func.
        'image_width': _int64_feature(image.size[0]),
        'image_height': _int64_feature(image.size[1]),
        'label': _int64_feature(label_num)
    }))
    tfrecord_writer.write(example.SerializeToString())
    tfrecord_writer.close()
    print("\n\nTFRecord has been genearated\n")


# read record file
def get_image_batch():
    recorder_file_list = tf.train.match_filenames_once(
        "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/test*")
    reader = tf.TFRecordReader()
    filename_queue = tf.train.string_input_producer(recorder_file_list,shuffle=True)

    _, serialized_example = reader.read(filename_queue)

    features = tf.parse_single_example(
        serialized_example,
        features={
            'image_raw_data': tf.FixedLenFeature([],tf.string),
            'image_width': tf.FixedLenFeature([],tf.int64),
            'image_height': tf.FixedLenFeature([], tf.int64),
            'label': tf.FixedLenFeature([], tf.int64)
        })

    image_raw = tf.decode_raw(features['image_raw_data'],tf.uint8)
    image_height = tf.cast(features['image_height'],tf.int32)
    image_width = tf.cast(features['image_width'],tf.int32)
    image = tf.reshape(image_raw,[image_height,image_width,3])

    #image = tf.image.resize_images(image,size=[227,227],method=0)

    label = tf.cast(features['label'], tf.int32)
    label_hot = tf.one_hot(label, 20, on_value=1, off_value=0, axis=-1)

    sess = tf.InteractiveSession()
    sess.run(tf.global_variables_initializer())

    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(sess=sess, coord=coord)

    #print sess.run(recorder_file_list) #[ '/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/test-tf-recorder']
    print sess.run(image)

    coord.request_stop()
    coord.join(threads)

def train():
    create_multi_TFRecordFile()
    get_image_batch()



def main(argv=None):
    train()

if __name__ == '__main__':
    tf.app.run()

运行信息显示从tf记录读取的特征长度（DecodeRaw的输入长度为55）不同于向其写入的特征长度（[227227,3]）。如果有人能够解释，我将不胜感激

/home/wuzheng/anaconda2/bin/python /home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties: 
name: GeForce GTX 960M
major: 5 minor: 0 memoryClockRate (GHz) 1.176
pciBusID 0000:01:00.0
Total memory: 3.95GiB
Free memory: 3.53GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 960M, pci bus id: 0000:01:00.0)
[ '/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/test-tf-recorder']
Traceback (most recent call last):
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 99, in <module>
    tf.app.run()
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 96, in main
    train()
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 91, in train
    get_image_batch(20,100)
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 84, in get_image_batch
    print sess.run(image_raw).shape
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
    run_metadata_ptr)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
    feed_dict_string, options, run_metadata)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
    target_list, options, run_metadata)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to DecodeRaw has length 55 that is not a multiple of 4, the size of float
     [[Node: DecodeRaw = DecodeRaw[little_endian=true, out_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](ParseSingleExample/Squeeze_image_raw_data)]]

Caused by op u'DecodeRaw', defined at:
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 99, in <module>
    tf.app.run()
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 96, in main
    train()
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 91, in train
    get_image_batch(20,100)
  File "/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py", line 69, in get_image_batch
    image_raw = tf.decode_raw(features['image_raw_data'], tf.float32)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_parsing_ops.py", line 101, in decode_raw
    little_endian=little_endian, name=name)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/wuzheng/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Input to DecodeRaw has length 55 that is not a multiple of 4, the size of float
     [[Node: DecodeRaw = DecodeRaw[little_endian=true, out_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](ParseSingleExample/Squeeze_image_raw_data)]]


Process finished with exit code 1

/home/wuzheng/anaconda2/bin/python/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py
I tensorflow/stream_executor/dso_loader.cc:135]在本地成功打开了CUDA库libcublas.so.8.0
I tensorflow/stream_executor/dso_loader.cc:135]在本地成功打开了CUDA库libcudnn.so.5
I tensorflow/stream_executor/dso_loader.cc:135]在本地成功打开了CUDA库libcuft.so.8.0
I tensorflow/stream_executor/dso_loader.cc:135]在本地成功打开了CUDA库libcuda.so.1
I tensorflow/stream_executor/dso_loader.cc:135]在本地成功打开了CUDA库libcurand.so.8.0
W tensorflow/core/platform/cpu_feature_guard.cc:45]tensorflow库的编译不是为了使用SSE3指令，但这些指令在您的机器上可用，可以加快cpu计算。
W tensorflow/core/platform/cpu_feature_guard.cc:45]tensorflow库的编译不是为了使用SSE4.1指令，但这些指令在您的机器上可用，可以加快cpu计算。
W tensorflow/core/platform/cpu_feature_guard.cc:45]tensorflow库的编译不是为了使用SSE4.2指令，但这些指令在您的机器上可用，可以加快cpu计算。
W tensorflow/core/platform/cpu\u feature\u guard.cc:45]tensorflow库的编译不是为了使用AVX指令，但这些指令在您的机器上可用，可以加快cpu计算。
W tensorflow/core/platform/cpu\u feature\u guard.cc:45]tensorflow库的编译不是为了使用AVX2指令，但这些指令在您的机器上可用，可以加快cpu计算。
W tensorflow/core/platform/cpu\u feature\u guard.cc:45]tensorflow库的编译不是为了使用FMA指令，但是这些指令可以在您的机器上使用，并且可以加快cpu计算。
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910]从SysFS成功读取的NUMA节点的值为负值（-1），但必须至少有一个NUMA节点，因此返回NUMA节点零
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885]找到了具有以下属性的设备0：
名称：GeForce GTX 960M
大调：5小调：0内存时钟频率（GHz）1.176
pciBusID 0000:01:00.0
总内存：3.95GiB
可用内存：3.53GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906]DMA:0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916]0:Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975]创建tensorflow设备（/gpu:0）->（设备：0，名称：GeForce GTX 960M，pci总线id:0000:01:00.0）
['/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/test tf recorder']
回溯（最近一次呼叫最后一次）：
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，第99行，在
tf.app.run（）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/platform/app.py”，第44行，正在运行
_系统出口（主（_sys.argv[：1]+标志_passthrough））
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，主文件第96行
列车（）
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，第91行，列车中
获取图像批处理（20100）
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，第84行，在get_image_批处理中
打印sess.run（图像_raw）.shape
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/client/session.py”，第767行，正在运行
运行_元数据_ptr）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/client/session.py”，第965行，正在运行
提要（dict字符串、选项、运行元数据）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/client/session.py”，第1015行，运行
目标\u列表、选项、运行\u元数据）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/client/session.py”，第1035行，在
提升类型（e）（节点定义、操作、消息）
tensorflow.python.framework.errors\u impl.InvalidArgumentError:DecodeRaw的输入长度为55，不是浮点大小4的倍数
[[Node:DecodeRaw=DecodeRaw[little_endian=true，out_type=DT_FLOAT，_device=“/job:localhost/replica:0/task:0/cpu:0”]（ParseSingleExample/squence_image_raw_data）]]
由op u“DecodeRaw”引起，定义为：
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，第99行，在
tf.app.run（）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/platform/app.py”，第44行，正在运行
_系统出口（主（_sys.argv[：1]+标志_passthrough））
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，主文件第96行
列车（）
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，第91行，列车中
获取图像批处理（20100）
文件“/home/wuzheng/PycharmProjects/Grammar/new_order/Examples/3/read.py”，第69行，在get_image_批处理中
image_raw=tf.decode_raw（特征['image_raw_data']，tf.float32）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/ops/gen_parsing_ops.py”，第101行，在decode_raw中
little_endian=little_endian，name=name）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/framework/op_def_library.py”，第763行，在apply_op
op_def=op_def）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/framework/ops.py”，第2327行，在create_op中
初始值=自身值。\默认值\初始值，初始值=初始值）
文件“/home/wuzheng/anaconda2/lib/python2.7/site packages/tensorflow/python/framework/ops.py”，第1226行，在__
自我追踪
def create_multi_TFRecordFile():
    ...
    resized = tf.image.resize_images(image_raw_data, [227, 227], method=0)
    resized = tf.image.encode_jpeg(resized)
    ...