Python 将TF记录加载到Keras中
我正在尝试将自定义TFRecord文件加载到我的keras模型中。我试图遵循本教程:,但适应我的使用 我的目标是让这些函数与Keras的ImageDataGenerator类似。我无法使用该函数,因为我从生成器无法获取的图像中获取特定元数据。我没有在这里包含元数据,因为我只需要先使用基本网络 我还希望能够将此应用于迁移学习应用程序 我一直遇到这样的错误:Python 将TF记录加载到Keras中,python,tensorflow,keras,tfrecord,Python,Tensorflow,Keras,Tfrecord,我正在尝试将自定义TFRecord文件加载到我的keras模型中。我试图遵循本教程:,但适应我的使用 我的目标是让这些函数与Keras的ImageDataGenerator类似。我无法使用该函数,因为我从生成器无法获取的图像中获取特定元数据。我没有在这里包含元数据,因为我只需要先使用基本网络 我还希望能够将此应用于迁移学习应用程序 我一直遇到这样的错误:TypeError:无法使用类型NoneType为None构建TypeSpec 我使用的是Tensorflow 2.2 def _parse_f
TypeError:无法使用类型NoneType为None构建TypeSpec
我使用的是Tensorflow 2.2
def _parse_function(serialized):
features = \
{
'image': tf.io.FixedLenFeature([], tf.string),
'label': tf.io.FixedLenFeature([], tf.int64),
'shapex': tf.io.FixedLenFeature([], tf.int64),
'shapey': tf.io.FixedLenFeature([], tf.int64),
}
parsed_example = tf.io.parse_single_example(serialized=serialized,
features=features)
shapex = tf.cast(parsed_example['shapex'], tf.int32)
shapey = tf.cast(parsed_example['shapey'], tf.int32)
image_shape = tf.stack([shapex, shapey, 3])
image_raw = parsed_example['image']
# Decode the raw bytes so it becomes a tensor with type.
image = tf.io.decode_raw(image_raw, tf.uint8)
image = tf.reshape(image, image_shape)
# Get labels
label = tf.cast(parsed_example['label'], tf.float32)
return image, label
def imgs_inputs(type, perform_shuffle=False):
records_dir = '/path/to/tfrecord/'
record_paths = [os.path.join(records_dir,record_name) for record_name in os.listdir(records_dir)]
full_dataset = tf.data.TFRecordDataset(filenames=record_paths)
full_dataset = full_dataset.map(_parse_function, num_parallel_calls=16)
dataset_length = (len(list(full_dataset))) #Gets length of datase
iterator = tf.compat.v1.data.make_one_shot_iterator(databatch)
image, label = iterator.get_next()
#labels saved as values ex: [1,2,3], and are now converted to one hot encoded
label = to_categorical(label)
return image, label
image, label = imgs_inputs(type ='Train',perform_shuffle=True)
#Combine it with keras
# base_model = MobileNet(weights='imagenet', include_top=False, input_shape=(200,200,3), dropout=.3)
model_input = Input(shape=[200,200,3])
#Build your network
model_output = Flatten(input_shape=(200, 200, 3))(model_input)
model_output = Dense(19, activation='relu')(model_output)
#Create your model
train_model = Model(inputs=model_input, outputs=model_output)
#Compile your model
optimizer = Adam(learning_rate=.001)
train_model.compile(optimizer=optimizer,loss='mean_squared_error',metrics=['accuracy'],target_tensors=[label])
#Train the model
train_model.fit(epochs=10,steps_per_epoch=2)
image
返回形状数组(100200200,3),它是一批100个图像
label
返回shape数组(100,19),这是一批100个标签(共19个标签)与shapex
和shapey
相关的问题,但我不知道确切原因。
我设置了shapex=200
和shapey=200
。然后我重写了模型,将迁移学习纳入其中
base_model = MobileNet(weights='imagenet', include_top=False, input_shape=(200,200,3), dropout=.3)
x = base_model.output
types = Dense(19,activation='softmax')(x)
model = Model(inputs=base_model.input,outputs=types)
model.compile(
optimizer='adam',
loss = 'sparse_categorical_crossentropy',
metrics=['accuracy']
history = model.fit(get_batches(), steps_per_epoch=1000, epochs=10)
I found everything I needed on this Google Colab:
[https://colab.research.google.com/github/GoogleCloudPlatform/training-data-analyst/blob/master/courses/fast-and-lean-data-science/04_Keras_Flowers_transfer_learning_solution.ipynb#scrollTo=XLJNVGwHUDy1][1]
[1]: https://colab.research.google.com/github/GoogleCloudPlatform/training-data-analyst/blob/master/courses/fast-and-lean-data-science/04_Keras_Flowers_transfer_learning_solution.ipynb#scrollTo=XLJNVGwHUDy1