Python 保存并加载带有附加属性的自定义图层的keras模型
我创建了一个自定义层Python 保存并加载带有附加属性的自定义图层的keras模型,python,tensorflow,keras,keras-layer,Python,Tensorflow,Keras,Keras Layer,我创建了一个自定义层DenseWithMask，它是Dense的子类。它还有几个属性，包括我称之为edge\u mask的属性。因此，这段代码工作得很好： new_layer = DenseWithMask(10) print(new_layer.edge_mask) 但是，如果我使用DenseWithMask创建模型，然后保存并再次加载，则层没有属性edge\u mask。（我添加的所有其他属性也丢失了。） 下面是一个例子（本文末尾的DenseWithMask代码）： 这将返回： edge_
DenseWithMaskDenseedge\u masknew_layer = DenseWithMask(10)
print(new_layer.edge_mask)
DenseWithMaskedge\u maskDenseWithMaskedge_mask: tf.Tensor(
[[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]
...
[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]], shape=(784, 128), dtype=bool)
INFO:tensorflow:Assets written to: model_with_custom_layers\assets
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-6-32e4901279b3> in <module>
23
24 # try accessing attributes of custom layer
---> 25 model2.layers[1].edge_mask
AttributeError: 'DenseWithMask' object has no attribute 'edge_mask'
我无法用您提供的代码重现您的错误。我还有几个其他错误，可以通过执行以下操作来修复：
/class\u name
/unit\u mask\u index
不应在edge\u mask\u index
返回的字典中，因为没有get\u config
参数\u init\u
/单元掩码
如果不是边缘掩码无
将tensorflow导入为tf
################################################################################
#定义允许永久修剪的keras图层类
################################################################################
#导入从keras.layers.core复制的内容
从tensorflow.python.eager导入上下文
从tensorflow.python.framework导入数据类型
从tensorflow.python.framework导入tensor_形状
从tensorflow.python.keras导入激活
从tensorflow.python.keras将后端导入为K
从tensorflow.python.keras导入约束
从tensorflow.python.keras导入初始值设定项
从tensorflow.python.keras导入正则化器
从tensorflow.python.keras.engine.base_层导入层
从tensorflow.python.keras.engine.input\u spec导入InputSpec
从tensorflow.python.ops导入数组
从tensorflow.python.ops导入gen_math_ops
从tensorflow.python.ops导入数学运算
从tensorflow.python.ops导入nn
从tensorflow.python.ops导入稀疏的
从tensorflow.python.ops导入标准操作
#其他进口
将numpy作为np导入
导入tensorflow作为tf
#从tensorflow.keras.layers导入*
################################################################################
#以下类别是keras中Dense的副本。我把所有的线都标上了
#我添加或更改了
类DensweithMask（层）：
“”“密集层，但具有可选的单元或边缘永久掩蔽。”“”
定义初始化__(
自己
单位,，
单位_掩码=无，#新
边缘遮罩=无，#新
激活=无，
使用_bias=True，
kernel\u initializer='glorot\u uniform'，
偏差\u初始值设定项='零'，
核正则化器=无，
偏差\正则化器=无，
活动\正则化器=无，
内核约束=无，
偏差约束=无，
**夸尔斯
):
如果“输入形状”不在kwargs中，而“输入尺寸”在kwargs中：
kwargs['input_shape']=（kwargs.pop（'input_dim'），）
超级（DenseWithMask，self）。u初始（#将“密集”更改为“DenseWithMask”
活动正则化器=正则化器。获取（活动正则化器））#，**kwargs）
self.units=int（单位），如果不是，则为instance（单位，int）else单位
#新增：将单位掩码添加到类属性
self.unit\u mask=np.array（unit\u mask），如果unit\u mask不是None-other-None
#新增：将边缘遮罩添加到类属性
self.edge\u mask=np.array（edge\u mask），如果edge\u mask不是None-other-None
#新增：向类属性添加单位掩码索引
self.unit\u掩码\u索引=无
#新增：向类属性添加边遮罩索引
self.edge\u mask\u索引=无
self.activation=activations.get（激活）
self.use\u bias=use\u bias
self.kernel\u initializer=initializers.get（kernel\u initializer）
self.bias\u初始值设定项=初始值设定项.get（bias\u初始值设定项）
self.kernel\u正则化器=正则化器.get（kernel\u正则化器）
self.bias\u正则化器=正则化器.get（bias\u正则化器）
self.kernel\u constraint=constraints.get（kernel\u constraint）
self.bias\u constraint=constraints.get（bias\u constraint）
self.supports\u masking=True
self.input\u spec=InputSpec（最小ndim=2）
超级（Densweithmask，self）。\uuuu初始值（**kwargs）
def get_配置（自身）：
配置={
“单位”：自我单位，
'unit_mask'：self.unit_mask.numpy（），#新：在配置中添加了unit_mask
“edge_mask”：self.edge_mask.numpy（），#新：将edge_mask添加到配置中
“激活”：激活。序列化（自激活），
“使用偏见”：self.use\u bias，
“kernel\u初始值设定项”：初始值设定项.serialize（self.kernel\u初始值设定项），
“bias_初始值设定项”：初始值设定项.serialize（self.bias_初始值设定项），
“kernel\u正则化器”：正则化器.serialize（self.kernel\u正则化器），
“bias_正则化器”：正则化器.serialize（self.bias_正则化器），
“活动\正则化器”：
正则化子.serialize（self.activity\u正则化子），
“内核约束”：约束.serialize（self.kernel\u约束），
“偏差约束”：约束.serialize（self.bias\u约束）
}
base_config=super（DenseWithMask，self）.get_config（）
return dict（list（base_config.items（））+list（config.items（））
def构建（自我，输入_形状）：
dtype=dtypes.as_dtype（self.dtype或K.floatx（））
如果不是（dtype.is_浮动或dtype.is_复杂）：
raise TypeError（'无法使用非浮点值构建'density'层'
“数据类型%s%”（数据类型，）
输入形状=张量形状。张量形状（输入形状）
if张量形状尺寸值（inp
edge_mask: tf.Tensor(
[[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]
...
[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]], shape=(784, 128), dtype=bool)
INFO:tensorflow:Assets written to: model_with_custom_layers\assets
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-6-32e4901279b3> in <module>
23
24 # try accessing attributes of custom layer
---> 25 model2.layers[1].edge_mask
AttributeError: 'DenseWithMask' object has no attribute 'edge_mask'
################################################################################
# Define a keras layer class that allows for permanent pruning
################################################################################
# imports copied from keras.layers.core
from tensorflow.python.eager import context
from tensorflow.python.framework import dtypes
from tensorflow.python.framework import tensor_shape
from tensorflow.python.keras import activations
from tensorflow.python.keras import backend as K
from tensorflow.python.keras import constraints
from tensorflow.python.keras import initializers
from tensorflow.python.keras import regularizers
from tensorflow.python.keras.engine.base_layer import Layer
from tensorflow.python.keras.engine.input_spec import InputSpec
from tensorflow.python.ops import array_ops
from tensorflow.python.ops import gen_math_ops
from tensorflow.python.ops import math_ops
from tensorflow.python.ops import nn
from tensorflow.python.ops import sparse_ops
from tensorflow.python.ops import standard_ops
# other imports
import numpy as np
import tensorflow as tf
#from tensorflow.keras.layers import *
################################################################################
# The following class is a copy of Dense in keras. I marked all the lines that
# I added or changed
class DenseWithMask(Layer):
"""Dense layer but with optional permanent masking of units or edges."""
def __init__(self,
units,
unit_mask=None, # NEW
edge_mask=None, # NEW
activation=None,
use_bias=True,
kernel_initializer='glorot_uniform',
bias_initializer='zeros',
kernel_regularizer=None,
bias_regularizer=None,
activity_regularizer=None,
kernel_constraint=None,
bias_constraint=None,
**kwargs):
if 'input_shape' not in kwargs and 'input_dim' in kwargs:
kwargs['input_shape'] = (kwargs.pop('input_dim'),)
super(DenseWithMask, self).__init__( # changed 'Dense' to 'DenseWithMask'
activity_regularizer=regularizers.get(activity_regularizer)) #, **kwargs)
self.units = int(units) if not isinstance(units, int) else units
# NEW: add unit_mask to class attributes
self.unit_mask = unit_mask
# NEW: add edge_mask to class attributes
self.edge_mask = edge_mask
# NEW: add unit_mask_indices to class attributes
self.unit_mask_indices = None
# NEW: add edge_mask_indices to class attributes
self.edge_mask_indices = None
self.activation = activations.get(activation)
self.use_bias = use_bias
self.kernel_initializer = initializers.get(kernel_initializer)
self.bias_initializer = initializers.get(bias_initializer)
self.kernel_regularizer = regularizers.get(kernel_regularizer)
self.bias_regularizer = regularizers.get(bias_regularizer)
self.kernel_constraint = constraints.get(kernel_constraint)
self.bias_constraint = constraints.get(bias_constraint)
self.supports_masking = True
self.input_spec = InputSpec(min_ndim=2)
super(DenseWithMask, self).__init__(**kwargs)
def get_config(self):
config = {
'class_name': 'DenseWithMask',
'units': self.units,
'unit_mask': self.unit_mask.numpy(), #NEW: added unit_mask to config
'unit_mask_indices': self.unit_mask_indices.numpy(), # NEW
'edge_mask': self.edge_mask.numpy(), #NEW: added edge_mask to config
'edge_mask_indices': self.edge_mask_indices.numpy(), # NEW
'activation': activations.serialize(self.activation),
'use_bias': self.use_bias,
'kernel_initializer': initializers.serialize(self.kernel_initializer),
'bias_initializer': initializers.serialize(self.bias_initializer),
'kernel_regularizer': regularizers.serialize(self.kernel_regularizer),
'bias_regularizer': regularizers.serialize(self.bias_regularizer),
'activity_regularizer':
regularizers.serialize(self.activity_regularizer),
'kernel_constraint': constraints.serialize(self.kernel_constraint),
'bias_constraint': constraints.serialize(self.bias_constraint)
}
base_config = super(DenseWithMask, self).get_config()
return dict(list(base_config.items()) + list(config.items()))
def build(self, input_shape):
dtype = dtypes.as_dtype(self.dtype or K.floatx())
if not (dtype.is_floating or dtype.is_complex):
raise TypeError('Unable to build `Dense` layer with non-floating point '
'dtype %s' % (dtype,))
input_shape = tensor_shape.TensorShape(input_shape)
if tensor_shape.dimension_value(input_shape[-1]) is None:
raise ValueError('The last dimension of the inputs to `Dense` '
'should be defined. Found `None`.')
last_dim = tensor_shape.dimension_value(input_shape[-1])
self.input_spec = InputSpec(min_ndim=2, axes={-1: last_dim})
# NEW: update masks in build
if self.unit_mask is None:
self.unit_mask = np.ones(self.units, dtype=bool)
else:
# check if previously set mask matches number of units
if not len(self.unit_mask) == self.units:
raise ValueError('Length of unit_mask must be equal to number of units.')
if self.edge_mask is None:
self.edge_mask = np.ones((last_dim, self.units), dtype=bool)
else:
# check if previously set mask matches input dimensions
if not self.edge_mask.shape == (last_dim, self.units):
raise ValueError('Dimensions of edge_mask must be equal to (last_input_dim, units).')
# NEW: incorporate unit_mask info into edge_mask
self.edge_mask = self.edge_mask * self.unit_mask
# NEW: incorporate edge_mask info into unit_mask
self.unit_mask = self.unit_mask * np.any(self.edge_mask, axis=0)
# NEW: update mask indices
self.edge_mask_indices = np.stack(self.edge_mask.nonzero()).T
self.edge_mask_indices = np.array(self.edge_mask_indices, dtype='int64')
self.unit_mask_indices = self.unit_mask.nonzero()[0]
# need to convert indices for bias to 2d array for sparse tensor
self.unit_mask_indices = np.stack([np.zeros(len(self.unit_mask_indices)),
self.unit_mask_indices]).T
self.unit_mask_indices = np.array(self.unit_mask_indices, dtype='int64')
# NEW: turn all new attributes into tensorflow constants
self.unit_mask = tf.constant(self.unit_mask)
self.edge_mask = tf.constant(self.edge_mask)
self.unit_mask_indices = tf.constant(self.unit_mask_indices)
self.edge_mask_indices = tf.constant(self.edge_mask_indices)
self.kernel = self.add_weight(
'kernel',
# NEW: trainable kernel weights may be fewer than before;
# they are now stored as 1D array,
shape=[int(np.sum(self.edge_mask))],
initializer=self.kernel_initializer,
regularizer=self.kernel_regularizer,
constraint=self.kernel_constraint,
dtype=self.dtype,
trainable=True)
if self.use_bias:
self.bias = self.add_weight(
'bias',
#NEW: trainable biases may be fewer than number of units
shape=[int(np.sum(self.unit_mask))],
initializer=self.bias_initializer,
regularizer=self.bias_regularizer,
constraint=self.bias_constraint,
dtype=self.dtype,
trainable=True)
else:
self.bias = None
self.built = True
def rebuild(self, edge_mask=None, unit_mask=None):
# NEW: This is a new function for rebuilding a DenseWithMask layer with a
# new edge mask and/or unit mask
# if none are given, get default values for masks from layer
if edge_mask is None:
edge_mask = self.edge_mask.numpy()
if unit_mask is None:
unit_mask = self.unit_mask.numpy()
# incorporate unit_mask info into edge_mask
edge_mask = edge_mask * unit_mask
# incorporate edge_mask info into unit_mask
unit_mask = unit_mask * np.any(edge_mask, axis=0)
# NOW: get new arrays of trainable weights for layer
# The stuff below contains slow, redundant lines but
# those might become handy when adding default values for
# new edges and nodes(?)
# create old kernel
kernel_old = np.zeros_like(self.edge_mask, dtype=float)
kernel_old[self.edge_mask.numpy()]=self.kernel.numpy()
# create new kernel
kernel_new = np.zeros_like(edge_mask, dtype=float)
for i in range(len(kernel_new)):
for j in range(len(kernel_new[0])):
if edge_mask[i,j]:
kernel_new[i,j] = kernel_old[i,j]
# create initializer for new kernel
vals = list(kernel_new[edge_mask])
initK = tf.compat.v1.keras.initializers.Constant(value=vals,
verify_shape=False)
# save new edge mask
self.edge_mask = edge_mask
# build new kernel
self.kernel = self.add_weight(
'kernel',
shape=[int(np.sum(self.edge_mask))],
initializer=initK,
regularizer=self.kernel_regularizer,
constraint=self.kernel_constraint,
dtype=self.dtype,
trainable=True)
# create old bias list
bias_old = np.zeros_like(self.unit_mask, dtype=float)
bias_old[self.unit_mask.numpy()] = self.bias.numpy()
# create new bias list
bias_new = np.zeros_like(unit_mask, dtype=float)
for i in range(len(bias_new)):
if unit_mask[i]:
bias_new[i] = bias_old[i]
# create initializer for new biases
vals = list(bias_new[unit_mask])
initB = tf.compat.v1.keras.initializers.Constant(value=vals,
verify_shape=False)
# save new unit mask
self.unit_mask = unit_mask
# build new biases
self.bias = self.add_weight(
'bias',
shape=[int(np.sum(self.unit_mask))],
initializer=initB,
regularizer=self.bias_regularizer,
constraint=self.bias_constraint,
dtype=self.dtype,
trainable=True)
# update mask indices
self.edge_mask_indices = np.stack(self.edge_mask.nonzero()).T
self.edge_mask_indices = np.array(self.edge_mask_indices, dtype='int64')
self.unit_mask_indices = self.unit_mask.nonzero()[0]
# need to convert indices for bias to 2d array for sparse tensor
self.unit_mask_indices = np.stack([np.zeros(len(self.unit_mask_indices)),
self.unit_mask_indices]).T
self.unit_mask_indices = np.array(self.unit_mask_indices, dtype='int64')
# turn all new attributes into tensorflow constants
self.unit_mask = tf.constant(self.unit_mask)
self.edge_mask = tf.constant(self.edge_mask)
self.unit_mask_indices = tf.constant(self.unit_mask_indices)
self.edge_mask_indices = tf.constant(self.edge_mask_indices)
def call(self, inputs):
# NEW: create a kernel (2D numpy array) from 1D list of trainable kernel weights
kernel = tf.SparseTensor(indices=self.edge_mask_indices,
values=self.kernel,
dense_shape=self.edge_mask.shape)
kernel = tf.sparse.to_dense(kernel)
# NEW: create a bias vector (1D numpy array) from 1D list of trainable biases
bias = tf.SparseTensor(indices=self.unit_mask_indices, values=self.bias,
dense_shape=[1,self.unit_mask.shape[0]])
bias = tf.squeeze(tf.sparse.to_dense(bias))
rank = inputs.shape.rank
if rank is not None and rank > 2:
# Broadcasting is required for the inputs.
outputs = standard_ops.tensordot(inputs, kernel, [[rank - 1], [0]]) # NEW: self.kernel -> kernel
# Reshape the output back to the original ndim of the input.
if not context.executing_eagerly():
shape = inputs.shape.as_list()
output_shape = shape[:-1] + [self.units]
outputs.set_shape(output_shape)
else:
inputs = math_ops.cast(inputs, self._compute_dtype)
if K.is_sparse(inputs):
outputs = sparse_ops.sparse_tensor_dense_matmul(inputs, kernel) # NEW: self.kernel -> kernel
else:
outputs = gen_math_ops.mat_mul(inputs, kernel) # NEW: self.kernel -> kernel
if self.use_bias:
outputs = nn.bias_add(outputs, bias) # NEW: self.bias -> bias
if self.activation is not None:
return self.activation(outputs)
return outputs
def compute_output_shape(self, input_shape):
input_shape = tensor_shape.TensorShape(input_shape)
input_shape = input_shape.with_rank_at_least(2)
if tensor_shape.dimension_value(input_shape[-1]) is None:
raise ValueError(
'The innermost dimension of input_shape must be defined, but saw: %s'
% input_shape)
return input_shape[:-1].concatenate(self.units)
edge_mask: tf.Tensor(
[[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]
...
[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]], shape=(784, 128), dtype=bool)
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
edge_mask of loaded model: tf.Tensor(
[[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]
...
[ True True True ... True True True]
[ True True True ... True True True]
[ True True True ... True True True]], shape=(784, 128), dtype=bool)