Keras 基于输出值最大值和最小值的阈值线性层
我正在研究一种神经网络结构,它有一个线性层,如果它高于某个阈值,我需要该层的输出与输入相同,即Keras 基于输出值最大值和最小值的阈值线性层,keras,Keras,我正在研究一种神经网络结构,它有一个线性层,如果它高于某个阈值,我需要该层的输出与输入相同,即 a(x) = x if x >= threshold else a(x) = 0 if x < threshold 因此,我在keras中的稠密层之后使用ThresholdeDrulu层。阈值取决于致密层输出值的最大值和最小值: threshold = delta*min{s} + (1-delta)*max{s} where min{s} is the minimum of
a(x) = x if x >= threshold else a(x) = 0 if x < threshold
因此,我在keras中的稠密层之后使用ThresholdeDrulu层。阈值取决于致密层输出值的最大值和最小值:
threshold = delta*min{s} + (1-delta)*max{s}
where min{s} is the minimum of the 100 output values of the Dense layer
and max{s} is the maximum of the 100 output values of the Dense layer
and delta is a value between [0,1]
是否有一种方法可以获得最大值和最小值,在每次历元和批更新后计算阈值,从而获得阈值输出您可以在其中定义并使用后端函数。我会这样做:
from keras.layers import Dense, Lambda
from keras.models import Sequential
import keras.backend as K
import numpy as np
def thresholded_relu(x, delta):
threshold = delta * K.min(x, axis=-1) + (1 - delta) * K.max(x, axis=-1)
return K.cast((x > threshold[:, None]), dtype=K.dtype(x)) * x
delta = 0.5
model = Sequential()
# model.add(Dense(100, input_shape=(100,)))
model.add(Lambda(lambda x: thresholded_relu(x, delta), input_shape=(100,)))
model.compile('sgd', 'mse')
x = np.arange(0, 100, 1)[None, :]
pred = model.predict(x)
for y, p in zip(x[0], pred[0]):
print('Input: {}. Pred: {}'.format(y, p))
from keras.layers import Dense, Lambda
from keras.models import Sequential
import keras.backend as K
import numpy as np
def thresholded_relu(x, delta):
threshold = delta * K.min(x, axis=-1) + (1 - delta) * K.max(x, axis=-1)
return K.cast((x > threshold[:, None]), dtype=K.dtype(x)) * x
delta = 0.5
model = Sequential()
# model.add(Dense(100, input_shape=(100,)))
model.add(Lambda(lambda x: thresholded_relu(x, delta), input_shape=(100,)))
model.compile('sgd', 'mse')
x = np.arange(0, 100, 1)[None, :]
pred = model.predict(x)
for y, p in zip(x[0], pred[0]):
print('Input: {}. Pred: {}'.format(y, p))