Python 可培训的多参数活动。函数（RBF）NeuPy/Theano

如何在Neupy或Theano中实现自定义激活函数（通过梯度下降调整均值和方差的RBF核），以便在Neupy中使用

{快速背景：梯度下降适用于网络中的每个参数。我想创建一个包含优化特征参数的专用特征空间，以便Neupy}

我认为我的问题在于参数的创建、它们的大小以及它们之间的连接方式

感兴趣的主要功能

激活函数类 径向基函数 功能塑造？

---

非常感谢您的帮助，因为这将为您提供有关如何定制neupy networks的深刻见解。文档可能需要在某些方面做一些工作，至少……

当层更改输入变量的形状时，它必须将更改通知后续层。对于这种情况，它必须具有自定义的

output\u shape

属性。例如：

from neupy import layers
from neupy.utils import as_tuple
import theano.tensor as T

class Flatten(layers.BaseLayer):
    """
    Slight modification of the Reshape layer from the neupy library:
    https://github.com/itdxer/neupy/blob/master/neupy/layers/reshape.py
    """
    @property 
    def output_shape(self):
        # Number of output feature depends on the input shape 
        # When layer receives input with shape (10, 3, 4)
        # than output will be (10, 12). First number 10 defines
        # number of samples which you typically don't need to
        # change during propagation
        n_output_features = np.prod(self.input_shape)
        return (n_output_features,)

    def output(self, input_value):
        n_samples = input_value.shape[0]
        return T.reshape(input_value, as_tuple(n_samples, self.output_shape))

如果您在终端中运行它，您将看到它工作正常

>>> network = layers.Input((3, 4)) > Flatten()
>>> predict = network.compile()
>>> predict(np.random.random((10, 3, 4))).shape
(10, 12)

在您的示例中，我可以看到一些问题：

rbf

函数不返回no表达式。它应该在函数编译期间失败

如果您不指定要计算范数的轴，像

np.linalg.norm这样的函数将返回标量
以下解决方案应该适合您

import numpy as np
from neupy import layers, init
import theano.tensor as T


def norm(value, axis=None):
    return T.sqrt(T.sum(T.square(value), axis=axis))


class RBF(layers.BaseLayer):
    def initialize(self):
        super(RBF, self).initialize()

        # It's more flexible when shape of the parameters
        # denend on the input shape
        self.add_parameter(
            name='mean', shape=self.input_shape,
            value=init.Constant(0.), trainable=True)

        self.add_parameter(
            name='std_dev', shape=self.input_shape,
            value=init.Constant(1.), trainable=True)

    def output(self, input_value):
        K = input_value - self.mean
        return T.exp(-norm(K, axis=0) / self.std_dev)


network = layers.Input(1) > RBF()
predict = network.compile()
print(predict(np.random.random((10, 1))))

network = layers.Input(4) > RBF()
predict = network.compile()
print(predict(np.random.random((10, 4))))

虽然itdxer充分回答了这个问题，但我想补充一下这个问题的确切解决方案

建筑创作
激活函数
径向基函数类
训练
如果你对培训感兴趣。就这么简单,

# Set training algorithm
gdnet = algorithms.Momentum(
    network,
    momenutm = 0.1
)

# Train. 
gdnet.train(x,y,max_iter=100)

通过适当的输入和目标进行编译，并在元素基础上更新均值和方差。
两项建议：（1）添加关于可培训性的评论/演示；（2）绘图总是有帮助的。但这非常有效，而且非常有指导意义。谢谢
>>> network = layers.Input((3, 4)) > Flatten()
>>> predict = network.compile()
>>> predict(np.random.random((10, 3, 4))).shape
(10, 12)

import numpy as np
from neupy import layers, init
import theano.tensor as T


def norm(value, axis=None):
    return T.sqrt(T.sum(T.square(value), axis=axis))


class RBF(layers.BaseLayer):
    def initialize(self):
        super(RBF, self).initialize()

        # It's more flexible when shape of the parameters
        # denend on the input shape
        self.add_parameter(
            name='mean', shape=self.input_shape,
            value=init.Constant(0.), trainable=True)

        self.add_parameter(
            name='std_dev', shape=self.input_shape,
            value=init.Constant(1.), trainable=True)

    def output(self, input_value):
        K = input_value - self.mean
        return T.exp(-norm(K, axis=0) / self.std_dev)


network = layers.Input(1) > RBF()
predict = network.compile()
print(predict(np.random.random((10, 1))))

network = layers.Input(4) > RBF()
predict = network.compile()
print(predict(np.random.random((10, 4))))

network = layers.Input(size) > RBF() > layers.Softmax(num_out)

    # Elementwise Gaussian (RBF)
    def rbf(value, mean, std):
        return T.exp(-.5*T.sqr(value-mean)/T.sqr(std))/(std*T.sqrt(2*np.pi))

    class RBF(layers.BaseLayer):
 
        def initialize(self):
 
            # Begin by initializing.
            super(RBF, self).initialize()
 
            # Add parameters to train
            self.add_parameter(name='means', shape=self.input_shape,
                           value=init.Normal(), trainable=True)
            self.add_parameter(name='std_dev', shape=self.input_shape,
                           value=init.Normal(), trainable=True)
 
        # Define output function for the RBF layer.
        def output(self, input_value):
            K = input_value - self.means
            return rbf(input_value,self.means,self.std_dev

# Set training algorithm
gdnet = algorithms.Momentum(
    network,
    momenutm = 0.1
)

# Train. 
gdnet.train(x,y,max_iter=100)