Python 3.x Numpy和tensorflow RNN形状表示不匹配

我正在tensorflow建立我的第一个RNN。在理解了有关3D输入形状的所有概念后，我遇到了这个问题

在我的numpy版本（1.15.4）中，三维阵列的形状表示如下：

（面板、行、列）

。我将使每个维度不同，以便更清晰：

In [1]: import numpy as np                                                                                                                  

In [2]: arr = np.arange(30).reshape((2,3,5))                                                                                                

In [3]: arr                                                                                                                                 
Out[3]: 
array([[[ 0,  1,  2,  3,  4],
        [ 5,  6,  7,  8,  9],
        [10, 11, 12, 13, 14]],

       [[15, 16, 17, 18, 19],
        [20, 21, 22, 23, 24],
        [25, 26, 27, 28, 29]]])

In [4]: arr.shape                                                                                                                           
Out[4]: (2, 3, 5)

In [5]: np.__version__                                                                                                                      
Out[5]: '1.15.4'

我的理解是：我有两个时间步，每个时间步有3个观察值，每个观察值有5个特征。

然而，在tensorflow“理论”（我相信它在numpy中有很强的基础）中，RNN单元期望形状

[批次大小、时间步长、特征]

的张量（即仅n维矩阵），在numpy“行话”中可以翻译为：

（行、面板、列）

可以看出，表示不匹配，导致将numpy数据馈送到占位符时出错，在大多数示例和理论中，占位符的定义如下：

x=tf.placeholder（tf.float32，shape=[None，N\u TIMESTEPS\u x，N\u FEATURES]，name='XPlaceholder'）

• np.reformate（）

  并不能解决这个问题，因为它只是重新排列维度，但会弄乱数据

• 我第一次使用了数据集API，但在会话中遇到了问题，而不是在数据集API ops中

• 我使用的是

  static\u rnn

  方法，在我必须将数据输入占位符之前，一切都很正常，这显然会导致形状错误

• 我已尝试将占位符形状更改为

  shape=[N\u TIMESTEPS\u X，None，N\u FEATURES]

  。但是，我使用的是dataset API，如果我将Xplaceholder更改为

  shape=[N\u TIMESTEPS\u X，None，N\u FEATURES]

  ，则在创建初始值设定项时会出错

因此，总结一下：

• 第一个问题：具有不同形状表示的形状错误
• 第二个问题：对形状表示进行等值时出现数据集错误（我认为，如果解决了这个问题，静态或动态表示都会起作用）

我的问题是：

'''The idea is to create xt, yt, xval and yval. My numpy arrays to 
be fed are of the following shapes: 

The 3D xt array has a shape of: (11, 69579, 74)
The 3D xval array has a shape of: (11, 7732, 74)

The yt array has a shape of: (69579, 3)
The yval array has a shape of: (7732, 3)

'''

N_TIMESTEPS_X = xt.shape[0] ## The stack number
BATCH_SIZE = 256
#N_OBSERVATIONS = xt.shape[1]
N_FEATURES = xt.shape[2]
N_OUTPUTS = yt.shape[1]
N_NEURONS_LSTM = 128 ## Number of units in the LSTMCell 
N_NEURONS_DENSE = 64 ## Number of units in the Dense layer
N_EPOCHS = 600
LEARNING_RATE = 0.1

### Define the placeholders anda gather the data.
train_data = (xt, yt)
validation_data = (xval, yval)

## We define the placeholders as a trick so that we do not break into memory problems, associated with feeding the data directly.
'''As an alternative, you can define the Dataset in terms of tf.placeholder() tensors, and feed the NumPy arrays when you initialize an Iterator over the dataset.'''
batch_size = tf.placeholder(tf.int64)
x = tf.placeholder(tf.float32, shape=[None, N_TIMESTEPS_X, N_FEATURES], name='XPlaceholder')
y = tf.placeholder(tf.float32, shape=[None, N_OUTPUTS], name='YPlaceholder')

# Creating the two different dataset objects.
train_dataset = tf.data.Dataset.from_tensor_slices((x,y)).batch(BATCH_SIZE).repeat()
val_dataset = tf.data.Dataset.from_tensor_slices((x,y)).batch(BATCH_SIZE)

# Creating the Iterator type that permits to switch between datasets.
itr = tf.data.Iterator.from_structure(train_dataset.output_types, train_dataset.output_shapes)
train_init_op = itr.make_initializer(train_dataset)
validation_init_op = itr.make_initializer(val_dataset)

next_features, next_labels = itr.get_next()

### Create the graph 
cellType = tf.nn.rnn_cell.LSTMCell(num_units=N_NEURONS_LSTM, name='LSTMCell')
inputs = tf.unstack(next_features, N_TIMESTEPS_X, axis=0)
'''inputs: A length T list of inputs, each a Tensor of shape [batch_size, input_size]'''
RNNOutputs, _ = tf.nn.static_rnn(cell=cellType, inputs=inputs, dtype=tf.float32)
predictionsLayer = tf.layers.dense(inputs=tf.layers.batch_normalization(RNNOutputs[-1]), units=N_NEURONS_DENSE, activation=None, name='Dense_Layer')

### Define the cost function, that will be optimized by the optimizer. 
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=predictionsLayer, labels=next_labels, name='Softmax_plus_Cross_Entropy'))
optimizer_type = tf.train.AdamOptimizer(learning_rate=LEARNING_RATE, name='AdamOptimizer')
optimizer = optimizer_type.minimize(cost)

### Model evaluation 
correctPrediction = tf.equal(tf.argmax(predictionsLayer,1), tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correctPrediction,tf.float32))
#confusionMatrix = tf.confusion_matrix(next_labels, predictionsLayer, num_classes=3, name='ConfMatrix')
N_BATCHES = train_data[0].shape[0] // BATCH_SIZE

## Saving variables so that we can restore them afterwards.
saver = tf.train.Saver()
save_dir = '/home/zmlaptop/Desktop/tfModels/{}_{}'.format(cellType.__class__.__name__, datetime.now().strftime("%Y%m%d%H%M%S"))
os.mkdir(save_dir)
varDict = {'nTimeSteps':N_TIMESTEPS_X, 'BatchSize': BATCH_SIZE, 'nFeatures':N_FEATURES,
           'nNeuronsLSTM':N_NEURONS_LSTM, 'nNeuronsDense':N_NEURONS_DENSE, 'nEpochs':N_EPOCHS,
           'learningRate':LEARNING_RATE, 'optimizerType': optimizer_type.__class__.__name__}
varDicSavingTxt = save_dir + '/varDict.txt'
modelFilesDir = save_dir + '/modelFiles'
os.mkdir(modelFilesDir)

logDir = save_dir + '/TBoardLogs'
os.mkdir(logDir)

acc_summary = tf.summary.scalar('Accuracy', accuracy)
loss_summary = tf.summary.scalar('Cost_CrossEntropy', cost)
summary_merged = tf.summary.merge_all()

with open(varDicSavingTxt, 'w') as outfile:
    outfile.write(repr(varDict))

with tf.Session() as sess:

    tf.set_random_seed(2)
    sess.run(tf.global_variables_initializer())
    train_writer = tf.summary.FileWriter(logDir + '/train', sess.graph)
    validation_writer = tf.summary.FileWriter(logDir + '/validation')

    # initialise iterator with train data
    sess.run(train_init_op, feed_dict = {x : train_data[0], y: train_data[1], batch_size: BATCH_SIZE})

    print('¡Training starts!')
    for epoch in range(N_EPOCHS):

        batchAccList = []
        tot_loss = 0

        for batch in range(N_BATCHES):

            optimizer_output, loss_value, summary = sess.run([optimizer, cost, summary_merged])
            accBatch = sess.run(accuracy)
            tot_loss += loss_value
            batchAccList.append(accBatch)

            if batch % 10 == 0:

                train_writer.add_summary(summary, batch)

        epochAcc = tf.reduce_mean(batchAccList)

        if epoch%10 == 0:

            print("Epoch: {}, Loss: {:.4f}, Accuracy: {}".format(epoch, tot_loss / N_BATCHES, epochAcc))

    #confM = sess.run(confusionMatrix)
    #confDic = {'confMatrix': confM}
    #confTxt = save_dir + '/confMDict.txt'
    #with open(confTxt, 'w') as outfile:
    #    outfile.write(repr(confDic))
    #print(confM)

    # initialise iterator with validation data
    sess.run(validation_init_op, feed_dict = {x : validation_data[0], y: validation_data[1], batch_size:len(validation_data[0])})
    print('Validation Loss: {:4f}, Validation Accuracy: {}'.format(sess.run(cost), sess.run(accuracy)))
    summary_val = sess.run(summary_merged)
    validation_writer.add_summary(summary_val)

    saver.save(sess, modelFilesDir)

在这种不同的表达逻辑方面，我有什么遗漏吗？这会让实践变得混乱吗

？切换到动态模式是否能解决问题？（虽然我遇到的关于形状的问题与数据集API初始值设定项与形状[N_TIMESTEPS_X，None，N_FEATURES]有关，但与RNN单元格本身无关

非常感谢您抽出时间

完整代码：

'''The idea is to create xt, yt, xval and yval. My numpy arrays to 
be fed are of the following shapes: 

The 3D xt array has a shape of: (11, 69579, 74)
The 3D xval array has a shape of: (11, 7732, 74)

The yt array has a shape of: (69579, 3)
The yval array has a shape of: (7732, 3)

'''

N_TIMESTEPS_X = xt.shape[0] ## The stack number
BATCH_SIZE = 256
#N_OBSERVATIONS = xt.shape[1]
N_FEATURES = xt.shape[2]
N_OUTPUTS = yt.shape[1]
N_NEURONS_LSTM = 128 ## Number of units in the LSTMCell 
N_NEURONS_DENSE = 64 ## Number of units in the Dense layer
N_EPOCHS = 600
LEARNING_RATE = 0.1

### Define the placeholders anda gather the data.
train_data = (xt, yt)
validation_data = (xval, yval)

## We define the placeholders as a trick so that we do not break into memory problems, associated with feeding the data directly.
'''As an alternative, you can define the Dataset in terms of tf.placeholder() tensors, and feed the NumPy arrays when you initialize an Iterator over the dataset.'''
batch_size = tf.placeholder(tf.int64)
x = tf.placeholder(tf.float32, shape=[None, N_TIMESTEPS_X, N_FEATURES], name='XPlaceholder')
y = tf.placeholder(tf.float32, shape=[None, N_OUTPUTS], name='YPlaceholder')

# Creating the two different dataset objects.
train_dataset = tf.data.Dataset.from_tensor_slices((x,y)).batch(BATCH_SIZE).repeat()
val_dataset = tf.data.Dataset.from_tensor_slices((x,y)).batch(BATCH_SIZE)

# Creating the Iterator type that permits to switch between datasets.
itr = tf.data.Iterator.from_structure(train_dataset.output_types, train_dataset.output_shapes)
train_init_op = itr.make_initializer(train_dataset)
validation_init_op = itr.make_initializer(val_dataset)

next_features, next_labels = itr.get_next()

### Create the graph 
cellType = tf.nn.rnn_cell.LSTMCell(num_units=N_NEURONS_LSTM, name='LSTMCell')
inputs = tf.unstack(next_features, N_TIMESTEPS_X, axis=0)
'''inputs: A length T list of inputs, each a Tensor of shape [batch_size, input_size]'''
RNNOutputs, _ = tf.nn.static_rnn(cell=cellType, inputs=inputs, dtype=tf.float32)
predictionsLayer = tf.layers.dense(inputs=tf.layers.batch_normalization(RNNOutputs[-1]), units=N_NEURONS_DENSE, activation=None, name='Dense_Layer')

### Define the cost function, that will be optimized by the optimizer. 
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=predictionsLayer, labels=next_labels, name='Softmax_plus_Cross_Entropy'))
optimizer_type = tf.train.AdamOptimizer(learning_rate=LEARNING_RATE, name='AdamOptimizer')
optimizer = optimizer_type.minimize(cost)

### Model evaluation 
correctPrediction = tf.equal(tf.argmax(predictionsLayer,1), tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correctPrediction,tf.float32))
#confusionMatrix = tf.confusion_matrix(next_labels, predictionsLayer, num_classes=3, name='ConfMatrix')
N_BATCHES = train_data[0].shape[0] // BATCH_SIZE

## Saving variables so that we can restore them afterwards.
saver = tf.train.Saver()
save_dir = '/home/zmlaptop/Desktop/tfModels/{}_{}'.format(cellType.__class__.__name__, datetime.now().strftime("%Y%m%d%H%M%S"))
os.mkdir(save_dir)
varDict = {'nTimeSteps':N_TIMESTEPS_X, 'BatchSize': BATCH_SIZE, 'nFeatures':N_FEATURES,
           'nNeuronsLSTM':N_NEURONS_LSTM, 'nNeuronsDense':N_NEURONS_DENSE, 'nEpochs':N_EPOCHS,
           'learningRate':LEARNING_RATE, 'optimizerType': optimizer_type.__class__.__name__}
varDicSavingTxt = save_dir + '/varDict.txt'
modelFilesDir = save_dir + '/modelFiles'
os.mkdir(modelFilesDir)

logDir = save_dir + '/TBoardLogs'
os.mkdir(logDir)

acc_summary = tf.summary.scalar('Accuracy', accuracy)
loss_summary = tf.summary.scalar('Cost_CrossEntropy', cost)
summary_merged = tf.summary.merge_all()

with open(varDicSavingTxt, 'w') as outfile:
    outfile.write(repr(varDict))

with tf.Session() as sess:

    tf.set_random_seed(2)
    sess.run(tf.global_variables_initializer())
    train_writer = tf.summary.FileWriter(logDir + '/train', sess.graph)
    validation_writer = tf.summary.FileWriter(logDir + '/validation')

    # initialise iterator with train data
    sess.run(train_init_op, feed_dict = {x : train_data[0], y: train_data[1], batch_size: BATCH_SIZE})

    print('¡Training starts!')
    for epoch in range(N_EPOCHS):

        batchAccList = []
        tot_loss = 0

        for batch in range(N_BATCHES):

            optimizer_output, loss_value, summary = sess.run([optimizer, cost, summary_merged])
            accBatch = sess.run(accuracy)
            tot_loss += loss_value
            batchAccList.append(accBatch)

            if batch % 10 == 0:

                train_writer.add_summary(summary, batch)

        epochAcc = tf.reduce_mean(batchAccList)

        if epoch%10 == 0:

            print("Epoch: {}, Loss: {:.4f}, Accuracy: {}".format(epoch, tot_loss / N_BATCHES, epochAcc))

    #confM = sess.run(confusionMatrix)
    #confDic = {'confMatrix': confM}
    #confTxt = save_dir + '/confMDict.txt'
    #with open(confTxt, 'w') as outfile:
    #    outfile.write(repr(confDic))
    #print(confM)

    # initialise iterator with validation data
    sess.run(validation_init_op, feed_dict = {x : validation_data[0], y: validation_data[1], batch_size:len(validation_data[0])})
    print('Validation Loss: {:4f}, Validation Accuracy: {}'.format(sess.run(cost), sess.run(accuracy)))
    summary_val = sess.run(summary_merged)
    validation_writer.add_summary(summary_val)

    saver.save(sess, modelFilesDir)

在这方面我有什么遗漏吗 是什么让实践变得混乱

事实上，

static\u rnn

和

dynamic\u rnn

的输入形状是错误的。

static\u rnn

的输入形状是

[timesteps，batch\u size，features]

（），这是形状[batch\u size，features]的二维张量列表。

dynamic\u rnn

的输入形状是

[timesteps，batch\u size，features]

或

[batch\u size，timesteps，features]

取决于

time\u major

是否为真（）

切换到动态网络是否可以解决问题

关键不在于您使用的是

static\u rnn

或

dynamic\u rnn

，而是您的数据形状与所需的形状匹配。占位符的一般格式类似于您的代码是

[None，N\u TIMESTEPS\u X，N\u FEATURES]

。使用dataset API也很方便。 您可以使用

transpose（）

（）而不是

reformate（）

，

transpose（）

将排列数组的维度，并且不会弄乱数据

所以你的代码需要修改

# permute the dimensions
xt = xt.transpose([1,0,2])
xval = xval.transpose([1,0,2])

# adjust shape,axis=1 represents timesteps
inputs = tf.unstack(next_features,  axis=1)

其他错误应该与rnn形状无关

在这方面我有什么遗漏吗 是什么让实践变得混乱

事实上，

static\u rnn

和

dynamic\u rnn

的输入形状是错误的。

static\u rnn

的输入形状是

[timesteps，batch\u size，features]

（），这是形状[batch\u size，features]的二维张量列表。

dynamic\u rnn

的输入形状是

[timesteps，batch\u size，features]

或

[batch\u size，timesteps，features]

取决于

time\u major

是否为真（）

切换到动态网络是否可以解决问题

关键不在于您使用的是

static\u rnn

或

dynamic\u rnn

，而是您的数据形状与所需的形状匹配。占位符的一般格式类似于您的代码是

[None，N\u TIMESTEPS\u X，N\u FEATURES]

。使用dataset API也很方便。 您可以使用

transpose（）

（）而不是

reformate（）

，

transpose（）

将排列数组的维度，并且不会弄乱数据

所以你的代码需要修改

# permute the dimensions
xt = xt.transpose([1,0,2])
xval = xval.transpose([1,0,2])

# adjust shape,axis=1 represents timesteps
inputs = tf.unstack(next_features,  axis=1)

---

其他错误应该与rnn形状无关。

非常感谢您的回答和时间@giser\u yugang.

tf.transpose（）

是我缺少的方法，同时在unstack中更改轴。但是，我认为您的回答中有一个关于

静态\u rnn

输入形状的拼写错误；它不是真正的3D输入，而是形状的2D张量列表[批量大小，特征]。列表的长度是时间步数。你能编辑一下吗？这样其他人就不会感到困惑了？我完全理解你的观点，但我认为这样做更清楚，就像在文档中一样。再次感谢你，回答正确！关于我上面的代码，运行到

优化器输出，loss\u value，summary=sess.run([optimizer，cost，summary_merged]）

。在那里，我遇到了一个Yplaceholder错误：InvalidArgumentError（se）