Python 此错误的含义是什么以及如何对其进行故障排除?-ValueError:形状必须为秩2,但为秩3
我发现至少还有6个类似的问题。这里的主要问题是:Python 此错误的含义是什么以及如何对其进行故障排除?-ValueError:形状必须为秩2,但为秩3,python,tensorflow,Python,Tensorflow,我发现至少还有6个类似的问题。这里的主要问题是: 我使用的是哪一个数据集?我使用的是4,堆栈跟踪掩盖了它所看到的任何细节 为什么我的数据集排名是3而不是2 我正在使用tensorflow推荐程序。错误的堆栈跟踪太模糊,我无法说出它所抱怨的数据集。我猜这是我训练过的事件数据集,看起来像这样: @@@ dataset: cached_train_event_ds in create_model - 2 <_OptionsDataset shapes: {item_id: (None,), us
@@@ dataset: cached_train_event_ds in create_model - 2
<_OptionsDataset shapes: {item_id: (None,), user_id: (None,)}, types: {item_id: tf.string, user_id: tf.string}>
@@@ record type: <class 'tensorflow.python.data.ops.dataset_ops.TakeDataset'>
@@@ x type: <class 'dict'>
{'item_id': array([b'music:12274071', b'music:12501193', b'music:7864297', ...,
b'music:11953766', b'music:10805147', b'music:11953766'],
dtype=object),
'user_id': array([b'artist:15523352', b'artist:12930551', b'artist:31057444', ...,
b'artist:32581820', b'artist:36023938', b'artist:30037204'],
dtype=object)}
问题是项目数据集需要批处理两次:一次加载
make\u csv\u数据集
,然后再次加载items\u ds.batch(128.map)(项目模型)
错误消息令人困惑(对多个人来说,看起来像),人们总是被它绊倒
首先,很难判断错误是为哪个数据集生成的,其次,消息本身是神秘的。如果它至少提到了形状尺寸,这将为用户提供关于发生了什么的即时线索,而不是
[?,0],,?,?,?,?],[]
。(也无法在IDE中设置断点)。这实际上应该是对TF中产生异常的部分的增强请求,以使其更清晰、更方便用户。Code。产生这些错误的代码在哪里?请看如何创建一个。可能重复的:或@gobrewers。我可以提供一些代码,但要使其可复制将很困难。我正在从数据湖加载数据集。我可以尝试,但这里的主要思想是从tf人员那里了解错误的可能原因。您可以看到我的数据集的形状。那么,它被视为等级3而不是等级2的原因是什么呢?有可能数据不完整吗?还有别的原因吗?这就是我要找的。@rootlikegroot是的,它可能是第二个链接的副本。然而,它所说的是,也许你需要重新塑造你的数据集。我只是想知道,为什么我必须这么做?这个错误太隐晦了,在可用性方面肯定需要一些改进。第一个链接不相关,因为我没有使用双向RNN。我在这里添加了一个最小的可复制示例:。包括代码、输入数据和关于如何运行示例的详细自述。在我的例子中,在映射项模型时,我(错误地)添加了一个额外维度。解决了这个问题,谢谢!
Traceback (most recent call last):
File "/mnt/tmp/spark-23c1419e-4a5c-4ec7-a86f-1f6f23be73d3/recsys_tfrs_songs.py", line 90, in <module>
main(sys.argv)
File "/mnt/tmp/spark-23c1419e-4a5c-4ec7-a86f-1f6f23be73d3/recsys_tfrs_songs.py", line 61, in main
model_maker.train_and_evaluate(model, NUM_TRAIN_EPOCHS)
File "/mnt/tmp/spark-23c1419e-4a5c-4ec7-a86f-1f6f23be73d3/recsys-deps.zip/recommender_system/recsys_tf/recsys_tfrs_model.py", line 151, in train_and_evaluate
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1152, in fit
tmp_logs = self.train_function(iterator)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 867, in __call__
result = self._call(*args, **kwds)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 911, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 749, in _initialize
*args, **kwds))
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3045, in _get_concrete_function_internal_garbage_collected
graph_function, _ = self._maybe_define_function(args, kwargs)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3439, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3284, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 998, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 657, in wrapped_fn
out = weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 985, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:847 train_function *
return step_function(self, iterator)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow_recommenders/tasks/retrieval.py:157 call *
update_op = self._factorized_metrics.update_state(query_embeddings,
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow_recommenders/metrics/factorized_top_k.py:83 update_state *
top_k_predictions, _ = self._candidates(query_embeddings, k=self._k)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow_recommenders/layers/factorized_top_k.py:224 top_k *
joined_scores = tf.concat([state_scores, x_scores], axis=1)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:206 wrapper **
return target(*args, **kwargs)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:1768 concat
return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py:1208 concat_v2
"ConcatV2", values=values, axis=axis, name=name)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:750 _apply_op_helper
attrs=attr_protos, op_def=op_def)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:600 _create_op_internal
compute_device)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:3554 _create_op_internal
op_def=op_def)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:2031 __init__
control_input_ops, op_def)
/home/hadoop/.local/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:1872 _create_c_op
raise ValueError(str(e))
ValueError: Shape must be rank 2 but is rank 3 for '{{node concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32](args_0, args_2, concat/axis)' with input shapes: [?,0], [?,?,?], [].