Tensorflow NLP用带CTC损耗的变压器
我试图理解tensorflow CTC损失，并将其用于NLP示例（） 我真的不明白发生了什么。当我使用Tensorflow NLP用带CTC损耗的变压器,tensorflow,nlp,Tensorflow,Nlp,我试图理解tensorflow CTC损失，并将其用于NLP示例（） 我真的不明白发生了什么。当我使用 with tf.GradientTape() as tape: predictions, _ = transformer(inp, tar_inp, True, enc_padding_mask, combi
with tf.GradientTape() as tape:
predictions, _ = transformer(inp, tar_inp,
True,
enc_padding_mask,
combined_mask,
dec_padding_mask)
loss = tf.reduce_mean(tf.nn.ctc_loss(tar_real, predictions, tar_inp.shape[0]))
> TypeError Traceback (most recent call
> last) <ipython-input-146-cb7fac4bee4b> in <module>
> 11 print('tar shape:', tar.shape, '\n', tar)
> 12
> ---> 13 train_step(inp, tar)
> 14
> 15 if batch % 50 == 0:
>
> <ipython-input-145-612c80b1a893> in train_step(inp, tar)
> 32 combined_mask,
> 33 dec_padding_mask)
> ---> 34 loss = tf.reduce_mean(tf.nn.ctc_loss(tar_real, predictions, tar_inp.shape[0]))
> 35
> 36
>
> TypeError: ctc_loss_v3() missing 1 required positional argument:
> 'logit_length'
> ValueError Traceback (most recent call
> last) <ipython-input-140-cb7fac4bee4b> in <module>
> 11 print('tar shape:', tar.shape, '\n', tar)
> 12
> ---> 13 train_step(inp, tar)
> 14
> 15 if batch % 50 == 0:
>
> <ipython-input-139-af0ccb4c431a> in train_step(inp, tar)
> 41 logit_length=logit_length,
> 42 logits_time_major=False,
> ---> 43 blank_index=None)
> 44 loss = tf.reduce_mean(loss)
> 45
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/ops/ctc_ops.py
> in ctc_loss_v3(labels, logits, label_length, logit_length,
> logits_time_major, unique, blank_index, name)
> 927 unique=unique,
> 928 blank_index=blank_index,
> --> 929 name=name)
> 930
> 931
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/ops/ctc_ops.py
> in ctc_loss_dense(labels, logits, label_length, logit_length,
> logits_time_major, unique, blank_index, name)
> 999 logits = ops.convert_to_tensor(logits, name="logits") 1000 labels = ops.convert_to_tensor(labels, name="labels")
> -> 1001 label_length = ops.convert_to_tensor(label_length, name="label_length") 1002 logit_length =
> ops.convert_to_tensor(logit_length, name="logit_length") 1003
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/ops.py
> in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype,
> dtype_hint, ctx, accepted_result_types) 1339 1340 if ret is
> None:
> -> 1341 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) 1342 1343 if ret is NotImplemented:
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in _constant_tensor_conversion_function(v, dtype, name, as_ref)
> 319 as_ref=False):
> 320 _ = as_ref
> --> 321 return constant(v, dtype=dtype, name=name)
> 322
> 323
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in constant(value, dtype, shape, name)
> 260 """
> 261 return _constant_impl(value, dtype, shape, name, verify_shape=False,
> --> 262 allow_broadcast=True)
> 263
> 264
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in _constant_impl(value, dtype, shape, name, verify_shape,
> allow_broadcast)
> 268 ctx = context.context()
> 269 if ctx.executing_eagerly():
> --> 270 t = convert_to_eager_tensor(value, ctx, dtype)
> 271 if shape is None:
> 272 return t
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in convert_to_eager_tensor(value, ctx, dtype)
> 94 dtype = dtypes.as_dtype(dtype).as_datatype_enum
> 95 ctx.ensure_initialized()
> ---> 96 return ops.EagerTensor(value, ctx.device_name, dtype)
> 97
> 98
>
> ValueError: Attempt to convert a value (None) with an unsupported type
> (<class 'NoneType'>) to a Tensor.
> TypeError Traceback (most recent call
> last) <ipython-input-146-cb7fac4bee4b> in <module>
> 11 print('tar shape:', tar.shape, '\n', tar)
> 12
> ---> 13 train_step(inp, tar)
> 14
> 15 if batch % 50 == 0:
>
> <ipython-input-145-612c80b1a893> in train_step(inp, tar)
> 32 combined_mask,
> 33 dec_padding_mask)
> ---> 34 loss = tf.reduce_mean(tf.nn.ctc_loss(tar_real, predictions, tar_inp.shape[0]))
> 35
> 36
>
> TypeError: ctc_loss_v3() missing 1 required positional argument:
> 'logit_length'
> ValueError Traceback (most recent call
> last) <ipython-input-140-cb7fac4bee4b> in <module>
> 11 print('tar shape:', tar.shape, '\n', tar)
> 12
> ---> 13 train_step(inp, tar)
> 14
> 15 if batch % 50 == 0:
>
> <ipython-input-139-af0ccb4c431a> in train_step(inp, tar)
> 41 logit_length=logit_length,
> 42 logits_time_major=False,
> ---> 43 blank_index=None)
> 44 loss = tf.reduce_mean(loss)
> 45
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/ops/ctc_ops.py
> in ctc_loss_v3(labels, logits, label_length, logit_length,
> logits_time_major, unique, blank_index, name)
> 927 unique=unique,
> 928 blank_index=blank_index,
> --> 929 name=name)
> 930
> 931
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/ops/ctc_ops.py
> in ctc_loss_dense(labels, logits, label_length, logit_length,
> logits_time_major, unique, blank_index, name)
> 999 logits = ops.convert_to_tensor(logits, name="logits") 1000 labels = ops.convert_to_tensor(labels, name="labels")
> -> 1001 label_length = ops.convert_to_tensor(label_length, name="label_length") 1002 logit_length =
> ops.convert_to_tensor(logit_length, name="logit_length") 1003
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/ops.py
> in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype,
> dtype_hint, ctx, accepted_result_types) 1339 1340 if ret is
> None:
> -> 1341 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) 1342 1343 if ret is NotImplemented:
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in _constant_tensor_conversion_function(v, dtype, name, as_ref)
> 319 as_ref=False):
> 320 _ = as_ref
> --> 321 return constant(v, dtype=dtype, name=name)
> 322
> 323
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in constant(value, dtype, shape, name)
> 260 """
> 261 return _constant_impl(value, dtype, shape, name, verify_shape=False,
> --> 262 allow_broadcast=True)
> 263
> 264
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in _constant_impl(value, dtype, shape, name, verify_shape,
> allow_broadcast)
> 268 ctx = context.context()
> 269 if ctx.executing_eagerly():
> --> 270 t = convert_to_eager_tensor(value, ctx, dtype)
> 271 if shape is None:
> 272 return t
>
> /opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
> in convert_to_eager_tensor(value, ctx, dtype)
> 94 dtype = dtypes.as_dtype(dtype).as_datatype_enum
> 95 ctx.ensure_initialized()
> ---> 96 return ops.EagerTensor(value, ctx.device_name, dtype)
> 97
> 98
>
> ValueError: Attempt to convert a value (None) with an unsupported type
> (<class 'NoneType'>) to a Tensor.
>ValueError回溯（最近的调用
>最后）在
>11打印（'tar形状：'，tar.shape'，\n'，tar）
> 12
>-->13列车步进（inp、tar）
> 14
>15如果批次%50==0：
>
>列车内步骤（inp、tar）
>41 logit_长度=logit_长度，
>42 logits\u time\u major=False，
>-->43空白（索引=无）
>44损失=tf.减少平均值（损失）
> 45
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/ops/ctc_ops.py
>在ctc_loss_v3中（标签、登录、标签长度、登录长度、，
>登录\u时间\u主要、唯一、空白\u索引、名称）
>927唯一=唯一，
>928空白索引=空白索引，
>-->929名称=名称）
> 930
> 931
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/ops/ctc_ops.py
>在ctc中（标签、登录、标签长度、登录长度、，
>登录\u时间\u主要、唯一、空白\u索引、名称）
>999 logits=ops.convert_to_tensor（logits，name=“logits”）1000 labels=ops.convert_to_tensor（labels，name=“labels”）
>->1001 label\u length=ops。将\u转换为\u张量（label\u length，name=“label\u length”）1002 logit\u length=
>运算。将\u转换为\u张量（logit\u length，name=“logit\u length”）1003
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/ops.py
>在转换为张量时（值、数据类型、名称、as ref、首选数据类型、，
>数据类型\提示、ctx、已接受\结果\类型）1339 1340（如果ret为
>无：
>->1341 ret=conversion\u func（值，dtype=dtype，name=name，as\u ref=as\u ref）1342 1343如果未实施ret：
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
>in _常量_张量_转换_函数（v，数据类型，名称，as_ref）
>319 as_ref=False）：
>320=作为参考
>-->321返回常量（v，dtype=dtype，name=name）
> 322
> 323
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
>常量（值、数据类型、形状、名称）
> 260 """
>261返回\u常量\u impl（值、数据类型、形状、名称、验证\u形状=False，
>-->262允许_广播=真）
> 263
> 264
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
>在_constant_impl（值、数据类型、形状、名称、验证形状、，
>允许广播）
>268 ctx=context.context（）
>269如果ctx.executing_急切地（）
>-->270 t=转换为张量（值、ctx、数据类型）
>271如果形状为“无”：
>272返回t
>
>/opt/anaconda3/envs/MyTransformers_py37/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py
>在转换为张量时（值、ctx、数据类型）
>94 dtype=dtypes.as\u dtype（dtype）.as\u datatype\u enum
>95 ctx.确保_已初始化（）
>-->96返回运算器（值，ctx.device\u名称，数据类型）
> 97
> 98
>
>ValueError:尝试转换类型不受支持的值（无）
>（）到张量。