Python 3.x ValueError:变量的形状不为';与检查点读取器中张量的形状不匹配

Python 3.x ValueError:变量的形状不为';与检查点读取器中张量的形状不匹配,python-3.x,tensorflow,machine-learning,pre-trained-model,Python 3.x,Tensorflow,Machine Learning,Pre Trained Model,我使用repo进行多标签分类,使用预训练的BERT模型。我有一个预训练的BERT模型,我想为我的分类问题进行微调。在运行提供的多标签分类脚本(run_bluebert_Multi_labels.py)时,我得到以下错误 ValueError: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embed

我使用repo进行多标签分类,使用预训练的BERT模型。我有一个预训练的BERT模型,我想为我的分类问题进行微调。在运行提供的多标签分类脚本(run_bluebert_Multi_labels.py)时,我得到以下错误

ValueError: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embeddings ([30522, 768]) from checkpoint reader.
以下是回溯:


WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:163: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f9756ef84d0>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Using config: {'_model_dir': './output', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f9756796310>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2), '_cluster': None}
INFO:tensorflow:_TPUContext: eval_on_tpu True
WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.
inside file based:
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:351: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:356: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

INFO:tensorflow:Writing example 0 of 20137
INFO:tensorflow:Writing example 10000 of 20137
INFO:tensorflow:Writing example 20000 of 20137
INFO:tensorflow:***** Running training *****
INFO:tensorflow:  Num examples = 20137
INFO:tensorflow:  Batch size = 32
INFO:tensorflow:  Num steps = 1887
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:389: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

WARNING:tensorflow:From /home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:425: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
WARNING:tensorflow:From /home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/contrib/data/python/ops/batching.py:273: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:398: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:405: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Running train on CPU
INFO:tensorflow:*** Features ***
INFO:tensorflow:  name = input_ids, shape = (32, 128)
INFO:tensorflow:  name = input_mask, shape = (32, 128)
INFO:tensorflow:  name = is_real_example, shape = (32,)
INFO:tensorflow:  name = label_ids, shape = (32, 19)
INFO:tensorflow:  name = segment_ids, shape = (32, 128)
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:172: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:411: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:492: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:359: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:673: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
WARNING:tensorflow:From /home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:546: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

ERROR:tensorflow:Error recorded from training_loop: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embeddings ([30522, 768]) from checkpoint reader.
INFO:tensorflow:training_loop marked as finished
WARNING:tensorflow:Reraising captured error
Traceback (most recent call last):
  File "bluebert/run_bluebert_multi_labels.py", line 876, in <module>
    main(args)
  File "bluebert/run_bluebert_multi_labels.py", line 781, in main
    estimator.train(input_fn=train_input_fn, max_steps=num_train_steps)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2876, in train
    rendezvous.raise_errors()
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 131, in raise_errors
    six.reraise(typ, value, traceback)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/six.py", line 696, in reraise
    raise value
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2871, in train
    saving_listeners=saving_listeners)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 367, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1158, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1188, in _train_model_default
    features, labels, ModeKeys.TRAIN, self.config)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2709, in _call_model_fn
    config)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1146, in _call_model_fn
    model_fn_results = self._model_fn(features=features, **kwargs)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2967, in _model_fn
    features, labels, is_export_mode=is_export_mode)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1549, in call_without_tpu
    return self._call_model_fn(features, labels, is_export_mode=is_export_mode)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1867, in _call_model_fn
    estimator_spec = self._model_fn(features=features, **kwargs)
  File "bluebert/run_bluebert_multi_labels.py", line 546, in model_fn
    tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 291, in init_from_checkpoint
    init_from_checkpoint_fn)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py", line 1684, in merge_call
    return self._merge_call(merge_fn, args, kwargs)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py", line 1691, in _merge_call
    return merge_fn(self._strategy, *args, **kwargs)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 286, in <lambda>
    ckpt_dir_or_file, assignment_map)
  File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 329, in _init_from_checkpoint
    tensor_name_in_ckpt, str(variable_map[tensor_name_in_ckpt])
ValueError: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embeddings ([30522, 768]) from checkpoint reader.

警告:tensorflow:From bluebert/run\u bluebert\u multi\u标签。py:163:不推荐使用名称tf.gfile.Open。请改用tf.io.gfile.gfile。
警告:tensorflow:Estimator的模型_fn()包含参数参数,但参数未传递给Estimator。
信息:tensorflow:使用配置:{“模型目录”:“./output”,“随机种子”:无”,“保存摘要步骤”:100,“保存检查点步骤”:1000,“保存检查点”:无,“会话配置”:允许软放置:true
图形选项{
重写选项{
元优化器迭代:一次
}
}
“保持检查点最大值”:5,“保持检查点每小时一次”:10000,“记录步骤计数步骤”:无,“训练分配”:无,“设备分配”:无,“协议分配”:无,“评估分配”:无,“实验分配”:无,“实验工工工工作延迟秒”:无,“服务分配”:无,“任务类型”:工作”,“任务id”:0,“任务分配”:0_全局\u id \u在\u集群中:0,'u master':'','u evaluation\u master':'','u是\u chief':True,'u num\u ps\u replications':0,'u num\u worker\u replications':1,'u tpu\u config':TPUConfig(迭代次数每循环=1000,碎片数=8,核心数每副本数=None,每主机输入训练=3,tpu作业名=None,初始输入睡眠秒=None,输入分区dims=None,评估训练输入配置=2),“\u集群”:None}
信息:tensorflow:\u TPUContext:eval\u on\u tpu True
警告:tensorflow:eval_on_tpu被忽略,因为use_tpu为False。
基于内部文件的:
警告:tensorflow:From bluebert/run\u bluebert\u multi\u标签。py:351:名称tf.python\u io.TFRecordWriter已被弃用。请改用tf.io.TFRecordWriter。
警告:tensorflow:From bluebert/run\u bluebert\u multi\u标签。py:356:名称tf.logging.info已被弃用。请改用tf.compat.v1.logging.info。
信息:tensorflow:编写20137的示例0
信息:tensorflow:编写20137的示例10000
信息:tensorflow:编写20137的示例20000
信息:tensorflow:****跑步训练*****
信息:tensorflow:Num示例=20137
信息:tensorflow:批量大小=32
信息:tensorflow:Num steps=1887
警告:tensorflow:来自bluebert/run\u bluebert\u multi\u标签。py:389:名称tf.FixedLenFeature已被弃用。请改用tf.io.FixedLenFeature。
警告:tensorflow:From/home/aditya/anaconda3/envs/RD/lib/python3.7/site packages/tensorflow/python/training/training_util.py:236:Variable.initialized_值(来自tensorflow.python.ops.variables)已弃用,将在未来版本中删除。
更新说明:
使用Variable.read_value。2.X中的变量在eager和graph(在tf.defun内)上下文中自动初始化。
警告:tensorflow:From bluebert/run\u bluebert\u multi\u labels.py:425:map\u和\u batch(From tensorflow.contrib.data.python.ops.batching)已弃用,将在未来版本中删除。
更新说明:
使用'tf.data.experimental.map_和_batch(…)`。
警告:tensorflow:From/home/aditya/anaconda3/envs/RD/lib/python3.7/site packages/tensorflow/contrib/data/python/ops/batching.py:273:map_和_batch(来自tensorflow.python.data.experimental.ops.batching)已被弃用,并将在未来版本中删除。
更新说明:
使用`tf.data.Dataset.map(map\u func,num\u parallel\u调用)`后跟`tf.data.Dataset.batch(batch\u size,drop\u remains)`。静态tf.data优化将使用融合实现。
警告:tensorflow:From bluebert/run\u bluebert\u multi\u labels.py:398:名称tf.parse\u single\u示例已弃用。请改用tf.io.parse\u single\u示例。
警告:tensorflow:From bluebert/run_bluebert_multi_labels.py:405:to_int32(From tensorflow.python.ops.math_ops)已弃用,将在未来版本中删除。
更新说明:
改用'tf.cast'。
信息:tensorflow:正在调用模型\u fn。
信息:tensorflow:在CPU上运行的列车
信息:tensorflow:**功能***
信息:tensorflow:name=input\u id,shape=(32128)
信息:tensorflow:name=input\u mask,shape=(32128)
信息:tensorflow:name=is\u real\u示例,shape=(32,)
信息:tensorflow:name=label_id,shape=(32,19)
信息:tensorflow:name=segment_id,shape=(32128)
警告:tensorflow:From/home/aditya/Projects/RD/ncbi\u bluebert/bert/modeling.py:172:已弃用名称tf.variable\u scope。请改用tf.compat.v1.variable\u scope。
警告:tensorflow:From/home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:411:名称tf.get_变量已被弃用。请改用tf.compat.v1.get_变量。
警告:tensorflow:From/home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:492:名称tf.assert_less_equal已被弃用。请改用tf.compat.v1.assert_less_equal。
警告:tensorflow:From/home/aditya/Projects/RD/ncbi\u bluebert/bert/modeling.py:359:使用keep\u prob调用退出(来自tensorflow.python.ops.nn\u ops)已被弃用,并将在未来版本中删除。
更新说明:
请使用'rate'而不是'keep_prob'。rate应设置为'rate=1-keep_prob'。
警告:tensorflow:From/home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:673:dense(来自tensorflow.python.layers.core)已弃用,将在未来版本中删除。
更新说明:
改用keras.layers.densed。
警告:tensorflow:From/home/aditya/anaconda3/envs/RD/lib/python3.7/site packages/tensorflow/python/ops/nn_impl.py:180:add_dispatch_support..wrapper(来自tensorflow.python.ops.array_ops)已弃用,将在未来版本中删除。
更新说明:
在2.0中使用tf.where,它与np.where具有相同的广播规则
警告:tensorflow:从bluebert/运行bluebert\u multi\u la