Python 如何根据自定义数据微调huggingface T5模型？

我有一个用于翻译的小文本数据集，我想用

t5 small

对其进行微调，下面是我试图用来微调的代码

import numpy as np
import tensorflow as tf
from transformers import TFT5ForConditionalGeneration, T5Tokenizer

model = TFT5ForConditionalGeneration.from_pretrained('t5-small')
tokenizer = T5Tokenizer.from_pretrained('t5-small')

def data_gen():
  for _ in range(256):
    x = np.random.randint(1,tokenizer.vocab_size, model.config.n_positions)
    attention = np.ones_like(x)
    yield ((x, attention), (x, attention))

output_type = ((tf.int32, tf.int32), (tf.int32, tf.int32)) 
ds = tf.data.Dataset.from_generator(data_gen, output_type).batch(2)

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=optimizer, loss=loss)

model.fit(ds, epochs=3, steps_per_epoch=128)

但是在运行这段代码时，我得到了以下错误

All model checkpoint layers were used when initializing TFT5ForConditionalGeneration.

All the layers of TFT5ForConditionalGeneration were initialized from the model checkpoint at t5-small.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFT5ForConditionalGeneration for predictions without further training.
Epoch 1/3
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-12c0ab7ab337> in <module>()
     19 model.compile(optimizer=optimizer, loss=loss)
     20 
---> 21 model.fit(ds, epochs=3, steps_per_epoch=128)

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    971           except Exception as e:  # pylint:disable=broad-except
    972             if hasattr(e, "ag_error_metadata"):
--> 973               raise e.ag_error_metadata.to_exception(e)
    974             else:
    975               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_t5.py:1285 call  *
        encoder_outputs = self.encoder(
    /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_t5.py:618 call  *
        input_shape = shape_list(input_ids)
    /usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py:1026 shape_list  *
        static = x.shape.as_list()
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_shape.py:1190 as_list  **
        raise ValueError("as_list() is not defined on an unknown TensorShape.")

    ValueError: as_list() is not defined on an unknown TensorShape

---

您能列出软件包

transformers

和

tensorflow

的版本吗？我个人认为这只是版本冲突，并且强烈建议升级Python版本！当我在colab上运行这些东西时，一切都是最新的。你能列出包的版本吗

transformers

和

tensorflow

？我个人认为这只是版本冲突，并且强烈建议升级Python版本！一切都是最新的，因为我在colab上运行这些东西。

Python 3.6.9
tensorflow==2.3.0
tensorflow-addons==0.8.3
tensorflow-datasets==4.0.1
tensorflow-estimator==2.3.0
tensorflow-gcs-config==2.3.0
tensorflow-hub==0.10.0
tensorflow-metadata==0.24.0
tensorflow-privacy==0.2.2
tensorflow-probability==0.11.0
transformers==3.5.0