如何通过tensorflow上的文本分类确定学习率？

我一直在用tensorflow keras编码情绪分析模型。 我使用的是csv数据集，第一行有标签（pos:1，neg:0），第二行有英文文本。 当我通过txt文件输入一些文本时，我期望的结果是显示0到1之间的数字。 然而，虽然我设置了模型，但损失率保持为负值，准确率没有增加，包括验证率。 我不知道是怎么回事。因此，我附上了我的代码。 多谢各位

import csv
import tensorflow as tf
import numpy as np
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

vocab_size = 20000
embedding_dim = 16
max_length = 120
trunc_type='post'
padding_type='post'
oov_tok = "<OOV>"
training_portion = .8

sentences = []
labels = []
stopwords = [ "a", "about", "above", "after", "again", "against", "all", "am", "an", "any", "are", 
"as", "at", "be", "because", "been", "before", "being", "below", "between", "both", "but", "by", 
"could", "did", "do", "does", "doing", "down", "during", "each", "few", "for", "from", "further", 
"had", "has", "have", "having", "he", "he'd", "he'll", "he's", "her", "here", "here's", "hers", 
"herself", "him", "himself", "his", "how", "how's", "i", "i'd", "i'll", "i'm", "i've", "if", "in", 
"into", "is", "it", "it's", "its", "itself", "let's", "me", "more", "most", "my", "myself", "nor", 
"of", "on", "once", "only", "or", "other", "ought", "our", "ours", "ourselves", "out", "over", "own", 
"same", "she", "she'd", "she'll", "she's", "should", "so", "some", "such", "than", "that", "that's", 
"the", "their", "theirs", "them", "themselves", "then", "there", "there's", "these", "they", 
"they'd", "they'll", "they're", "they've", "this", "those", "through", "to", "too", "under", "until", 
"up", "very", "was", "we", "we'd", "we'll", "we're", "we've", "were", "what", "what's", "when", 
"when's", "where", "where's", "which", "while", "who", "who's", "whom", "why", "why's", "with", 
"would", "you", "you'd", "you'll", "you're", "you've", "your", "yours", "yourself", "yourselves" ]
print(len(stopwords)

with open("/train.csv", 'r', encoding='latin1') as csvfile:
reader = csv.reader(csvfile, delimiter=',')
next(reader)
for row in reader:
    labels.append(row[0])
    sentence = row[1]
    for word in stopwords:
        token = " " + word + " "
        sentence = sentence.replace(token, " ")
    sentences.append(sentence)

train_size = int(len(sentences) * training_portion)

train_sentences = sentences[:train_size]
train_labels = labels[:train_size]

validation_sentences = sentences[train_size:]
validation_labels = labels[train_size:]

tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)
tokenizer.fit_on_texts(train_sentences)
word_index = tokenizer.word_index

train_sequences = tokenizer.texts_to_sequences(train_sentences)
train_padded = pad_sequences(train_sequences, padding=padding_type, maxlen=max_length)

validation_sequences = tokenizer.texts_to_sequences(validation_sentences)
validation_padded = pad_sequences(validation_sequences, padding=padding_type, maxlen=max_length)

label_tokenizer = Tokenizer()
label_tokenizer.fit_on_texts(labels)

training_label_seq = np.array(label_tokenizer.texts_to_sequences(train_labels))
validation_label_seq = np.array(label_tokenizer.texts_to_sequences(validation_labels))

model = tf.keras.Sequential([
tf.keras.layers.Embedding(vocab_size, 64),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(128,  return_sequences=True)),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),
tf.keras.layers.Dense(64, activation='relu'),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Dense(16, activation='relu'),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Dense(1)
])
model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
          optimizer=tf.keras.optimizers.Adam(1e-4),
          metrics=['accuracy'])
model.summary()

history = model.fit(train_padded, training_label_seq, epochs=6,
                validation_data=(validation_padded, validation_label_seq),
                validation_steps=30)

导入csv
导入tensorflow作为tf
将numpy作为np导入
来自tensorflow.keras.preprocessing.text导入标记器
从tensorflow.keras.preprocessing.sequence导入pad_序列
vocab_尺寸=20000
嵌入_dim=16
最大长度=120
trunc_type='post'
padding_type='post'
oov_tok=“”
培训部分=.8
句子=[]
标签=[]
stopwords=[“a”、“关于”、“以上”、“之后”、“再次”、“反对”、“全部”、“am”、“an”、“任何”、“是”，
“as”、“at”、“be”、“because”、“been”、“before”、“being”、“bewer”、“between”、“between”、“twitter”、“but”、“by”，
“可以”、“做过”、“做过”、“做过”、“做过”、“下来”、“在”、“每一次”、“很少”、“为了”、“从”、“进一步”，
“had”、“has”、“have”、“have”、“have”、“he”、“he'd”、“he'll”、“he's”、“her”、“here”、“heres”、“hers”，
“她自己”、“他”、“他自己”、“他的”、“怎么样”、“怎么样”、“我”、“我会”、“我是”、“我曾经”、“如果”、“在”，
“进入”、“是”、“它”、“它的”、“它的”、“自身”、“让我们”、“我”、“更多”、“大多数”、“我的”、“我自己”、“或”，
“of”，“on”，“once”，“only”，“or”，“other”，“should”，“ours”，“ours”，“your”，“out”，“over”，“own”，
“相同”、“她”、“她会”、“她会”、“她是”、“应该”、“所以”、“一些”、“这样”、“比”、“那个”、“那是”，
“他们的”，“他们的”，“他们的”，“他们”，“他们自己”，“然后”，“那里”，“那里”，“这些”，“他们”，
“他们会”，“他们会”，“他们是”，“他们曾经”，“这个”，“那些”，“通过”，“到”，“太”，“在”，“直到”，
“向上”、“非常”、“过去”、“我们”、“我们会”、“我们现在”、“我们曾经”、“过去”、“什么”、“什么”、“什么时候”，
“何时”、“何地”、“何地”、“何人”、“何人”、“何人”、“何人”、“何人”、“何人”、“何人”、“何人”、“何人”，
“愿意”、“你”、“你会”、“你会”、“你是”、“你曾经”、“你的”、“你的”、“你自己”、“你自己”]
打印（len（停止字）
将open（“/train.csv”，'r'，encoding='latin1'）作为csvfile：
reader=csv.reader（csvfile，分隔符='，'）
下一位（读者）
对于读取器中的行：
labels.append（第[0]行）
句子=行[1]
对于stopwords中的单词：
token=”“+单词+“”
句子=句子。替换（标记“”）
句子。附加（句子）
训练规模=整数（长度（句子）*训练部分）
训练句子=句子[：训练大小]
列车标签=标签[：列车尺寸]
验证句子=句子[训练大小：]
验证标签=标签[序列尺寸：]
标记器=标记器（num_words=vocab_size，oov_标记=oov_tok）
标记器。在文本上匹配（训练句子）
word\u index=tokenizer.word\u index
序列=标记器。文本到序列（序列句子）
填充序列=填充序列（填充序列=填充类型，最大长度=最大长度）
验证序列=标记器。文本到序列（验证句子）
验证\填充=填充\序列（验证\序列，填充=填充\类型，最大长度）
标签\标记器=标记器（）
标签标记器。在文本（标签）上匹配
training_label_seq=np.array（label_tokenizer.text_to_序列（train_标签））
验证\标签\顺序=np.数组（标签\标记器.文本\到\序列（验证\标签））
模型=tf.keras.Sequential([
tf.keras.layers.Embedding（vocab_尺寸，64），
双向（tf.keras.layers.LSTM（128，返回序列=True）），
双向（tf.keras.layers.LSTM（32）），
tf.keras.layers.Dense（64，活化='relu'），
tf.keras.层压降（0.5），
tf.keras.layers.Dense（16，活化='relu'），
tf.keras.层压降（0.5），
tf.keras.致密层（1）
])
compile（loss=tf.keras.loss.BinaryCrossentropy（from_logits=True），
优化器=tf.keras.optimizers.Adam（1e-4），
指标=[‘准确度’]）
model.summary（）
历史=型号.配合（带衬垫的列车，列车标签，历代=6，
验证\数据=（验证\填充，验证\标签\顺序），
验证（步骤=30）

---

问题在于，您也在标签上应用标记器，这将标签

0

和

1

转换为

1

和

2

，这会混淆分类器，因为tf.keras标记器word.index从索引1（而不是0）开始

下面是由于混淆分类器而导致负面损失的目标标签

label_tokenizer.word_index

{0'：1，'1'：2}

下面是样本数据的修改代码，该代码给出了预期的正损失值

import csv
import tensorflow as tf
import numpy as np
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
import pandas as pd 

vocab_size = 20000
embedding_dim = 16
max_length = 120
trunc_type='post'
padding_type='post'
oov_tok = "<OOV>"
training_portion = .8

sentences = []
labels = []
stopwords = [ "a", "about", "above", "after", "again", "against", "all", "am", "an", "any", "are", 
"as", "at", "be", "because", "been", "before", "being", "below", "between", "both", "but", "by", 
"could", "did", "do", "does", "doing", "down", "during", "each", "few", "for", "from", "further", 
"had", "has", "have", "having", "he", "he'd", "he'll", "he's", "her", "here", "here's", "hers", 
"herself", "him", "himself", "his", "how", "how's", "i", "i'd", "i'll", "i'm", "i've", "if", "in", 
"into", "is", "it", "it's", "its", "itself", "let's", "me", "more", "most", "my", "myself", "nor", 
"of", "on", "once", "only", "or", "other", "ought", "our", "ours", "ourselves", "out", "over", "own", 
"same", "she", "she'd", "she'll", "she's", "should", "so", "some", "such", "than", "that", "that's", 
"the", "their", "theirs", "them", "themselves", "then", "there", "there's", "these", "they", 
"they'd", "they'll", "they're", "they've", "this", "those", "through", "to", "too", "under", "until", 
"up", "very", "was", "we", "we'd", "we'll", "we're", "we've", "were", "what", "what's", "when", 
"when's", "where", "where's", "which", "while", "who", "who's", "whom", "why", "why's", "with", 
"would", "you", "you'd", "you'll", "you're", "you've", "your", "yours", "yourself", "yourselves" ]
print(len(stopwords)) 

with open("train.csv", 'r', encoding='latin1') as csvfile:
  reader = csv.reader(csvfile, delimiter=',')
  next(reader)
  for row in reader:
      labels.append(int(row[0]))
      sentence = row[1]
      for word in stopwords:
          token = " " + word + " "
          sentence = sentence.replace(token, " ")
      sentences.append(sentence) 

train_size = int(len(sentences) * training_portion)

train_sentences = sentences[:train_size]
train_labels = np.array(labels[:train_size])

validation_sentences = sentences[train_size:]
validation_labels = np.array(labels[train_size:])

tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)
tokenizer.fit_on_texts(train_sentences)
word_index = tokenizer.word_index

train_sequences = tokenizer.texts_to_sequences(train_sentences)
train_padded = pad_sequences(train_sequences, padding=padding_type, maxlen=max_length)

validation_sequences = tokenizer.texts_to_sequences(validation_sentences)
validation_padded = pad_sequences(validation_sequences, padding=padding_type, maxlen=max_length) 


model = tf.keras.Sequential([
tf.keras.layers.Embedding(vocab_size, 64),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(128,  return_sequences=True)),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),
tf.keras.layers.Dense(64, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(16, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(1,activation="sigmoid")
])
model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
          optimizer=tf.keras.optimizers.Adam(1e-4),
          metrics=['accuracy'])
model.summary()

history = model.fit(train_padded, train_labels, epochs=6,
                validation_data=(validation_padded, validation_labels),
                validation_steps=30)

---

你应该在最后一层中使用乙状激活，这将解决问题。我在最后一层中添加了乙状激活，即'tf.keras.layers.Dense（1）“.然而，尽管数字变低了，但仍然是负数，准确率保持在不低于0.5000。能否请您提供您使用的CSV数据集，以便我们可以检查和复制它。@Chu SeongYeub-如果您的问题通过上述答案得到解决，请接受并对答案进行投票。”。

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_2 (Embedding)      (None, None, 64)          1280000   
_________________________________________________________________
bidirectional_4 (Bidirection (None, None, 256)         197632    
_________________________________________________________________
bidirectional_5 (Bidirection (None, 64)                73984     
_________________________________________________________________
dense_6 (Dense)              (None, 64)                4160      
_________________________________________________________________
dropout_4 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_7 (Dense)              (None, 16)                1040      
_________________________________________________________________
dropout_5 (Dropout)          (None, 16)                0         
_________________________________________________________________
dense_8 (Dense)              (None, 1)                 17        
=================================================================
Total params: 1,556,833
Trainable params: 1,556,833
Non-trainable params: 0
_________________________________________________________________
Epoch 1/6
191/191 [==============================] - 71s 371ms/step - loss: 0.7135 - accuracy: 0.5783 - val_loss: 0.6934 - val_accuracy: 0.5345
Epoch 2/6
191/191 [==============================] - 71s 369ms/step - loss: 0.6943 - accuracy: 0.5793 - val_loss: 0.6932 - val_accuracy: 0.5345
Epoch 3/6
191/191 [==============================] - 71s 370ms/step - loss: 0.6936 - accuracy: 0.5791 - val_loss: 0.6932 - val_accuracy: 0.5345
Epoch 4/6
191/191 [==============================] - 70s 369ms/step - loss: 0.6934 - accuracy: 0.5793 - val_loss: 0.6932 - val_accuracy: 0.5345
Epoch 5/6
191/191 [==============================] - 71s 371ms/step - loss: 0.6934 - accuracy: 0.5793 - val_loss: 0.6932 - val_accuracy: 0.5345
Epoch 6/6
191/191 [==============================] - 72s 375ms/step - loss: 0.6933 - accuracy: 0.5793 - val_loss: 0.6931 - val_accuracy: 0.5345