Question

我正在使用LSTM架构创建聊天机器人。我正在使用GloVe嵌入。在我的训练过程中，我的训练准确性停留在非常低的值（0.1969），并且没有进度。我在下面附上我的代码。你能告诉我如何做才能改善训练吗？

from keras.models import Sequential
from keras.layers import Embedding, Flatten, Dense, LSTM
from keras.optimizers import Adam


model=Sequential()
model.add(Embedding(max_words,embedding_dim,input_length=maxlen))
model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid'))
model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid'))
model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid'))
model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid'))
model.summary()

model.layers[0].set_weights([embedding_matrix])
model.layers[0].trainable = False

model.compile(loss='cosine_proximity', optimizer='adam', metrics=['accuracy'])

model.fit(x_train, y_train,
epochs = 500,
batch_size = 32,
validation_data=(x_val,y_val))

Epoch 498/500
60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909
Epoch 499/500
60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909
Epoch 500/500
60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909

对同一会话数据集进行进一步的训练不会提高准确性。

Answer 1

在嵌入和LSTM之后也添加BatchNormalization层。这有助于规范学习。我添加了它，对我有帮助。另外，查看数据。也可能有一些问题。

#clear session
keras.backend.clear_session()
model = Sequential()
#embedding layer
model.add(Embedding(vocab_size, 50, input_length=pad_length))

#Batch Norm layer
model.add(BatchNormalization())

#First LSTM layer
model.add(LSTM(units=100, return_sequences=True))

#Batch Norm layer
model.add(BatchNormalization())
#add dropout too
model.add(Dropout(0.25))

#Second LSTM layer
model.add(LSTM(units=100))

#Batch Norm
model.add(BatchNormalization())

model.add(Dense(1, activation="sigmoid"))

Keras LSTM训练精度在NLP任务中处于较低水平

1 个答案: