将elmo嵌入与keras一起使用时，训练损失和验证损失不会减少

Question

我正在使用带有keras的elmo嵌入构建LSTM网络。我的目标是最小化RMSE。 elmo嵌入是使用以下代码段获得的：

def ElmoEmbedding(x):
    return elmo_model(inputs={
                            "tokens": tf.squeeze(tf.cast(x, tf.string)),
                            "sequence_len": tf.constant(batch_size*[max_len])
                      },
                      signature="tokens",
                      as_dict=True)["elmo"]

模型定义如下：

def create_model(max_len):
    input_text = Input(shape=(max_len,), dtype=tf.string)
    embedding = Lambda(ElmoEmbedding, output_shape=(max_len, 1024))(input_text)
    x = Bidirectional(LSTM(units=512, return_sequences=False,
                   recurrent_dropout=0.2, dropout=0.2))(embedding)
    out = Dense(1, activation = "relu")(x)

    model = Model(input_text, out)

    return model

模型编译为：

model.compile(optimizer = "rmsprop", loss = root_mean_squared_error,
          metrics =[root_mean_squared_error])

然后训练为：

model.fit(np.array(X_tr), y_tr, validation_data=(np.array(X_val), y_val),
                batch_size=batch_size, epochs=5, verbose=1)

root_mean_square_error定义为：

def root_mean_squared_error(y_true, y_pred):
    return K.sqrt(K.mean(K.square(y_pred - y_true), axis=-1))

我拥有的数据集大小为9652，由句子组成，标签为数字值。数据集分为训练集和验证集。最大句子长度为142。我添加了填充（ PAD ），以使每个句子的长度为142。因此，一个句子如下所示：

['france', 'is', 'hunting', 'down', 'its', 'citizens', 'who', 'joined', 'twins', 'without', 'trial', 'in', 'iraq']
['france', 'is', 'hunting', 'down', 'its', 'citizens', 'who', 'joined', 'twins', 'without', 'trial', 'in', 'iraq', '__PAD__', '__PAD__', '__PAD__',...., '__PAD__']

训练该模型时，得到以下输出

Train on 8704 samples, validate on 928 samples
Epoch 1/5
8704/8704 [==============================] - 655s 75ms/step - loss: 0.9960 - 
root_mean_squared_error: 0.9960 - val_loss: 0.9389 - val_root_mean_squared_error: 0.9389
Epoch 2/5
8704/8704 [==============================] - 650s 75ms/step - loss: 0.9354 - 
root_mean_squared_error: 0.9354 - val_loss: 0.9389 - val_root_mean_squared_error: 0.9389
Epoch 3/5
8704/8704 [==============================] - 650s 75ms/step - loss: 0.9354 - 
root_mean_squared_error: 0.9354 - val_loss: 0.9389 - val_root_mean_squared_error: 0.9389
Epoch 4/5
8704/8704 [==============================] - 650s 75ms/step - loss: 0.9354 - 
root_mean_squared_error: 0.9354 - val_loss: 0.9389 - val_root_mean_squared_error: 0.9389
Epoch 5/5
8704/8704 [==============================] - 650s 75ms/step - loss: 0.9354 - 
root_mean_squared_error: 0.9354 - val_loss: 0.9389 - val_root_mean_squared_error: 0.9389

损失和指标均未改善，并且在第2-5阶段保持不变。

我不确定这是怎么回事？任何帮助将不胜感激。