Question

我正在使用tfprof来获取模型转发路径所需的翻转次数。我的模型是3层LSTM和之后完全连接的层。我观察到完全连接层的计算次数呈线性增长，而LSTM层则没有变化。怎么可能呢？

tfprof报告1个时间戳前进路径。

==================Model Analysis Report======================
_TFProfRoot (0/2.71m flops)
  rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops)
  rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops)
  rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/MatMul (606.21k/606.21k flops)
  fc_layer/MatMul (1.54k/1.54k flops)
  rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops)
  rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops)
  rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops)
  fc_layer/BiasAdd (3/3 flops)

tfprof报告2个时间戳前进路径。

==================Model Analysis Report======================
_TFProfRoot (0/2.71m flops)
  rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops)
  rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops)
  rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/MatMul (606.21k/606.21k flops)
  fc_layer/MatMul (3.07k/3.07k flops)
  rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops)
  rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops)
  rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops)
  fc_layer/BiasAdd (6/6 flops)

Answer 1

tfprof对图进行静态分析，并计算每个图节点的浮点运算。

我假设您正在使用dynamic_rnn或类似于tf.while_loop的东西。在这种情况下，图形节点会出现在图形中一次但实际上是在运行时运行多次。

在这种情况下，tfprof无法静态计算出有多少将运行步骤（单词中的时间戳）。因此，它只是重要的浮动操作一次。

现在解决的问题可能是你自己的倍数时间步。

Tensorflow tfprof LSTMCell

1 个答案: