Question

我正在尝试使用TensorFlow计算矩阵中每列与所有其他列（不包括其自身）之间的最小欧几里德距离：

with graph.as_default():
  ...
  def get_diversity(matrix):
       num_rows = matrix.get_shape()[0].value
       num_cols = matrix.get_shape()[1].value
       identity = tf.ones([1, num_cols], dtype=tf.float32)
       diversity = 0

       for i in range(num_cols):
           col = tf.reshape(matrix[:, i], [num_rows, 1])
           col_extended_to_matrix = tf.matmul(neuron_matrix, identity)
           difference_matrix = (col_extended_to_matrix - matrix) ** 2
           sum_vector = tf.reduce_sum(difference_matrix, 0)
           mask = tf.greater(sum_vector, 0)
           non_zero_vector = tf.select(mask, sum_vector, tf.ones([num_cols], dtype=tf.float32) * 9e99)
           min_diversity = tf.reduce_min(non_zero_vector)
           diversity += min_diversity

       return diversity / num_cols
  ...

  diversity = get_diversity(matrix1)

  ...

当我每1000次迭代调用get_diversity()一次（在300k的范围内）时它可以正常工作。但是当我尝试在每次迭代时调用它时，解释器返回：

W tensorflow/core/common_runtime/bfc_allocator.cc:271] Ran out of memory trying to allocate 2.99MiB.  See logs for memory state.

我原以为是因为TF每次调用get_diversity()时都会创建一组新的变量。我试过这个：

def get_diversity(matrix, scope):
    scope.reuse_variables()
...
with tf.variable_scope("diversity") as scope:
    diversity = get_diversity(matrix1, scope)

但它没有解决问题。

如何修复此分配问题并使用get_diversity()进行大量迭代？

Answer 1

假设您在训练循环中多次致电get_diversity()，Aaron's comment是一个不错的选择：相反，您可以执行以下操作：

diversity_input = tf.placeholder(tf.float32, [None, None], name="diversity_input")
diversity = get_diversity(matrix)

# ...

with tf.Session() as sess:
  for _ in range(NUM_ITERATIONS):
    # ...

    diversity_val = sess.run(diversity, feed_dict={diversity_input: ...})

这将避免每次循环创建新操作，这应该可以防止内存泄漏。这个answer有更多细节。

tensorflow：＆＃39; for＆＃39;的内存分配周期

1 个答案: