Question

我想清理数据以减少重复次数。我不想删除所有重复项。

如何获取具有一定重复次数的numpy数组？

假设我有

x = np.array([[1,2,3],[1,2,3],[5,5,5],[1,2,3],[1,2,3]])

我将重复项的数量设置为2。

输出应该像

x
>>[[1,2,3],[1,2,3],[5,5,5]]

或

x
>>[[5,5,5],[1,2,3],[1,2,3]]

它不计入我的任务

Answer 1

即使在已经有numpy数组的情况下，使用列表附加作为中间步骤并不总是一个好主意，但在这种情况下，这是迄今为止最干净的方法：

def n_uniques(arr, max_uniques):
    uniq, cnts = np.unique(arr, axis=0, return_counts=True)
    arr_list = []
    for i in range(cnts.size):
        num = cnts[i] if cnts[i] <= max_uniques else max_uniques
        arr_list.extend([uniq[i]] * num)
    return np.array(arr_list)

x = np.array([[1,2,3],
              [1,2,3],
              [1,2,3],
              [5,5,5],
              [1,2,3],
              [1,2,3],])

reduced_arr = n_uniques(x, 2)

Answer 2

这有点棘手，但实际上您可以在没有循环的情况下做到这一点，并使用类似的方法在原始数组中保留相对顺序（在这种情况下，将保留第一个重复项）：

import numpy as np

def drop_extra_repetitions(x, max_reps):
    # Find unique rows
    uniq, idx_inv, counts = np.unique(x, axis=0, return_inverse=True, return_counts=True)
    # Compute number of repetitions of each different row
    counts_clip = np.minimum(counts, max_reps)
    # Array alternating between valid unique row indices and -1 ([0, -1, 1, -1, ...])
    idx_to_repeat = np.stack(
        [np.arange(len(uniq)), -np.ones(len(uniq), dtype=int)], axis=1).ravel()
    # Number of repetitions for each of the previous indices
    idx_repeats_clip = np.stack([counts_clip, counts - counts_clip], axis=1).ravel()
    # Valid unique row indices are repetead at most max_reps,
    # extra repetitions are filled with -1
    idx_clip_sorted = np.repeat(idx_to_repeat, idx_repeats_clip)
    # Sorter for inverse index - that is, sort the indices in the input array
    # according to their corresponding unique row index
    sorter = np.argsort(idx_inv)
    # The final inverse index is the same as the original but with -1 on extra repetitions
    idx_inv_final = np.empty(len(sorter), dtype=int)
    idx_inv_final[sorter] = idx_clip_sorted
    # Return the array reconstructed from the inverse index without the positions with -1
    return uniq[idx_inv_final[idx_inv_final >= 0]]

x = [[5, 5, 5], [1, 2, 3], [1, 2, 3], [5, 5, 5], [1, 2, 3], [1, 2, 3]]
max_reps = 2

print(drop_extra_repetitions(x, max_reps))
# [[5 5 5]
#  [1 2 3]
#  [1 2 3]
#  [5 5 5]]

如果您根本不需要保留订单，则只需执行以下操作：

import numpy as np

def drop_extra_repetitions(x, max_reps):
    uniq, counts = np.unique(x, axis=0, return_counts=True)
    # Repeat each unique row index at most max_reps
    ret_idx = np.repeat(np.arange(len(uniq)), np.minimum(counts, max_reps))
    return uniq[ret_idx]

x = [[5, 5, 5], [1, 2, 3], [1, 2, 3], [5, 5, 5], [1, 2, 3], [1, 2, 3]]
max_reps = 2

print(drop_extra_repetitions(x, max_reps))
# [[1 2 3]
#  [1 2 3]
#  [5 5 5]
#  [5 5 5]]

如何减少Numpy数组中的行重复次数

2 个答案: