Question

例如，如果我这样做：

cdef np.ndarray[np.int64_t, ndim=1] my_array

我的my_array存放在哪里？我认为，因为我没有告诉cython存储在堆上它会存储在堆栈上，但是在进行下面的实验之后，它似乎存储在堆上，或者以某种方式有效地管理内存。如何管理my_array的内存？也许我错过了一些明显的东西，但我找不到任何文件。

import numpy as np
cimport cython
cimport numpy as np

from libc.stdlib cimport malloc, free

def big_sum():
    # freezes up:
    # "a" is created on the stack
    # space on the stack is limited, so it runs out

    cdef int a[10000000]

    for i in range(10000000):
        a[i] = i

    cdef int my_sum
    my_sum = 0
    for i in range(10000000):
        my_sum += a[i]
    return my_sum

def big_sum_malloc():
    # runs fine:
    # "a" is stored on the heap, no problem

    cdef int *a
    a = <int *>malloc(10000000*cython.sizeof(int))

    for i in range(10000000):
        a[i] = i

    cdef int my_sum
    my_sum = 0
    for i in range(10000000):
        my_sum += a[i]

    with nogil:
        free(a) 
    return my_sum    

def big_numpy_array_sum():
    # runs fine:
    # I don't know what is going on here
    # but given that the following code runs fine,
    # it seems that entire array is NOT stored on the stack

    cdef np.ndarray[np.int64_t, ndim=1] my_array
    my_array = np.zeros(10000000, dtype=np.int64)

    for i in range(10000000):
        my_array[i] = i

    cdef int my_sum
    my_sum = 0
    for i in range(10000000):
        my_sum += my_array[i]
    return my_sum

Answer 1

Cython在这里没有做任何神奇的事情。 Numpy有一个完整的C-api，这就是cython与之交互的东西 - cython本身没有执行内存管理，而numpy数组中的内存处理方式与使用python中的numpy数组时的处理方式相同。 @Bakuriu是对的 - 这绝对是在堆上。

考虑一下这个cython代码：

cimport numpy as np
def main():
    zeros = np.zeros
    cdef np.ndarray[dtype=np.double_t, ndim=1] array
    array = zeros(10000)

这将在等效的main函数中转换为以下C.我删除了声明和错误处理代码，使其更清晰。

PyArrayObject *__pyx_v_array = 0;
PyObject *__pyx_v_zeros = NULL;
PyObject *__pyx_t_1 = NULL;
PyObject *__pyx_t_2 = NULL;

// zeros = np.zeros             # <<<<<<<<<<<<<<
// get the numpy module object
__pyx_t_1 = __Pyx_GetModuleGlobalName(__pyx_n_s__np);
// get the "zeros" function
__pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s__zeros)
__pyx_v_zeros = __pyx_t_2;

// array = zeros(10000)             # <<<<<<<<<<<<<<
// (__pyx_k_tuple_1 is a static global variable containing the literal python tuple
// (10000, ) that was initialized during the __Pyx_InitCachedConstants function)
__pyx_t_2 = PyObject_Call(__pyx_v_zeros, ((PyObject *)__pyx_k_tuple_1), NULL);
__pyx_v_array = ((PyArrayObject *)__pyx_t_2);

如果你查看numpy C api文档，你会发现PyArrayObject是numpy ndarray的C-api结构。这里的关键点是看到cython根本没有明确地处理内存分配。相同的面向对象的设计原则适用于python和numpy C apis，这里的内存管理是PyArrayObject的责任。这种情况与在python中使用numpy数组没什么不同。

如何在cython中为np.ndarray处理内存？

1 个答案: