分裂一个大的ndarray

时间:2015-09-14 17:25:49

标签: python numpy pandas split gps

我对python很新,对pandas来说更新,numpy。我正在尝试格式化GPS RINEX文件,以便将文件拆分为卫星(总共32个)。然后应该通过纪元(30秒间隔)对每个文件(即卫星)进行格式化,其中每个信号的数据(总共7个)然后显示在相应的列中。例如:

SV1
2014-11-07 00:00:00 L1    L2    P1    P2    C1    S1    S2 
2014-11-07 00:00:30 L1    L2    P1    P2    C1    S1    S2 
2014-11-07 00:00:30 L1    L2    P1    P2    C1    S1    S2

我正在处理的代码,特别是函数是:

def read_data_chunk(self, RINEXfile, CHUNK_SIZE = 10000):
    obss = np.empty((CHUNK_SIZE, TOTAL_SATS, len(self.obs_types)), dtype=np.float64) * np.NaN
    llis = np.zeros((CHUNK_SIZE, TOTAL_SATS, len(self.obs_types)), dtype=np.uint8)
    signal_strengths = np.zeros((CHUNK_SIZE, TOTAL_SATS, len(self.obs_types)), dtype=np.uint8)
    epochs = np.zeros(CHUNK_SIZE, dtype='datetime64[us]')
    flags = np.zeros(CHUNK_SIZE, dtype=np.uint8)

    i = 0
    while True:
        hdr = self.read_epoch_header(RINEXfile)
        #print hdr
        if hdr is None:
            break
        epoch, flags[i], sats = hdr
        epochs[i] = np.datetime64(epoch)
        sat_map = np.ones(len(sats)) * -1
        for n, sat in enumerate(sats):
            if sat[0] == 'G':
                sat_map[n] = int(sat[1:]) - 1
        obss[i], llis[i], signal_strengths[i] = self.read_obs(RINEXfile, len(sats), sat_map)
        i += 1
        if i >= CHUNK_SIZE:
            break

    print "obss.ndim: {0}".format(obss.ndim)
    print "obss.shape: {0}" .format(obss.shape)
    print "obss.size: {0}".format(obss.size)
    print "obss.dtype: {0}".format(obss.dtype)
    print "obss.itemsize: {0}".format(obss.itemsize)
    print "obss: {0}".format(obss)

    y = np.split(obss, 32, 1)
    print "y.ndim: {0}".format(y[3].ndim)
    print "y.shape: {0}" .format(y[3].shape)
    print "y.size: {0}".format(y[3].size)
    print "y_0: {0}".format(y[3])

    return obss[:i], llis[:i], signal_strengths[:i], epochs[:i], flags[:i]

打印陈述只是为了理解所涉及的维度,其结果如下:

obss.ndim: 3
obss.shape: (10000L, 32L, 7L)
obss.size: 2240000
obss.dtype: float64
obss.itemsize: 8
y.ndim: 3
y.shape: (10000L, 1L, 7L)
y.size: 70000

我遇到的确切问题是如何精确操作,以便将阵列分成后续的32个部分(即卫星)。下面是目前为止的输出示例:

sats = np.rollaxis(obss, 1, 0) 
sat = sats[5] #sv6 
sat.shape: (10000L, 7L) 
sat.ndim: 2 
sat.size: 70000 
sat.dtype: float64 
sat.item
size: 8 
sat: [[ -7.28308440e+06 -5.66279406e+06 2.38582902e+07 ..., 2.38582906e+07 4.70000000e+01 4.20000000e+01] [ -7.32362993e+06 -5.69438797e+06 2.38505736e+07 ..., 2.38505742e+07 4.70000000e+01 4.20000000e+01] [ -7.36367675e+06 -5.72559325e+06 2.38429526e+07 ..., 2.38429528e+07 4.60000000e+01 4.20000000e+01] 

上面的输出是针对第6颗卫星(“sat”)并显示前3个时期的信号。我尝试下面的代码单独打开新文件,但生成的文本文件只显示下面的输出:

代码:

for i in range(32): 
    sat = obss[:, i] 
    open(((("sv{0}").format(sat)),'w').writelines(sat)) 

文本文件中的输出:

ø ø ø ø ø ø ø 

显然,我正在忽略的数组操作有问题。 read_data_chunk函数从read_data函数调用:

def read_data(self, RINEXfile): 
    obs_data_chunks = [] 
    while True: 
        obss, _, _, epochs, _ = self.read_data_chunk(RINEXfile) 
        if obss.shape[0] == 0: 
            break 

        obs_data_chunks.append(pd.Panel( np.rollaxis(obss, 1, 0), items=['G%02d' % d for d in range(1, 33)], major_axis=epochs,minor_axis=self.obs_types).dropna(axis=0, how='all').dropna(axis=2, how='all'))   

    print "obs_data_chunks: {0}".format(obs_data_chunks) 
    self.data = pd.concat(obs_data_chunks, axis=1) 

我尝试的下一步是在上面的代码中,因为我认为这个数组可能是正确的操作。最后的印刷声明:

obs_data_chunks: [<class 'pandas.core.panel.Panel'> 
Dimensions: 32 (items) x 2880 (major_axis) x 7 (minor_axis) 
Items axis: G01 to G32 
Major_axis axis: 2014-04-27 00:00:00 to 2014-04-27 23:59:30 
Minor_axis axis: L1 to S2] 

我试图找出如何使用以下方法处理obs_data_chunks数组:

odc = np.rollaxis(obs_data_chunks, 1) 
odc_temp = odc[5]   

但收到错误:AttributeError: 'list' object has no attribute 'ndim'

1 个答案:

答案 0 :(得分:1)

这取决于您对这32个卫星子集的确切要求。据我所知,你目前的方式obss,形状为(10000, 32, 7),你已经在某种程度上“分裂”了。以下是您可以访问它们的方法:

  1. 沿着'卫星'维度切片,即axis=1

    sat = obss[:, 0]  # all the data for satellite 0, with shape (10000, 7)
    sat = obss[:, i]  # for any i from 0 through 31.
    sats = obss[:, :3] # the first three satellites
    
  2. 如果您发现主要是通过卫星编制索引,则可以使用np.rollaxis将其轴移到前面:

    sats = np.rollaxis(obss, 1)
    sats.shape
    # (32, 10000, 7)
    sat = sats[i]  # satellite i, equivalent to obss[:, i]
    sat = sats[:3] # first three satellites
    
  3. 如果您想循环通过卫星,就像在y = np.split(obss)示例中一样,更简单的方法是:

    for i in range(32):
        sat = obss[:, i]
        ...
    

    或者,如果你滚动sats的轴,你可以这样做:

    sats = np.rollaxis(obss, 1)
    for sat in sats:
        ...
    
  4. 最后,如果你真的想要一个卫星列表,你可以做

    sats = np.rollaxis(obss, 1)
    satlist = list(sats)