Question

尝试通过pd.HDFStore('mystore.h5', mode='a').append(my_frame, chunk)以小块的形式向hdf5发送数据帧时遇到问题。这些块在列和类型方面都是相同的（它们来自相同的数据帧）但是它适用于大量的块，然后是中途的炸弹。

ValueError: cannot match existing table structure for [Net_Bal_Amt,Loan_Current_Rate] on appending data

我打印出导致此失败的数据帧块，它们共同的一个特点是特定列的所有“无”值（它们原来是从源无效的）。不知道如何纠正这个问题。只要它们是空的，它们应该保持None或NaN或null。谢谢。

Traceback (most recent call last):
  File "[...]\Anaconda3\lib\site-packages\pandas\io\pytables.py", line 3381, in create_axes
    b, b_items = by_items.pop(items)
KeyError: ('Net_Bal_Amt', 'Loan_Current_Rate')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]\crd_test.py", line 8, in <module>
    credit.CRD.hdf_install(overwrite=True, tablenames=['loans_uscrd', 'loans_uscrd_a'])
  File "[...]\credit_base.py", line 62, in hdf_install
    cls._hdf_creation(map_)
  File "[...]\credit_base.py", line 80, in _hdf_creation
    cls._hdf_processing(v, chunk)
  File "[...]\credit_base.py", line 88, in _hdf_processing
    cls.crd.append(frame, chunk)   
  File "[...]\Anaconda3\lib\site-packages\pandas\io\pytables.py", line 903, in append
    **kwargs)
  File "[...]\lib\site-packages\pandas\io\pytables.py", line 1259, in _write_to_group
    s.write(obj=value, append=append, complib=complib, **kwargs)
  File "[...]\lib\site-packages\pandas\io\pytables.py", line 3751, in write
    **kwargs)
  File "[...]\Anaconda3\lib\site-packages\pandas\io\pytables.py", line 3388, in create_axes
    item in items))
ValueError: cannot match existing table structure for [Net_Bal_Amt,Loan_Current_Rate] on appending data

dtypes：

pd.read_hdf(r'[...]\crd_test.h5','loans').dtypes
Out[4]: 
Customer_Id                  object
As_of_Date           datetime64[ns]
Net_Bal_Amt                 float64
Loan_Current_Rate           float64
dtype: object

版本： pytables：3.1.1 大熊猫：0.15.2 蟒：3.4

崩溃后附加的块的dtypes：

Customer_Id                  object
As_of_Date           datetime64[ns]
Net_Bal_Amt                 float64
Loan_Current_Rate            object
dtype: object

附加HDFStore失败，无法匹配现有的表结构

0 个答案: