Question

我想将csv文件导入cassandra所以，首先我创建了keypace和columnfamily，就像这样

CREATE COLUMNFAMILY Consumer_complaints(
    Date_received varchar,
    Product varchar,
    Sub_product varchar,
    Issue varchar,
    Sub_issue varchar,
    Consumer_complaint_narrative varchar,
    Company_public_response varchar,
    Company varchar,
    State varchar,
    ZIP_code varint,
    Tags varchar,
    Consumer_consent_provided varchar,
    Submitted_via varchar,
    Date_sent_to_company varchar,
    Company_response_to_consumer varchar,
    Timely_response varchar,
    Consumer_disputed varchar,
    Complaint_ID varint,
    PRIMARY KEY(Complaint_ID)
);

我从www.data.gov获得了一个名为消费者投诉的csv文件然后我输入了命令行

COPY consumer_complaints (Date_received,Product,Sub_product, Issue, Sub_issue, Consumer_complaint_narrative, Company_public_response, Company, State, ZIP_code, Tags, Consumer_consent_provided, Submitted_via, Date_sent_to_company, Company_response_to_consumer, Timely_response, Consumer_disputed, Complaint_ID) FROM 'consumer_complaints.csv';

示例输入

3/21/2017,Credit reporting,,Incorrect information on credit report,Information is not mine,,Company has responded to the consumer and the CFPB and chooses not to provide a public response,EXPERIAN DELAWARE GP,TX,77075,Older American,N/A,Phone,03/21/2017,Closed with non-monetary relief,Yes,No,2397100
04/19/2017,Debt collection,"Other (i.e. phone, health club, etc.)",Disclosure verification of debt,Not disclosed as an attempt to collect,,,"Security Credit Services, LLC",IL,60643,,,Web,04/20/2017,Closed with explanation,Yes,No,2441777

错误

Failed to import 1 rows: ParseError - Failed to parse 797XX : invalid lit for int() with base 10: '797XX',  given up without retries
Failed to import 1 rows: ParseError - Failed to parse 354XX : invalid lit for int() with base 10: '354XX',  given up without retries
Failed to import 2 rows: ParseError - Failed to parse 313XX : invalid lit for int() with base 10: '313XX',  given up without retries
Failed to import 2 rows: ParseError - Failed to parse 054XX : invalid lit for int() with base 10: '054XX',  given up without retries

我该如何解决？

Answer 1

Cassandra在创建时不保留列的顺序。导入数据时需要指定列名。

尝试此命令：

COPY consumer_complaints (Date_received,Product,Sub_product, Issue, Sub_issue, Consumer_complaint_narrative, Company_public_response, Company, State, ZIP_code, Tags, Consumer_consent_provided, Submitted_via, Date_sent_to_company, Company_response_to_consumer, Timely_response, Consumer_disputed, Complaint_ID) FROM 'c.csv' WITH HEADER = true;

示例输入：

Date_received,Product,Sub_product, Issue, Sub_issue, Consumer_complaint_narrative, Company_public_response, Company, State, ZIP_code, Tags, Consumer_consent_provided, Submitted_via, Date_sent_to_company, Company_response_to_consumer, Timely_response, Consumer_disputed, Complaint_ID
07/26/2013,Mortgage,FHA mortgage,"Loan servicing, payments, escrow account",,,,"CITIBANK, N.A.",NC,28056,,N/A,Web,07/29/2013,Closed with explanation,Yes,No,467750
09/26/2014,Consumer Loan,Vehicle loan,Managing the loan or lease,,,,HSBC NORTH AMERICA HOLDINGS INC.,NY,12572,,N/A,Web,09/26/2014,Closed with explanation,Yes,No,1046323

输出：

complaint_id  | company                          | company_public_response | company_response_to_consumer | consumer_complaint_narrative | consumer_consent_provided | consumer_disputed | date_received | date_sent_to_company | issue                                    | product       | state | sub_issue | sub_product  | submitted_via | tags | timely_response | zip_code
--------------+----------------------------------+-------------------------+------------------------------+------------------------------+---------------------------+-------------------+---------------+----------------------+------------------------------------------+---------------+-------+-----------+--------------+---------------+------+-----------------+----------
      1046323 | HSBC NORTH AMERICA HOLDINGS INC. |                    null |      Closed with explanation |                         null |                       N/A |                No |    09/26/2014 |           09/26/2014 |               Managing the loan or lease | Consumer Loan |    NY |      null | Vehicle loan |           Web | null |             Yes |    12572
       467750 |                   CITIBANK, N.A. |                    null |      Closed with explanation |                         null |                       N/A |                No |    07/26/2013 |           07/29/2013 | Loan servicing, payments, escrow account |      Mortgage |    NC |      null | FHA mortgage |           Web | null |             Yes |    28056

<强>被修改

我检查来自https://catalog.data.gov/dataset/consumer-complaint-database的数据，部分邮政编码具有非整数值，如797XX，354XX，313XX和054XX。你可以看到它显然不是整数。您可以将这些值更改为整数或更改表格，并将ZIP_code字段的类型更改为varchar

Answer 2

您的错误消息显示：

无法导入1行：ParseError - 无法解析11/08/2013：int（）的无效文字，基数为10：'11 / 08/2013'，没有重试就放弃了

看起来Cassandra正试图将日期字符串作为整数插入。如果没有看到CSV，我猜测列的顺序不正确，并且日期被解析为varint字段之一。如果您共享CSV样本，则可能更容易帮助调试。

如何将csv文件导入cassandra

2 个答案: