Question

for row in codecs.getreader(self.encoding)(self.response[u'Body']).readlines():
            row_string = StringIO(row)
            print ("Row read from the data is: ")
            print (row_string.getvalue())
            df = pd.read_csv(row_string, sep=",")

我已经编写了上面的代码，以逐行从S3传输csv文件。但是，csv文件中有一行，其中一行中有一个 enter 。在本地下载文件时，Pandas可以读取它，但是在上面的代码中，它会生成错误：

[2018-11-12 14:11:45,586] {models.py:1595} ERROR - Error tokenizing data. C error: EOF inside string starting at line 0

忽略我在代码中看到的第0行备注，我读了一行并形成了它的数据框。

完整的错误回溯是：

[2018-11-12 14:11:45,586]



{models.py:1595} ERROR - Error tokenizing data. C error: EOF inside
 string starting at line 0 Traceback (most recent call last):   File
 "/usr/local/lib/python3.5/dist-packages/airflow/models.py", line 1493,
 in _run_raw_task
     result = task_copy.execute(context=context)   File "/usr/local/lib/python3.5/dist-packages/airflow/operators/python_operator.py",
 line 89, in execute
     return_value = self.execute_callable()   File "/usr/local/lib/python3.5/dist-packages/airflow/operators/python_operator.py",
 line 94, in execute_callable
     return self.python_callable(*self.op_args, **self.op_kwargs)   File
 "/usr/local/lib/python3.5/dist-packages/pallet-0.0.0-py3.5.egg/pallet/tasks/versionator.py", line 228, in driver_de_versionator
     a.index_patch()   File "/usr/local/lib/python3.5/dist-packages/pallet-0.0.0-py3.5.egg/pallet/tasks/versionator.py", line 202, in index_patch
     DB.process(self.form_candidate_version, self.destination_of_kch_file_to_be_downloaded)   File
 "/usr/local/lib/python3.5/dist-packages/pallet-0.0.0-py3.5.egg/pallet/tasks/versionator.py", line 144, in form_candidate_version
     df = pd.read_csv(row_string, sep=",")   File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line
 678, in parser_f
     return _read(filepath_or_buffer, kwds)   File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line
 440, in _read
     parser = TextFileReader(filepath_or_buffer, **kwds)   File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line
 787, in __init__
     self._make_engine(self.engine)   File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line
 1014, in _make_engine
     self._engine = CParserWrapper(self.f, **self.options)   File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line
 1708, in __init__
     self._reader = parsers.TextReader(src, **kwds)   File "pandas/_libs/parsers.pyx", line 539, in
 pandas._libs.parsers.TextReader.__cinit__   File
 "pandas/_libs/parsers.pyx", line 737, in
 pandas._libs.parsers.TextReader._get_header   File
 "pandas/_libs/parsers.pyx", line 932, in
 pandas._libs.parsers.TextReader._tokenize_rows   File
 "pandas/_libs/parsers.pyx", line 2112, in
 pandas._libs.parsers.raise_parser_error pandas.errors.ParserError:
 Error tokenizing data. C error: EOF inside string starting at line 0

Answer 1

遇到麻烦的快速Google出现this link，表明：

解决方案是在engine='python'中使用参数read_csv 函数调用。 Pandas CSV解析器可以使用两个不同的“引擎” 解析CSV文件– Python或C（默认）。

pandas.read_csv(filepath_or_buffer, sep=', ', delimiter=None, 
                header='infer', names=None, 
                index_col=None, usecols=None, squeeze=False, 
                ..., engine=None, ...)

引擎 ：{'c'，'python'}，可选

要使用的解析器引擎。 C引擎更快，而python引擎更快   目前功能更完善。

如何通过连续处理换行符之类的案例读取csv文件？

1 个答案: