Question

我正在处理一个函数，该函数接收CSV的文件名并将每行转换为字典，然后返回创建的字典列表（以便能够在以后的函数中进行迭代和组织。我已经通过做以下事情得到它做我想要的但是觉得必须有更好的方法。有任何改进的建议吗？

import re

def import_incidents(filename):
    """Imports CSV and returns list of dictionaries for each incident"""
    with open(filename, 'r') as file:
        data = file.read()
        data = data.split('\n')
        list_of_data = []
        headers = True
        for line in data:
            line = line.split('","')
            if headers == True:
                #Skip header and set to false
                headers = False
            elif len(line) == 1 or line[3] == '':
                #File always has a 1 lenth final line, skip it.
                #Events can leave blank policies, skip those too.
                pass
            else:
                temp_dict = {}
                temp_dict['id'] = re.sub('"', '', line[0])
                temp_dict['time'] = re.sub('GMT-0600','',line[1])
                temp_dict['source'] = line[2]
                temp_dict['policy'] = line[3]
                temp_dict['destination'] = line[5]
                temp_dict['status'] = line[10]
                list_of_data.append(temp_dict)

return list_of_data

print(import_incidents('Incidents (Yesterday Only).csv'))

CSV内容示例：

"ID","Incident Time","Source","Policies","Channel","Destination","Severity","Action","Maximum Matches","Transaction Size","Status",
"9511564","29 Dec. 2015, 08:33:59 AM GMT-0600","Doe, John","Encrypted files","HTTPS","blah.blah.com","Medium","Permitted","0","47.7 KB","Closed - Authorized",
"1848446","29 Dec. 2015, 08:23:36 AM GMT-0600","Smith, Joe","","HTTP","google.com","Low","Permitted","0","775 B","Closed"

Answer 1

你重新发明了csv.DictReader() class，我害怕：

import csv

def import_incidents(filename):
    with open(filename, 'r', newline='') as file:
        reader = csv.DictReader(file)
        for row in reader:
            if not row or not row['Policies']:
                continue
            row['Incident Time'] = re.sub('GMT-0600', '', row['Incident Time'])
            yield row

这依赖于字典键的标题行。您可以使用fieldnames的{{1}}参数定义自己的字典键（DictReader()字段按顺序与文件中的列匹配），但是后面的第一行文件仍然像任何其他行一样被读取。您可以使用fieldnames函数跳过行（请参阅Skip the headers when editing a csv file using Python）。

Answer 2

原始任务：词典列表

您可以使用pandas。它很快，可以在几行中完成;

import pandas as pd
df = pd.read_csv('incidents.csv')
df['Incident Time'] = df['Incident Time'].str.replace('GMT-0600', '')
list_of_data = df.dropna(subset=['Policies']).to_dict(orient='records')

现在list_of_data包含：

[{'Action': 'Permitted',
  'Channel': 'HTTPS',
  'Destination': 'blah.blah.com',
  'ID': 9511564,
  'Incident Time': '29 Dec. 2015, 08:33:59 AM ',
  'Maximum Matches': 0,
  'Policies': 'Encrypted files',
  'Severity': 'Medium',
  'Source': 'Doe, John',
  'Status': 'Closed - Authorized',
  'Transaction Size': '47.7 KB',
  'Unnamed: 11': nan}]

.dropna(subset='Policies')删除NaN列中Policies s的行，即缺少值。

仅限Pandas的解决方案：

如果您不想要dicts列表，请保留数据框：

df = pd.read_csv('incidents.csv', parse_dates=[1]).dropna(subset=['Policies'])

这将Incident Time读取为非常方便的datetime64[ns]对象。数据框如下所示：

CSV到词典列表 - 更好的方式？

2 个答案:

原始任务：词典列表

仅限Pandas的解决方案：