用于根据数组值清除.csv文件的Python脚本

时间:2016-06-03 03:37:57

标签: python shell csv pandas export-to-csv

我是Python的新手,所以请原谅我。我已经通过网上找到的东西将它拼凑在一起,然而,它仍然没有完全正常工作。

我想要一个将在给定电子表格(list.csv)中查找的python脚本,为任何" key_words"解析它,然后只导出不包含的行的文件任何" key_words"叫做#34; cleaning.csv"。我想它只能看第一列,[0]。如果可能的话,我希望它还向我输出包含关键字的第二个电子表格,只是为了验证它的内容是什么。

这个当前代码查看整个csv文件,我看到它没有把一些行放在" cleaning.csv"中,技术​​上应该是,除非我的数组有问题。

这是我目前的代码......

key_words = [ 'Dog', 'Cat', 'Bird', 'Cow', ]

with open('list.csv') as oldfile, open('cleaned.csv', 'w') as newfile:
    for line in oldfile:
        if not any(key_word in line for key_word in key_words):
            newfile.write(line)

前几行数据是......

Dog,Walks,Land,4legs,
Fish,Swims,Water,fins,
Kangaroo,Hops,Land,2legs,
Cow,Walks,Land,4legs,
Bird,Flies,Air,2legs,

Cleaned.csv应该显示:

Fish,Swims,Water,fins,
Kangaroo,Hops,Land,2legs,

Other.csv(坏的,匹配的数组)应显示:

Dog,Walks,Land,4legs,
Cow,Walks,Land,4legs,
Bird,Flies,Air,2legs,

2 个答案:

答案 0 :(得分:1)

好的代码看起来很好并且对我有用,所以它本身没有任何问题。

如果您只想签入第一行,则必须将该行拆分为“,”:

key_words = ['Dog', 'Cat', 'Bird', 'Cow', ]

with open('list.csv') as oldfile, open('cleaned.csv', 'w') as cleaned, open("matched.csv", "w") as matched:
    for line in oldfile:
        if not any(key_word in line.split(",", 1)[0] for key_word in key_words):
            cleaned.write(line)
        else:
            matched.write(line)

如果第一列始终是“单词”而不是“句子”(如Dog is out),那么您可以像这样改进测试:

if not line.split(",", 1)[0] in key_words:

注意:对于字符串测试,请注意区分大小写。

请注意,在此处maxsplit=1提供line.split(",", 1)将提高字符串解析性能,尤其是如果您有更长的行,因为它会在找到第一个,后停止解析并返回2个项目的列表。第一项将是您的第一栏。在这里阅读更多内容:

https://docs.python.org/2/library/stdtypes.html#str.split

测试结果:

mac: cat list.csv
Dog,Walks,Land,4legs,
Fish,Swims,Water,fins,
Kangaroo,Hops,Land,2legs,
Cow,Walks,Land,4legs,
Bird,Flies,Air,2legs,

mac: cat cleaned.csv
Fish,Swims,Water,fins,
Kangaroo,Hops,Land,2legs,

mac: cat matched.csv
Dog,Walks,Land,4legs,
Cow,Walks,Land,4legs,
Bird,Flies,Air,2legs,

答案 1 :(得分:0)

这是一个纯pandas方法:

In [51]:
key_words = [ 'Dog', 'Cat', 'Bird', 'Cow']
t="""Dog,Walks,Land,4legs
Fish,Swims,Water,fins
Kangaroo,Hops,Land,2legs
Cow,Walks,Land,4legs
Bird,Flies,Air,2legs"""
df = pd.read_csv(io.StringIO(t), header=None)
df

Out[51]:
          0      1      2      3
0       Dog  Walks   Land  4legs
1      Fish  Swims  Water   fins
2  Kangaroo   Hops   Land  2legs
3       Cow  Walks   Land  4legs
4      Bird  Flies    Air  2legs

我们可以创建一个正则表达式模式并将其传递给str.contains并取消布尔条件以在调用to_csv之前屏蔽df:

In [55]:    
pat = '|'.join(key_words)
df[df.apply(lambda x: ~x.str.contains(pat).any(), axis=1)]

Out[55]:
          0      1      2      3
1      Fish  Swims  Water   fins
2  Kangaroo   Hops   Land  2legs

因此我们使用apply和param axis=1来逐行应用我们的lambda,我们使用str.contains测试否定的any以查看是否有任何列不包含我们的关键字:

In [56]:
df.apply(lambda x: ~x.str.contains(pat).any(), axis=1)

Out[56]:
0    False
1     True
2     True
3    False
4    False
dtype: bool