Question

我有一个包含数千行的大型CSV文件，没有标题，每行都有一个单独的网址。

一些示例行：

http://www.whitehouse.gov/the-press-office/2012/01/27/remarks-president-college-affordability-ann-arbor-michigan 
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-first-lady-dnc-event-palm-beach-fl 
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-president-american-energy-aurora-colorado
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-first-lady-dnc-event-sarasota-fl
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-first-lady-goya-foods-miplato-announcement-tampa-fl 
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-president-american-made-energy
http://www.whitehouse.gov/the-press-office/2012/01/25/remarks-president-intel-ocotillo-campus-chandler-az 
http://www.whitehouse.gov/the-press-office/2012/01/25/remarks-first-lady-school-lunch-standards-announcement 
http://www.whitehouse.gov/the-press-office/2012/01/25/remarks-president-conveyor-engineering-and-manufacturing-cedar-rapids-io
http://www.whitehouse.gov/the-press-office/2012/01/24/remarks-president-state-union-address 
http://www.whitehouse.gov/the-press-office/2012/01/23/remarks-president-welcoming-2011-stanley-cup-champion-boston-bruins 
http://www.whitehouse.gov/the-press-office/2012/01/21/weekly-address-creating-jobs-boosting-tourism
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event-2 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event-1 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event-0
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-unveiling-strategy-help-boost-travel-and-tourism
http://www.whitehouse.gov/the-press-office/2012/01/17/remarks-president-and-first-lady-honoring-2011-world-champion-st-louis-c

我想过滤这些网址，以便将结果通过管道传输到单独的CSV文件中。我已经尝试了多个grep和awk选项，但是我得到的结果太多，与我引用的字符串不匹配。

例如，我想

grep "remarks-president" speechurls.csv >> remarks-president_urls.csv

返回URL中只有“remarks-president”的所有网址。例如：

http://www.whitehouse.gov/the-press-office/2012/01/27/remarks-president-college-affordability-ann-arbor-michigan 
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-president-american-energy-aurora-colorado
http://www.whitehouse.gov/the-press-office/2012/01/26/remarks-president-american-made-energy
http://www.whitehouse.gov/the-press-office/2012/01/25/remarks-president-intel-ocotillo-campus-chandler-az 
http://www.whitehouse.gov/the-press-office/2012/01/25/remarks-president-conveyor-engineering-and-manufacturing-cedar-rapids-io
http://www.whitehouse.gov/the-press-office/2012/01/24/remarks-president-state-union-address 
http://www.whitehouse.gov/the-press-office/2012/01/23/remarks-president-welcoming-2011-stanley-cup-champion-boston-bruins 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event-2 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event-1 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event-0
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-campaign-event 
http://www.whitehouse.gov/the-press-office/2012/01/19/remarks-president-unveiling-strategy-help-boost-travel-and-tourism
http://www.whitehouse.gov/the-press-office/2012/01/17/remarks-president-and-first-lady-honoring-2011-world-champion-st-louis-c

类似地

grep "remarks-first-lady"  speechurls.csv >> remarks-first-lady_urls.csv

应该在网址中以“remarks-first-lady”返回所有这些演讲。

我尝试过的其他规格，没有帮助。

grep -w -l "remarks-president" speechurls.csv >> remarks-president_urls.csv

我也没有太多运气试过以下。

awk -F, '$1 ~ /remarks-president|president-obama/ {print}' speechurls.csv

fgrep -w "remarks-vice-president" speechurls.csv

我不完全确定如何解决这个问题。任何帮助将非常感激。如果有更好的方法在Python中执行此操作，我也对此解决方案持开放态度。

Answer 1

我不太明白这个问题。＆＃34; grep＆＃34;评论 - 第一夫人＆＃34; speechurls.csv＆＃34;在这种情况下应该可以正常工作。

您遇到的问题可能来自＆＃34;＆gt;＆gt;＆＃34;，＆＃34;＆gt;＆gt;＆＃34;表示将新行附加到现有文件，如果您想要一个仅包含该命令输出的文件，则需要使用＆＃34;＆gt;＆＃34;而不是＆＃34;＆gt;＆gt;＆＃34;。

如果您还可以指出代码出了什么问题，我可能会更好地识别您的问题。

Answer 2

像这样的情况很有趣，可以快速创建一个快速而又脏的python脚本。我相信以下内容应该有效。

import csv 
with open('speechurls.csv', 'r') as f:
    for row in csv.reader(f):
        if 'remarks-president' in row[0]:
            with open('remarks-president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))
        elif 'remarks-first-lady' in row[0]:
            with open('remarks-first-lady_urls.csv', 'a') as f2: f2.write("{}\n".format(row[0]))
        else:
            pass

它并不漂亮，它没有优雅的设计，但它的工作原理似乎符合您的要求。

Answer 3

我只是想发布我的问题的更新。感谢@Muttonchop的帮助，我已经能够解决CSV过滤问题了。

这个python解决方案效果很好。修改@Muttonchop的初始响应，这是我最终得到的完整代码：

    def filterSpeechURL():
        import csv 
        with open('speechurls.csv', 'rU') as f:
            for row in csv.reader(f):

                #Filter President Obama
                if 'remarks-president' in row[0]:
                    with open('__president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))
                elif 'weekly-address' in row[0]:
                    with open('__president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))
                elif 'letter' in row[0]:
                    with open('__president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))
                elif 'statement-president' in row[0]:
                    with open('__president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))
                elif 'president-obama' in row[0]:
                    with open('__president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))
                elif 'excerpts-president' in row[0]:
                    with open('__president_urls.csv','a') as f1: f1.write("{}\n".format(row[0]))


                #Filter First Lady
                elif 'remarks-first-lady' in row[0]:
                    with open('__first-lady_urls.csv', 'a') as f2: f2.write("{}\n".format(row[0]))


                #Filter VP
                elif 'vice-president' in row[0]:
                    with open('__vice_president_urls.csv', 'a') as f2: f2.write("{}\n".format(row[0]))


                #Filter Jill Biden
                elif 'jill' in row[0]:
                    with open('__second-lady_urls.csv', 'a') as f2: f2.write("{}\n".format(row[0]))
                elif 'dr-biden' in row[0]:
                    with open('__second-lady_urls.csv', 'a') as f2: f2.write("{}\n".format(row[0]))


                #Filter Everthing Else
                else:
                    with open('__other_urls.csv', 'a') as f2: f2.write("{}\n".format(row[0]))

    filterSpeechURL()

根据字符串匹配过滤URLS的CSV文件

3 个答案: