使用Python Tweepy从Twitter趋势主题数据中提取主题标签

时间:2014-04-16 12:29:29

标签: python twitter hashtag tweepy trending

我遇到以下问题:

使用Twitter API和tweepy模块,我想监控趋势主题并从数据中提取主题标签。

此代码:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import tweepy, json

CONSUMER_KEY = 'key'
CONSUMER_SECRET = 'secret'
ACCESS_KEY = 'key'
ACCESS_SECRET = 'secret'
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)

trends1 = api.trends_place(1)
print trends1

为我提供了有关全球趋势主题的数据,其结构如下:

[{u'created_at': u'2014-04-16T12:13:15Z', u'trends': [{u'url': u'http://twitter.com/search?q=%22South+Korea%22', u'query': u'%22South+Korea%22', u'name': u'South Korea', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%23FETUSONEDIRECTIONDAY', u'query': u'%23FETUSONEDIRECTIONDAY', u'name': u'#FETUSONEDIRECTIONDAY', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%23PrayForSouthKorea', u'query': u'%23PrayForSouthKorea', u'name': u'#PrayForSouthKorea', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%23GaraGaraRP', u'query': u'%23GaraGaraRP', u'name': u'#GaraGaraRP', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%23%D8%A5%D8%B3%D9%85_%D8%A3%D9%85%D9%8A_%D8%A8%D8%AC%D9%88%D8%A7%D9%84%D9%8A', u'query': u'%23%D8%A5%D8%B3%D9%85_%D8%A3%D9%85%D9%8A_%D8%A8%D8%AC%D9%88%D8%A7%D9%84%D9%8A', u'name': u'#\u0625\u0633\u0645_\u0623\u0645\u064a_\u0628\u062c\u0648\u0627\u0644\u064a', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%23Kad%C4%B1nlarKamyon%C5%9Eof%C3%B6r%C3%BCOlursa', u'query': u'%23Kad%C4%B1nlarKamyon%C5%9Eof%C3%B6r%C3%BCOlursa', u'name': u'#Kad\u0131nlarKamyon\u015eof\xf6r\xfcOlursa', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%22Dear+My+BestFriend%22', u'query': u'%22Dear+My+BestFriend%22', u'name': u'Dear My BestFriend', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%22%D0%A1%D0%B0%D0%BC%D0%BE%D0%BE%D0%B1%D0%BE%D1%80%D0%BE%D0%BD%D0%B0+100%22', u'query': u'%22%D0%A1%D0%B0%D0%BC%D0%BE%D0%BE%D0%B1%D0%BE%D1%80%D0%BE%D0%BD%D0%B0+100%22', u'name': u'\u0421\u0430\u043c\u043e\u043e\u0431\u043e\u0440\u043e\u043d\u0430 100', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=%22If+I+Stay%22', u'query': u'%22If+I+Stay%22', u'name': u'If I Stay', u'promoted_content': None}, {u'url': u'http://twitter.com/search?q=Gabashvili', u'query': u'Gabashvili', u'name': u'Gabashvili', u'promoted_content': None}], u'as_of': u'2014-04-16T12:20:29Z', u'locations': [{u'woeid': 1, u'name': u'Worldwide'}]}]

这是一个包含多个词典的python列表吗?如何从该数据中提取主题标签并将其保存到新变量中?

我是python的新手,所以请解释一下你的选择。

谢谢!

2 个答案:

答案 0 :(得分:3)

您的数据是包含一个字典的列表。这本词典中的一个关键词叫做趋势。此键的值是字典列表。这些字典中的每一个都包含一个名为name的键,它包含一个包含#标签的字符串。以下是访问数据的示例:

hashtags = []
trends = data[0]['trends']
for trend in trends:
    name = trend['name']
    if name.startswith('#'):
        hashtags.append(name)

这可以压缩为:

hashtags = [trend['name'] for trend in data[0]['trends'] if trend['name'].startswith('#')]

前三行输出:

>>> for hashtag in hashtags:
        print(hashtag)
#FETUSONEDIRECTIONDAY
#PrayForSouthKorea
#GaraGaraRP

答案 1 :(得分:2)

在你的例子中,你的列表中有一个条目,包括带有键值'trend'的嵌套dicts,每个值是另一个dict,你感兴趣的是'name',特别是如果它以''开头' #':

In [180]:

[x for x in temp[0]['trends'] if x['name'].find('#') ==0]
Out[180]:
[{'name': '#FETUSONEDIRECTIONDAY',
  'promoted_content': None,
  'query': '%23FETUSONEDIRECTIONDAY',
  'url': 'http://twitter.com/search?q=%23FETUSONEDIRECTIONDAY'},
 {'name': '#PrayForSouthKorea',
  'promoted_content': None,
  'query': '%23PrayForSouthKorea',
  'url': 'http://twitter.com/search?q=%23PrayForSouthKorea'},
 {'name': '#GaraGaraRP',
  'promoted_content': None,
  'query': '%23GaraGaraRP',
  'url': 'http://twitter.com/search?q=%23GaraGaraRP'},
 {'name': '#إسم_أمي_بجوالي',
  'promoted_content': None,
  'query': '%23%D8%A5%D8%B3%D9%85_%D8%A3%D9%85%D9%8A_%D8%A8%D8%AC%D9%88%D8%A7%D9%84%D9%8A',
  'url': 'http://twitter.com/search?q=%23%D8%A5%D8%B3%D9%85_%D8%A3%D9%85%D9%8A_%D8%A8%D8%AC%D9%88%D8%A7%D9%84%D9%8A'},
 {'name': '#KadınlarKamyonŞoförüOlursa',
  'promoted_content': None,
  'query': '%23Kad%C4%B1nlarKamyon%C5%9Eof%C3%B6r%C3%BCOlursa',
  'url': 'http://twitter.com/search?q=%23Kad%C4%B1nlarKamyon%C5%9Eof%C3%B6r%C3%BCOlursa'}]

修改 为了得到hastags:

In [181]:

[x['name'] for x in temp[0]['trends'] if x['name'].find('#') ==0]
Out[181]:
['#FETUSONEDIRECTIONDAY',
 '#PrayForSouthKorea',
 '#GaraGaraRP',
 '#إسم_أمي_بجوالي',
 '#KadınlarKamyonŞoförüOlursa']

您可以使用startswith代替find

[x['name'] for x in temp[0]['trends'] if x['name'].startswith('#')]