Question

在这个来自https://letterboxd.com/shesnicky/list/top-50-favourite-films/的html片段中，我正在尝试浏览所有不同的li标签并从'data-target-link'获取信息，以便我可以使用它来创建一个新链接带我到那部电影的页面，但每次我尝试获取数据时，它只返回None或沿这些行的错误。

<li class="poster-container numbered-list-item" data-owner-rating="10"> <div class="poster film-poster really-lazy-load" data-image-width="125" data-image-height="187" data-film-slug="/film/donnie-darko/" data-linked="linked" data-menu="menu" data-target-link="/film/donnie-darko/" > <img src="https://s3.ltrbxd.com/static/img/empty-poster-125.c6227b2a.png" class="image" width="125" height="187" alt="Donnie Darko"/><span class="frame"><span class="frame-title"></span></span> </div> <p class="list-number">1</p> </li>

我将使用链接为twitter机器人抓取imgs，所以我尝试在我的代码中执行此操作：

class BotStreamer(tweepy.StreamListener):

    print "Bot Streamer"
    #on_data method of Tweepy’s StreamListener 
    #passes data from statuses to the on_status method
    def on_status(self, status):
        print "on status"
        link = 'https://letterboxd.com/shesnicky/list/top-50-favourite-films/'
        page = requests.get(link)
        soup = BS(page.content, 'html.parser')
        movies_ul = soup.find('ul', {'class':'poster-list -p125 -grid film-list'})

        movies = []
        for mov in movies_ul.find('data-film-slug'):
            movies.append(mov)

        rand = randint(0,51)
        newLink = "https://letterboxd.com%s" % (str(movies[rand]))
        newPage = requests.get(newLink)
        code = BS(newPage.content, 'html.parser')
        code_div = code.find\
                   ('div', {'class':'react-component film-poster film-poster-51910 poster'})

        image = code_div.find('img')
        url = image.get('src')

        username = status.user.screen_name
        status_id = status.id
        tweet_reply(url, username, status_id)

但是，我不断收到列表超出范围的错误，或者无法迭代NoneType。所以我做了一个测试prgrm，看看我是否能以某种方式得到数据：

 import requests
 from bs4 import BeautifulSoup as BS

 link = 'https://letterboxd.com/shesnicky/list/top-50-favourite-films/'
 page = requests.get(link)
 soup = BS(page.content, 'html.parser')
 movies_ul = soup.find('ul', {'class':'poster-list -p125 -grid film-list'})
 more = movies_ul.find('li', {'class':'poster-container numbered-list-item'})
 k = more.find('data-target-link')
 print k

而且，我得到的只是无。任何帮助非常感谢。

Answer 1

阅读doc：find()，因为第一个参数需要标记名称，而不是属性。

你可以做

soup.find('div', {'data-target-link': True})

或

soup.find(attrs={'data-target-link': True})

完整示例

import requests
from bs4 import BeautifulSoup as BS

link = 'https://letterboxd.com/shesnicky/list/top-50-favourite-films/'
page = requests.get(link)
soup = BS(page.content, 'html.parser')

all_items = soup.find_all('div', {'data-target-link': True})

for item in all_items:
    print(item['data-target-link'])

尝试获取标记值时没有返回

1 个答案: