Question

我正在努力理解为什么BeautifulSoup与HTML源代码中存在的某些类名称不匹配。

原始HTML代码包含以下要匹配的行：

<span class="Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)" data-reactid="34">1,148.52</span>

<div class="Py(16px)">...

<div class="Mt(15px)" data-reactid="4">...

以下是我的代码：

# ref: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#searching-by-css-class 

import requests
from bs4 import BeautifulSoup

page = requests.get("https://finance.yahoo.com/quote/GOOGL/")
soup = BeautifulSoup(page.content, 'html.parser')

# the folowing matches ok
print(soup.find('div', class_="Mt(15px)"))

# yet, the following lines fail to match:
print(soup.find("span", class_="Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)"))
print(soup.find('span', {"class": "Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)"}))
print(soup.find('div', class_="Py(16px)"))

我以为也许类名不能与BeautifulSoup一起使用，所以我尝试使用page.content.replace（“ old class name”，“ new”）来简化和简化BeautifulSoup函数的类名。不行。

Answer 1

并不完全健壮，但是您可以从脚本选项卡之一中获取该信息

import requests
import json
from bs4 import BeautifulSoup as bs

res = requests.get('https://finance.yahoo.com/quote/GOOGL?p=GOOGL')
soup = bs(res.content, 'lxml')
scripts = [script.text for script in soup.select('script') if '/* -- Data -- */' in script.text]
script = scripts[0].split('App.main = ')[1].replace('\n','').replace(';}(this));','')
data = json.loads(script)
content = data['context']['dispatcher']['stores']['StreamDataStore']['quoteData']['GOOGL']

print(content['longName'])
print(content['regularMarketPrice']['fmt'])
print(content['regularMarketChange']['fmt'])
print(content['regularMarketChangePercent']['fmt'])

为什么BeautifulSoup匹配某些类名而不匹配其他类名

1 个答案: