Question

我正在尝试从cnn文章中提取元标记

import httplib2
from bs4 import BeautifulSoup

http = httplib2.Http()
status, response = http.request(http://www.cnn.com/2016/08/09/health/chagas-sleeping-sickness-leishmaniasis-drug/index.html)
soup = BeautifulSoup(response)
print(soup.select('body > div.pg-right-rail-tall.pg-wrapper.pg__background__image > article > meta'))

我试图将其缩小到这个输出

<meta content="health" itemprop="articleSection"><meta content="2016-08-09T12:10:24Z" itemprop="dateCreated"><meta content="2016-08-09T12:10:24Z" itemprop="datePublished"><meta content="2016-08-09T12:10:24Z" itemprop="dateModified"><meta content="http://www.cnn.com/2016/08/09/health/chagas-sleeping-sickness-leishmaniasis-drug/index.html" itemprop="url"><meta content="Meera Senthilingam, for CNN" itemprop="author"><meta content="Could one discovery take on three deadly parasites?  - CNN.com" itemprop="headline"><meta content="Three seemingly different diseases infect 20 million people each year: Chagas disease, leishmaniasis and African sleeping sickness. But one drug could be developed to fight all three." itemprop="description"><meta content="sleeping sickness, disease, drug, drug development, chagas disease, leishmaniasis, Novartis, health, Could one discovery take on three deadly parasites?  - CNN.com" itemprop="keywords"><meta content="http://i2.cdn.turner.com/cnnnext/dam/assets/150812101743-chagas-bug-large-tease.jpg" itemprop="image"><meta content="http://i2.cdn.turner.com/cnnnext/dam/assets/150812101743-chagas-bug-large-tease.jpg" itemprop="thumbnailUrl"><meta content="Could one discovery take on three deadly parasites? " itemprop="alternativeHeadline">

但由于某种原因，BeautifulSoup.select()方法返回的内容大约是我想要的hx的100倍。我真的很感激有关如何解决这个问题的任何建议。

Answer 1

问题是解析器/ html， lxml 和 html5lib 为您提供了所需的内容。

soup = BeautifulSoup(response,"lxml")

或者：

 soup = BeautifulSoup(response,"html5lib")

如果您没有安装 lxml 或 html5lib ，则可以使用pip， lxml html5lib >根据您的操作系统，它会涉及更多，因为它有一些依赖关系，但绝对值得安装。

您还可以简化您的选择：

soup.select('div.pg-right-rail-tall.pg-wrapper.pg__background__image meta')

试图从新闻文章中提取元数据

1 个答案: