BeautifulSoup无法解析没有关闭元素的html标签

时间:2014-06-04 14:17:38

标签: python beautifulsoup

以下是我处理的HTML代码

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>sdasdsadsad</title>
<link rel="alternate" media="only screen and (max-width: 640px)" href="local:80" />
<meta name="description" content="sdddsdsdsdsdsd">
<meta name="keywords" content="3333333333333333">
<meta property="og:title" content="444444444444444444444444">
<meta property="og:type" content="article">
<meta property="og:description" content="dsdsdsdsddsds">

</head>
<body></body>
</html>

我希望该行包含标记&#34; <meta name = description&#34; ,它没有关闭元素</meta>。有我的代码

import glob, os, re, urllib2, codecs
from bs4 import BeautifulSoup
from bs4 import SoupStrainer


html_doc = """
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>sdasdsadsad</title>
<link rel="alternate" media="only screen and (max-width: 640px)" href="local:80" />
<meta name="description" content="sdddsdsdsdsdsd">
<meta name="keywords" content="3333333333333333">
<meta property="og:title" content="444444444444444444444444">
<meta property="og:type" content="article">
<meta property="og:description" content="dsdsdsdsddsds">

</head>
<body></body>
</html>
"""



soup = BeautifulSoup(html_doc)
aa = soup.find("meta", {"name":"description"})
print aa.encode("utf-8")

运行Python代码,但控制台显示

<meta content="sdddsdsdsdsdsd" name="description">
<meta content="3333333333333333" name="keywords">
<meta content="444444444444444444444444" property="og:title">
<meta content="article" property="og:type">
<meta content="dsdsdsdsddsds" property="og:description">
</meta></meta></meta></meta></meta>

但是如果&#34; <meta content="sdddsdsdsdsdsd" name="description">&#34;有一个紧密的元素</meta>,我可以得到完全一行:

<meta content="sdddsdsdsdsdsd" name="description"> </meta>

您是否想告诉我为什么BeautifulSoup获取<meta name = description下的所有HTML标记,以及如何获取该行包含<meta name = description

感谢。

1 个答案:

答案 0 :(得分:2)

使用lxml模块作为parser并且它会起作用,我已对其进行了测试。

from bs4 import BeautifulSoup

soup = BeautifulSoup(html_doc, 'lxml')
aa = soup.find("meta", {"name":"description"})

print aa.encode('utf-8')

# console output
<meta content="sdddsdsdsdsdsd" name="description"/>