如何过滤掉标签中不需要的标签

时间:2016-02-02 00:30:40

标签: python tags beautifulsoup

我正在尝试从此页面中删除项目符号列表:http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/

具体而言,下面的屏幕截图中的子弹以黄色突出显示。

enter image description here

首先,我使用漂亮的汤来过滤掉所有没有属性的<ul>标签:

text = BeautifulSoup(requests.get('http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/', timeout=7.00).text)
bullets = text.find_all(lambda tag: tag.name == 'ul' and not tag.attrs) 

以下是返回的两个<ul>代码:

<ul>
<li>You are experiencing a decrease in sales and customers</li>
<li>If your brand design does not reflect what you deliver</li>
<li>If you want to attract a new target audience</li>
<li>Management change</li>
<li><a href="http://www.risingabovethenoise.com/how-to-rebrand-19-questions-ask-before-you-start/" onclick="__gaTracker('send', 'event', 'outbound-article', 'http://www.risingabovethenoise.com/how-to-rebrand-19-questions-ask-before-you-start/', '19 Questions to Ask Yourself Before You Start Rebranding');">19 Questions to Ask Yourself Before You Start Rebranding</a></li>
</ul>

<ul><li class="share-item share-fb" data-title="What is Causing your Headaches?- Startup Pain Points" data-type="facebook" data-url="http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/" title="Facebook"></li><li class="share-item share-tw" data-title="What is Causing your Headaches?- Startup Pain Points" data-type="twitter" data-url="http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/" title="Twitter"></li><li class="share-item share-gp" data-lang="en-US" data-title="What is Causing your Headaches?- Startup Pain Points" data-type="googlePlus" data-url="http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/" title="Google+"></li><li class="share-item share-pn" data-media="http://bodetree.com/wp-content/uploads/2015/04/pain-points.png" data-title="What is Causing your Headaches?- Startup Pain Points" data-type="pinterest" data-url="http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/" title="Pinterest"></li></ul>

我只想提取页面正文中出现的<ul>标记,因此我想过滤掉第二个带有垃圾的<ul>标记。看来,页面正文中没有出现的<ul>标记包含<li>个标记,其中包含属性,因此我们可以根据该标记进行过滤。基本上我想要的只是<ul><li>string</li></ul>形式的标签结构。所以在这种情况下,我想要返回的唯一<ul>是:

<ul> 
<li>You are experiencing a decrease in sales and customers</li> 
<li>If your brand design does not reflect what you deliver</li> 
<li>If you want to attract a new target audience</li> 
<li>Management change</li> 
<li>19 Questions to Ask Yourself Before You Start Rebranding</li>
</ul> 

有没有办法用find_all()做到​​这一点?

1 个答案:

答案 0 :(得分:0)

在文章中搜索uldiv class="entry-content"

from bs4 import BeautifulSoup

soup = BeautifulSoup(requests.get('http://bodetree.com/what-is-causing-your-headaches-startup-pain-points/', timeout=7.00).text)

bullets = soup.select("div.entry-content ul li")
print([bullet.get_text() for bullet in bullets])

打印:

[
    'You are experiencing a decrease in sales and customers', 
    'If your brand design does not reflect what you deliver', 
    'If you want to attract a new target audience', 
    'Management change', 
    '19 Questions to Ask Yourself Before You Start Rebranding'
]