Scrapy获取关于下一个兄弟的类型/信息

时间:2017-07-31 00:05:48

标签: python python-2.7 xpath web-scraping scrapy

我正在使用html从以下网站中提取信息:

<h2> <span class="mw-headline" id="Cast">Cast</span></h2>
<ul><li> <a href="/Shota_Sometani" title="Shota Sometani">Shota Sometani</a> - Shinichi Izumi
</li><li> <a href="/Ai_Hashimoto" title="Ai Hashimoto">Ai Hashimoto</a> - Satomi Murano
</li><li> <a href="/Sadao_Abe" title="Sadao Abe">Sadao Abe</a> - Migi
</li><li> <a href="/Eri_Fukatsu" title="Eri Fukatsu">Eri Fukatsu</a> - Teacher Ryoko Tamiya
</li><li> <a href="/Masahiro_Higashide" title="Masahiro Higashide">Masahiro Higashide</a> - Hideo Shimada (mysterious transfer student)
</li><li> <a href="/Nao_Omori" title="Nao Omori">Nao Omori</a> - Kuramori (freelance journalist)
</li><li> <a href="/Pierre_Taki" title="Pierre Taki">Pierre Taki</a> - Miki
</li><li> <a href="/Hirofumi_Arai" title="Hirofumi Arai">Hirofumi Arai</a> - Uragami (serial killer)
</li><li> <a href="/Kazuki_Kitamura" title="Kazuki Kitamura">Kazuki Kitamura</a> - Takeshi Hirokawa (mysterious mayor candidate)
</li><li> <a href="/Tadanobu_Asano" title="Tadanobu Asano">Tadanobu Asano</a> - Goto (Takeshi Hirokawa’s bodyguard)
</li><li> <a href="/Kimiko_Yo" title="Kimiko Yo">Kimiko Yo</a> - Nobuko Izumi (Shinichi Izumi’s mother)
</li><li> <a href="/Jun_Kunimura" title="Jun Kunimura">Jun Kunimura</a> - Detective Hirama
</li><li> <a href="/Wataru_Ichinose_(1985)" title="Wataru Ichinose (1985)">Wataru Ichinose</a> - gang member
</li><li> <a href="/Roy_Kishima" title="Roy Kishima">Roy Kishima</a> - Roy Kishima
</li></ul>

或者,html也可能如下所示:

<h2> <span class="mw-headline" id="Cast">Cast</span></h2>
<table border="0" cellspacing="3" cellpadding="0" style="text-align:center">
<tr>
<td width="100pt">
</td>
<td width="100pt">
</td>
<td width="100pt">
</td>
<td width="100pt">
</td>
<td width="100pt">
</td></tr>
<tr>
<td> <a href="/Ryosuke_Yamada" title="Ryosuke Yamada"><img alt="Assassination Classroom-Graduation-Ryosuke Yamada.jpg" src="/images/1/12/Assassination_Classroom-Graduation-Ryosuke_Yamada.jpg" width="90" height="83"/></a>
</td>
<td> <a href="/Masaki_Suda" title="Masaki Suda"><img alt="Assassination Classroom-Graduation-Masaki Suda.jpg" src="/images/4/45/Assassination_Classroom-Graduation-Masaki_Suda.jpg" width="90" height="90"/></a>
</td>
<td> <a href="/Maika_Yamamoto" title="Maika Yamamoto"><img alt="Assassination Classroom-Graduation-Maika Yamamoto.jpg" src="/images/e/e7/Assassination_Classroom-Graduation-Maika_Yamamoto.jpg" width="90" height="90"/></a>
</td>
<td> <a href="/Seika_Taketomi" title="Seika Taketomi"><img alt="Assassination Classroom-Graduation-Seika Taketomi.jpg" src="/images/c/ce/Assassination_Classroom-Graduation-Seika_Taketomi.jpg" width="90" height="90"/></a>
</td>
<td> <a href="/Mio_Yuki" title="Mio Yuki"><img alt="Assassination Classroom-Graduation-Mio Yuki.jpg" src="/images/b/b4/Assassination_Classroom-Graduation-Mio_Yuki.jpg" width="90" height="90"/></a>
</td></tr>
...
</table>

我不知道紧跟在h2之后作为兄弟节点的元素类型。 在大多数情况下,h2元素后面的元素应该是一个表,这就是我想要检查的内容。对于此示例,我需要检测h2之后的元素不是表。为此,我使用

response.xpath('//h2/span[@id="Cast"]/../following-sibling::table')

抓住下表。我通常会期待一个空列表。然而,这并不起作用,因为恰好有一张桌子,但是它的许多元素都与完全不相关的内容有关。

有没有办法获取紧跟节点后跟随兄弟的信息?

2 个答案:

答案 0 :(得分:1)

只是为了获得下一个兄弟姐妹,就像这样:

(//h2/span[@id="Cast"]/../following-sibling::table)[1]

答案 1 :(得分:1)

如果要获取紧随的兄弟节点的类型(名称),可以尝试

//h2/following-sibling::*[1]/name()
相关问题