需要帮助弄清楚如何遍历索引

时间:2019-01-03 22:07:03

标签: python for-loop web-scraping indices

我正在从事一个涉及大量数据收集的项目。我现在正在编写一个相当长的脚本,但是遇到了我的for循环问题。

我正在尝试从9行表中抓取信息。我试图建立一个for循环,以便从每一行中刮取相同的信息。为了访问第一行,我将该表拆分为一个列表。第一行从第三个索引开始。

这是我的代码:

当我运行它时,在“ Aa”行上出现一个“ AttributeError”。错误显示为“'NoneType'对象没有属性'text'”

当我将那行代码分别输入到控制台时,不会出现这种情况,我得到了所需的文本。当我取出for循环时,我可以抓取第一个indaplaybox。

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url ='Myurl/=' + page

uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

page_soup = soup(page_html, "html.parser")
boxes = page_soup.findAll("table",{"class":"TableTable tableBody"})
box = boxes[0]
playboxes = box.find_all('tr')
indaplaybox = playboxes[3]

filename = "QBS.csv"
f = open(filename, "a")

headers= "Aa, Ab, Ac, Ad\n"
f.write(headers)

for indaplaybox in playboxes:

    Aa = indaplaybox.find('td', attrs = {'style' : 'font-weight: bold;'}).text

    c = indaplaybox.find('td', attrs = {'class' : 'tablePlayName'})
    cl = c.text.split() 
    Ab = cl[0] + " " + cl[1]
    Ac = cl[2]
    Ad = indaplaybox.div.a.text



    print("Aa:" + Aa)
    print("Ab:" + Ab)
    print("Ac:" + Ac)
    print("Ad:" + Ad)


    with open (filename, "a") as myfile:
        myfile.write(Aa + "," + Ab + "," + Ac.replace(",", "|") + "," + Ad + "\n")
f.close()

我想遍历播放箱索引3-11。

我不太熟悉索引,因此尝试执行以下操作:

p = [str(i) for i in range (3,12)] 
indaplaybox = playboxes[p]

for indaplaybox in playboxes:

    rest of code

但这不起作用,因为对大多数人来说很明显的是列表索引必须是整数。

我真的可以用一些帮助来思考如何使此for循环平稳运行。谢谢!

2 个答案:

答案 0 :(得分:2)

您可以这样做:

方法1:

# p has all the values from playboxes at these indexes
p = [playboxes[i] for i in range(3,12)]

# now simple loop
for indaplaybox in p:
    ......

方法2:

for indaplaybox in playboxes[3:12]:
    ....

答案 1 :(得分:0)

p = [str(i) for i in range (3,12)] 
for i in p:
    indaplaybox = playboxes[i]
    ...
    rest of the code