需要数据收集循环帮助

时间:2019-09-02 13:52:07

标签: webdriver

我需要一个循环,该循环将进入产品页面,收集数据,然后转到下一个产品,依此类推。

该循环应该存在于我已经拥有的循环中(见下文)

webdriver.ChromeOptions()
driver=webdriver.Chrome(executable_path='chromedriver', service_args=['--ssl-protocol=any', '--ignore-ssl-errors=true'])
driver.get("https://www.amazon.com/s?k=galaxy+s10&qid=1564815305&ref=sr_pg_2")

page = driver.page_source
soup = BeautifulSoup(page, 'html5lib')
# print(soup.prettify())
wait = WebDriverWait(driver, 15)



page_links=[]
links=[]
name=[]
product_price=[]


var=0
while True:
    while True: #clicks on product and exists but the 1st product only!
        product=driver.find_element_by_class_name('s-search-results')
        time.sleep(2)
        product.find_element_by_class_name('a-text-normal').click()
        wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "a-text-normal")))
        time.sleep(2)
        driver.execute_script("window.history.go(-1)")
#         driver.back() did not work
        continue

    driver.find_element_by_class_name('a-last').click()
    wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "a-last")))
    var+=1
    if var == 3:
        driver.close()

0 个答案:

没有答案
相关问题