Question

我正在尝试从某个网站获取数据。我正在使用Selenium，以便我可以自己登录，然后开始解析数据。

我有3个主要错误：

最后一页＃未正确加载。在这里我加载“1”应该是“197”并且我相信这是因为与网站相关的负载而发生
元素'test'xpath未正确找到。我在最后评论了循环。

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[1]/div[@class='col-lg-3 col-sm-3 result-info' and 2]/span[@class='brand-name' and 1]"}

最后，我试图点击最后一页来测试它是否有效，但是我收到一个错误，即找不到Element。

selenium.common.exceptions.ElementNotVisibleException: Message: element not visible

这是我的代码

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException

url = "https://marketplace.refersion.com/"
username = "jupoxar@b2bx.net"
password = "testpass123"
driver = webdriver.Chrome("/Users/xxx/Downloads/chromedriver")

if __name__ == "__main__":
   driver.get(url)

   driver.find_element_by_xpath("/html/body/div[@class='wrapper']/div[@class='top-block']/header[@class='header clearfix']/div[@class='login-button']/a[@class='login-link']").click()

   driver.find_element_by_id("email").send_keys(username)  # enters the username in textbox

   driver.find_element_by_xpath("/html/body/div[@id='app']/div[@class='top-block']/div[@class='row']/div[@class='col-xs-12 col-sm-10 col-sm-offset-1 col-md-8 col-md-offset-2 col-lg-6 col-lg-offset-3 main-section']/div[@class='main-section-content']/div/form[@class='form-horizontal']/div[@class='form-group ']/div[@class='col-xs-12 col-sm-10 col-sm-offset-1 input-group input-group-lg']/input[@id='password']").send_keys(password)  # enters the password in textbox

   # Find the submit button using class name and click on it.
   driver.find_element_by_class_name("btn-primary").click()

   driver.find_element_by_link_text("Find Offers").click()

   driver.find_element_by_id("sorting-dropdown").click()  # enters the username in textbox

   driver.find_element_by_link_text("Newest First").click()

   last_page = driver.find_element_by_class_name("right-center").text
   print(last_page)

   # try:
   #     last_page = WebDriverWait(driver, 3).until(EC.presence_of_element_located((By.CLASS_NAME, 'right-center')))
   #     print("Page is ready!")
   # except TimeoutException:
   #     print("Loading took too much time!")

   for i in range(1, 10):
     #   test = driver.find_element_by_xpath("//div[1]/div[@class='col-lg-3 col-sm-3 result-info' and 2]/span[@class='brand-name' and 1]")
      #  print(test)

     WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'hover-link'))).click()

我认为这与页面的加载方式有关。我的问题是，有什么工作可以解决这个问题吗？

Answer 1

您的代码中应该有explicit waits来处理页面的动态加载。按＆＃34;最新优先＆＃34;对页面进行排序使它刷新结果并引入一个微调器来指示排序。

<i class="fa fa-spinner fa-spin" aria-hidden="true" style="font-size: 48px;"></i>

等待微调器消失应该给你正确的页数。以下几行：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
.....
# your login code
.....

driver.find_element_by_link_text("Newest First").click()
element = WebDriverWait(driver, 10).until(
    EC.invisibility_of_element_located((By.XPATH, "//i[@class='fa fa-spinner fa-spin']"))
)
last_page = driver.find_element_by_class_name("right-center").text

要查找页面上列出的所有品牌名称，您需要通过调用方法 find_elements_by_xpath （复数，元素）<{1}}找到所有span标记< / p>

class='brand-name'

Selenium加载时间错误 - 寻找可能的解决方法

1 个答案: