开发者社区> 问答> 正文

我想要点击下一页直到最后一页

from selenium import webdriver
from bs4 import BeautifulSoup as bs
import time

url = 'https://curecity.in/vendor-list.php?category=Doctor&filters_location=Jaipur&filters%5Bsubareas_global%5D=&filters_speciality='

driver = webdriver.Chrome('C:\chromedriver.exe')
driver.get(url)
driver.maximize_window()


next_page_number=1
next_page = True
while next_page == True:
     soup = bs(driver.page_source, 'html.parser')
     for link in soup.find_all('div',class_='col-md-9 feature-info'):
        link1 = link.find('a')
        print(link1['href'])
     try:
        driver.find_element_by_link_text(">").click()
        next_page_number+=1
        time.sleep(1)
    except:
        print ('No more pages')
        next_page=False

driver.close()

我正试图点击下一页,但上面写的代码是点击替代页..我想点击从第一页到最后一页。 问题来源StackOverflow 地址:/questions/59386434/selenium-webdriver-i-want-to-click-on-the-next-page-till-last-page

展开
收起
kun坤 2019-12-25 21:56:32 595 0
1 条回答
写回答
取消 提交回答
  • 我在这页上发现了两个问题。 1)它加载页面非常慢,所以我不得不在获取数据和点击()按钮之前睡眠10秒。 2)按钮>的工作方式与我预期的不同——它会跳转3个页面(即使我在浏览器中手动点击它),所以我用下一页的编号搜索按钮,然后点击它。

     driver.find_element_by_xpath('//a[@data-page="{}"]'.format(next_page_number)).click()
    

    完整的代码。即使没有漂亮的汤也能工作

    from selenium import webdriver
    #from bs4 import BeautifulSoup as bs
    import time
    
    url = 'https://curecity.in/vendor-list.php?category=Doctor&filters_location=Jaipur&filters%5Bsubareas_global%5D=&filters_speciality='
    
    driver = webdriver.Chrome('C:\chromedriver.exe')
    #driver = webdriver.Firefox()
    driver.maximize_window()
    
    driver.get(url)
    next_page_number = 1
    
    while True:
    
        print('page:', next_page_number)
        time.sleep(10)
    
        #soup = bs(driver.page_source, 'html.parser')
        #for link in soup.find_all('div',class_='col-md-9 feature-info'):
        #    link1 = link.find('a')
        #    print(link1['href'])
    
        for link in driver.find_elements_by_xpath('//div[@class="col-md-2 feature-icon"]/a'):
            print(link.get_attribute('href'))
    
        try:
            next_page_number += 1
            driver.find_element_by_xpath('//a[@data-page="{}"]'.format(next_page_number)).click()
        except:
            print('No more pages')
            break # exit loop
    
    #driver.close()
    
    2019-12-25 21:56:42
    赞同 展开评论 打赏
问答地址:
问答排行榜
最热
最新

相关电子书

更多
低代码开发师(初级)实战教程 立即下载
冬季实战营第三期:MySQL数据库进阶实战 立即下载
阿里巴巴DevOps 最佳实践手册 立即下载