通过Python Selenium Web驱动程序在Quora上的某个问题下发布答案

通过Python Selenium Web驱动程序在Quora上的某个问题下发布答案,python,selenium-webdriver,web-scraping,Python,Selenium Webdriver,Web Scraping,我是Python Selenium scraping的初学者。我想从一个问题下面贴出的所有Quora答案中找出永久性的答案。到目前为止,我已经创建了以下代码段。但当我运行它时,它只给我输出中的一个链接。我想这是因为页面没有完全加载。我应该怎么做才能从页面来源获得至少100个永久链接 from selenium import webdriver from selenium.webdriver.common.by import from selenium.webdriver.support.ui i

我是Python Selenium scraping的初学者。我想从一个问题下面贴出的所有Quora答案中找出永久性的答案。到目前为止,我已经创建了以下代码段。但当我运行它时,它只给我输出中的一个链接。我想这是因为页面没有完全加载。我应该怎么做才能从页面来源获得至少100个永久链接

from selenium import webdriver
from selenium.webdriver.common.by import
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

driver_option = webdriver.ChromeOptions()
driver_option.add_argument(" — incognito")
chromedriver_path = './chromedriver'
def create_webdriver():
 return webdriver.Chrome(executable_path=chromedriver_path, chrome_options=driver_option)

f = open('file_text.txt', 'w')
# Open the website
browser = create_webdriver()
browser.get("https://www.quora.com/How-do-I-prove-the-flat-earth-theory")

projects = browser.find_elements_by_xpath("//a[@class='answer_permalink']")

for proj in projects:
 anslink = proj.get_attribute('href')
 f.write(anslink)

f.close()
输出:

/How-do-I-prove-the-plat-earth-theory/answer/Dave-Morgan-14
/How-do-I-prove-the-Plat-earth-theory/答案/肯·纳科
/我如何证明平坦的地球理论/答案/马修作家
/How-do-I-prove-the-plat-earth-theory/答案/机遇-汤普森-13
/How-do-I-prove-the-plat-earth-theory/answers/272223260
/How-do-I-prove-the-plat-earth-theory/answers/26836797
/我如何证明平坦地球理论/答案/弗里达·席斯
/我如何证明平坦的地球理论/答案/皮埃尔·里普林格
/我如何证明平地理论/答案/雅各布·傅
/How-do-I-prove-the-plat-earth-theory/answer/Mike-Howells-4
/如何证明平坦的地球理论/答案/米克·斯图特
/如何证明平坦的地球理论/答案/杰西·布里奇斯III
/如何证明平坦地球理论/答案/勒纳德·勒布朗
/How-do-I-prove-the-plat-earth-theory/answers/26831140
/How-do-I-prove-the-plat-earth-theory/answers/27158717
/How-do-I-prove-the-Plat-earth-theory/answer/Chris-Lockwood-4
/如何证明平坦地球理论/答案/大卫·明格
/How-do-I-prove-the-plat-earth-theory/answer/Rick-Brown-50
/How-do-I-prove-the-Plat-earth-theory/答案/雅克·马兰-4
/How-do-I-prove-the-plat-earth-theory/答案/罗伯特·伦特1
/How-do-I-prove-the-plat-earth-theory/answers/79419339
/How-do-I-prove-the-Plat-earth-theory/答案/戴夫·康塞利奥
/How-do-I-prove-the-plat-earth-theory/answers/65113366
/如何证明平坦的地球理论/答案/克里希纳布·梅迪
硒方法:

从selenium导入webdriver
从bs4导入BeautifulSoup
从selenium.webdriver.firefox.options导入选项
从bs4导入BeautifulSoup
导入时间
选项=选项()
options.add_参数('--headless')
driver=webdriver.Firefox(options=options)
司机,快(
'https://www.quora.com/How-do-I-prove-the-flat-earth-theory')
lenOfPage=driver.execute\u脚本(
“window.scrollTo(0,document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;”
匹配=错误
while(match==False):
lastCount=Lenof页
lenOfPage=driver.execute\u脚本(
“window.scrollTo(0,document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;”
如果lastCount>=51000:
打破
soup=BeautifulSoup(driver.page_源代码'html.parser')
计数=0
对于soup.findAll(“a”,{'class':'answer_permalink'})中的项目:
计数+=1
打印(item.get(“href”))
打印(计数)
driver.quit()
输出:

/How-do-I-prove-the-plat-earth-theory/answer/Dave-Morgan-14
/How-do-I-prove-the-Plat-earth-theory/答案/肯·纳科
/我如何证明平坦的地球理论/答案/马修作家
/How-do-I-prove-the-plat-earth-theory/答案/机遇-汤普森-13
/How-do-I-prove-the-plat-earth-theory/answers/272223260
/How-do-I-prove-the-plat-earth-theory/answers/26836797
/我如何证明平坦地球理论/答案/弗里达·席斯
/我如何证明平坦的地球理论/答案/皮埃尔·里普林格
/我如何证明平地理论/答案/雅各布·傅
/How-do-I-prove-the-plat-earth-theory/answer/Mike-Howells-4
/如何证明平坦的地球理论/答案/米克·斯图特
/如何证明平坦的地球理论/答案/杰西·布里奇斯III
/如何证明平坦地球理论/答案/勒纳德·勒布朗
/How-do-I-prove-the-plat-earth-theory/answers/26831140
/How-do-I-prove-the-Plat-earth-theory/答案/丹雅·罗斯
/How-do-I-prove-the-Plat-earth-theory/answer/Chris-Lockwood-4
/如何证明平坦地球理论/答案/大卫·明格
/How-do-I-prove-the-plat-earth-theory/answer/Rick-Brown-50
/How-do-I-prove-the-Plat-earth-theory/答案/雅克·马兰-4
/How-do-I-prove-the-plat-earth-theory/答案/罗伯特·伦特1
/How-do-I-prove-the-Plat-earth-theory/答案/约翰·林德22
/How-do-I-prove-the-Plat-earth-theory/答案/戴夫·康塞利奥
/How-do-I-prove-the-plat-earth-theory/answers/65113366
/如何证明平坦的地球理论/答案/克里希纳布·梅迪
/How-do-I-prove-the-plat-earth-theory/answers/44569062
/How-do-I-prove-the-Plat-earth-theory/答案/阿布德·拉赫曼·洛马克斯
/How-do-I-prove-the-plat-earth-theory/答案/赫尔穆特·沃勒
/How-do-I-prove-the-plat-earth-theory/answer/Ed-Kohlwey-1
/How-do-I-prove-the-plat-earth-theory/answer/Jason-Ree-4
/How-do-I-prove-the-plat-earth-theory/答案/德鲁·库里
/我如何证明平坦的地球理论/答案/Darrel-Blakely-2
/How-do-I-prove-the-plate-earth-theory/答案/亚历山大·昆茨-2
/How-do-I-prove-the-Plat-earth-theory/answer/Michael-Greenberg-61
/如何证明平坦地球理论/答案/马修·申克
/How-do-I-prove-the-plat-earth-theory/answer/Gregory-Hart-8
/How-do-I-prove-the-Plat-earth-theory/答案/马克·贾马蒂
/How-do-I-prove-the-Plat-earth理论/答案/弗农·本德
/如何证明平坦地球理论/答案/布雷特·埃维尔
/如何证明平坦地球理论/答案/库尔特·麦格
/How-do-I-prove-the-Plat-earth-theory/答案/迈克尔·布伦纳-13
/How-do-I-prove-the-plat-earth-theory/answer/Luke-Anderson-87
/How-do-I-prove-the-Plat-earth-theory/答案/萨萨·纽夫
/How-do-I-prove-the-flat-earth理论/答案/斯潘丹•马立克
/How-do-I-prove-the-plat-earth-theory/answers/58252346
/我如何证明平坦地球理论/答案/蒂莫西·拉莫特
/How-do-I-prove-the-Plat-earth-theory/答案/埃里克·施韦特费格
/How-do-I-prove-the-plat-earth-theory/answers/70843234
/How-do-I-prove-the-plat-earth-theory/答案/托尼·弗洛里
/How-do-I-prove-the-plat-earth-theory/答案/Aji Jijo
/How-do-I-prove-the-Plat-earth-theory/答案/Tia Eastlake
/How-do-I-prove-the-plat-earth-theory/answer/Michael-Grace-53
/How-do-I-prove-the-plat-earth-theory/answer/Ray-Mason-30
/我该怎么办-
import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.quora.com/How-do-I-prove-the-flat-earth-theory")

soup = BeautifulSoup(r.text, 'html.parser')

for item in soup.findAll("a", {'class': 'answer_permalink'}):
    print(item.get("href"))