前言

• 本文是個(gè)人使用Python Selenium 獲取動(dòng)態(tài)網(wǎng)頁指定元素的超鏈接的電子筆記，由于水平有限，難免出現(xiàn)錯(cuò)漏，敬請(qǐng)批評(píng)改正。
• 更多精彩內(nèi)容，可點(diǎn)擊進(jìn)入Python日常小操作專欄或我的個(gè)人主頁查看

前提條件

• 熟悉Python
• 熟悉Requests
• 熟悉XPath
• 熟悉Selenium

相關(guān)介紹

• Python是一種跨平臺(tái)的計(jì)算機(jī)程序設(shè)計(jì)語言。是一個(gè)高層次的結(jié)合了解釋性、編譯性、互動(dòng)性和面向?qū)ο蟮哪_本語言。最初被設(shè)計(jì)用于編寫自動(dòng)化腳本(shell)，隨著版本的不斷更新和語言新功能的添加，越多被用于獨(dú)立的、大型項(xiàng)目的開發(fā)。
• Requests是一個(gè)很實(shí)用的Python HTTP客戶端庫。
• Lxml是XML和HTML的解析器，其主要功能是解析和提取XML和HTML中的數(shù)據(jù)；lxml和正則一樣，也是用C語言實(shí)現(xiàn)的，是一款高性能的python HTML、XML解析器，也可以利用XPath語法，來定位特定的元素及節(jié)點(diǎn)信息。
• XPath 是一門在 XML 文檔中查找信息的語言， XSLT 中的主要元素，XQuery 和 XPointer 均構(gòu)建于 XPath 表達(dá)式之上。
• Selenium 是一個(gè)用于Web應(yīng)用程序測(cè)試的工具。Selenium測(cè)試直接運(yùn)行在瀏覽器中，就像真正的用戶在操作一樣。支持的瀏覽器包括IE（7, 8, 9, 10, 11），Mozilla Firefox，Safari，Google Chrome，Opera，Edge等。這個(gè)工具的主要功能包括：測(cè)試與瀏覽器的兼容性——測(cè)試應(yīng)用程序看是否能夠很好得工作在不同瀏覽器和操作系統(tǒng)之上。測(cè)試系統(tǒng)功能——?jiǎng)?chuàng)建回歸測(cè)試檢驗(yàn)軟件功能和用戶需求。支持自動(dòng)錄制動(dòng)作和自動(dòng)生成.Net、Java、Perl等不同語言的測(cè)試腳本。

實(shí)驗(yàn)環(huán)境

• Python 3.x （面向?qū)ο蟮母呒?jí)語言）
• Requests（Python第三方庫）pip3 install requests
• Lxml（python第三方庫）pip3 install lxml
• Selenium 4.1.0（Python第三方庫） pip3 install selenium==4.1.0

獲取動(dòng)態(tài)網(wǎng)頁指定元素的超鏈接

目標(biāo)網(wǎng)址

https://www.zhipin.com/

代碼實(shí)現(xiàn)

import time
from selenium import webdriver
from selenium.webdriver.common.by import By

# chrome_options = webdriver.ChromeOptions()
# # # 增加無界面選項(xiàng)
# # chrome_options.headless = True
# # 增加無界面選項(xiàng)
# chrome_options.add_argument('--headless')
# # 如果不加這個(gè)選項(xiàng)，有時(shí)定位會(huì)出現(xiàn)問題
# chrome_options.add_argument('--disable-gpu')
# driver = webdriver.Chrome(options=chrome_options)

options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])

# 打開谷歌驅(qū)動(dòng)
driver = webdriver.Chrome(options=options)
# 打開火狐驅(qū)動(dòng), 下載的是哪個(gè)驅(qū)動(dòng)就用所對(duì)應(yīng)的驅(qū)動(dòng)名打開，這里我使用的是谷歌
# driver = webdriver.Firefox()
# 輸入需要搜索的網(wǎng)址，這里直接輸入boos的首頁網(wǎng)址，也可以根據(jù)自己的需要來輸入對(duì)應(yīng)的網(wǎng)址，修改括號(hào)中的網(wǎng)址即可
driver.get("https://www.zhipin.com/web/geek/job?query=AI算法工程師")
# implicitly_wait(): 隱式等待
# 當(dāng)使用了隱士等待執(zhí)行測(cè)試的時(shí)候，如果
# WebDriver沒有在
# DOM中找到元素，將繼續(xù)等待，超出設(shè)定時(shí)間后則拋出找不到元素的異常
# 換句話說，當(dāng)查找元素或元素并沒有立即出現(xiàn)的時(shí)候，隱式等待將等待一段時(shí)間再查找
# DOM，默認(rèn)的時(shí)間是0
# 一旦設(shè)置了隱式等待，則它存在整個(gè)
# WebDriver
# 對(duì)象實(shí)例的聲明周期中，隱式的等到會(huì)讓一個(gè)正常響應(yīng)的應(yīng)用的測(cè)試變慢，
# 它將會(huì)在尋找每個(gè)元素的時(shí)候都進(jìn)行等待，這樣會(huì)增加整個(gè)測(cè)試執(zhí)行的時(shí)間。
driver.implicitly_wait(10)
all_links = []
# 超鏈接
links=driver.find_elements_by_xpath('//*[@id="wrap"]/div[2]/div[2]/div/div[1]/div[2]/ul/li/div[1]/a')
for i in links:
    all_links.append(i.get_attribute('href'))
print('all_links:\n':all_links)
# 關(guān)閉瀏覽器
driver.quit()

all_links:
['https://www.zhipin.com/job_detail/51130f7cf022920c1nZ72tm4F1BS.html?lid=2u4NYccXIT6.search.1&securityId=Wrf6Ju0q891dB-v1-KH4S3MEZ6XB5SqDB_1Fp7sEOGhhdMQpoPnFeQBPYifubMCJ1iOa-PXgN5UiRSKqV-XZZwnL_uoZQiluwP8Kq8exEJgIsfQsKPZu&sessionId=',
'https://www.zhipin.com/job_detail/7634a64b01109ed51XF52tu5FFpU.html?lid=2u4NYccXIT6.search.2&securityId=oDlK2lguTb4eX-M1yrqXbbh5TH1j7RS3eqy-ArQEneYjgbFolwR2PPyGbbRBloLpBJHbT3AnRsU8emKTjqSEpENaaxCGAQ70oGfyDQDrEG21vV4BTw~~&sessionId=', 
'https://www.zhipin.com/job_detail/cc2370d01d570b5f0XZ80967ElQ~.html?lid=2u4NYccXIT6.search.3&securityId=MZnPCmBEEgEuJ-Y1qbNCS-1p9d4JkX3sVCSWLjjds10wKf95TOWnoYxqMzKlHZ1CLO-1TnUjabnGixzyNUceSvr6FyPPuSUxX2tTv8O7EsEjv3O_&sessionId=', 
'https://www.zhipin.com/job_detail/4c9e6e7ede9dd95a1XF62d26EVRT.html?lid=2u4NYccXIT6.search.4&securityId=JGkGN673lbp85-W149dR6DaRzOf5wcDIewGU3c52-EIyZ8s5EsHNaqNdc5PjHnXc5WWwJECjAikjAuzRvN2kZSt80dPGnOkLO_o8fgsuGNoIZerW5JM~&sessionId=', 
'https://www.zhipin.com/job_detail/e5501e5949ed4c0c1nZz2tm8ElZT.html?lid=2u4NYccXIT6.search.5&securityId=Tg1bEND7CL5dr-N1ocgrg1AjE8Hiy_0W3jeqblLj_cJ91dZ9K_fUN_vlKCHW-eJGHZeCaoxTy6MRUk_30GfKekqof1l1NBuk3CgAF0BRMis7FWdqgw~~&sessionId=', 
'https://www.zhipin.com/job_detail/b083288fd86f60341Xd40968GVBT.html?lid=2u4NYccXIT6.search.6&securityId=lOb153Refp1jY-j124U-9vdL8OVpfDoeJsYd3qAaszGlUga35frIuFzQFr28GXkSNHHS6Xerv1HtjEEIpIVOsnfkoyN7JoCowgtIP9y0c3cIJ5fBBA~~&sessionId=', 
'https://www.zhipin.com/job_detail/f1935b91115865b71Xd73Nm7E1BZ.html?lid=2u4NYccXIT6.search.7&securityId=no9awCmwc9ENF-c1Yhm-ooEXLFmUPEmSrVVBHb486rL6sF8nQR-gg3fJ8CoXn3r-M_NYvZMyguu7E2-Vtr5OB1SYuzTOUp7td8eMhHhnEsWAmDV81A~~&sessionId=', 
'https://www.zhipin.com/job_detail/c17bb1992367d5391nFy39y5E1JT.html?lid=2u4NYccXIT6.search.8&securityId=8yDZPiRjHPNeV-X19P8eh-81r_7dUDzXpv32APys-CFyvc2RtJVu1UU2Y1z176G0HvM9eZYLwIz1HX_vHFBwiAOhjhKDFuCDGWB4u740_jYg0cRxS6I~&sessionId=', 
'https://www.zhipin.com/job_detail/1426b6ff57d2db7e1XB92N6-GFNW.html?lid=2u4NYccXIT6.search.9&securityId=oEnznOvwjDI3W-W1VWvfVyHw2kFqcyE_tbWHZquef1mvOsZImuE6oxLU-PUuLGpJOXfseJ15796BYtm2cE84vG9xuXZ3j-nyx3wnIKSH0SY9pnBThkY~&sessionId=', 
'https://www.zhipin.com/job_detail/8d14dd627bd80af81n173tq9FFpY.html?lid=2u4NYccXIT6.search.10&securityId=3qGNKBq74pbCs-813w9Exyl9z3fvSQQy4ZVTZWHGFH0s-O8xvQZf596-b7Gsegf5RxmTPeE1IwwoIEIMh49VBbo1dV-kugJRV9rU7rCgA9w0FBgBuA~~&sessionId=', 
'https://www.zhipin.com/job_detail/6538d1ae0f1e07ea0X1_29i8EVQ~.html?lid=2u4NYccXIT6.search.11&securityId=xLw96mBuLIPX4-u1m5FsIID-G0-2uMwuBMeowYgXwohjjkIK3_8OCZC_awlG2qik8ZgOBKOjK9fporTq8CdFYYNO_neuNyoeC43RY7vJ17ji4A9z&sessionId=', 
'https://www.zhipin.com/job_detail/c7d67e5791040d2f1XB63dy4FVdW.html?lid=2u4NYccXIT6.search.12&securityId=hVNq6XqfJIYpB-J1nN5KVNIgzSVefGZVXFRrfnQZmV1jyUgd2NEHzEElOdnZPfic1dHx1jWV9ohIllZeJm0sYQl-Ci3IkiEVuxXzSR2XdKztHeqoSw~~&sessionId=', 
'https://www.zhipin.com/job_detail/11307d89f7002a641XF42N24F1FY.html?lid=2u4NYccXIT6.search.13&securityId=1mIvbYuaUlVf7-s192IXpv2ovP66NRGNzB5SVdWJ6DD2pZMEc_F8pKC34iSljZJQs6cGodVIVG5jUzk9IriSLocMO4CxvO-SyoaWyatEPtPaWTQi77w~&sessionId=', 
'https://www.zhipin.com/job_detail/648e24692691756b1XF63N-5GFpU.html?lid=2u4NYccXIT6.search.14&securityId=-VF0MDVs3u35I-I1Yok_LwTiYiSpXsxsxkPoUjrvDXcjGqc65CyWlmTKeqTiUUnvfdBBZbtoKj8YkMfLMInGc3ca028svXTuUlI9qMJ70Z2E5TMG&sessionId=', 
'https://www.zhipin.com/job_detail/876263a9f6ff67821nF62Nu9EVJQ.html?lid=2u4NYccXIT6.search.15&securityId=ID_TMnru4dmH3-Q1sMsOu_tAOyyr28xA4atYMSOLih2MSZglfPmbJnDey9fGoX3H7Ajhq2gfFDsIRePStIL-YBf3ejuyhWHxJwnK2YnV5th0touNuQ~~&sessionId=', 
'https://www.zhipin.com/job_detail/93fa6b26874889b11XB_2t-6FltR.html?lid=2u4NYccXIT6.search.16&securityId=vSB0onzr-1bRT-E1D6QlWdJlVlZb-3_lJiW_WYNJ1chOIs_WVlOa2hIS8r9IRQZMOWP7xN_aaxl7ZgFmend2TFjHvtK5jmJ9VRi-Z_1tM8qWQ9s7ia5v&sessionId=', 
'https://www.zhipin.com/job_detail/3b16bfb732d48f0e1XR509--EFJQ.html?lid=2u4NYccXIT6.search.17&securityId=QS2k4km47ReZ7-b1crpZw5BA_3fVtifvRorS5gSWW-QzbNuyhdoI3CALKpRwby045LY8smmC12g5ZfN5tA9vRVVPzraQmh6OCwY3HoaUEBvvWvmIOpw~&sessionId=', 
'https://www.zhipin.com/job_detail/620ab2873e7b6d611n1_0920F1BS.html?lid=2u4NYccXIT6.search.18&securityId=eg8rJyP34wrgf-v1TShFLmFg-l7q-7NP7XxbyBtBjes9CFZkpChq4NhRS1uibKk-9yd95m5kC7mjFVwVtbswVD1YAMRMVKgZliiyrZ7uM3a3s1sM9e4~&sessionId=', 
'https://www.zhipin.com/job_detail/7507d3c13c19d0ea1nJ709i0EVNU.html?lid=2u4NYccXIT6.search.19&securityId=FkIOur07GILxV-j1srQmAMPHo9e2QL5kVyqeugujI25YkJUz_JhtwSYZ_QBN_9CczLaxTa88snUAH2uMmBHT7mLWm_Bejs-rI_68maLixkgKGyGoFQ~~&sessionId=', 
'https://www.zhipin.com/job_detail/3eb26c73fdfea05e1Xdy09m1EFNW.html?lid=2u4NYccXIT6.search.20&securityId=mrMOyVV1-rLpZ-L11YRlCh6iLQ51-EybD8SyZKxieRBiaXX_AcyH--tF0jVgR8Pt7ZKY3ONVihWHc-G68A_s1yE3ys5W3-ZfsIwfBHtD4E1USZZEX6A~&sessionId=', 
'https://www.zhipin.com/job_detail/214537274f3b8def1XV90t-_FldZ.html?lid=2u4NYccXIT6.search.21&securityId=yDUKAWzTIUMON-E1ANnN9oKEXVrJVaXj-B_Gqx7nvEQ_nBfL5wAIWVCRPywfJOnxkbk3S1vLwpud1RcttKJrrKsSkG6flVJvR-GWSW_ZBpH8U_CDww~~&sessionId=', 
'https://www.zhipin.com/job_detail/2a728edb6c98854e3nV93tm7ElU~.html?lid=2u4NYccXIT6.search.22&securityId=7lACZthfL5-2k-A1XD9zd3G4LpMdfUIhSgdcon9UDOKcD3VSa3_UIUlNEeEVqS31c81XB929W-EltX.html?lid=2u4NYccXIT6.search.23&securityId=ydly6xajXCxz2-Z13QPs7clJQhwEk2brx0pFH94T-twmDmSIBKM3dLo-z4crjdbWC18wxj4xrQZ2ZdLoZwfLWZfOqC_CRtMKKlCVVq4GCX0VUnFZZC4~&sessionId=', 
'https://www.zhipin.com/job_detail/4b34a687816d4b6a1XZ529S0FlZZ.html?lid=2u4NYccXIT6.search.24&securityId=XPdhBlAs2a7Wr-_1Crcu_FVbIl9kUjigiapTYQTbBERz-QVKPmeTf7TT_YPia0y8RDv5crn5EJJOHXtOizRW_p_Pbnvxyppb2hYzcs7b-yTD60BqJQ~~&sessionId=', 
'https://www.zhipin.com/job_detail/e0f13a2d5afb919e1XZ43dm6FlFX.html?lid=2u4NYccXIT6.search.25&securityId=JHpoTGIllDTWz-41j8IDGteCA5sRXCCXdZQdU3_MeAJOio-rv0HMsrAvqxOlghX6M7Z62CrH7v-Rv7BJJqfu4UcHF4I1JZrN6QdfR3tR5WP14b4J5Q~~&sessionId=', 
'https://www.zhipin.com/job_detail/7d3d524dbd64b92f1XB-3N61EFNR.html?lid=2u4NYccXIT6.search.26&securityId=I67CL5DLTqhbT-E12sAj57OTTRY1oX0E3xFqs_l0miBCoeKsxOWPt_j1YJGe0e0KgEwFdVZKYe4TbikUec4RV_1_JcmVwWwD1qK6qWFN87kXduBDPOs~&sessionId=', 
'https://www.zhipin.com/job_detail/75a1767b3779781e1n142d-0F1JS.html?lid=2u4NYccXIT6.search.27&securityId=jW7SHWIWK67xH-G1_UkokmVqoZqvWqfP3RRCqK1hrnHZeTR_LbgRFGnJmHKnOEoqKYUvNEzvK6Z3g-2_OvGGJgGCmLzJUxgXvfLm8o10H8w0DBnNxsmL&sessionId=', 
'https://www.zhipin.com/job_detail/74084650d3ef7c7f1XR72tu7EFNX.html?lid=2u4NYccXIT6.search.28&securityId=gaYBuF_s7iF5x-s1XWg07odOcl11GrbSNEyXlRWLenHjAvYT__lPujMYtkUsso3tDjQ2OyaMRRUYFosM4TmP2mSRrlYrRRzTZ1TK41d6OTTTdvZ7-lM~&sessionId=', 
'https://www.zhipin.com/job_detail/c10b1224f7fb3a6e1XB_3dW8FFpU.html?lid=2u4NYccXIT6.search.29&securityId=4GxZox2AmPZg--W1TK4akbtADgYHAm6UcC44faW4HMZiJgqUNYQmd_HytPJ2ZMzCEjs68nJ0tPaz6khdpCGY5NAdxFnYfgIFHT4LI-ELFc-HP1_RZ34~&sessionId=', 
'https://www.zhipin.com/job_detail/19ce985daea5046c1Xd50t68E1VU.html?lid=2u4NYccXIT6.search.30&securityId=LpJtJoDlZq1tc-k1yPTHpOkWYneArdICQ14ktVUWO8FaW-xqcwBHAtnNB38iamjFBCLE3l6zDGBPET338fnepvzyaXHlczk2XTn4C-wwi3n9FbwVAQ~~&sessionId=']

更多精彩內(nèi)容，可點(diǎn)擊進(jìn)入Python日常小操作專欄或我的個(gè)人主頁查看文章來源地址http://www.zghlxwxcb.cn/news/detail-806421.html

到了這里，關(guān)于Python Selenium 獲取動(dòng)態(tài)網(wǎng)頁指定元素的超鏈接的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！