Selenium爬虫Driver的选择

简介:    由于Chrome速度快,因此很早便使用Chrome Driver淘汰了IE Driver和PhantomJS Driver。最近的抓取工作出现了一个令人头疼的事情,单开一个Driver做while 1循环,每隔5分钟扫描抓取目标对象。

   由于Chrome速度快,因此很早便使用Chrome Driver淘汰了IE Driver和PhantomJS Driver。最近的抓取工作出现了一个令人头疼的事情,单开一个Driver做while 1循环,每隔5分钟扫描抓取目标对象。然而Chrome Driver总是会在4~5个小时僵死掉。我做了性能和时间的记录:
  1. Turn 1:硕博家园
  2. 2016-1-6 16:45:00
  3. Mem 1.45GB
  4. 2016-1-6 17:06:45
  5. Mem 1.43GB
  6. chrome 38,000k
  7. 2016-1-6 17:14:14
  8. Turn 2:考研版块
  9. 2016-1-6 17:19:14
  10. Mem 1.43GB
  11. chrome 53,780k
  12. 2016-1-6 17:28:17
  13. Mem 1.32GB
  14. chrome 49,672k
  15. Turn 3:xxxx
  16. 2016-1-6 17:33:14
  17. Mem 1.33GB
  18. chrome 56,000k
  19. 2016-1-6 17:35:30
  20. Mem 1.33GB
  21. chrome 55,728k
  22. Turn 4:lunwentougao
  23. 2016-1-6 17:33:14
  24. Mem 1.33GB
  25. chrome 56,000k
  26. 2016-1-6 17:28:17
  27. Mem 1.32GB
  28. chrome 49,672k
  29. ------------------------------------------
  30. 2016-01-11 13:26:56
  31. 2016-01-11 17:20:42
  32. 22:23:30
  33. 02:22:28
  34. 2016-01-12 06:50:12
  35. 2016-01-12 11:30:37
  36. 2016-01-12 13:13:28
  37. 2016-01-12 18:05:44
    所以,我打算重新换个Driver试试,到官网的Download页面下(www.seleniumhq.org/download/) ,能够看到现在陆续出现了更多浏览器的Drvier,包括opera、Edge等。

    我下载的是Firefox。当然必须首先在服务器上安装好完整的Firefox浏览器。然后可以在Python中简单调用。capabilities我还不太会用,所以注销了:
  1. from selenium import webdriver
  2. from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

  3. #driver = webdriver.Chrome('C:\Python27\Scripts\chromedriver')
  4. #firefox_capabilities = DesiredCapabilities.FIREFOX
  5. #firefox_capabilities['marionette'] = True
  6. #firefox_capabilities['binary'] = u'C:\Python27\Scripts'

  7. #driver = webdriver.Firefox(capabilities=firefox_capabilities)
  8. driver = webdriver.Firefox()
      经过一晚的测试,发现效果不错,没有什么问题,机器一直在正确的轨道上运行着:
  1. Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32
  2. Type "copyright", "credits" or "license()" for more information.
  3. >>>
  4. ================== RESTART: C:\datascrawl\ccnu_hf\ccnuHf.py ==================
  5. 2016-01-12 19:27:25
  6. 2016-01-12 19:27:45
  7. 2016-01-12 19:28:47
  8. 2016-01-12 19:29:47
  9. 2016-01-12 19:30:09
  10. 2016-01-12 19:30:31
  11. 2016-01-12 19:30:53
  12. 2016-01-12 19:31:15
  13. Again_______________________:
  14. 2016-01-12 19:36:15
  15. 2016-01-12 19:36:37
  16. 2016-01-12 19:36:57
  17. 2016-01-12 19:37:38
  18. 2016-01-12 19:38:00
  19. 2016-01-12 19:38:22
  20. 2016-01-12 19:38:44
  21. 2016-01-12 19:39:06
  22. 2016-01-12 19:39:26
  23. Again_______________________:
  24. 2016-01-12 19:44:26
  25. 2016-01-12 19:44:48
  26. 2016-01-12 19:45:08
  27. 2016-01-12 19:45:30
  28. 2016-01-12 19:46:11
  29. 2016-01-12 19:46:33
  30. 2016-01-12 19:46:55
  31. 2016-01-12 19:47:17
  32. 2016-01-12 19:47:39
  33. Again_______________________:
  34. 2016-01-12 19:52:39
  35. 2016-01-12 19:53:20
  36. 2016-01-12 19:53:40
  37. 2016-01-12 19:54:02
  38. 2016-01-12 19:54:24
  39. 2016-01-12 19:54:46
  40. 2016-01-12 19:55:46
  41. 2016-01-12 19:56:08
  42. 2016-01-12 19:56:30
  43. Again_______________________:
  44. 2016-01-12 20:01:30
  45. 2016-01-12 20:01:52
  46. 2016-01-12 20:02:12
  47. 2016-01-12 20:02:35
  48. 2016-01-12 20:02:57
  49. 2016-01-12 20:03:18
  50. 2016-01-12 20:03:59
  51. 2016-01-12 20:04:21
  52. 2016-01-12 20:04:43
  53. Again_______________________:
  54. 2016-01-12 20:09:43
  55. 2016-01-12 20:10:25
  56. 2016-01-12 20:11:03
  57. 2016-01-12 20:11:24
  58. 2016-01-12 20:11:46
  59. 2016-01-12 20:12:08
  60. 2016-01-12 20:12:30
  61. 2016-01-12 20:12:52
  62. 2016-01-12 20:13:33
  63. Again_______________________:
  64. 2016-01-12 20:18:33
  65. 2016-01-12 20:18:55
  66. 2016-01-12 20:19:32
  67. 2016-01-12 20:19:55
  68. 2016-01-12 20:20:17
  69. 2016-01-12 20:20:39
  70. 2016-01-12 20:21:13
  71. 2016-01-12 20:21:48
  72. 2016-01-12 20:22:23
  73. Again_______________________:
  74. 2016-01-12 20:27:23
  75. 2016-01-12 20:28:17
  76. 2016-01-12 20:28:48
  77. 2016-01-12 20:29:23
  78. 2016-01-12 20:29:57
  79. 2016-01-12 20:30:32
  80. 2016-01-12 20:31:07
  81. 2016-01-12 20:32:01
  82. 2016-01-12 20:32:37
  83. Again_______________________:
  84. 2016-01-12 20:37:37
  85. 2016-01-12 20:38:12
  86. 2016-01-12 20:38:32
  87. 2016-01-12 20:38:54
  88. 2016-01-12 20:39:16
  89. 2016-01-12 20:39:38
  90. 2016-01-12 20:40:00
  91. 2016-01-12 20:40:21
  92. 2016-01-12 20:40:43
  93. Again_______________________:
  94. 2016-01-12 20:45:43
  95. 2016-01-12 20:46:06
  96. 2016-01-12 20:46:26
  97. 2016-01-12 20:46:48
  98. 2016-01-12 20:47:10
  99. 2016-01-12 20:47:32
  100. 2016-01-12 20:47:53
  101. 2016-01-12 20:48:15
  102. 2016-01-12 20:48:38
  103. Again_______________________:
  104. 2016-01-12 20:53:38
  105. 2016-01-12 20:53:59
  106. 2016-01-12 20:54:20
  107. 2016-01-12 20:54:42
  108. 2016-01-12 20:55:03
  109. 2016-01-12 20:55:25
  110. 2016-01-12 20:55:47
  111. 2016-01-12 20:56:09
  112. 2016-01-12 20:56:31
  113. Again_______________________:
  114. 2016-01-12 21:01:31
  115. 2016-01-12 21:01:53
  116. 2016-01-12 21:02:31
  117. 2016-01-12 21:02:53
  118. 2016-01-12 21:03:15
  119. 2016-01-12 21:03:37
  120. 2016-01-12 21:03:59
  121. 2016-01-12 21:04:21
  122. 2016-01-12 21:04:46
  123. Again_______________________:
  124. 2016-01-12 21:09:46
  125. 2016-01-12 21:10:08
  126. 2016-01-12 21:10:28
  127. 2016-01-12 21:11:09
  128. 2016-01-12 21:11:31
  129. 2016-01-12 21:12:12
  130. 2016-01-12 21:12:34
  131. 2016-01-12 21:12:56
  132. 2016-01-12 21:13:18
  133. Again_______________________:
  134. 2016-01-12 21:18:18
  135. 2016-01-12 21:18:40
  136. 2016-01-12 21:19:00
  137. 2016-01-12 21:19:41
  138. 2016-01-12 21:20:03
  139. 2016-01-12 21:20:45
  140. 2016-01-12 21:21:07
  141. 2016-01-12 21:21:29
  142. 2016-01-12 21:21:51
  143. Again_______________________:
  144. 2016-01-12 21:26:51
  145. 2016-01-12 21:27:32
  146. 2016-01-12 21:27:52
  147. 2016-01-12 21:28:14
  148. 2016-01-12 21:28:36
  149. 2016-01-12 21:28:58
  150. 2016-01-12 21:29:20
  151. 2016-01-12 21:29:42
  152. 2016-01-12 21:30:04
  153. Again_______________________:
  154. 2016-01-12 21:35:04
  155. 2016-01-12 21:35:44
  156. 2016-01-12 21:36:05
  157. 2016-01-12 21:36:45
  158. 2016-01-12 21:37:07
  159. 2016-01-12 21:37:29
  160. 2016-01-12 21:37:51
  161. 2016-01-12 21:38:13
  162. 2016-01-12 21:38:35
  163. Again_______________________:
  164. 2016-01-12 21:43:35
  165. 2016-01-12 21:43:57
  166. 2016-01-12 21:44:17
  167. 2016-01-12 21:44:39
  168. 2016-01-12 21:45:01
  169. 2016-01-12 21:45:23
  170. 2016-01-12 21:46:03
  171. 2016-01-12 21:46:25
  172. 2016-01-12 21:46:47
  173. Again_______________________:
  174. 2016-01-12 21:51:47
  175. 2016-01-12 21:52:09
  176. 2016-01-12 21:52:29
  177. 2016-01-12 21:52:51
  178. 2016-01-12 21:53:13
  179. 2016-01-12 21:53:34
  180. 2016-01-12 21:53:57
  181. 2016-01-12 21:54:19
  182. 2016-01-12 21:54:59
  183. Again_______________________:
  184. 2016-01-12 21:59:59
  185. 2016-01-12 22:01:00
  186. 2016-01-12 22:01:37
  187. 2016-01-12 22:02:01
  188. 2016-01-12 22:02:23
  189. 2016-01-12 22:02:44
  190. 2016-01-12 22:03:26
  191. 2016-01-12 22:03:48
  192. 2016-01-12 22:04:10
  193. Again_______________________:
  194. 2016-01-12 22:09:10
  195. 2016-01-12 22:09:32
  196. 2016-01-12 22:09:52
  197. 2016-01-12 22:10:14
  198. 2016-01-12 22:10:34
  199. 2016-01-12 22:10:55
  200. 2016-01-12 22:11:51
  201. 2016-01-12 22:12:13
  202. 2016-01-12 22:12:35
  203. Again_______________________:
  204. 2016-01-12 22:17:35
  205. 2016-01-12 22:17:57
  206. 2016-01-12 22:18:18
  207. 2016-01-12 22:18:39
  208. 2016-01-12 22:19:01
  209. 2016-01-12 22:19:43
  210. 2016-01-12 22:20:05
  211. 2016-01-12 22:20:27
  212. 2016-01-12 22:20:48
  213. Again_______________________:
  214. 2016-01-12 22:25:49
  215. 2016-01-12 22:26:11
  216. 2016-01-12 22:26:31
  217. 2016-01-12 22:27:12
  218. 2016-01-12 22:27:34
  219. 2016-01-12 22:27:55
  220. 2016-01-12 22:28:17
  221. 2016-01-12 22:28:39
  222. 2016-01-12 22:29:01
  223. Again_______________________:
  224. 2016-01-12 22:34:01
  225. 2016-01-12 22:34:42
  226. 2016-01-12 22:35:03
  227. 2016-01-12 22:35:25
  228. 2016-01-12 22:35:46
  229. 2016-01-12 22:36:08
  230. 2016-01-12 22:36:49
  231. 2016-01-12 22:37:11
  232. 2016-01-12 22:37:33
  233. Again_______________________:
  234. 2016-01-12 22:42:33
  235. 2016-01-12 22:43:15
  236. 2016-01-12 22:43:35
  237. 2016-01-12 22:43:57
  238. 2016-01-12 22:44:19
  239. 2016-01-12 22:44:41
  240. 2016-01-12 22:45:21
  241. 2016-01-12 22:45:43
  242. 2016-01-12 22:46:05
  243. Again_______________________:
  244. 2016-01-12 22:51:05
  245. 2016-01-12 22:51:27
  246. 2016-01-12 22:51:48
  247. 2016-01-12 22:52:10
  248. 2016-01-12 22:52:31
  249. 2016-01-12 22:53:13
  250. 2016-01-12 22:53:54
  251. 2016-01-12 22:54:34
  252. 2016-01-12 22:54:56
  253. Again_______________________:
  254. 2016-01-12 22:59:56
  255. 2016-01-12 23:00:18
  256. 2016-01-12 23:00:38
  257. 2016-01-12 23:01:00
  258. 2016-01-12 23:01:22
  259. 2016-01-12 23:01:44
  260. 2016-01-12 23:02:06
  261. 2016-01-12 23:02:28
  262. 2016-01-12 23:02:50
  263. Again_______________________:
  264. 2016-01-12 23:07:50
  265. 2016-01-12 23:08:12
  266. 2016-01-12 23:08:49
  267. 2016-01-12 23:09:30
  268. 2016-01-12 23:09:51
  269. 2016-01-12 23:10:13
  270. 2016-01-12 23:10:35
  271. 2016-01-12 23:10:57
  272. 2016-01-12 23:11:19
  273. Again_______________________:
  274. 2016-01-12 23:16:19
  275. 2016-01-12 23:17:00
  276. 2016-01-12 23:17:20
  277. 2016-01-12 23:17:42
  278. 2016-01-12 23:18:23
  279. 2016-01-12 23:19:05
  280. 2016-01-12 23:19:26
  281. 2016-01-12 23:19:48
  282. 2016-01-12 23:20:29
  283. Again_______________________:
  284. 2016-01-12 23:25:29
  285. 2016-01-12 23:26:09
  286. 2016-01-12 23:26:30
  287. 2016-01-12 23:26:52
  288. 2016-01-12 23:28:54
  289. 2016-01-12 23:29:16
  290. 2016-01-12 23:29:57
  291. 2016-01-12 23:30:18
  292. 2016-01-12 23:30:40
  293. Again_______________________:
  294. 2016-01-12 23:35:40
  295. 2016-01-12 23:36:02
  296. 2016-01-12 23:36:22
  297. 2016-01-12 23:36:44
  298. 2016-01-12 23:37:06
  299. 2016-01-12 23:37:47
  300. 2016-01-12 23:38:10
  301. 2016-01-12 23:38:31
  302. 2016-01-12 23:38:54
  303. Again_______________________:
  304. 2016-01-12 23:43:54
  305. 2016-01-12 23:44:15
  306. 2016-01-12 23:44:36
  307. 2016-01-12 23:44:58
  308. 2016-01-12 23:45:20
  309. 2016-01-12 23:45:42
  310. 2016-01-12 23:46:03
  311. 2016-01-12 23:46:44
  312. 2016-01-12 23:47:06
  313. Again_______________________:
  314. 2016-01-12 23:52:06
  315. 2016-01-12 23:52:28
  316. 2016-01-12 23:52:48
  317. 2016-01-12 23:53:09
  318. 2016-01-12 23:53:29
  319. 2016-01-12 23:53:50
  320. 2016-01-12 23:54:28
  321. 2016-01-12 23:54:50
  322. 2016-01-12 23:55:12
  323. Again_______________________:
  324. 2016-01-13 00:00:12
  325. 2016-01-13 00:00:34
  326. 2016-01-13 00:00:54
  327. 2016-01-13 00:01:36
  328. 2016-01-13 00:01:57
  329. 2016-01-13 00:02:19
  330. 2016-01-13 00:02:41
  331. 2016-01-13 00:03:03
  332. 2016-01-13 00:03:25
  333. Again_______________________:
  334. 2016-01-13 00:08:25
  335. 2016-01-13 00:08:47
  336. 2016-01-13 00:09:07
  337. 2016-01-13 00:09:29
  338. 2016-01-13 00:09:51
  339. 2016-01-13 00:10:13
  340. 2016-01-13 00:10:35
  341. 2016-01-13 00:10:57
  342. 2016-01-13 00:11:19
  343. Again_______________________:
  344. 2016-01-13 00:16:19
  345. 2016-01-13 00:16:41
  346. 2016-01-13 00:17:01
  347. 2016-01-13 00:17:23
  348. 2016-01-13 00:17:45
  349. 2016-01-13 00:18:07
  350. 2016-01-13 00:18:47
  351. 2016-01-13 00:19:09
  352. 2016-01-13 00:19:32
  353. Again_______________________:
  354. 2016-01-13 00:24:32
  355. 2016-01-13 00:25:33
  356. 2016-01-13 00:25:53
  357. 2016-01-13 00:26:15
  358. 2016-01-13 00:26:37
  359. 2016-01-13 00:26:59
  360. 2016-01-13 00:27:20
  361. 2016-01-13 00:27:43
  362. 2016-01-13 00:28:05
  363. Again_______________________:
  364. 2016-01-13 00:33:05
  365. 2016-01-13 00:33:27
  366. 2016-01-13 00:33:48
  367. 2016-01-13 00:34:10
  368. 2016-01-13 00:34:32
  369. 2016-01-13 00:34:56
  370. 2016-01-13 00:35:18
  371. 2016-01-13 00:35:40
  372. 2016-01-13 00:36:02
  373. Again_______________________:
  374. 2016-01-13 00:41:02
  375. 2016-01-13 00:41:24
  376. 2016-01-13 00:41:45
  377. 2016-01-13 00:42:25
  378. 2016-01-13 00:42:48
  379. 2016-01-13 00:43:09
  380. 2016-01-13 00:43:32
  381. 2016-01-13 00:44:12
  382. 2016-01-13 00:44:34
  383. Again_______________________:
  384. 2016-01-13 00:49:34
  385. 2016-01-13 00:49:56
  386. 2016-01-13 00:50:16
  387. 2016-01-13 00:50:38
  388. 2016-01-13 00:51:00
  389. 2016-01-13 00:51:22
  390. 2016-01-13 00:51:44
  391. 2016-01-13 00:52:06
  392. 2016-01-13 00:52:27
  393. Again_______________________:
  394. 2016-01-13 00:57:27
  395. 2016-01-13 00:57:50
  396. 2016-01-13 00:58:10
  397. 2016-01-13 00:58:32
  398. 2016-01-13 00:58:54
  399. 2016-01-13 00:59:16
  400. 2016-01-13 00:59:38
  401. 2016-01-13 00:59:59
  402. 2016-01-13 01:00:41
  403. Again_______________________:
  404. 2016-01-13 01:05:41
  405. 2016-01-13 01:06:03
  406. 2016-01-13 01:06:24
  407. 2016-01-13 01:06:45
  408. 2016-01-13 01:07:07
  409. 2016-01-13 01:07:29
  410. 2016-01-13 01:07:51
  411. 2016-01-13 01:08:13
  412. 2016-01-13 01:08:35
  413. Again_______________________:
  414. 2016-01-13 01:13:35
  415. 2016-01-13 01:13:57
  416. 2016-01-13 01:14:17
  417. 2016-01-13 01:14:39
  418. 2016-01-13 01:15:01
  419. 2016-01-13 01:15:23
  420. 2016-01-13 01:15:45
  421. 2016-01-13 01:16:06
  422. 2016-01-13 01:16:28
  423. Again_______________________:
  424. 2016-01-13 01:21:28
  425. 2016-01-13 01:21:50
  426. 2016-01-13 01:22:10
  427. 2016-01-13 01:22:32
  428. 2016-01-13 01:22:54
  429. 2016-01-13 01:23:16
  430. 2016-01-13 01:23:38
  431. 2016-01-13 01:23:59
  432. 2016-01-13 01:24:21
  433. Again_______________________:
  434. 2016-01-13 01:29:21
  435. 2016-01-13 01:29:43
  436. 2016-01-13 01:30:03
  437. 2016-01-13 01:30:25
  438. 2016-01-13 01:30:47
  439. 2016-01-13 01:31:09
  440. 2016-01-13 01:31:31
  441. 2016-01-13 01:31:52
  442. 2016-01-13 01:32:14
  443. Again_______________________:
  444. 2016-01-13 01:37:14
  445. 2016-01-13 01:37:36
  446. 2016-01-13 01:37:57
  447. 2016-01-13 01:38:18
  448. 2016-01-13 01:38:40
  449. 2016-01-13 01:39:02
  450. 2016-01-13 01:39:24
  451. 2016-01-13 01:39:46
  452. 2016-01-13 01:40:07
  453. Again_______________________:
  454. 2016-01-13 01:45:08
  455. 2016-01-13 01:45:29
  456. 2016-01-13 01:45:50
  457. 2016-01-13 01:46:12
  458. 2016-01-13 01:46:33
  459. 2016-01-13 01:46:55
  460. 2016-01-13 01:47:17
  461. 2016-01-13 01:47:39
  462. 2016-01-13 01:48:01
  463. Again_______________________:
  464. 2016-01-13 01:53:01
  465. 2016-01-13 01:53:23
  466. 2016-01-13 01:53:43
  467. 2016-01-13 01:54:05
  468. 2016-01-13 01:54:27
  469. 2016-01-13 01:54:48
  470. 2016-01-13 01:55:10
  471. 2016-01-13 01:55:32
  472. 2016-01-13 01:55:53
  473. Again_______________________:
  474. 2016-01-13 02:00:53
  475. 2016-01-13 02:01:15
  476. 2016-01-13 02:01:36
  477. 2016-01-13 02:01:58
  478. 2016-01-13 02:02:19
  479. 2016-01-13 02:02:41
  480. 2016-01-13 02:03:03
  481. 2016-01-13 02:03:25
  482. 2016-01-13 02:03:47
  483. Again_______________________:
  484. 2016-01-13 02:08:47
  485. 2016-01-13 02:09:08
  486. 2016-01-13 02:09:29
  487. 2016-01-13 02:09:51
  488. 2016-01-13 02:10:12
  489. 2016-01-13 02:10:35
  490. 2016-01-13 02:11:16
  491. 2016-01-13 02:11:37
  492. 2016-01-13 02:11:59
  493. Again_______________________:
  494. 2016-01-13 02:16:59
  495. 2016-01-13 02:17:21
  496. 2016-01-13 02:17:41
  497. 2016-01-13 02:18:03
  498. 2016-01-13 02:18:25
  499. 2016-01-13 02:18:47
  500. 2016-01-13 02:19:09
  501. 2016-01-13 02:19:30
  502. 2016-01-13 02:19:52
  503. Again_______________________:
  504. 2016-01-13 02:24:52
  505. 2016-01-13 02:25:14
  506. 2016-01-13 02:25:34
  507. 2016-01-13 02:25:56
  508. 2016-01-13 02:26:18
  509. 2016-01-13 02:26:40
  510. 2016-01-13 02:27:02
  511. 2016-01-13 02:27:23
  512. 2016-01-13 02:27:45
  513. Again_______________________:
  514. 2016-01-13 02:32:45
  515. 2016-01-13 02:33:07
  516. 2016-01-13 02:33:27
  517. 2016-01-13 02:33:49
  518. 2016-01-13 02:34:11
  519. 2016-01-13 02:34:33
  520. 2016-01-13 02:34:55
  521. 2016-01-13 02:35:16
  522. 2016-01-13 02:35:38
  523. Again_______________________:
  524. 2016-01-13 02:40:38
  525. 2016-01-13 02:41:00
  526. 2016-01-13 02:41:21
  527. 2016-01-13 02:41:42
  528. 2016-01-13 02:42:04
  529. 2016-01-13 02:42:26
  530. 2016-01-13 02:42:47
  531. 2016-01-13 02:43:09
  532. 2016-01-13 02:43:31
  533. Again_______________________:
  534. 2016-01-13 02:48:31
  535. 2016-01-13 02:48:53
  536. 2016-01-13 02:49:13
  537. 2016-01-13 02:49:35
  538. 2016-01-13 02:49:57
  539. 2016-01-13 02:50:18
  540. 2016-01-13 02:50:40
  541. 2016-01-13 02:51:02
  542. 2016-01-13 02:51:24
  543. Again_______________________:
  544. 2016-01-13 02:56:24
  545. 2016-01-13 02:56:46
  546. 2016-01-13 02:57:06
  547. 2016-01-13 02:57:28
  548. 2016-01-13 02:57:50
  549. 2016-01-13 02:58:11
  550. 2016-01-13 02:58:33
  551. 2016-01-13 02:58:55
  552. 2016-01-13 02:59:17
  553. Again_______________________:
  554. 2016-01-13 03:04:17
  555. 2016-01-13 03:04:38
  556. 2016-01-13 03:04:59
  557. 2016-01-13 03:05:21
  558. 2016-01-13 03:05:42
  559. 2016-01-13 03:06:23
  560. 2016-01-13 03:06:45
  561. 2016-01-13 03:07:07
  562. 2016-01-13 03:07:29
  563. Again_______________________:
  564. 2016-01-13 03:12:29
  565. 2016-01-13 03:12:50
  566. 2016-01-13 03:13:11
  567. 2016-01-13 03:13:33
  568. 2016-01-13 03:13:54
  569. 2016-01-13 03:14:16
  570. 2016-01-13 03:14:38
  571. 2016-01-13 03:15:00
  572. 2016-01-13 03:15:22
  573. Again_______________________:
  574. 2016-01-13 03:20:22
  575. 2016-01-13 03:20:43
  576. 2016-01-13 03:21:04
  577. 2016-01-13 03:21:25
  578. 2016-01-13 03:21:47
  579. 2016-01-13 03:22:09
  580. 2016-01-13 03:22:31
  581. 2016-01-13 03:22:53
  582. 2016-01-13 03:23:15
  583. Again_______________________:
  584. 2016-01-13 03:28:15
  585. 2016-01-13 03:28:36
  586. 2016-01-13 03:28:57
  587. 2016-01-13 03:29:18
  588. 2016-01-13 03:29:40
  589. 2016-01-13 03:30:02
  590. 2016-01-13 03:30:24
  591. 2016-01-13 03:30:45
  592. 2016-01-13 03:31:07
  593. Again_______________________:
  594. 2016-01-13 03:36:07
  595. 2016-01-13 03:36:29
  596. 2016-01-13 03:36:49
  597. 2016-01-13 03:37:11
  598. 2016-01-13 03:37:33
  599. 2016-01-13 03:37:54
  600. 2016-01-13 03:38:16
  601. 2016-01-13 03:38:38
  602. 2016-01-13 03:39:00
  603. Again_______________________:
  604. 2016-01-13 03:44:00
  605. 2016-01-13 03:44:22
  606. 2016-01-13 03:44:42
  607. 2016-01-13 03:45:04
  608. 2016-01-13 03:45:26
  609. 2016-01-13 03:45:47
  610. 2016-01-13 03:46:09
  611. 2016-01-13 03:46:31
  612. 2016-01-13 03:46:53
  613. Again_______________________:
  614. 2016-01-13 03:51:53
  615. 2016-01-13 03:52:15
  616. 2016-01-13 03:52:35
  617. 2016-01-13 03:52:57
  618. 2016-01-13 03:53:19
  619. 2016-01-13 03:53:41
  620. 2016-01-13 03:54:01
  621. 2016-01-13 03:54:22
  622. 2016-01-13 03:54:43
  623. Again_______________________:
  624. 2016-01-13 03:59:43
  625. 2016-01-13 04:00:05
  626. 2016-01-13 04:00:25
  627. 2016-01-13 04:00:47
  628. 2016-01-13 04:01:09
  629. 2016-01-13 04:01:31
  630. 2016-01-13 04:01:53
  631. 2016-01-13 04:02:14
  632. 2016-01-13 04:02:36
  633. Again_______________________:
  634. 2016-01-13 04:07:36
  635. 2016-01-13 04:07:58
  636. 2016-01-13 04:08:18
  637. 2016-01-13 04:08:40
  638. 2016-01-13 04:09:02
  639. 2016-01-13 04:09:24
  640. 2016-01-13 04:09:45
  641. 2016-01-13 04:10:07
  642. 2016-01-13 04:10:29
  643. Again_______________________:
  644. 2016-01-13 04:15:29
  645. 2016-01-13 04:15:51
  646. 2016-01-13 04:16:11
  647. 2016-01-13 04:16:33
  648. 2016-01-13 04:16:55
  649. 2016-01-13 04:17:16
  650. 2016-01-13 04:17:38
  651. 2016-01-13 04:18:00
  652. 2016-01-13 04:18:22
  653. Again_______________________:
  654. 2016-01-13 04:23:22
  655. 2016-01-13 04:23:44
  656. 2016-01-13 04:24:04
  657. 2016-01-13 04:24:26
  658. 2016-01-13 04:24:49
  659. 2016-01-13 04:25:11
  660. 2016-01-13 04:25:33
  661. 2016-01-13 04:25:55
  662. 2016-01-13 04:26:17
  663. Again_______________________:
  664. 2016-01-13 04:31:17
  665. 2016-01-13 04:31:39
  666. 2016-01-13 04:31:59
  667. 2016-01-13 04:32:21
  668. 2016-01-13 04:32:43
  669. 2016-01-13 04:33:05
  670. 2016-01-13 04:33:26
  671. 2016-01-13 04:33:48
  672. 2016-01-13 04:34:10
  673. Again_______________________:
  674. 2016-01-13 04:39:10
  675. 2016-01-13 04:39:32
  676. 2016-01-13 04:39:53
  677. 2016-01-13 04:40:15
  678. 2016-01-13 04:40:36
  679. 2016-01-13 04:40:58
  680. 2016-01-13 04:41:20
  681. 2016-01-13 04:41:42
  682. 2016-01-13 04:42:03
  683. Again_______________________:
  684. 2016-01-13 04:47:03
  685. 2016-01-13 04:47:25
  686. 2016-01-13 04:47:45
  687. 2016-01-13 04:48:07
  688. 2016-01-13 04:48:29
  689. 2016-01-13 04:48:51
  690. 2016-01-13 04:49:13
  691. 2016-01-13 04:49:34
  692. 2016-01-13 04:49:56
  693. Again_______________________:
  694. 2016-01-13 04:54:56
  695. 2016-01-13 04:55:18
  696. 2016-01-13 04:55:38
  697. 2016-01-13 04:56:00
  698. 2016-01-13 04:56:22
  699. 2016-01-13 04:56:44
  700. 2016-01-13 04:57:06
  701. 2016-01-13 04:57:28
  702. 2016-01-13 04:57:50
  703. Again_______________________:
  704. 2016-01-13 05:02:50
  705. 2016-01-13 05:03:11
  706. 2016-01-13 05:03:32
  707. 2016-01-13 05:03:54
  708. 2016-01-13 05:04:15
  709. 2016-01-13 05:04:37
  710. 2016-01-13 05:04:59
  711. 2016-01-13 05:05:21
  712. 2016-01-13 05:05:43
  713. Again_______________________:
  714. 2016-01-13 05:10:43
  715. 2016-01-13 05:11:05
  716. 2016-01-13 05:11:25
  717. 2016-01-13 05:11:47
  718. 2016-01-13 05:12:09
  719. 2016-01-13 05:12:30
  720. 2016-01-13 05:12:52
  721. 2016-01-13 05:13:14
  722. 2016-01-13 05:13:36
  723. Again_______________________:
  724. 2016-01-13 05:18:36
  725. 2016-01-13 05:18:57
  726. 2016-01-13 05:19:18
  727. 2016-01-13 05:19:40
  728. 2016-01-13 05:20:02
  729. 2016-01-13 05:20:23
  730. 2016-01-13 05:20:45
  731. 2016-01-13 05:21:07
  732. 2016-01-13 05:21:29
  733. Again_______________________:
  734. 2016-01-13 05:26:29
  735. 2016-01-13 05:26:51
  736. 2016-01-13 05:27:11
  737. 2016-01-13 05:27:33
  738. 2016-01-13 05:27:55
  739. 2016-01-13 05:28:16
  740. 2016-01-13 05:28:38
  741. 2016-01-13 05:29:00
  742. 2016-01-13 05:29:22
  743. Again_______________________:
  744. 2016-01-13 05:34:22
  745. 2016-01-13 05:34:44
  746. 2016-01-13 05:35:04
  747. 2016-01-13 05:35:26
  748. 2016-01-13 05:35:47
  749. 2016-01-13 05:36:09
  750. 2016-01-13 05:36:31
  751. 2016-01-13 05:36:53
  752. 2016-01-13 05:37:15
  753. Again_______________________:
  754. 2016-01-13 05:42:15
  755. 2016-01-13 05:42:37
  756. 2016-01-13 05:42:57
  757. 2016-01-13 05:43:19
  758. 2016-01-13 05:43:41
  759. 2016-01-13 05:44:02
  760. 2016-01-13 05:44:24
  761. 2016-01-13 05:44:46
  762. 2016-01-13 05:45:08
  763. Again_______________________:
  764. 2016-01-13 05:50:08
  765. 2016-01-13 05:50:30
  766. 2016-01-13 05:50:50
  767. 2016-01-13 05:51:12
  768. 2016-01-13 05:51:34
  769. 2016-01-13 05:51:56
  770. 2016-01-13 05:52:17
  771. 2016-01-13 05:52:39
  772. 2016-01-13 05:53:01
  773. Again_______________________:
  774. 2016-01-13 05:58:01
  775. 2016-01-13 05:58:23
  776. 2016-01-13 05:58:43
  777. 2016-01-13 05:59:05
  778. 2016-01-13 05:59:46
  779. 2016-01-13 06:00:08
  780. 2016-01-13 06:00:29
  781. 2016-01-13 06:00:51
  782. 2016-01-13 06:01:13
  783. Again_______________________:
  784. 2016-01-13 06:06:13
  785. 2016-01-13 06:07:13
  786. 2016-01-13 06:07:34
  787. 2016-01-13 06:07:56
  788. 2016-01-13 06:08:18
  789. 2016-01-13 06:08:39
  790. 2016-01-13 06:09:01
  791. 2016-01-13 06:09:23
  792. 2016-01-13 06:09:45
  793. Again_______________________:
  794. 2016-01-13 06:14:45
  795. 2016-01-13 06:15:07
  796. 2016-01-13 06:15:27
  797. 2016-01-13 06:15:49
  798. 2016-01-13 06:16:11
  799. 2016-01-13 06:16:33
  800. 2016-01-13 06:16:54
  801. 2016-01-13 06:17:16
  802. 2016-01-13 06:17:38
  803. Again_______________________:
  804. 2016-01-13 06:22:38
  805. 2016-01-13 06:23:00
  806. 2016-01-13 06:23:20
  807. 2016-01-13 06:23:42
  808. 2016-01-13 06:24:04
  809. 2016-01-13 06:24:26
  810. 2016-01-13 06:24:47
  811. 2016-01-13 06:25:09
  812. 2016-01-13 06:25:31
  813. Again_______________________:
  814. 2016-01-13 06:30:31
  815. 2016-01-13 06:30:53
  816. 2016-01-13 06:31:13
  817. 2016-01-13 06:31:35
  818. 2016-01-13 06:31:57
  819. 2016-01-13 06:32:19
  820. 2016-01-13 06:32:41
  821. 2016-01-13 06:33:03
  822. 2016-01-13 06:33:24
  823. Again_______________________:
  824. 2016-01-13 06:38:24
  825. 2016-01-13 06:38:46
  826. 2016-01-13 06:39:07
  827. 2016-01-13 06:39:29
  828. 2016-01-13 06:39:50
  829. 2016-01-13 06:40:12
  830. 2016-01-13 06:40:34
  831. 2016-01-13 06:40:55
  832. 2016-01-13 06:41:17
  833. Again_______________________:
  834. 2016-01-13 06:46:17
  835. 2016-01-13 06:46:39
  836. 2016-01-13 06:47:00
  837. 2016-01-13 06:47:22
  838. 2016-01-13 06:47:43
  839. 2016-01-13 06:48:05
  840. 2016-01-13 06:48:27
  841. 2016-01-13 06:48:49
  842. 2016-01-13 06:49:11
  843. Again_______________________:
  844. 2016-01-13 06:54:11
  845. 2016-01-13 06:54:33
  846. 2016-01-13 06:54:53
  847. 2016-01-13 06:55:14
  848. 2016-01-13 06:55:36
  849. 2016-01-13 06:55:58
  850. 2016-01-13 06:56:19
  851. 2016-01-13 06:56:41
  852. 2016-01-13 06:57:21
  853. Again_______________________:
  854. 2016-01-13 07:02:21
  855. 2016-01-13 07:02:43
  856. 2016-01-13 07:03:04
  857. 2016-01-13 07:03:25
  858. 2016-01-13 07:03:47
  859. 2016-01-13 07:04:09
  860. 2016-01-13 07:04:31
  861. 2016-01-13 07:04:53
  862. 2016-01-13 07:05:15
  863. Again_______________________:
  864. 2016-01-13 07:10:15
  865. 2016-01-13 07:10:37
  866. 2016-01-13 07:10:57
  867. 2016-01-13 07:11:19
  868. 2016-01-13 07:11:41
  869. 2016-01-13 07:12:03
  870. 2016-01-13 07:12:25
  871. 2016-01-13 07:12:47
  872. 2016-01-13 07:13:09
  873. Again_______________________:
  874. 2016-01-13 07:18:09
  875. 2016-01-13 07:18:31
  876. 2016-01-13 07:18:51
  877. 2016-01-13 07:19:13
  878. 2016-01-13 07:19:35
  879. 2016-01-13 07:19:57
  880. 2016-01-13 07:20:37
  881. 2016-01-13 07:20:59
  882. 2016-01-13 07:21:21
  883. Again_______________________:
  884. 2016-01-13 07:26:21
  885. 2016-01-13 07:26:43
  886. 2016-01-13 07:27:03
  887. 2016-01-13 07:27:25
  888. 2016-01-13 07:27:47
  889. 2016-01-13 07:28:09
  890. 2016-01-13 07:28:31
  891. 2016-01-13 07:28:52
  892. 2016-01-13 07:29:14
  893. Again_______________________:
  894. 2016-01-13 07:34:14
  895. 2016-01-13 07:34:36
  896. 2016-01-13 07:34:56
  897. 2016-01-13 07:35:18
  898. 2016-01-13 07:35:40
  899. 2016-01-13 07:36:02
  900. 2016-01-13 07:36:24
  901. 2016-01-13 07:36:46
  902. 2016-01-13 07:37:08
  903. Again_______________________:
  904. 2016-01-13 07:42:08
  905. 2016-01-13 07:42:30
  906. 2016-01-13 07:42:50
  907. 2016-01-13 07:43:12
  908. 2016-01-13 07:43:34
  909. 2016-01-13 07:43:56
  910. 2016-01-13 07:44:18
  911. 2016-01-13 07:44:39
  912. 2016-01-13 07:45:01
  913. Again_______________________:
  914. 2016-01-13 07:50:01
  915. 2016-01-13 07:50:23
  916. 2016-01-13 07:50:44
  917. 2016-01-13 07:51:04
  918. 2016-01-13 07:51:26
  919. 2016-01-13 07:51:48
  920. 2016-01-13 07:52:10
  921. 2016-01-13 07:52:32
  922. 2016-01-13 07:53:13
  923. Again_______________________:
  924. 2016-01-13 07:58:13
  925. 2016-01-13 07:58:35
  926. 2016-01-13 07:58:56
  927. 2016-01-13 07:59:17
  928. 2016-01-13 07:59:39
  929. 2016-01-13 08:00:01
  930. 2016-01-13 08:00:23
  931. 2016-01-13 08:00:45
  932. 2016-01-13 08:01:07
  933. Again_______________________:
  934. 2016-01-13 08:06:07
  935. 2016-01-13 08:06:29
  936. 2016-01-13 08:06:49
  937. 2016-01-13 08:07:11
  938. 2016-01-13 08:07:33
  939. 2016-01-13 08:07:55
  940. 2016-01-13 08:08:17
  941. 2016-01-13 08:08:39
  942. 2016-01-13 08:09:01
  943. Again_______________________:
  944. 2016-01-13 08:14:01
  945. 2016-01-13 08:15:01
  946. 2016-01-13 08:15:21
  947. 2016-01-13 08:15:44
  948. 2016-01-13 08:16:05
  949. 2016-01-13 08:16:27
  950. 2016-01-13 08:16:50
  951. 2016-01-13 08:17:11
  952. 2016-01-13 08:17:33
  953. Again_______________________:
  954. 2016-01-13 08:22:33
  955. 2016-01-13 08:22:55
  956. 2016-01-13 08:23:16
  957. 2016-01-13 08:23:38
  958. 2016-01-13 08:24:00
  959. 2016-01-13 08:24:22
  960. 2016-01-13 08:24:44
  961. 2016-01-13 08:25:06
  962. 2016-01-13 08:25:47
  963. Again_______________________:
  964. 2016-01-13 08:30:47
  965. 2016-01-13 08:31:09
  966. 2016-01-13 08:31:29
  967. 2016-01-13 08:31:51
  968. 2016-01-13 08:32:13
  969. 2016-01-13 08:32:35
  970. 2016-01-13 08:32:57
  971. 2016-01-13 08:33:19
  972. 2016-01-13 08:33:41
  973. Again_______________________:
  974. 2016-01-13 08:38:41
  975. 2016-01-13 08:39:03
  976. 2016-01-13 08:39:23
  977. 2016-01-13 08:40:04
  978. 2016-01-13 08:40:26
  979. 2016-01-13 08:40:48
  980. 2016-01-13 08:41:10
  981. 2016-01-13 08:41:32
  982. 2016-01-13 08:41:54
  983. Again_______________________:
  984. 2016-01-13 08:46:54
  985. 2016-01-13 08:47:16
  986. 2016-01-13 08:47:36
  987. 2016-01-13 08:47:58
  988. 2016-01-13 08:48:43
  989. 2016-01-13 08:49:05
  990. 2016-01-13 08:49:27
  991. 2016-01-13 08:49:49
  992. 2016-01-13 08:50:11
  993. Again_______________________:
  994. 2016-01-13 08:55:11
  995. 2016-01-13 08:55:33
  996. 2016-01-13 08:55:53
  997. 2016-01-13 08:56:16
  998. 2016-01-13 08:56:38
  999. 2016-01-13 08:57:01
  1000. 2016-01-13 08:57:42
  1001. 2016-01-13 08:58:04
  1002. 2016-01-13 08:58:45
  1003. Again_______________________:
  1004. 2016-01-13 09:03:45
  1005. 2016-01-13 09:04:23
  1006. 2016-01-13 09:04:43
  1007. 2016-01-13 09:05:06
  1008. 2016-01-13 09:05:28
  1009. 2016-01-13 09:05:49
  1010. 2016-01-13 09:06:12
  1011. 2016-01-13 09:06:32
  1012. 2016-01-13 09:06:53
  1013. Again_______________________:
  1014. 2016-01-13 09:11:53
  1015. 2016-01-13 09:12:54
  1016. 2016-01-13 09:13:14
  1017. 2016-01-13 09:13:36
  1018. 2016-01-13 09:13:58
  1019. 2016-01-13 09:14:20
  1020. 2016-01-13 09:15:20
  1021. 2016-01-13 09:15:42
  1022. 2016-01-13 09:16:04
  1023. Again_______________________:
  1024. 2016-01-13 09:21:04
  1025. 2016-01-13 09:21:46
  1026. 2016-01-13 09:22:07
  1027. 2016-01-13 09:22:29
  1028. 2016-01-13 09:23:10
  1029. 2016-01-13 09:23:32
  1030. 2016-01-13 09:23:54
  1031. 2016-01-13 09:24:16
  1032. 2016-01-13 09:24:38

     So,写这篇博文其实可以总结为一句话: Driver建议用Firefox
相关文章
|
6月前
|
数据采集 Web App开发 数据挖掘
利用Python和Selenium实现定时任务爬虫
利用Python和Selenium实现定时任务爬虫
|
6月前
|
数据采集 Web App开发 存储
Selenium库编写爬虫详细案例
Selenium库编写爬虫详细案例
|
7天前
|
数据采集 Web App开发 前端开发
Python爬虫进阶:Selenium在动态网页抓取中的实战
【10月更文挑战第26天】动态网页抓取是网络爬虫的难点,因为数据通常通过JavaScript异步加载。Selenium通过模拟浏览器行为,可以加载和执行JavaScript,从而获取动态网页的完整内容。本文通过实战案例,介绍如何使用Selenium在Python中抓取动态网页。首先安装Selenium库和浏览器驱动,然后通过示例代码展示如何抓取英国国家美术馆的图片信息。
26 6
|
21天前
|
数据采集 Web App开发 JavaScript
Selenium爬虫技术:如何模拟鼠标悬停抓取动态内容
本文介绍了如何使用Selenium爬虫技术抓取抖音评论,通过模拟鼠标悬停操作和结合代理IP、Cookie及User-Agent设置,有效应对动态内容加载和反爬机制。代码示例展示了具体实现步骤,帮助读者掌握这一实用技能。
Selenium爬虫技术:如何模拟鼠标悬停抓取动态内容
|
26天前
|
数据采集 Web App开发 数据可视化
Python爬虫教程:Selenium可视化爬虫的快速入门
Python爬虫教程:Selenium可视化爬虫的快速入门
|
4月前
|
数据采集 Web App开发 XML
爬虫进阶:Selenium与Ajax的无缝集成
爬虫进阶:Selenium与Ajax的无缝集成
|
6月前
|
数据采集 Web App开发 搜索推荐
突破目标网站的反爬虫机制:Selenium策略分析
突破目标网站的反爬虫机制:Selenium策略分析
|
3月前
|
数据采集 监控 数据安全/隐私保护
掌握Selenium爬虫的日志管理:调整–log-level选项的用法
在Selenium Web数据采集时,日志管理至关重要。通过调整`–log-level`参数可优化日志详细度,如设置为`INFO`记录一般操作信息。结合代理IP、Cookie及user-agent配置,不仅能提高采集成功率,还能规避反爬机制。合理选择日志级别有助于调试与性能平衡,在复杂的数据采集任务中保持程序稳定与可控。
113 1
掌握Selenium爬虫的日志管理:调整–log-level选项的用法
|
3月前
|
数据采集 人工智能 数据可视化
Python selenium爬虫被检测到,该怎么破?
Python selenium爬虫被检测到,该怎么破?
|
6月前
|
数据采集 Web App开发 JavaScript
Selenium与PhantomJS:自动化测试与网页爬虫的完美结合
Selenium与PhantomJS:自动化测试与网页爬虫的完美结合