开发者社区> 问答> 正文

爬虫项目用代理ip爬https网站就报错,急急急,求高手不吝赐教。错误信息:(S?400报错

import requests

https_ip_link = "这里是代理IP的链接"
https_ip_list = requests.get(https_ip_link).text.split("\r\n")
headers = {
            "User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36"
        }
for proxies in https_ip_list:
    proxies = {"https": 'http://201806091859344402:30494667@{}'.format(proxies)}
    response = requests.get("https://www.baidu.com/", headers=headers, proxies=proxies, timeout=10)
    print(response.content.decode())

这是我的爬虫测试代码,在线等,在线等,在线等,希望解决过这个错误或者知道原因的朋友回复我,真心万分感谢。很急切的想知道解决方案。

 

一直报错如下:

Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\contrib\pyopenssl.py", line 441, in wrap_socket
    cnx.do_handshake()
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\OpenSSL\SSL.py", line 1907, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\OpenSSL\SSL.py", line 1632, in _raise_ssl_error
    raise SysCallError(-1, "Unexpected EOF")
OpenSSL.SSL.SysCallError: (-1, 'Unexpected EOF')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\connectionpool.py", line 595, in urlopen
    self._prepare_proxy(conn)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\connectionpool.py", line 816, in _prepare_proxy
    conn.connect()
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\connection.py", line 326, in connect
    ssl_context=context)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\util\ssl_.py", line 329, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\contrib\pyopenssl.py", line 448, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\adapters.py", line 445, in send
    timeout=timeout
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\urllib3\util\retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.baidu.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:/project_job/itslaw-spider/trunk/QiChaCha/test/test.py", line 14, in <module>
    response = requests.get("https://www.baidu.com/", headers=headers, proxies=proxies, timeout=10)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\adapters.py", line 511, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError:HTTPSConnectionPool(host='www.baidu.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

 

我已经找了2周这个问题的解决方案,网上都没找到,在解决不了饭碗就没了,急急急急急急求高手解答。万分感谢了

我的版本控制:

ndg-httpsclient   0.5.0

pyasn1            0.4.3

pyOpenSSL         18.0.0

requests          2.19.1

urllib3           1.22

展开
收起
爱吃鱼的程序员 2020-06-06 20:58:40 1071 0
1 条回答
写回答
取消 提交回答
  • https://developer.aliyun.com/profile/5yerqm5bn5yqg?spm=a2c6h.12873639.0.0.6eae304abcjaIB
                        <p>不要沉,顶起来</p>
    
                    
    
                        这份代码爬别的网站正常么?另外有单独研究ssl unexpected eof么?有在stackoverflow上问过了么? 
                    
    
                            这个代码跑任何https 都这个错,没有去其他地方问
                        
    
                        弄一个ssl证书context 
                    
    
                        <p>我也遇到这个问题了,是不是代理不好呢</p>
    
                    
    
                        <p>楼主解决没,</p>
    
                    
    
                        <p>老哥,我可能也是遇到类似的问题,</p> 
    

    requests默认的编码可能与网站的编码不一样

    req=requests.get(url=target)
    #查看request默认的编码,发现与网站response不符,改为网站使用的gbk
    req.encoding='gbk'

    老哥可以试一试

                        <p>代理ip支不支持https?</p>
    
    2020-06-06 20:58:53
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
Python第五讲——关于爬虫如何做js逆向的思路 立即下载
CDN助力企业网站进入HTTPS时代 立即下载
低代码开发师(初级)实战教程 立即下载