协程-gevent-阿里云开发者社区

协程-gevent

2017-11-12 928

版权

本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。

简介：

协程用gevent实现, 依赖libevent和greenlet
###安装
1. 安装libevent
   yum install libevent

2. 安装easy_install
【1】wget -q http://peak.telecommunity.com/dist/ez_setup.py
【2】python ez_setup.py
【3】使用easy_install 查看命令是否可用，如果不可用可以讲路径加入到PATH中

3. 安装greenlet
    【1】yum install python-devel
    【2】easy_install greenlet
4. 安装pip
   wget https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9
    tar zxf pip-9.0.1.tar.gz
    cd pip-9.0.1
    python setup.py build
    python setup.py install

5. 安装gevent
   wget http://pypi.python.org/packages/source/g/gevent/gevent-0.13.1.tar.gz#md5=5c1b03d9ce39fee4cfe5ea8befb1d4c4
   tar zxf gevent-0.13.1.tar.gz
   cd gevent-0.13
   python fetch_libevent.py
   python setup.py build
   python setup.py install

6. 安装request模块
    pip install requests

7. 安装dnspython模块
    wget http://www.dnspython.org/kits/1.12.0/dnspython-1.12.0.tar.gz
     tar -zxf dnspython-1.12.0.tar.gz
     cd dnspython-1.12.0
     python setup.py install

###案例1
#!/usr/bin/env python
# _*_coding:utf-8_*_

from gevent import monkey
import os
import gevent
import time
monkey.patch_socket()

class MyGeven:

   def __init__(self):
       self.data = {}

   def get_dig_domain(self, domain):
                ret = os.system('dig %s |egrep -v "^$|;" >>/tmp/tmpcheck' % (domain) )
       if ret:
           self.data[domain] = ret

   def gprobe(self, domain_list):
       jobs = []
       for domain in domain_list:
           jobs.append(gevent.spawn(self.get_dig_domain, domain))
       gevent.joinall(jobs)

if __name__ == '__main__':
   start_ts = time.time()
   print "start....."
   domain_list = [
            "www.baidu.com",
                "www.qq.com",
                "www.163.com",
   ]
   obj = MyGeven()
   obj.gprobe(domain_list)
   print "请求数:", len(domain_list), "耗时:", time.time() - start_ts, "s"
   print obj.data

###案例2:
#!/usr/bin/env python
# _*_coding:utf-8_*_
from gevent import monkey
import gevent
import time
monkey.patch_socket()
import requests

class MyGeven:

   def __init__(self):
       self.data = {}

   def get_url_data(self, url):

       ret = requests.get(url)
       if ret:
           self.data[url] = ret

   def gprobe(self, url_list):
       jobs = []
       for url in url_list:
           jobs.append(gevent.spawn(self.get_url_data, url))
       gevent.joinall(jobs)

if __name__ == '__main__':
   start_ts = time.time()
   print "start….."
   url_list = [
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
            "http://www.baidu.com",
                "http://www.qq.com",
                "http://www.163.com",
   ]
   obj = MyGeven()
   obj.gprobe(url_list)
   print "请求数:", len(url_list), "耗时:", time.time() - start_ts, "s"

print obj.data

本文转自cloves 51CTO博客，原文链接:http://blog.51cto.com/yeqing/1925164

文章标签：

Python

关键词：

协程gevent

协程-gevent

热门文章

最新文章

相关电子书