# Python 分布式计算框架 PP (Parallel Python)：集群模式下的实践探索

#### 实战案例：求解质数和

Python

#!/usr/bin/python
# File: sum_primes.py
# Author: VItalii Vanovschi
# Desc: This program demonstrates parallel computations with pp module
# It calculates the sum of prime numbers below a given integer in parallel
# Parallel Python Software: http://www.parallelpython.com

import math, sys, time, datetime
import pp

def isprime(n):
"""Returns True if n is prime and False otherwise"""
if not isinstance(n, int):
raise TypeError("argument passed to is_prime is not of 'int' type")
if n < 2:
return False
if n == 2:
return True
max = int(math.ceil(math.sqrt(n)))
i = 2
while i <= max:
if n % i == 0:
return False
i += 1
return True

def sum_primes(n):
"""Calculates sum of all primes below given integer n"""
return sum([x for x in xrange(2,n) if isprime(x)])

print """Usage: python sum_primes.py [ncpus]
[ncpus] - the number of workers to run in parallel,
if omitted it will be set to the number of processors in the system
"""

# tuple of all parallel python servers to connect with
#ppservers = ()
ppservers = ("192.168.1.104:35000",)
#ppservers=("*",)

if len(sys.argv) > 1:
ncpus = int(sys.argv[1])
# Creates jobserver with ncpus workers
job_server = pp.Server(ncpus, ppservers=ppservers, secret="123456")
else:
# Creates jobserver with automatically detected number of workers
job_server = pp.Server(ppservers=ppservers, secret="123456")

print "Starting pp with", job_server.get_ncpus(), "workers"

# Submit a job of calulating sum_primes(100) for execution.
# sum_primes - the function
# (100,) - tuple with arguments for sum_primes
# (isprime,) - tuple with functions on which function sum_primes depends
# ("math",) - tuple with module names which must be imported before sum_primes execution
# Execution starts as soon as one of the workers will become available
job1 = job_server.submit(sum_primes, (100,), (isprime,), ("math",))

# Retrieves the result calculated by job1
# The value of job1() is the same as sum_primes(100)
# If the job has not been finished yet, execution will wait here until result is available
result = job1()

print "Sum of primes below 100 is", result

start_time = time.time()

# The following submits 8 jobs and then retrieves the results
inputs = (500000, 500100, 500200, 500300, 500400, 500500, 500600, 500700, 500000, 500100, 500200, 500300, 500400, 500500, 500600, 500700)
#inputs = (1000000, 1000100, 1000200, 1000300, 1000400, 1000500, 1000600, 1000700)
jobs = [(input, job_server.submit(sum_primes,(input,), (isprime,), ("math",))) for input in inputs]
for input, job in jobs:
print datetime.datetime.now()
print "Sum of primes below", input, "is", job()

print "Time elapsed: ", time.time() - start_time, "s"
job_server.print_stats()

c:\Python27\python.exe test_pp_official.py
Usage: python sum_primes.py [ncpus]
[ncpus] - the number of workers to run in parallel,
if omitted it will be set to the number of processors in the system
Starting pp with 4 workers
Sum of primes below 100 is 1060
2016-08-28 19:07:26.579000
Sum of primes below 500000 is 9914236195
2016-08-28 19:07:33.032000
Sum of primes below 500100 is 9917236483
2016-08-28 19:07:33.035000
Sum of primes below 500200 is 9922237979
2016-08-28 19:07:33.296000
Sum of primes below 500300 is 9926740220
2016-08-28 19:07:33.552000
Sum of primes below 500400 is 9930743046
2016-08-28 19:07:33.821000
Sum of primes below 500500 is 9934746636
2016-08-28 19:07:34.061000
Sum of primes below 500600 is 9938250425
2016-08-28 19:07:37.199000
Sum of primes below 500700 is 9941254397
2016-08-28 19:07:37.202000
Sum of primes below 500000 is 9914236195
2016-08-28 19:07:41.640000
Sum of primes below 500100 is 9917236483
2016-08-28 19:07:41.742000
Sum of primes below 500200 is 9922237979
2016-08-28 19:07:41.746000
Sum of primes below 500300 is 9926740220
2016-08-28 19:07:41.749000
Sum of primes below 500400 is 9930743046
2016-08-28 19:07:41.752000
Sum of primes below 500500 is 9934746636
2016-08-28 19:07:41.756000
Sum of primes below 500600 is 9938250425
2016-08-28 19:07:43.846000
Sum of primes below 500700 is 9941254397
Time elapsed:  17.2770001888 s
Job execution statistics:
job count | % of all jobs | job time sum | time per job | job server
6 |         35.29 |      27.4460 |     4.574333 | 192.168.1.104:35000
11 |         64.71 |      60.2950 |     5.481364 | local
Time elapsed since server creation 17.2849998474
0 active tasks, 4 cores

#### 参考文献

|
6天前
|

12 0
|
5天前
|

22 5
|
4天前
|

【6月更文挑战第21天】探索Python Web几个流行框架，了解各框架特性以适应不同场景需求。
24 1
|
7天前
|

Python streamlit框架开发数据分析网站并免费部署

23 1
|
3天前
|
Java 网络安全

10 0
|
3天前
|

15 0
|
11天前
|

Zookeeper 详解：分布式协调服务的核心概念与实践
Zookeeper 详解：分布式协调服务的核心概念与实践
14 0
|
11天前
|

Python进行自动化测试测试框架的选择与应用
【6月更文挑战第9天】本文介绍了Python自动化测试的重要性及选择测试框架的考量因素，如功能丰富性、易用性、灵活性和集成性。文中列举了常用的Python测试框架，包括unittest、pytest、nose2和Robot Framework，并提供了使用pytest进行单元测试的示例代码。此外，还展示了如何使用Robot Framework进行验收测试和Web UI测试。选择合适的测试框架对提升测试效率和软件质量至关重要，团队应根据项目需求、社区支持、集成性和学习曲线等因素进行选择。通过不断学习和实践，可以优化自动化测试流程，确保软件的稳定性和可靠性。
21 0
|
1月前
|
NoSQL Java 关系型数据库
【Redis系列笔记】分布式锁

430 2
|
11天前
|
NoSQL 算法 Java

16 1