使用多进程
multiprocessing
模块提供了使用进程的方法,使用起来和线程threading
模块非常类似。
multiprocessing
模块包含一个与Thread类非常相似的Process类。
你可以将Python中的并发编程(2)线程的实现的Thread
直接替换成Process
。
import itertools import time # 从multiprocessing导入 from multiprocessing import Process, Event from multiprocessing import synchronize # 旋转 def spin(msg: str, done: synchronize.Event ) -> None: for char in itertools.cycle(r'\|/-'): status = f'\r{char} {msg}' print(status, end='', flush=True) if done.wait(.1): break blanks = ' ' * len(status) print(f'\r{blanks}\r', end='') # 阻塞3秒,并返回42 def slow() -> int: time.sleep(3) return 42 def supervisor() -> int: done = Event() # Thread 替换成Process spinner = Process(target=spin, args=('thinking!', done)) print(f'spinner object: {spinner}') spinner.start() result = slow() done.set() spinner.join() return result def main() -> None: result = supervisor() print(f'Answer: {result}') if __name__ == '__main__': main()
同样,我们用进程改写线程Python中的并发编程(2)线程的实现的计算。
import time from multiprocessing import Process # 计算number的因子 def factorize(number): for i in range(1, number + 1): if number % i == 0: yield i numbers = [2139079, 1214759, 1516637, 1852285, 14256346, 12456533] def get_factor(number): factors = list(factorize(number)) return factors if __name__ == '__main__': start = time.time() processes = [] for number in numbers: process = Process(target=get_factor, args=(number,)) process.start() # 启动 processes.append(process) for process in processes: process.join() # 等待完成 end = time.time() delta = end - start print(f'Process {delta:.3f} 秒')
我们发现修改为进程后,计算耗费时间减少了一些:
(之前的)顺序执行花费 2.478 秒 Process 1.744 秒
由于进程启动和通信需要耗费一定时间,所以并不明显。
如果把numbers中的数字加大,时间减少的会更明显:
numbers = [4139079, 2214759, 4516637, 6852285, 44256346, 62456533]
顺序执行花费 11.079 秒 Process 6.870 秒
multiprocessing
还提供了进程池Pool
,可以方便地处理一系列输入:
from multiprocessing import Pool def f(x): return x*x if __name__ == '__main__': with Pool(5) as p: print(p.map(f, [1, 2, 3]))