开发者社区> 问答> 正文

使用ProcessPoolExecutor进行并行处理不起作用而且不返回错误

python小能手 2019-01-23 16:05:48 874

目前我正在尝试加速我的模拟。我已经尝试过使用线程并且它有效。现在我想尝试使用并行进程来比较两种方式。那对我使用futures.ProcessPoolExecutor。当我开始我的脚本时,模拟时间被打印(它非常低),但我的程序不能正常工作。通常它应该生成几个文件,但不会生成它们。此外,没有错误消息。我已经在书本和互联网上对它进行了一些研究,但我无法弄清楚问题。

这是我的代码:

def main(setting):

cfg_path = generate(settings[setting])
run_simulation(cfg_path)

if name == '__main__':

settings = get_wrapper_input_json("Szenarioanalyse.json")
typ = "processes"
start = time.perf_counter()
if typ == "threads":
    with futures.ThreadPoolExecutor(cpu_count()-1) as e:
        e.map(main,settings)
elif typ == "processes":
    with futures.ProcessPoolExecutor(cpu_count()-1) as e:
        e.map(main,settings)
else:
    for setting in settings:
        main(setting)

print("Simulationtime: "+str(time.perf_counter()-1))
分享到
取消 提交回答
全部回答(1)
  • python小能手
    2019-07-17 23:26:41

    settings = get_wrapper_input_json("Szenarioanalyse.json") #get the settings
    parameters = {"Threads":True,"Processes":False,"Serial":False}

    def simulate(setting):

    cfg_path = generate(settings[setting])
    run_simulation(cfg_path)
    

    if name == '__main__':

    for key,value in parameters.items():
        if key == "Threads" and value == True:
            start_threads = time.perf_counter()
            with futures.ThreadPoolExecutor(10) as e:
                e.map(simulate,settings)
            print("Simulationtime "+key+": "+str(time.perf_counter()-start_threads))
        elif key == "Processes" and value == True:
            start_processes = time.perf_counter()
            pool = multiprocessing.Pool(multiprocessing.cpu_count())
            pool.map(simulate,settings)
            print("Simulationtime "+key+": "+str(time.perf_counter()-start_processes))
        elif key == "Serial" and value == True:
            start_serial = time.perf_counter()
            for setting in settings:
                simulate(setting)
            print("Simulationtime "+key+": "+str(time.perf_counter()-start_serial))
    
        #save_dataframe("Szenarioanalyse.json")
        #file_management()
    
    0 0
+ 订阅

时时分享云计算技术内容,助您降低 IT 成本,提升运维效率,使您更专注于核心业务创新。

推荐文章