遇到的错误:
1. RuntimeError: 2. Attempt to start a new process before the current process 3. has finished its bootstrapping phase. 4. This probably means that you are on Windows and you have 5. forgotten to use the proper idiom in the main module: 6. if __name__ == '__main__': 7. freeze_support() 8. ... 9. The "freeze_support()" line can be omitted if the program 10. is not going to be frozen to produce a Windows executable.
查阅:https://stackoverflow.com/questions/18204782/runtimeerror-on-windows-trying-python-multiprocessing
对应的解答:
On Windows the subprocesses will import (i.e. execute) the main module at start. You need to insert an if __name__ == '__main__':
guard in the main module to avoid creating subprocesses recursively.
Modified testMain.py
:
1. import parallelTestModule 2. 3. if __name__ == '__main__': 4. extractor = parallelTestModule.ParallelExtractor() 5. extractor.runInParallel(numProcesses=2, numThreads=4)
找到自己问题:
通过分析应该是系统的多线程问题
找到代码中使用到的多线程代码进行修改:
trainloader = DataLoader(db_train, batch_size=p['trainBatch'], shuffle=True, num_workers=2)
设置num_workers为0,也就是用主进程读取数据后,模型训练程序运行正常。
解答:
PyTorch——num_workers参数的设置
https://blog.csdn.net/songyuc/article/details/103365591
Soumith Chintala也在《Deep Learning with PyTorch: A 60 Minute Blitz》中说到过这个问题:
1. Soumith Chintala也在《Deep Learning with PyTorch: A 60 Minute Blitz》中说到过这个问题: 2. 3. If running on Windows and you get a BrokenPipeError, try setting 4. the num_worker of torch.utils.data.DataLoader() to 0.
AIEarth是一个由众多领域内专家博主共同打造的学术平台,旨在建设一个拥抱智慧未来的学术殿堂!【平台地址:https://devpress.csdn.net/aiearth】 很高兴认识你!加入我们共同进步!