ForkingPickler(file, protocol).dump(obj) TypeError: can‘t pickle Environment objects

简介: ForkingPickler(file, protocol).dump(obj) TypeError: can‘t pickle Environment objects

问题描述

在win系统下复现SPSR代码出现这种错误,查询资料发现是windows系统的问题。

解决方案:

因为windows操作系统的原因,在Windows中,多进程multiprocessing使用的是序列化pickle来在多进程之间转移数据,而socket对象是不能被序列化的,但是在linux操作系统上却没问题,因为在linux上多进程multiprocessing使用的是fork,所以在windows上可以改用多线程。因为网络通信属于io密集型的操作,对cpu计算要求不高,不用多进程,用多线程就行。

num_workers=0,记得无论是参数设置还是数据集读取处理的代码,都要重新设置。

loader = torch.utils.data.DataLoader(dataset=dataset,
                                              batch_size=self.batch,
                                              shuffle=self.shuf,
                                              num_workers=0,
                                              drop_last=True)
 
parser.add_argument('--num_workers', type=int, default=0)

还有一些其他的解决方案:python3 PicklingError: Can't pickle  at......> attribute lookup  on __main_can't pickle  at 0x000001ed8215d-CSDN博客

参考:成功解决can‘t pickle Environment objects和Ran out of input_forkingpickler(file, protocol).dump(obj) typeerror-CSDN博客

目录
相关文章
成功解决ForkingPickler(file, protocol).dump(obj) TypeError: can't pickle Environment objects
成功解决ForkingPickler(file, protocol).dump(obj) TypeError: can't pickle Environment objects
成功解决ForkingPickler(file, protocol).dump(obj) TypeError: can't pickle Environment objects
|
5月前
|
开发者 Python
【Python】已解决:TypeError: descriptor ‘index‘ for ‘list‘ objects doesn‘t apply to a ‘str‘ object
【Python】已解决:TypeError: descriptor ‘index‘ for ‘list‘ objects doesn‘t apply to a ‘str‘ object
151 0
|
5月前
|
Python
【Python】已解决:TypeError: write._Log() takes 0 positional arguments but 1 was given
【Python】已解决:TypeError: write._Log() takes 0 positional arguments but 1 was given
199 0
|
7月前
报错:cannot read properties of undefined(reading ‘forEach‘)
报错:cannot read properties of undefined(reading ‘forEach‘)
650 1
报错:cannot read properties of undefined(reading ‘forEach‘)
|
7月前
【报错】 “TypeError: Cannot read properties of undefined (reading ‘split‘)“
【报错】 “TypeError: Cannot read properties of undefined (reading ‘split‘)“
965 0
|
7月前
|
JavaScript
【报错】:Cannot read properties of undefined (reading ‘prototype‘)
【报错】:Cannot read properties of undefined (reading ‘prototype‘)
414 0
|
7月前
|
JavaScript
Cannot read properties of undefined (reading ‘install‘) TypeError: Cannot read properties of……
Cannot read properties of undefined (reading ‘install‘) TypeError: Cannot read properties of……
|
TensorFlow 算法框架/工具 Python
成功解决File "frozen importlib._bootstrap", line 219, in _call_with_frames_removed ImportError: DLL lo
成功解决File "frozen importlib._bootstrap", line 219, in _call_with_frames_removed ImportError: DLL lo
成功解决File "frozen importlib._bootstrap", line 219, in _call_with_frames_removed ImportError: DLL lo
解决AttributeError: ‘model‘ object has no attribute ‘copy‘
原因是没有使用model.state_dict()进行模型保存,但是使用了model.load_state_dict()加载模型。这两者要配套使用。
480 0
|
编解码 搜索推荐 算法
Data-Data Objects and Attribute Types| 学习笔记
快速学习 Data-Data Objects and Attribute Types。
Data-Data Objects and Attribute Types| 学习笔记