modelscope-funasr1.0 微调热词模型,单机多卡出现上述问题,会是啥原因?单机单卡可以跑通环境安装:
python==3.8.5
torch==2.0.1
funasr==1.0.19
使用模型:
[[https://www.modelscope.cn/models/damo/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary]]
参考示例:
[[https://github.com/alibaba-damo-academy/FunASR/blob/5e7eb6f160c48861cbcd39825a0cb98f98538772/examples/industrial_data_pretraining/seaco_paraformer/finetune.sh]]
代码修改将GPU卡设置成2卡 (单卡可以跑通)
export CUDA_VISIBLE_DEVICES="0,1"
++dataset_conf.batch_type="example"
++dataset_conf.batch_size=10
错误信息:
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one等