开发者社区 > ModelScope模型即服务 > 正文

如果modelscope中微调数据集位置改变了,会造成这个报错嘛?

"如果modelscope中微调数据集位置改变了,会造成这个报错嘛? 完整的报错:
Traceback (most recent call last):
File ""/data/PAI/swift/swift/cli/export.py"", line 5, in
export_main()
File ""/data/PAI/swift/swift/utils/run_utils.py"", line 27, in x_main
result = llm_x(args, kwargs)
^^^^^^^^^^^^^^^^^^^^^
File ""/data/PAI/swift/swift/llm/export.py"", line 114, in llm_export
awq_model_quantize(model, template.tokenizer)
File ""/data/PAI/swift/swift/llm/export.py"", line 71, in awq_model_quantize
awq_model.quantize(tokenizer, quant_config=quant_config)
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/torch/utils/_contextlib.py"", line 115, in decorate_context
return func(*args,
kwargs)
^^^^^^^^^^^^^^^^^^^^^
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/awq/models/base.py"", line 170, in quantize
self.quantizer = AwqQuantizer(
^^^^^^^^^^^^^
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/awq/quantize/quantizer.py"", line 61, in init
self.modules, self.module_kwargs, self.inps = self.init_quant()
^^^^^^^^^^^^^^^^^
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/awq/quantize/quantizer.py"", line 440, in init_quant
samples = get_calib_dataset(
^^^^^^^^^^^^^^^^^^
File ""/data/PAI/swift/swift/llm/export.py"", line 25, in _get_dataset
dataset = get_dataset(
^^^^^^^^^^^^
File ""/data/PAI/swift/swift/llm/utils/dataset.py"", line 2415, in get_dataset
dataset = get_function(
^^^^^^^^^^^^^
File ""/data/PAI/swift/swift/llm/utils/dataset.py"", line 412, in get_dataset_from_repo
dataset = load_ms_dataset(dataset_id, subset_split_list, use_hf)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/data/PAI/swift/swift/llm/utils/dataset.py"", line 326, in load_ms_dataset
dataset = MsDataset.load(dataset_id, subset_name=subset_name, split=split, download_mode=download_mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/data/PAI/swift/swift/llm/utils/utils.py"", line 95, in _msdataset_ddp_load
dataset = _old_msdataset_load(args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/modelscope/msdatasets/ms_dataset.py"", line 288, in load
dataset_id_on_hub, dataset_type = _api.get_dataset_id_and_type(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/modelscope/hub/api.py"", line 707, in get_dataset_id_and_type
datahub_raise_on_error(datahub_url, resp, r)
File ""/root/anaconda3/envs/myswift/lib/python3.11/site-packages/modelscope/hub/errors.py"", line 157, in datahub_raise_on_error
raise RequestError(
modelscope.hub.errors.RequestError: Url = https://www.modelscope.cn/api/v1/datasets/modelscope/_custom_dataset, Request id= Message = 不存在的数据集, Please specify correct dataset_name and namespace."

展开
收起
小小爱吃香菜 2024-07-23 22:20:40 19 0
1 条回答
写回答
取消 提交回答
  • " -dataset 可以指定数据集 参考以下图片 945af154eb6fc19a6a2259859d31f64e.png
    此回答整理自钉群“魔搭ModelScope开发者联盟群 ①”
    "

    2024-07-24 16:01:00
    赞同 展开评论 打赏

ModelScope旨在打造下一代开源的模型即服务共享平台,为泛AI开发者提供灵活、易用、低成本的一站式模型服务产品,让模型应用更简单!欢迎加入技术交流群:微信公众号:魔搭ModelScope社区,钉钉群号:44837352

相关电子书

更多
视觉AI能力的开放现状及ModelScope实战 立即下载
ModelScope助力语音AI模型创新与应用 立即下载
低代码开发师(初级)实战教程 立即下载