如何加载自己的数据集,进行微调。我是说:自定义数据集的格式如何,才能load?

如何加载自己的数据集,进行微调。我是说:自定义数据集的格式如何,才能load?

展开
收起
1220793969477760 2023-04-04 16:02:07 364 分享 版权
2 条回答
写回答
取消 提交回答
  • 查看

    train_dataset = MsDataset.load('people_daily_ner_1998_tiny', namespace='damo', split='train', download_mode=DownloadMode.FORCE_REDOWNLOAD)
    eval_dataset = MsDataset.load('people_daily_ner_1998_tiny', namespace='damo', split='validation', download_mode=DownloadMode.FORCE_REDOWNLOAD)
    

    的缓存文件,类容格式为

    id,text,info_list,schema
    test,好用,很适合画内眼线。,"[[{""type"": ""属性词"", ""span"": ""画内眼线"", ""offset"": [6, 10]}, {""type"": ""正向情感(情感词)"", ""span"": ""很适合"", ""offset"": [3, 6]}]]","{""属性词"": {""正向情感(情感>词)"": null, ""负向情感(情感词)"": null, ""中性情感(情感词)"": null}}"
    test,很好,很好用,物流很快,能搜索到方圆一百五十米内的信号,"[[{""type"": ""属性词"", ""span"": ""物流"", ""offset"": [7, 9]}, {""type"": ""正向情感(情感词)"", ""span"": ""很快"", ""offset"": [9, 11]}]]","{"">属性词"": {""正向情感(情感词)"": null, ""负向情感(情感词)"": null, ""中性情感(情感词)"": null}}"
    test,一般般还可以用也可以看,[],"{""属性词"": {""正向情感(情感词)"": null, ""负向情感(情感词)"": null, ""中性情感(情感词)"": null}}"
    test,死都不在这里买东西,烂烂烂,[],"{""属性词"": {""正向情感(情感词)"": null, ""负向情感(情感词)"": null, ""中性情感(情感词)"": null}}"
    test,质量不怎么样,价格贵,"[[{""type"": ""属性词"", ""span"": ""质量"", ""offset"": [0, 2]}, {""type"": ""负向情感(情感词)"", ""span"": ""不怎么样"", ""offset"": [2, 6]}], [{""type"": ""属性词"", ""span"": "">价格"", ""offset"": [7, 9]}, {""type"": ""负向情感(情感词)"", ""span"": ""贵"", ""offset"": [9, 10]}]]","{""属性词"": {""正向情感(情感词)"": null, ""负向情感(情感词)"": null, ""中性情感(情感词)"": null}}"
    test,物美价廉 服务上乘 交易顺利 perfect,"[[{""type"": ""属性词"", ""span"": ""服务"", ""offset"": [5, 7]}, {""type"": ""正向情感(情感词)"", ""span"": ""上乘"", ""offset"": [7, 9]}], [{""type"": ""属性词"", ""span"": ""交易"", ""offset"": [10, 12]}, {""type"": ""正向情感(情感词)"", ""span"": ""顺利"", ""offset"": [12, 14]}]]","{""属性词"": {""正向情感(情感词)"": null, ""负向情感(情感词)"": null, ""中性情感(情感词)"": null}}"
    
    2023-07-26 16:30:37
    赞同 展开评论
  • 同问,楼主解决了么

    2023-06-06 17:25:54
    赞同 展开评论

包含命名实体识别、文本分类、分词、关系抽取、问答、推理、文本摘要、情感分析、机器翻译等多个领域

热门讨论

热门文章

还有其他疑问?
咨询AI助理