开发者社区 问答 正文

高阳用户通过datax脚本模式,想要写入参数,一般放在哪里?

高阳用户通过datax脚本模式,想要写入参数set odps.sql.cfile2.field.maxsize=16384; 这种参数,一般需要放在哪里呢#!/bin/bash @resource_reference{"aksk_resource"} eval $(cat aksk_resource) echo ' { "job" : { "content":[ { "reader":{ "name":"postgresqlreader", "parameter":{ "column":[                             "id",                             "party_uid",                             "label_id", "label_value :: text",                             "created_at",                             "updated_at" ], "connection":[ { "jdbcUrl":[ "jdbc:postgresql://pgr-uf687c6ih6bv1f27.pg.rds.aliyuncs.com:1921/mpp_ciam_prod_db?useUnicode=true&characterEncoding=utf8" ], "table":[ "identity_labels" ] } ], "password":"'${ikeaid_password}'", "username":"'${ikeaid_username}'" } }, "writer":{ "name":"odpswriter", "parameter":{ "accessId":"'${accessId}'", "accessKey":"'${accessKey}'", "column":[                             "id",                             "party_uid",                             "label_id",                             "label_value",                             "created_at",                             "updated_at" ], "odpsServer":"http://service.cn.maxcompute.aliyun-inc.com/api", "partition":"ds='${bizdate}'", "project":"${dataphin_ikea_etl}", "table":"ikea_id_identity_labels", "truncate":true } } } ], "setting":{ "errorLimit":{ "record":0 }, "speed":{ "byte":1048576, "channel":3 } } } } ' > datax-pg-ikea_id_identity_labels.json

python $DATAX_HOME/bin/datax.py datax-pg-ikea_id_identity_labels.json

展开
收起
冰激凌甜筒 2023-04-04 15:22:32 301 分享 版权
1 条回答
写回答
取消 提交回答
  • 目前只能让客户看一下是否可以拆分一下数据,此回答整理自钉群“Dataphin二线技术支持群”

    2023-04-04 16:38:57
    赞同 展开评论