环境信息

canal version : 1.1.4.RELEASE

canal admin/server HA

mysql version

问题描述

使用admin管理集群不能自定义instance的spring.xml

步骤重现

以下只写主要相关配置：

1. canal admin的集群配置文件canal.properties

canal.admin.manager = 10.100.12.12:8089 canal.instance.global.mode = manager canal.instance.global.spring.xml = classpath:spring/default-instance.xml

自定义instance的spring.xml

canal.instance.shop-item.spring.xml = classpath:spring/group-instance.xml

由于未自动从zk获取写手动填写

canal.destinations = shop-item

1. 导入默认instance模板，修改设置并启动

期待结果

以自定义的group-instance.xml加载正常启动运行

实际执行情况

仍然使用global配置的default-instance.xml加载启动，由于instance以group的形式配置，无法获取到mysql地址报错

Debug

从代码中发现：

1、配置了canal.admin.manager参数就会强制设置InstanceConfig.setMode(InstanceMode.MANAGER)

2、当Mode == InstanceMode.MANAGER时不会设置自定义的example.spring.xml

// com.alibaba.otter.canal.deployer.CanalController

private void initInstanceConfig(Properties properties) {
    // 实例名称从admin配置的canal.properties中获取
    String destinationStr = getProperty(properties, CanalConstants.CANAL_DESTINATIONS);
    String[] destinations = StringUtils.split(destinationStr, CanalConstants.CANAL_DESTINATION_SPLIT);

    for (String destination : destinations) {
        InstanceConfig config = parseInstanceConfig(properties, destination);
        InstanceConfig oldConfig = instanceConfigs.put(destination, config);

        if (oldConfig != null) {
            logger.warn("destination:{} old config:{} has replace by new config:{}", destination, oldConfig, config);
        }
    }
}

private InstanceConfig parseInstanceConfig(Properties properties, String destination) {
    String adminManagerAddress = getProperty(properties, CanalConstants.CANAL_ADMIN_MANAGER);
    InstanceConfig config = new InstanceConfig(globalInstanceConfig);
    String modeStr = getProperty(properties, CanalConstants.getInstanceModeKey(destination));
   
    // 由于使用canal admin管理集群，必定会配置该参数，此时MODE会被强制设置为MANAGER
    if (StringUtils.isNotEmpty(adminManagerAddress)) {
        // 如果指定了manager地址,则强制适用manager
        config.setMode(InstanceMode.MANAGER);
    } else if (StringUtils.isNotEmpty(modeStr)) {
        config.setMode(InstanceMode.valueOf(StringUtils.upperCase(modeStr)));
    }

    String lazyStr = getProperty(properties, CanalConstants.getInstancLazyKey(destination));
    if (!StringUtils.isEmpty(lazyStr)) {
        config.setLazy(Boolean.valueOf(lazyStr));
    }

    if (config.getMode().isManager()) {
        String managerAddress = getProperty(properties, CanalConstants.getInstanceManagerAddressKey(destination));
        if (StringUtils.isNotEmpty(managerAddress)) {
            if (StringUtils.equals(managerAddress, "${canal.admin.manager}")) {
                managerAddress = adminManagerAddress;
            }
            config.setManagerAddress(managerAddress);
        }
        
        // 由于已强制被设置为MANAGER, 不会进入该段逻辑，即不能自定义spring.xml，只能一个集群都使用global.spring.xml
    } else if (config.getMode().isSpring()) {
        String springXml = getProperty(properties, CanalConstants.getInstancSpringXmlKey(destination));
        if (StringUtils.isNotEmpty(springXml)) {
            config.setSpringXml(springXml);
        }
    }

    return config;
}

Log

2020-05-21 09:58:45.341 [pool-50-thread-1] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [canal.properties] 2020-05-21 09:58:45.341 [pool-50-thread-1] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [shop-item/instance.properties] 2020-05-21 09:58:45.341 [pool-50-thread-1] WARN c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Could not load properties from class path resource [shop-item/instance.properties]: class path resource [shop-item/instance.properties] cannot be opened because it does not exist 2020-05-21 09:58:45.345 [pool-50-thread-1] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start CannalInstance for 1-shop-item 2020-05-21 09:58:45.349 [pool-50-thread-1] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^...$ 2020-05-21 09:58:45.349 [pool-50-thread-1] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table black filter : 2020-05-21 09:58:45.349 [destination = shop-item , address = null , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - parse events has an error com.alibaba.otter.canal.parse.exception.CanalParseException: illegal connection is null

原提问者GitHub用户ashulin