服务发现和负载均衡的自定义路由-支持 TCP 协议

阿里云容器服务在使用的过程中，针对 TCP 负载均衡的场景，会遇到这样的问题：如果一个应用的客户端镜像和服务端镜像均部署在同一个节点（ECS）上面，由于受负载均衡的限制，该应用的客户端不能通过负载均衡访问本机的服务端。本文档以常用的基于 TCP 协议的 redis 为例，通过自定义路由 acs/proxy 来解决这一问题。
[backcolor=transparent]注意：任何两个不同的服务均不能共享使用同一个负载均衡，否则会导致负载均衡后端机器被删除，服务不可用。

解法一：通过调度容器，避免客户端和服务端容器部署在同一个节点


示例应用模板（使用了 lb 标签和 swarm filter 功能）：

1. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]master[backcolor=transparent]:
2. [backcolor=transparent]    ports[backcolor=transparent]:
3. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] [backcolor=transparent]6379[backcolor=transparent]:[backcolor=transparent]6379[backcolor=transparent]/[backcolor=transparent]tcp
4. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
5. [backcolor=transparent]    labels[backcolor=transparent]:
6. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]lb[backcolor=transparent].[backcolor=transparent]port_6379[backcolor=transparent]:[backcolor=transparent] tcp[backcolor=transparent]:[backcolor=transparent]//proxy_test:6379
7. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]client[backcolor=transparent]:
8. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
9. [backcolor=transparent]    links[backcolor=transparent]:
10. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]master
11. [backcolor=transparent]    environment[backcolor=transparent]:[backcolor=transparent]
12. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] [backcolor=transparent]'affinity:aliyun.lb.port_6379!=tcp://proxy_test:6379'
13. [backcolor=transparent]    command[backcolor=transparent]:[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]cli [backcolor=transparent]-[backcolor=transparent]h [backcolor=transparent]120.25[backcolor=transparent].[backcolor=transparent]131.64
14. [backcolor=transparent]    stdin_open[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]true
15. [backcolor=transparent]    tty[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]true

[backcolor=transparent]注意：

• 如果发现调度不生效，在容器服务管理控制台，单击左侧导航栏的 [backcolor=transparent]服务 进入服务列表页面 > 选择您需要调度的服务 > 单击 [backcolor=transparent]重新调度 > 在弹出的对话框中勾选 [backcolor=transparent]强制重新调度 > 单击 [backcolor=transparent]确定。
• [backcolor=transparent]强制重新调度 会丢弃已有容器的 volume，请做好相应的备份迁移工作。

1. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]master[backcolor=transparent]:
2. [backcolor=transparent]    ports[backcolor=transparent]:
3. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] [backcolor=transparent]6379[backcolor=transparent]:[backcolor=transparent]6379[backcolor=transparent]/[backcolor=transparent]tcp
4. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
5. [backcolor=transparent]    labels[backcolor=transparent]:
6. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]lb[backcolor=transparent].[backcolor=transparent]port_6379[backcolor=transparent]:[backcolor=transparent] tcp[backcolor=transparent]:[backcolor=transparent]//proxy_test:6379
7. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]client[backcolor=transparent]:
8. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
9. [backcolor=transparent]    links[backcolor=transparent]:
10. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]master
11. [backcolor=transparent]    command[backcolor=transparent]:[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]cli [backcolor=transparent]-[backcolor=transparent]h redis[backcolor=transparent]-[backcolor=transparent]master
12. [backcolor=transparent]    stdin_open[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]true
13. [backcolor=transparent]    tty[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]true

1. [backcolor=transparent]lb[backcolor=transparent]:
2. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  registry[backcolor=transparent].[backcolor=transparent]aliyuncs[backcolor=transparent].[backcolor=transparent]com[backcolor=transparent]/[backcolor=transparent]acs[backcolor=transparent]/[backcolor=transparent]proxy[backcolor=transparent]:[backcolor=transparent]0.5
3. [backcolor=transparent]    ports[backcolor=transparent]:
4. [backcolor=transparent]            [backcolor=transparent]-[backcolor=transparent]  [backcolor=transparent]'6379:6379/tcp'
5. [backcolor=transparent]    restart[backcolor=transparent]:[backcolor=transparent]  always
6. [backcolor=transparent]    labels[backcolor=transparent]:
7. [backcolor=transparent]        [backcolor=transparent]# addon 使得 proxy 镜像有订阅注册中心的能力，动态加载服务的路由
8. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]custom_addon[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]"proxy"
9. [backcolor=transparent]        [backcolor=transparent]# 每台 vm 部署一个该镜像的容器
10. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]global[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]"true"
11. [backcolor=transparent]        [backcolor=transparent]#  前端绑定负载均衡，使用 lb 标签
12. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]lb[backcolor=transparent].[backcolor=transparent]port_6379[backcolor=transparent]:[backcolor=transparent] tcp[backcolor=transparent]:[backcolor=transparent]//proxy_test:6379
13. [backcolor=transparent]        [backcolor=transparent]# 告诉系统，自定义路由需要等待 master 和 slave 启动之后再启动，并且对 master 和 slave 有依赖
14. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]depends[backcolor=transparent]:[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]master[backcolor=transparent],[backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]slave
15. [backcolor=transparent]    environment[backcolor=transparent]:
16. [backcolor=transparent]        [backcolor=transparent]#  支持加载路由的后端容器的范围，"*"表示整个集群，默认为应用内的服务
17. [backcolor=transparent]        ADDITIONAL_SERVICES[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]"*"
18. [backcolor=transparent]        EXTRA_DEFAULT_SETTINGS[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]"log rsyslog local0,log global,option httplog"
19. [backcolor=transparent]        [backcolor=transparent]# 配置 HAProxy 工作于 tcp 模式
20. [backcolor=transparent]        MODE[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]"tcp"
21. [backcolor=transparent]    links[backcolor=transparent]:
22. [backcolor=transparent]        [backcolor=transparent]-[backcolor=transparent] rsyslog[backcolor=transparent]:[backcolor=transparent]rsyslog
23. [backcolor=transparent]rsyslog[backcolor=transparent]:
24. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent] registry[backcolor=transparent].[backcolor=transparent]cn[backcolor=transparent]-[backcolor=transparent]hangzhou[backcolor=transparent].[backcolor=transparent]aliyuncs[backcolor=transparent].[backcolor=transparent]com[backcolor=transparent]/[backcolor=transparent]linhuatest[backcolor=transparent]/[backcolor=transparent]rsyslog[backcolor=transparent]:[backcolor=transparent]latest
25. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]master[backcolor=transparent]:
26. [backcolor=transparent]    ports[backcolor=transparent]:
27. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] [backcolor=transparent]6379[backcolor=transparent]/[backcolor=transparent]tcp
28. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
29. [backcolor=transparent]    labels[backcolor=transparent]:
30. [backcolor=transparent]        [backcolor=transparent]# 告诉自定义路由需要暴露 6379 端口
31. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]proxy[backcolor=transparent].[backcolor=transparent]TCP_PORTS[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]"6379"
32. [backcolor=transparent]        [backcolor=transparent]# 告诉系统，该服务的路由需要添加到自定义路由服务中
33. [backcolor=transparent]        aliyun[backcolor=transparent].[backcolor=transparent]proxy[backcolor=transparent].[backcolor=transparent]required[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]"true"
34. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]slave[backcolor=transparent]:
35. [backcolor=transparent]    ports[backcolor=transparent]:
36. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent]  [backcolor=transparent]6379[backcolor=transparent]/[backcolor=transparent]tcp
37. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
38. [backcolor=transparent]    links[backcolor=transparent]:
39. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]master
40. [backcolor=transparent]    labels[backcolor=transparent]:
41. [backcolor=transparent]      [backcolor=transparent]# 告诉自定义路由需要暴露 6379 端口
42. [backcolor=transparent]      aliyun[backcolor=transparent].[backcolor=transparent]proxy[backcolor=transparent].[backcolor=transparent]TCP_PORTS[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]"6379"
43. [backcolor=transparent]      [backcolor=transparent]# 告诉系统，该服务的路由需要添加到自定义路由服务中
44. [backcolor=transparent]      aliyun[backcolor=transparent].[backcolor=transparent]proxy[backcolor=transparent].[backcolor=transparent]required[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]"true"
45. [backcolor=transparent]      [backcolor=transparent]# 告诉系统，slave 需要等待 master 启动之后再启动，并且对 master 有依赖
46. [backcolor=transparent]      aliyun[backcolor=transparent].[backcolor=transparent]depends[backcolor=transparent]:[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]master
47. [backcolor=transparent]    command[backcolor=transparent]:[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]server [backcolor=transparent]--[backcolor=transparent]slaveof redis[backcolor=transparent]-[backcolor=transparent]master [backcolor=transparent]6379
48. [backcolor=transparent]redis[backcolor=transparent]-[backcolor=transparent]client[backcolor=transparent]:
49. [backcolor=transparent]    image[backcolor=transparent]:[backcolor=transparent]  [backcolor=transparent]'redis:alpine'
50. [backcolor=transparent]    links[backcolor=transparent]:
51. [backcolor=transparent]      [backcolor=transparent]-[backcolor=transparent] lb[backcolor=transparent]:[backcolor=transparent]www[backcolor=transparent].[backcolor=transparent]example[backcolor=transparent].[backcolor=transparent]com
52. [backcolor=transparent]    labels[backcolor=transparent]:
53. [backcolor=transparent]      aliyun[backcolor=transparent].[backcolor=transparent]depends[backcolor=transparent]:[backcolor=transparent] lb
54. [backcolor=transparent]    command[backcolor=transparent]:[backcolor=transparent] redis[backcolor=transparent]-[backcolor=transparent]cli [backcolor=transparent]-[backcolor=transparent]h www[backcolor=transparent].[backcolor=transparent]example[backcolor=transparent].[backcolor=transparent]com
55. [backcolor=transparent]    stdin_open[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]true
56. [backcolor=transparent]    tty[backcolor=transparent]:[backcolor=transparent] [backcolor=transparent]true