本文介绍如何在阿里云服务网格ASM中,结合Lua Filter和ASMGlobalRateLimiter,实现基于调用来源身份(Service Account)和请求API路径的多维度精细化全局限流。该方案可满足"不同客户端访问同一服务的不同API接口,适用不同限流配额"的业务需求。
前提条件
- 已创建ASM实例,版本≥1.25.6.74(本文使用1.28版本)。
- 已将ACK集群添加到ASM实例,并为default命名空间开启Sidecar自动注入。
- 集群中已启用mTLS(STRICT或PERMISSIVE模式),确保服务间调用携带客户端证书。
- 已安装kubectl工具,并配置好集群的kubeconfig。
背景信息
在微服务架构中,不同的调用方对同一服务的不同接口可能有不同的QPS需求。例如:
- 客户端A调用服务B的
/api/query接口,限制为50 QPS。 - 客户端A调用服务B的
/api/list接口,限制为100 QPS。 - 客户端B不受限制。
传统的全局限流只能实现"对某个服务端口统一限流"或"对某个路径统一限流",无法同时区分调用来源。本方案通过以下技术组合解决该问题:
- Lua Filter(EnvoyFilter):从mTLS连接的
x-forwarded-client-cert头中提取调用方的Service Account身份,与请求路径组合后写入自定义Headerx-rate-key。 - ASMGlobalRateLimiter:基于
x-rate-key头的精确值匹配,为不同的"调用来源+路径"组合配置独立的限流配额。
方案架构
┌─────────────────────────────┐ │ Rate Limit Service (gRPC) │ │ + Redis (计数存储) │ └──────────────▲──────────────┘ │ gRPC 查询限流 │ ┌───────────┐ ┌─────────────────────────────┼───────────────────────┐ │ sleep │──mTLS──│ httpbin Sidecar (Envoy) │ │ (客户端A) │ │ │ └───────────┘ │ ① Lua Filter: 提取 SA + Path → x-rate-key │ │ ② Rate Limit Filter: 根据 x-rate-key 查询限流服务 │ ┌───────────┐ │ ③ 限流通过 → 转发到 httpbin; 超限 → 返回 429 │ │ notsleep │──mTLS──│ │ │ (客户端B) │ └─────────────────────────────────────────────────────┘ └───────────┘
操作步骤
步骤一:部署示例服务
部署httpbin(服务端)、sleep(客户端A)和notsleep(客户端B)三个服务。
1. 部署httpbin服务
# httpbin.yaml apiVersion: v1 kind: ServiceAccount metadata: name: httpbin --- apiVersion: v1 kind: Service metadata: name: httpbin labels: app: httpbin spec: ports: - name: http port: 8000 targetPort: 80 selector: app: httpbin --- apiVersion: apps/v1 kind: Deployment metadata: name: httpbin spec: replicas: 1 selector: matchLabels: app: httpbin template: metadata: labels: app: httpbin spec: serviceAccountName: httpbin containers: - image: registry.cn-hangzhou.aliyuncs.com/acs/httpbin:latest imagePullPolicy: IfNotPresent name: httpbin ports: - containerPort: 80
2. 部署sleep服务(客户端A)
# sleep.yaml apiVersion: v1 kind: ServiceAccount metadata: name: sleep --- apiVersion: v1 kind: Service metadata: name: sleep labels: app: sleep spec: ports: - name: http port: 80 targetPort: 80 selector: app: sleep --- apiVersion: apps/v1 kind: Deployment metadata: name: sleep spec: replicas: 1 selector: matchLabels: app: sleep template: metadata: labels: app: sleep spec: terminationGracePeriodSeconds: 0 serviceAccountName: sleep containers: - name: sleep image: registry.cn-hangzhou.aliyuncs.com/acs/curl:8.1.2 command: ["/bin/sleep", "infinity"] imagePullPolicy: IfNotPresent volumeMounts: - mountPath: /etc/sleep/tls name: secret-volume volumes: - name: secret-volume secret: secretName: sleep-secret optional: true
3. 部署notsleep服务(客户端B)
# notsleep.yaml apiVersion: v1 kind: ServiceAccount metadata: name: notsleep --- apiVersion: v1 kind: Service metadata: name: notsleep labels: app: notsleep spec: ports: - name: http port: 80 targetPort: 80 selector: app: notsleep --- apiVersion: apps/v1 kind: Deployment metadata: name: notsleep spec: replicas: 1 selector: matchLabels: app: notsleep template: metadata: labels: app: notsleep spec: terminationGracePeriodSeconds: 30 serviceAccountName: notsleep containers: - name: notsleep image: registry.cn-hangzhou.aliyuncs.com/acs/curl:8.1.2 command: ["/bin/sleep", "infinity"] imagePullPolicy: IfNotPresent volumeMounts: - mountPath: /etc/sleep/tls name: notsecret-volume volumes: - name: notsecret-volume secret: secretName: notsleep-secret optional: true
执行以下命令部署:
kubectl apply -f httpbin.yaml kubectl apply -f sleep.yaml kubectl apply -f notsleep.yaml
等待所有Pod变为Running状态(含2/2 Sidecar注入):
kubectl get pods -n default
预期输出:
NAME READY STATUS RESTARTS AGE httpbin-5669bf6c74-ztddh 2/2 Running 0 1m notsleep-584ccb7dcf-26jdv 2/2 Running 0 1m sleep-7f78fffb9b-5cmk6 2/2 Running 0 1m
步骤二:部署限流服务
在数据面集群中部署Rate Limit Service和Redis。
# ratelimit-svc.yaml apiVersion: v1 kind: ServiceAccount metadata: name: redis --- apiVersion: v1 kind: Service metadata: name: redis labels: app: redis spec: ports: - name: redis port: 6379 selector: app: redis --- apiVersion: apps/v1 kind: Deployment metadata: name: redis spec: replicas: 1 selector: matchLabels: app: redis template: metadata: labels: app: redis sidecar.istio.io/inject: "false" spec: containers: - image: registry-cn-hangzhou.ack.aliyuncs.com/dev/redis:alpine imagePullPolicy: Always name: redis ports: - name: redis containerPort: 6379 restartPolicy: Always serviceAccountName: redis --- apiVersion: v1 kind: ConfigMap metadata: name: ratelimit-config data: config.yaml: | {} --- apiVersion: v1 kind: Service metadata: name: ratelimit labels: app: ratelimit spec: ports: - name: http-port port: 8080 targetPort: 8080 protocol: TCP - name: grpc-port port: 8081 targetPort: 8081 protocol: TCP - name: http-debug port: 6070 targetPort: 6070 protocol: TCP selector: app: ratelimit --- apiVersion: apps/v1 kind: Deployment metadata: name: ratelimit spec: replicas: 1 selector: matchLabels: app: ratelimit strategy: type: Recreate template: metadata: labels: app: ratelimit sidecar.istio.io/inject: "false" spec: containers: - image: registry-cn-hangzhou.ack.aliyuncs.com/dev/envoyproxy/ratelimit:e059638d imagePullPolicy: Always name: ratelimit command: ["/bin/ratelimit"] env: - name: LOG_LEVEL value: debug - name: REDIS_SOCKET_TYPE value: tcp - name: REDIS_URL value: redis:6379 - name: USE_STATSD value: "false" - name: RUNTIME_ROOT value: /data - name: RUNTIME_SUBDIRECTORY value: ratelimit - name: RUNTIME_WATCH_ROOT value: "false" - name: RUNTIME_IGNOREDOTFILES value: "true" ports: - containerPort: 8080 - containerPort: 8081 - containerPort: 6070 volumeMounts: - name: config-volume mountPath: /data/ratelimit/config volumes: - name: config-volume configMap: name: ratelimit-config
执行以下命令部署:
kubectl apply -f ratelimit-svc.yaml
说明:限流服务的Pod标签设置了sidecar.istio.io/inject: "false",不注入Sidecar,避免限流服务自身的gRPC通信受到mTLS影响。
等待限流服务就绪:
kubectl get pods -l app=ratelimit kubectl get pods -l app=redis
步骤三:验证mTLS和客户端身份
在配置限流规则之前,先确认服务间mTLS已生效,且能正确识别调用来源。
从sleep调用httpbin,检查x-forwarded-client-cert头:
kubectl exec deploy/sleep -c sleep -- curl -s httpbin:8000/headers
预期输出:
{ "headers": { "Accept": "*/*", "Host": "httpbin:8000", "User-Agent": "curl/8.1.2", "X-Envoy-Attempt-Count": "1", "X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=...;Subject=\"\";URI=spiffe://cluster.local/ns/default/sa/sleep" } }
XFCC头中的URI=spiffe://cluster.local/ns/default/sa/sleep即为调用方的SPIFFE身份,其中sleep为Service Account名称。
同样验证notsleep:
kubectl exec deploy/notsleep -c notsleep -- curl -s httpbin:8000/headers
确认XFCC头中包含URI=spiffe://cluster.local/ns/default/sa/notsleep。
步骤四:配置Lua Filter提取调用来源标识
创建EnvoyFilter,在httpbin的入向Sidecar上部署Lua脚本。该脚本从XFCC头中提取调用方Service Account名称,与请求路径拼接后写入x-rate-key头,供后续限流规则匹配使用。
# lua-filter-inject-rate-key.yaml apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: inject-source-principal namespace: default spec: workloadSelector: labels: app: httpbin configPatches: - applyTo: HTTP_FILTER match: context: SIDECAR_INBOUND listener: filterChain: filter: name: envoy.filters.network.http_connection_manager subFilter: name: envoy.filters.http.router patch: operation: INSERT_BEFORE value: name: envoy.filters.http.lua typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua defaultSourceCode: inlineString: | function envoy_on_request(request_handle) -- 从 x-forwarded-client-cert 头提取调用方 Service Account local xfcc = request_handle:headers():get("x-forwarded-client-cert") local source_sa = "unknown" if xfcc then local sa = string.match(xfcc, "URI=spiffe://[^/]+/ns/[^/]+/sa/([^;,]+)") if sa then source_sa = sa end end -- 提取请求路径(去除 query string) local path = request_handle:headers():get(":path") or "/" local clean_path = string.match(path, "([^?]+)") or "/" -- 注入 x-source-principal 头(可选,用于可观测性) request_handle:headers():add("x-source-principal", source_sa) -- 注入组合限流键:格式为 "service-account:path" request_handle:headers():add("x-rate-key", source_sa .. ":" .. clean_path) end
执行以下命令应用:
kubectl apply -f lua-filter-inject-rate-key.yaml
等待配置生效(约5~10秒),验证Header注入是否成功:
kubectl exec deploy/sleep -c sleep -- curl -s httpbin:8000/headers
预期输出中包含:
{ "headers": { "X-Rate-Key": "sleep:/headers", "X-Source-Principal": "sleep", ... } }
kubectl exec deploy/notsleep -c notsleep -- curl -s httpbin:8000/get | grep "X-Rate-Key"
预期输出:
"X-Rate-Key": "notsleep:/get",
说明:x-rate-key的格式为<service-account>:<path>。由于该Header基于mTLS证书的SPIFFE ID提取,调用方无法伪造,安全性有保障。
步骤五:配置ASMGlobalRateLimiter限流规则
创建ASMGlobalRateLimiter资源,定义基于x-rate-key精确匹配的限流规则。
本示例的限流需求如下:
调用来源 |
目标接口 |
限流配额 |
sleep(客户端A) |
/get |
1次/分钟 |
sleep(客户端A) |
/headers |
5次/分钟 |
notsleep(客户端B) |
/get |
10次/分钟 |
notsleep(客户端B) |
/headers |
20次/分钟 |
其他 |
其他路径 |
不限制(100000次/秒) |
# asmglobalratelimiter.yaml apiVersion: istio.alibabacloud.com/v1 kind: ASMGlobalRateLimiter metadata: name: httpbin-per-client-path namespace: default spec: workloadSelector: labels: app: httpbin applyToTraffic: sidecar_inbound rateLimitService: host: ratelimit.default.svc.cluster.local port: 8081 timeout: seconds: 5 configs: - name: httpbin-ratelimit target_services: - name: httpbin port: 8000 limit: unit: SECOND quota: 100000 limit_overrides: - request_match: header_match: - name: "x-rate-key" exact_match: "sleep:/get" limit: unit: MINUTE quota: 1 - request_match: header_match: - name: "x-rate-key" exact_match: "sleep:/headers" limit: unit: MINUTE quota: 5 - request_match: header_match: - name: "x-rate-key" exact_match: "notsleep:/get" limit: unit: MINUTE quota: 10 - request_match: header_match: - name: "x-rate-key" exact_match: "notsleep:/headers" limit: unit: MINUTE quota: 20
执行以下命令应用:
kubectl apply -f asmglobalratelimiter.yaml
步骤六:同步限流配置到限流服务
ASMGlobalRateLimiter创建后,ASM控制面会自动reconcile并在资源的.status.config.yaml字段中生成限流服务所需的配置。
查看生成的配置:
kubectl get asmglobalratelimiter httpbin-per-client-path -o jsonpath='{.status}' | python3 -m json.tool
预期输出:
{ "config.yaml": "descriptors:\n- descriptors:\n - key: header_match\n rate_limit:\n requests_per_unit: 1\n unit: MINUTE\n value: RateLimit[httpbin-per-client-path.default]-Id[174941530]\n - key: header_match\n rate_limit:\n requests_per_unit: 5\n unit: MINUTE\n value: RateLimit[httpbin-per-client-path.default]-Id[3018745594]\n - key: header_match\n rate_limit:\n requests_per_unit: 10\n unit: MINUTE\n value: RateLimit[httpbin-per-client-path.default]-Id[2971100891]\n - key: header_match\n rate_limit:\n requests_per_unit: 20\n unit: MINUTE\n value: RateLimit[httpbin-per-client-path.default]-Id[1626499192]\n key: generic_key\n rate_limit:\n requests_per_unit: 100000\n unit: SECOND\n value: RateLimit[httpbin-per-client-path.default]-Id[699415551]\ndomain: ratelimit.default.svc.cluster.local\n", "message": "ok", "status": "successful" }
确认status为successful后,将生成的config.yaml内容更新到ratelimit-config ConfigMap中:
# ratelimit-config.yaml apiVersion: v1 kind: ConfigMap metadata: name: ratelimit-config namespace: default data: config.yaml: | descriptors: - descriptors: - key: header_match rate_limit: requests_per_unit: 1 unit: MINUTE value: RateLimit[httpbin-per-client-path.default]-Id[174941530] - key: header_match rate_limit: requests_per_unit: 5 unit: MINUTE value: RateLimit[httpbin-per-client-path.default]-Id[3018745594] - key: header_match rate_limit: requests_per_unit: 10 unit: MINUTE value: RateLimit[httpbin-per-client-path.default]-Id[2971100891] - key: header_match rate_limit: requests_per_unit: 20 unit: MINUTE value: RateLimit[httpbin-per-client-path.default]-Id[1626499192] key: generic_key rate_limit: requests_per_unit: 100000 unit: SECOND value: RateLimit[httpbin-per-client-path.default]-Id[699415551] domain: ratelimit.default.svc.cluster.local
重要:请将实际.status.config.yaml字段的内容复制到ConfigMap的data.config.yaml中。不同环境生成的Id值可能不同,请以实际输出为准。
执行以下命令更新ConfigMap并重启限流服务以加载新配置:
kubectl apply -f ratelimit-config.yaml kubectl rollout restart deploy/ratelimit kubectl rollout status deploy/ratelimit --timeout=60s
验证限流服务成功加载配置:
kubectl logs deploy/ratelimit --tail=5
预期输出包含:
level=info msg="Successfully loaded new configuration" level=info msg="Successfully loaded the initial ratelimit configs" level=warning msg="Listening for gRPC on '0.0.0.0:8081'"
步骤七:验证限流效果
等待约10秒使Envoy配置完全生效后,开始验证各场景。
验证1:sleep访问/get(限制1次/分钟)
for i in $(seq 1 3); do echo -n "Request $i: " kubectl exec deploy/sleep -c sleep -- curl -s -o /dev/null -w "%{http_code}" httpbin:8000/get echo "" done
预期输出:
Request 1: 200 Request 2: 429 Request 3: 429
验证2:sleep访问/headers(限制5次/分钟)
for i in $(seq 1 7); do echo -n "Request $i: " kubectl exec deploy/sleep -c sleep -- curl -s -o /dev/null -w "%{http_code}" httpbin:8000/headers echo "" done
预期输出:
Request 1: 200 Request 2: 200 Request 3: 200 Request 4: 200 Request 5: 200 Request 6: 429 Request 7: 429
验证3:notsleep访问/get(限制10次/分钟)
for i in $(seq 1 12); do echo -n "Request $i: " kubectl exec deploy/notsleep -c notsleep -- curl -s -o /dev/null -w "%{http_code}" httpbin:8000/get echo "" done
预期输出:
Request 1: 200 ... Request 10: 200 Request 11: 429 Request 12: 429
验证4:notsleep访问/headers(限制20次/分钟)
for i in $(seq 1 22); do echo -n "Request $i: " kubectl exec deploy/notsleep -c notsleep -- curl -s -o /dev/null -w "%{http_code}" httpbin:8000/headers echo "" done
预期输出:
Request 1: 200 ... Request 20: 200 Request 21: 429 Request 22: 429
验证5:未配置规则的接口不受限制
for i in $(seq 1 5); do echo -n "Request $i: " kubectl exec deploy/sleep -c sleep -- curl -s -o /dev/null -w "%{http_code}" httpbin:8000/status/200 echo "" done
预期输出:
Request 1: 200 Request 2: 200 Request 3: 200 Request 4: 200 Request 5: 200
验证6:查看限流响应头
当请求被限流时,响应中会包含以下标准限流头:
kubectl exec deploy/sleep -c sleep -- curl -s -D - httpbin:8000/get | grep -i "x-ratelimit\|x-envoy-ratelimited\|HTTP/"
预期输出(被限流时):
HTTP/1.1 429 Too Many Requests x-envoy-ratelimited: true x-ratelimit-limit: 1, 1;w=60, 100000;w=1 x-ratelimit-remaining: 0 x-ratelimit-reset: 49
响应头说明:
响应头 |
说明 |
|
表示请求被限流 |
|
限流配额为1次/60秒 |
|
剩余可用配额为0 |
|
配额将在49秒后重置 |
工作原理
数据流
- 客户端(如sleep)发起对httpbin:8000的请求,经Envoy Sidecar出向代理发出。
- 请求到达httpbin的Envoy Sidecar入向代理。由于mTLS已建立,XFCC头包含调用方SPIFFE身份。
- Lua Filter解析XFCC头,提取Service Account名(如
sleep),与请求路径(如/get)拼接为sleep:/get,写入x-rate-key头。 - Rate Limit Filter根据路由上配置的actions,生成descriptor(包含
generic_key和header_value_match结果),通过gRPC发送到Rate Limit Service。 - Rate Limit Service查询Redis中的计数,判断是否超限,返回OVER_LIMIT或OK。
- 如果超限,Envoy返回HTTP 429;否则请求正常转发到httpbin服务。
ASMGlobalRateLimiter Reconcile机制
ASMGlobalRateLimiter CRD创建后:
- ASM控制面Reconcile该资源,自动生成两部分配置:
- EnvoyFilter:注入Rate Limit Filter和route上的rate_limits actions到目标Sidecar。
- config.yaml:生成Rate Limit Service的descriptor配置,写入资源的
.status.config.yaml字段。
- 用户需手动将
.status.config.yaml内容同步到ratelimit-configConfigMap中。 - 重启Rate Limit Service使配置生效。
为什么使用组合Header而非多Header匹配
ASMGlobalRateLimiter的limit_overrides.request_match.header_match支持配置多个Header条件。当配置多个Header时,控制器会生成多层嵌套的descriptor结构,导致Envoy发送的descriptor层级与RLS期望的层级不匹配。
通过Lua Filter将多个维度(来源身份+路径)组合为单一Header值(x-rate-key),使每个limit_override只需匹配一个Header,生成正确的2层descriptor结构(generic_key → header_match),确保Envoy与RLS之间的descriptor完全对齐。
配置字段说明
ASMGlobalRateLimiter核心字段
字段 |
类型 |
说明 |
|
map |
选择生效的工作负载,本例为 |
|
string |
流量方向。 |
|
string |
限流服务的集群内域名 |
|
int |
限流服务的gRPC端口 |
|
object |
默认限流配额(未匹配任何override时生效) |
|
array |
针对特定请求的限流覆盖配置 |
|
array |
请求头匹配条件 |
|
array |
限流生效的目标服务(ASM≥1.25) |
EnvoyFilter Lua脚本关键逻辑
逻辑 |
说明 |
从XFCC提取SA |
|
去除path中的query string |
|
组合rate key |
|
扩展场景
新增客户端限流规则
当需要为新客户端(如service-c)配置限流时,只需在ASMGlobalRateLimiter的limit_overrides中增加条目:
- request_match: header_match: - name: "x-rate-key" exact_match: "service-c:/get" limit: unit: MINUTE quota: 30
然后重新同步config.yaml到ConfigMap并重启限流服务。
按路径前缀匹配
如需对路径前缀进行限流(如/api/v1/开头的所有接口),可使用prefix_match:
- request_match: header_match: - name: "x-rate-key" prefix_match: "sleep:/api/v1/" limit: unit: MINUTE quota: 100
注意:使用prefix_match时需注意匹配优先级。Rate Limit Service按descriptor精确匹配,多条规则可能同时命中。建议优先使用exact_match,或确保前缀路径不存在重叠。
监控限流指标
在httpbin的ProxyConfig中开启限流相关指标采集:
kubectl exec deploy/httpbin -c istio-proxy -- curl -s localhost:15090/stats/prometheus | grep envoy_cluster_ratelimit
关键指标:
指标名 |
说明 |
|
通过限流检查的请求数 |
|
被限流的请求数 |
|
限流服务调用失败数 |
注意事项
- Rate Limit Service高可用:生产环境建议部署多副本Rate Limit Service,Redis使用集群模式或Sentinel模式,确保限流服务本身不成为单点故障。
- 性能影响:全局限流会为每个请求增加一次gRPC调用(约1~5ms延迟)。如对延迟敏感,可调整
rateLimitService.timeout或考虑结合本地限流(ASMLocalRateLimiter)使用。 - failure_mode_deny:默认为
false,即当限流服务不可用时请求放行。如需严格限流(限流服务故障时拒绝所有请求),需设置为true。 - ConfigMap同步:修改ASMGlobalRateLimiter后,需手动同步
.status.config.yaml到ConfigMap并重启限流服务。后续版本可能支持自动同步。 - EnvoyFilter版本兼容:Lua Filter的EnvoyFilter为底层API,ASM版本升级时建议重新验证兼容性。
- Header安全性:
x-rate-key由服务端Sidecar的Lua脚本在入向流量上生成,基于mTLS证书信息,客户端无法伪造。但如果httpbin对外暴露(通过IngressGateway),需确保网关层不会透传外部伪造的x-rate-key头。
清理资源
如需删除本文创建的所有资源:
kubectl delete asmglobalratelimiter httpbin-per-client-path -n default kubectl delete envoyfilter inject-source-principal -n default kubectl delete -f ratelimit-svc.yaml kubectl delete -f notsleep.yaml kubectl delete -f sleep.yaml kubectl delete -f httpbin.yaml