背景
入口流量的tag标签，一般有两种方式：
1）客户端发送请求带上特定的标签( 如 HTTP Header) —— 需要客户端根据灰度修改代码
2）入口网关根据流量特征进行打标
网关层面的打标一般基于网关插件化能力，将请求流量进行打标。 比如外部用户访问 Base 环境，内部用户访问 Gray 环境；比如将userid处于一定范围的打上代表灰度的tag； 比如根据登录用户 token 用户组 group 进行打标。
通过本文实现的 流量打标 + ASM 全链路灰度，可以实现完整的从 入口网关 到 服务的 无侵入的ASM全链路灰度发布。
实现方式
通过 EnvoyFilter 采用 Lua 脚本根据 HTTP 请求的 来源IP 或者 JWT 在 route 到 upstream 服务之前附加新的 HTTP Headers。
场景一：根据内外网来源请求对流量打标
EnvoyFilter 配置
来源于内部的请求将打上 x-asm-prefer-tag = gray 标签，来源于外部的请求将打上 x-asm-prefer-tag = base 的标签
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: http-request-labelling-according-source
namespace: istio-system
spec:
workloadSelector:
labels:
app: istio-ingressgateway
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
function envoy_on_request(request_handle)
local jwtInterHeaderName = "X-Envoy-Internal"
local jwtExterHeaderName = "X-Envoy-External-Address"
headers = request_handle:headers()
jwtinter = headers:get(jwtInterHeaderName)
jwtexter = headers:get(jwtExterHeaderName)
if (jwtinter ~= nil) then
headers:add("x-asm-prefer-tag","gray")
elseif (jwtexter ~= nil) then
headers:add("x-asm-prefer-tag","base")
else
headers:add("x-asm-prefer-tag","notmatch")
end
end
httpbin 应用部署
部署 httpbin 应用来检验 request HTTP header 是否如愿打上 header。
##################################################################################################
# httpbin service
##################################################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: httpbin
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
labels:
app: httpbin
service: httpbin
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
version: v1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
serviceAccountName: httpbin
containers:
- image: docker.io/kennethreitz/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: direct-httpbin-simple-http-vs
spec:
gateways:
- direct-httpbin-simple-http-gw
hosts:
- direct-httpbin-simple-http.asmworkshop.io
- httpbin
http:
- route:
- destination:
host: httpbin
port:
number: 8000
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: direct-httpbin-simple-http-gw
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- direct-httpbin-simple-http.asmworkshop.io
port:
name: http
number: 80
protocol: HTTP
测试 Header 打标情况
从外部访问(访问 istio-ingressgateway 公网 SLB 入口 )：可见带上了 X-Asm-Prefer-Tag": "base"
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 120.24.87.135/get?show_env=true
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "base",
"X-B3-Parentspanid": "bfea77423107a397",
"X-B3-Sampled": "1",
"X-B3-Spanid": "78967b70d984995d",
"X-B3-Traceid": "601184bf39e6f5e5bfea77423107a397",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "47.115.37.91",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=a1b644096de33600b9b59a006afd0a087cc2f2612c85f4f2da8738e3d829f108;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "47.115.37.91",
"X-Forwarded-Proto": "http",
"X-Request-Id": "e2cd8489-22cd-979d-b4a8-db9f494c0982"
},
"origin": "47.115.37.91",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
从VPC 内部访问(访问 istio-ingressgateway 私网 SLB 入口 )：可见带上了 "X-Asm-Prefer-Tag": "gray"
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 192.168.4.143/get?show_env=true
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "gray",
"X-B3-Parentspanid": "78c3503c97a46b22",
"X-B3-Sampled": "1",
"X-B3-Spanid": "c0ee0a47deb148eb",
"X-B3-Traceid": "1593be6d6dece99678c3503c97a46b22",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=a1b644096de33600b9b59a006afd0a087cc2f2612c85f4f2da8738e3d829f108;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "192.168.0.240",
"X-Forwarded-Proto": "http",
"X-Request-Id": "f77e6a52-4532-9596-a8a4-aa5d152c9a7a"
},
"origin": "192.168.0.240",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
根据标签路由
部署一个新的应用，分别 base 版本 和 gray 版本，测试按照上一步骤打上的 x-asm-prefer-tag 来路由：
apiVersion: v1
kind: ServiceAccount
metadata:
name: http-a
---
apiVersion: v1
kind: Service
metadata:
labels:
app: http-a
name: http-a
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
selector:
app: http-a
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: http-a
name: lua-http-a-base
spec:
replicas: 1
selector:
matchLabels:
app: http-a
version: base
template:
metadata:
labels:
app: http-a
version: base
spec:
serviceAccountName: http-a
containers:
- env:
- name: "LISTEN_ADDR"
value: "0.0.0.0:8080"
- name: "SERVER_TYPE"
value: "http"
- name: "NAME"
value: "http-a"
- name: "MESSAGE"
value: "Web response from http-a-Base"
- name: "HTTP_CLIENT_APPEND_REQUEST"
value: "true"
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/containerdemo/nicholasjackson-fake-service:v0.17.0
imagePullPolicy: IfNotPresent
name: http-a
ports:
- containerPort: 8080
name: http
protocol: TCP
securityContext:
privileged: false
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: http-a
name: lua-http-a-gray
spec:
replicas: 1
selector:
matchLabels:
app: http-a
version: gray
template:
metadata:
labels:
app: http-a
version: gray
spec:
serviceAccountName: http-a
containers:
- env:
- name: "LISTEN_ADDR"
value: "0.0.0.0:8080"
- name: "SERVER_TYPE"
value: "http"
- name: "NAME"
value: "http-a"
- name: "MESSAGE"
value: "Web response from http-a-Gray"
- name: "HTTP_CLIENT_APPEND_REQUEST"
value: "true"
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/containerdemo/nicholasjackson-fake-service:v0.17.0
imagePullPolicy: IfNotPresent
name: http-a
ports:
- containerPort: 8080
name: http
protocol: TCP
securityContext:
privileged: false
配置 GW, VS, DR 暴露服务：
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: lua-simple-http
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- lua-simple-http.asmworkshop.io
port:
name: http
number: 80
protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: lua-simple-http-a-vs
spec:
gateways:
- lua-simple-http
hosts:
- lua-simple-http.asmworkshop.io
- http-a
http:
- match:
- headers:
x-asm-prefer-tag:
exact: gray
route:
- destination:
host: http-a
subset: version-gray
- route:
- destination:
host: http-a
subset: version-base
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: lua-simple-http-a-dr
spec:
host: http-a
subsets:
- name: version-gray
labels:
version: gray
- name: version-base
labels:
version: base
测试
从VPC 内部访问(访问 istio-ingressgateway 私网 SLB 入口 )：可见内部用户访问到 灰度服务
# curl -H "Host: lua-simple-http.asmworkshop.io" 192.168.4.143
{
"name": "http-a",
"uri": "/",
"type": "HTTP",
"ip_addresses": [
"192.168.86.68"
],
"start_time": "2021-10-24T12:54:16.883832",
"end_time": "2021-10-24T12:54:16.883950",
"duration": "117.748µs",
"body": "Web response from http-a-Gray",
"code": 200
}
从外部访问(访问 istio-ingressgateway 公网 SLB 入口 )：可见外部用户 访问到 正常环境
# curl -H "Host: lua-simple-http.asmworkshop.io" 120.24.87.135
{
"name": "http-a",
"uri": "/",
"type": "HTTP",
"ip_addresses": [
"192.168.86.67"
],
"start_time": "2021-10-24T12:54:31.610400",
"end_time": "2021-10-24T12:54:31.610771",
"duration": "370.758µs",
"body": "Web response from http-a-Base",
"code": 200
}
场景二：根据 JWT Claim 对流量进行打标
EnvoyFilter 配置
Lua 将 decode 从 HTTP Header Authorization 带的 JWT Token ，并将 Token 中的 iss, sub, group 等信息附加到新的 Authorization-Iss, Authorization-Sub, Authorization-Group 的 Header 里：
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: parse-request-jwt-for-labelling
namespace: istio-system
spec:
workloadSelector:
labels:
app: istio-ingressgateway
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
-- Json Parsing based on https://gist.github.com/tylerneylon/59f4bcf316be525b30ab
-- Base64 decoding based on wikipedia description of 8/6bit encoding.
-- base64 char array.. note final 2 chars are for RFC4648-URL encoding
-- as per JWT spec section 2 terminology 'Base64url Encoding'
local alpha='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
-- convert to 6 char long binary string. (max int 64!)
function toBinaryString(int)
if int > 64 then
error("Bad number "..int.." to convert to binary")
end
local remaining = tonumber(int)
local bits = ''
for i = 5, 0, -1 do
local pow = 2 ^ i
if remaining >= pow then
bits = bits .. '1'
remaining = remaining - pow
else
bits = bits .. '0'
end
end
return bits
end
function fromBinaryString(bits)
return tonumber(bits, 2)
end
function decodeBase64(encoded)
local bitstr = ''
local decoded = ''
-- decode chars into bitstring
for i = 1, string.len(encoded) do
local offset, _ = string.find(alpha, string.sub(encoded, i, i))
if offset == nil then
error("Bad base64 character " .. string.sub(encoded, i, i))
end
bitstr = bitstr .. toBinaryString(offset-1)
end
-- decode bitstring back to chars
for i = 1, string.len(bitstr), 8 do
decoded = decoded .. string.char(fromBinaryString(string.sub(bitstr, i, i+7)))
end
return decoded
end
-- json handling
local json = {}
local function kind_of(obj)
if type(obj) ~= 'table' then return type(obj) end
local i = 1
for _ in pairs(obj) do
if obj[i] ~= nil then i = i + 1 else return 'table' end
end
if i == 1 then return 'table' else return 'array' end
end
local function escape_str(s)
local in_char = {'\\', '"', '/', '\b', '\f', '\n', '\r', '\t'}
local out_char = {'\\', '"', '/', 'b', 'f', 'n', 'r', 't'}
for i, c in ipairs(in_char) do
s = s:gsub(c, '\\' .. out_char[i])
end
return s
end
-- Returns pos, did_find; there are two cases:
-- 1. Delimiter found: pos = pos after leading space + delim; did_find = true.
-- 2. Delimiter not found: pos = pos after leading space; did_find = false.
-- This throws an error if err_if_missing is true and the delim is not found.
local function skip_delim(str, pos, delim, err_if_missing)
pos = pos + #str:match('^%s*', pos)
if str:sub(pos, pos) ~= delim then
if err_if_missing then
error('Expected ' .. delim .. ' near position ' .. pos)
end
return pos, false
end
return pos + 1, true
end
-- Expects the given pos to be the first character after the opening quote.
-- Returns val, pos; the returned pos is after the closing quote character.
local function parse_str_val(str, pos, val)
val = val or ''
local early_end_error = 'End of input found while parsing string.'
if pos > #str then error(early_end_error) end
local c = str:sub(pos, pos)
if c == '"' then return val, pos + 1 end
if c ~= '\\' then return parse_str_val(str, pos + 1, val .. c) end
-- We must have a \ character.
local esc_map = {b = '\b', f = '\f', n = '\n', r = '\r', t = '\t'}
local nextc = str:sub(pos + 1, pos + 1)
if not nextc then error(early_end_error) end
return parse_str_val(str, pos + 2, val .. (esc_map[nextc] or nextc))
end
-- Returns val, pos; the returned pos is after the number's final character.
local function parse_num_val(str, pos)
local num_str = str:match('^-?%d+%.?%d*[eE]?[+-]?%d*', pos)
local val = tonumber(num_str)
if not val then error('Error parsing number at position ' .. pos .. '.') end
return val, pos + #num_str
end
json.null = {} -- one-off table to represent the null value.
function json.parse(str, pos, end_delim)
pos = pos or 1
if pos > #str then error('Reached unexpected end of input.') end
local pos = pos + #str:match('^%s*', pos) -- Skip whitespace.
local first = str:sub(pos, pos)
if first == '{' then -- Parse an object.
local obj, key, delim_found = {}, true, true
pos = pos + 1
while true do
key, pos = json.parse(str, pos, '}')
if key == nil then return obj, pos end
if not delim_found then error('Comma missing between object items.') end
pos = skip_delim(str, pos, ':', true) -- true -> error if missing.
obj[key], pos = json.parse(str, pos)
pos, delim_found = skip_delim(str, pos, ',')
end
elseif first == '[' then -- Parse an array.
local arr, val, delim_found = {}, true, true
pos = pos + 1
while true do
val, pos = json.parse(str, pos, ']')
if val == nil then return arr, pos end
if not delim_found then error('Comma missing between array items.') end
arr[#arr + 1] = val
pos, delim_found = skip_delim(str, pos, ',')
end
elseif first == '"' then -- Parse a string.
return parse_str_val(str, pos + 1)
elseif first == '-' or first:match('%d') then -- Parse a number.
return parse_num_val(str, pos)
elseif first == end_delim then -- End of an object or array.
return nil, pos + 1
else -- Parse true, false, or null.
local literals = {['true'] = true, ['false'] = false, ['null'] = json.null}
for lit_str, lit_val in pairs(literals) do
local lit_end = pos + #lit_str - 1
if str:sub(pos, lit_end) == lit_str then return lit_val, lit_end + 1 end
end
local pos_info_str = 'position ' .. pos .. ': ' .. str:sub(pos, pos + 10)
error('Invalid json syntax starting at ' .. pos_info_str)
end
end
function decode_jwt(jwt)
i=0
result = {}
for match in (jwt..'.'):gmatch("(.-)%.") do
result[i]=decodeBase64(match)
i=i+1
end
-- header
head = json.parse(result[0])
-- claims
claims = json.parse(result[1])
return {head=head,claims=claims}
end
function add_header(k,v,prefix,headers)
if "number" == type (k) then
headers:add(prefix,v)
else
headers:add(prefix.."-"..k,v)
end
end
function add_table_as_headers(table, prefix, headers)
for k,v in pairs(table) do
if "string" == type( v ) then
add_header(k,v,prefix,headers)
elseif "table" == type( v ) then
add_table_as_headers(v,prefix.."-"..k,headers)
end
end
end
function envoy_on_request(request_handle)
local jwtHeaderName = "Authorization"
headers = request_handle:headers()
jwt = headers:get(jwtHeaderName)
if jwt == nil then
headers:add("jwt","headernotfound")
else
content = decode_jwt(jwt)
add_table_as_headers(content["claims"],jwtHeaderName,headers)
end
end
测试 httpbin 服务
ADMIN_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwNzEsImdyb3VwIjoiYWRtaW4iLCJpYXQiOjE1OTE1NDUwNzEsImlzcyI6ImF1dGhAaXN0aW9pbmFjdGlvbi5pbyIsInN1YiI6IjIxOGQzZmI5LTQ2MjgtNGQyMC05NDNjLTEyNDI4MWM4MGU3YiJ9.MEL9ANwx4kvxkK90cdkUBejn-cLIrACdvGiE9T4RE3F1FRc4et4EZ79s-tbb7OJgnOCkTcvB-Q4V_9WaeAU_kNvzM1rGGh1a0ahQI01Iipt0c6RUlWk1GUr5eUul7xw5MoR-kKDuB-fB0qG2_WQfyiqez6uO9OGJxipTwfhoWJfq_9sZ3p7d8iwJzIcCleTb6ywKmIa4gJb0UhaVcs77HP7KTq9PzTj2adOa2KtfH0BTFjAymZKJVEsV64A_XdNAybiVmEmd8kqTuIbHob-ZT9Mlyl3ER_A6rbIzx6myD9F8m1GIaz2fgtMCJyawuxd_YK4L1cvWhJ2BkbyCtC1znQ
echo $ADMIN_TOKEN | cut -d '.' -f2 | base64 -d
{"exp":4745145071,"group":"admin","iat":1591545071,"iss":"auth@istioinaction.io","sub":"218d3fb9-4628-4d20-943c-124281c80e7b"}
不带 Authorizaiton Header：可见带上了 "Jwt": "headernotfound" 的 Header
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 192.168.4.143/get?show_env=true
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"Jwt": "headernotfound",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "gray",
"X-B3-Parentspanid": "bd40ccc0158d33bd",
"X-B3-Sampled": "1",
"X-B3-Spanid": "2a222c8963b9c868",
"X-B3-Traceid": "7c0ecbb3b1894e18bd40ccc0158d33bd",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=a1b644096de33600b9b59a006afd0a087cc2f2612c85f4f2da8738e3d829f108;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "192.168.0.240",
"X-Forwarded-Proto": "http",
"X-Request-Id": "4ec96591-230f-9858-b961-6d741f7c7188"
},
"origin": "192.168.0.240",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
带 Authorization Header(访问 私网 SLB）: 可见带上了以下 Header
"Authorization-Group": "admin",
"Authorization-Iss": "auth@istioinaction.io",
"Authorization-Sub": "218d3fb9-4628-4d20-943c-124281c80e7b",
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 192.168.4.143/get?show_env=true -H "Authorization: $ADMIN_TOKEN"
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Authorization": "eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwNzEsImdyb3VwIjoiYWRtaW4iLCJpYXQiOjE1OTE1NDUwNzEsImlzcyI6ImF1dGhAaXN0aW9pbmFjdGlvbi5pbyIsInN1YiI6IjIxOGQzZmI5LTQ2MjgtNGQyMC05NDNjLTEyNDI4MWM4MGU3YiJ9.MEL9ANwx4kvxkK90cdkUBejn-cLIrACdvGiE9T4RE3F1FRc4et4EZ79s-tbb7OJgnOCkTcvB-Q4V_9WaeAU_kNvzM1rGGh1a0ahQI01Iipt0c6RUlWk1GUr5eUul7xw5MoR-kKDuB-fB0qG2_WQfyiqez6uO9OGJxipTwfhoWJfq_9sZ3p7d8iwJzIcCleTb6ywKmIa4gJb0UhaVcs77HP7KTq9PzTj2adOa2KtfH0BTFjAymZKJVEsV64A_XdNAybiVmEmd8kqTuIbHob-ZT9Mlyl3ER_A6rbIzx6myD9F8m1GIaz2fgtMCJyawuxd_YK4L1cvWhJ2BkbyCtC1znQ",
"Authorization-Group": "admin",
"Authorization-Iss": "auth@istioinaction.io",
"Authorization-Sub": "218d3fb9-4628-4d20-943c-124281c80e7b",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "gray",
"X-B3-Parentspanid": "121a2ec37208dfdf",
"X-B3-Sampled": "1",
"X-B3-Spanid": "42ee3aca9dc339d0",
"X-B3-Traceid": "fee9135efff4cf33121a2ec37208dfdf",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=a1b644096de33600b9b59a006afd0a087cc2f2612c85f4f2da8738e3d829f108;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "192.168.0.240",
"X-Forwarded-Proto": "http",
"X-Request-Id": "c9e0c85b-1d18-92d9-8a15-6b217476d8f0"
},
"origin": "192.168.0.240",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
带 Authorization Header(访问 公网 SLB）: 可见带上了以下 Header
"Authorization-Group": "admin",
"Authorization-Iss": "auth@istioinaction.io",
"Authorization-Sub": "218d3fb9-4628-4d20-943c-124281c80e7b",
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 120.24.87.135/get?show_env=true -H "Authorization: $ADMIN_TOKEN"
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Authorization": "eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwNzEsImdyb3VwIjoiYWRtaW4iLCJpYXQiOjE1OTE1NDUwNzEsImlzcyI6ImF1dGhAaXN0aW9pbmFjdGlvbi5pbyIsInN1YiI6IjIxOGQzZmI5LTQ2MjgtNGQyMC05NDNjLTEyNDI4MWM4MGU3YiJ9.MEL9ANwx4kvxkK90cdkUBejn-cLIrACdvGiE9T4RE3F1FRc4et4EZ79s-tbb7OJgnOCkTcvB-Q4V_9WaeAU_kNvzM1rGGh1a0ahQI01Iipt0c6RUlWk1GUr5eUul7xw5MoR-kKDuB-fB0qG2_WQfyiqez6uO9OGJxipTwfhoWJfq_9sZ3p7d8iwJzIcCleTb6ywKmIa4gJb0UhaVcs77HP7KTq9PzTj2adOa2KtfH0BTFjAymZKJVEsV64A_XdNAybiVmEmd8kqTuIbHob-ZT9Mlyl3ER_A6rbIzx6myD9F8m1GIaz2fgtMCJyawuxd_YK4L1cvWhJ2BkbyCtC1znQ",
"Authorization-Group": "admin",
"Authorization-Iss": "auth@istioinaction.io",
"Authorization-Sub": "218d3fb9-4628-4d20-943c-124281c80e7b",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "base",
"X-B3-Parentspanid": "e67a410d8a5aaf9e",
"X-B3-Sampled": "1",
"X-B3-Spanid": "8f3049fa4b729cb2",
"X-B3-Traceid": "6d4cc5da57581e34e67a410d8a5aaf9e",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "192.168.0.240",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=aa897e5e1f68f47f20baf3071b4473e1a433fc04a6a54e17bfea902d01b656bd;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "56.5.6.7, 72.9.5.6, 98.1.2.3,192.168.0.240",
"X-Forwarded-Proto": "http",
"X-Request-Id": "d64d5e70-1d58-99f2-8ef1-3dff2e9eb809"
},
"origin": "56.5.6.7, 72.9.5.6, 98.1.2.3,192.168.0.240",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
根据 JWT claim[group] 对流量进行打标
根据 Lua Decode 的 JWT 信息，如果 group = admin，则访问到正常环境；如果 group=user，则转发请求附加 x-asm-traffic-tag=gray，然后根据 VirtualService 规则访问到灰度环境。
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: parse-request-jwt-for-labelling
namespace: istio-system
spec:
workloadSelector:
labels:
app: istio-ingressgateway
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
-- Json Parsing based on https://gist.github.com/tylerneylon/59f4bcf316be525b30ab
-- Base64 decoding based on wikipedia description of 8/6bit encoding.
-- base64 char array.. note final 2 chars are for RFC4648-URL encoding
-- as per JWT spec section 2 terminology 'Base64url Encoding'
local alpha='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
-- convert to 6 char long binary string. (max int 64!)
function toBinaryString(int)
if int > 64 then
error("Bad number "..int.." to convert to binary")
end
local remaining = tonumber(int)
local bits = ''
for i = 5, 0, -1 do
local pow = 2 ^ i
if remaining >= pow then
bits = bits .. '1'
remaining = remaining - pow
else
bits = bits .. '0'
end
end
return bits
end
function fromBinaryString(bits)
return tonumber(bits, 2)
end
function decodeBase64(encoded)
local bitstr = ''
local decoded = ''
-- decode chars into bitstring
for i = 1, string.len(encoded) do
local offset, _ = string.find(alpha, string.sub(encoded, i, i))
if offset == nil then
error("Bad base64 character " .. string.sub(encoded, i, i))
end
bitstr = bitstr .. toBinaryString(offset-1)
end
-- decode bitstring back to chars
for i = 1, string.len(bitstr), 8 do
decoded = decoded .. string.char(fromBinaryString(string.sub(bitstr, i, i+7)))
end
return decoded
end
-- json handling
local json = {}
local function kind_of(obj)
if type(obj) ~= 'table' then return type(obj) end
local i = 1
for _ in pairs(obj) do
if obj[i] ~= nil then i = i + 1 else return 'table' end
end
if i == 1 then return 'table' else return 'array' end
end
local function escape_str(s)
local in_char = {'\\', '"', '/', '\b', '\f', '\n', '\r', '\t'}
local out_char = {'\\', '"', '/', 'b', 'f', 'n', 'r', 't'}
for i, c in ipairs(in_char) do
s = s:gsub(c, '\\' .. out_char[i])
end
return s
end
-- Returns pos, did_find; there are two cases:
-- 1. Delimiter found: pos = pos after leading space + delim; did_find = true.
-- 2. Delimiter not found: pos = pos after leading space; did_find = false.
-- This throws an error if err_if_missing is true and the delim is not found.
local function skip_delim(str, pos, delim, err_if_missing)
pos = pos + #str:match('^%s*', pos)
if str:sub(pos, pos) ~= delim then
if err_if_missing then
error('Expected ' .. delim .. ' near position ' .. pos)
end
return pos, false
end
return pos + 1, true
end
-- Expects the given pos to be the first character after the opening quote.
-- Returns val, pos; the returned pos is after the closing quote character.
local function parse_str_val(str, pos, val)
val = val or ''
local early_end_error = 'End of input found while parsing string.'
if pos > #str then error(early_end_error) end
local c = str:sub(pos, pos)
if c == '"' then return val, pos + 1 end
if c ~= '\\' then return parse_str_val(str, pos + 1, val .. c) end
-- We must have a \ character.
local esc_map = {b = '\b', f = '\f', n = '\n', r = '\r', t = '\t'}
local nextc = str:sub(pos + 1, pos + 1)
if not nextc then error(early_end_error) end
return parse_str_val(str, pos + 2, val .. (esc_map[nextc] or nextc))
end
-- Returns val, pos; the returned pos is after the number's final character.
local function parse_num_val(str, pos)
local num_str = str:match('^-?%d+%.?%d*[eE]?[+-]?%d*', pos)
local val = tonumber(num_str)
if not val then error('Error parsing number at position ' .. pos .. '.') end
return val, pos + #num_str
end
json.null = {} -- one-off table to represent the null value.
function json.parse(str, pos, end_delim)
pos = pos or 1
if pos > #str then error('Reached unexpected end of input.') end
local pos = pos + #str:match('^%s*', pos) -- Skip whitespace.
local first = str:sub(pos, pos)
if first == '{' then -- Parse an object.
local obj, key, delim_found = {}, true, true
pos = pos + 1
while true do
key, pos = json.parse(str, pos, '}')
if key == nil then return obj, pos end
if not delim_found then error('Comma missing between object items.') end
pos = skip_delim(str, pos, ':', true) -- true -> error if missing.
obj[key], pos = json.parse(str, pos)
pos, delim_found = skip_delim(str, pos, ',')
end
elseif first == '[' then -- Parse an array.
local arr, val, delim_found = {}, true, true
pos = pos + 1
while true do
val, pos = json.parse(str, pos, ']')
if val == nil then return arr, pos end
if not delim_found then error('Comma missing between array items.') end
arr[#arr + 1] = val
pos, delim_found = skip_delim(str, pos, ',')
end
elseif first == '"' then -- Parse a string.
return parse_str_val(str, pos + 1)
elseif first == '-' or first:match('%d') then -- Parse a number.
return parse_num_val(str, pos)
elseif first == end_delim then -- End of an object or array.
return nil, pos + 1
else -- Parse true, false, or null.
local literals = {['true'] = true, ['false'] = false, ['null'] = json.null}
for lit_str, lit_val in pairs(literals) do
local lit_end = pos + #lit_str - 1
if str:sub(pos, lit_end) == lit_str then return lit_val, lit_end + 1 end
end
local pos_info_str = 'position ' .. pos .. ': ' .. str:sub(pos, pos + 10)
error('Invalid json syntax starting at ' .. pos_info_str)
end
end
function decode_jwt(jwt)
i=0
result = {}
for match in (jwt..'.'):gmatch("(.-)%.") do
result[i]=decodeBase64(match)
i=i+1
end
-- header
head = json.parse(result[0])
-- claims
claims = json.parse(result[1])
return {head=head,claims=claims}
end
function add_header(k,v,prefix,headers)
if "number" == type (k) then
headers:add(prefix,v)
else
headers:add(prefix.."-"..k,v)
end
end
function add_table_as_headers(table, prefix, headers)
for k,v in pairs(table) do
if "string" == type( v ) then
add_header(k,v,prefix,headers)
elseif "table" == type( v ) then
add_table_as_headers(v,prefix.."-"..k,headers)
end
end
end
function envoy_on_request(request_handle)
local jwtHeaderName = "Authorization"
headers = request_handle:headers()
jwt = headers:get(jwtHeaderName)
if jwt == nil then
headers:add("jwt","headernotfound")
else
content = decode_jwt(jwt)
usergroup = content["claims"]["group"]
if usergroup == "user" then
headers:add("x-asm-traffic-tag", "gray")
end
add_table_as_headers(content["claims"],jwtHeaderName,headers)
end
end
测试 jwt group = admin 时，不打 x-asm-traffic-tag 标签
# ADMIN_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwNzEsImdyb3VwIjoiYWRtaW4iLCJpYXQiOjE1OTE1NDUwNzEsImlzcyI6ImF1dGhAaXN0aW9pbmFjdGlvbi5pbyIsInN1YiI6IjIxOGQzZmI5LTQ2MjgtNGQyMC05NDNjLTEyNDI4MWM4MGU3YiJ9.MEL9ANwx4kvxkK90cdkUBejn-cLIrACdvGiE9T4RE3F1FRc4et4EZ79s-tbb7OJgnOCkTcvB-Q4V_9WaeAU_kNvzM1rGGh1a0ahQI01Iipt0c6RUlWk1GUr5eUul7xw5MoR-kKDuB-fB0qG2_WQfyiqez6uO9OGJxipTwfhoWJfq_9sZ3p7d8iwJzIcCleTb6ywKmIa4gJb0UhaVcs77HP7KTq9PzTj2adOa2KtfH0BTFjAymZKJVEsV64A_XdNAybiVmEmd8kqTuIbHob-ZT9Mlyl3ER_A6rbIzx6myD9F8m1GIaz2fgtMCJyawuxd_YK4L1cvWhJ2BkbyCtC1znQ
# echo $ADMIN_TOKEN | cut -d '.' -f2 | base64 -d
{"exp":4745145071,"group":"admin","iat":1591545071,"iss":"auth@istioinaction.io","sub":"218d3fb9-4628-4d20-943c-124281c80e7b"}
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 192.168.4.143/get?show_env=true -H "Authorization: $ADMIN_TOKEN"
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Authorization": "eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwNzEsImdyb3VwIjoiYWRtaW4iLCJpYXQiOjE1OTE1NDUwNzEsImlzcyI6ImF1dGhAaXN0aW9pbmFjdGlvbi5pbyIsInN1YiI6IjIxOGQzZmI5LTQ2MjgtNGQyMC05NDNjLTEyNDI4MWM4MGU3YiJ9.MEL9ANwx4kvxkK90cdkUBejn-cLIrACdvGiE9T4RE3F1FRc4et4EZ79s-tbb7OJgnOCkTcvB-Q4V_9WaeAU_kNvzM1rGGh1a0ahQI01Iipt0c6RUlWk1GUr5eUul7xw5MoR-kKDuB-fB0qG2_WQfyiqez6uO9OGJxipTwfhoWJfq_9sZ3p7d8iwJzIcCleTb6ywKmIa4gJb0UhaVcs77HP7KTq9PzTj2adOa2KtfH0BTFjAymZKJVEsV64A_XdNAybiVmEmd8kqTuIbHob-ZT9Mlyl3ER_A6rbIzx6myD9F8m1GIaz2fgtMCJyawuxd_YK4L1cvWhJ2BkbyCtC1znQ",
"Authorization-Group": "admin",
"Authorization-Iss": "auth@istioinaction.io",
"Authorization-Sub": "218d3fb9-4628-4d20-943c-124281c80e7b",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "gray",
"X-B3-Parentspanid": "ba38f3a6f6e3b4a1",
"X-B3-Sampled": "1",
"X-B3-Spanid": "ff276929069963f9",
"X-B3-Traceid": "c7ead271417ec943ba38f3a6f6e3b4a1",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=a1b644096de33600b9b59a006afd0a087cc2f2612c85f4f2da8738e3d829f108;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "192.168.0.240",
"X-Forwarded-Proto": "http",
"X-Request-Id": "571b89d3-df43-9b75-9221-03824e06dac1"
},
"origin": "192.168.0.240",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
测试 jwt group = user 时，打上 x-asm-traffic-tag = gray 标签
# USER_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwMzgsImdyb3VwIjoidXNlciIsImlhdCI6MTU5MTU0NTAzOCwiaXNzIjoiYXV0aEBpc3Rpb2luYWN0aW9uLmlvIiwic3ViIjoiOWI3OTJiNTYtN2RmYS00ZTRiLWE4M2YtZTIwNjc5MTE1ZDc5In0.jNDoRx7SNm8b1xMmPaOEMVgwdnTmXJwD5jjCH9wcGsLisbZGcR6chkirWy1BVzYEQDTf8pDJpY2C3H-aXN3IlAcQ1UqVe5lShIjCMIFTthat3OuNgu-a91csGz6qtQITxsOpMcBinlTYRsUOICcD7UZcLugxK4bpOECohHoEhuASHzlH-FYESDB-JYrxmwXj4xoZ_jIsdpuqz_VYhWp8e0phDNJbB6AHOI3m7OHCsGNcw9Z0cks1cJrgB8JNjRApr9XTNBoEC564PX2ZdzciI9BHoOFAKx4mWWEqW08LDMSZIN5Ui9ppwReSV2ncQOazdStS65T43bZJwgJiIocSCg
echo $USER_TOKEN | cut -d '.' -f2 | base64 -d
{"exp":4745145038,"group":"user","iat":1591545038,"iss":"auth@istioinaction.io","sub":"9b792b56-7dfa-4e4b-a83f-e20679115d79"}
# curl -H "Host: direct-httpbin-simple-http.asmworkshop.io" 192.168.4.143/get?show_env=true -H "Authorization: $USER_TOKEN"
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": "*/*",
"Authorization": "eyJhbGciOiJSUzI1NiIsImtpZCI6IkNVLUFESkpFYkg5YlhsMHRwc1FXWXVvNEV3bGt4RlVIYmVKNGNra2FrQ00iLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ3NDUxNDUwMzgsImdyb3VwIjoidXNlciIsImlhdCI6MTU5MTU0NTAzOCwiaXNzIjoiYXV0aEBpc3Rpb2luYWN0aW9uLmlvIiwic3ViIjoiOWI3OTJiNTYtN2RmYS00ZTRiLWE4M2YtZTIwNjc5MTE1ZDc5In0.jNDoRx7SNm8b1xMmPaOEMVgwdnTmXJwD5jjCH9wcGsLisbZGcR6chkirWy1BVzYEQDTf8pDJpY2C3H-aXN3IlAcQ1UqVe5lShIjCMIFTthat3OuNgu-a91csGz6qtQITxsOpMcBinlTYRsUOICcD7UZcLugxK4bpOECohHoEhuASHzlH-FYESDB-JYrxmwXj4xoZ_jIsdpuqz_VYhWp8e0phDNJbB6AHOI3m7OHCsGNcw9Z0cks1cJrgB8JNjRApr9XTNBoEC564PX2ZdzciI9BHoOFAKx4mWWEqW08LDMSZIN5Ui9ppwReSV2ncQOazdStS65T43bZJwgJiIocSCg",
"Authorization-Group": "user",
"Authorization-Iss": "auth@istioinaction.io",
"Authorization-Sub": "9b792b56-7dfa-4e4b-a83f-e20679115d79",
"Content-Length": "0",
"Host": "direct-httpbin-simple-http.asmworkshop.io",
"User-Agent": "curl/7.29.0",
"X-Asm-Prefer-Tag": "gray",
"X-Asm-Traffic-Tag": "gray",
"X-B3-Parentspanid": "412847ab2f501b48",
"X-B3-Sampled": "1",
"X-B3-Spanid": "2dc41eb50c073b38",
"X-B3-Traceid": "0781bb560d8f47b3412847ab2f501b48",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/simple-http/sa/httpbin;Hash=a1b644096de33600b9b59a006afd0a087cc2f2612c85f4f2da8738e3d829f108;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"X-Forwarded-For": "192.168.0.240",
"X-Forwarded-Proto": "http",
"X-Request-Id": "dc591104-1ed4-9770-8a73-f690ffeeba6b"
},
"origin": "192.168.0.240",
"url": "http://direct-httpbin-simple-http.asmworkshop.io/get?show_env=true"
}
测试按标路由
修改 VirtualService 按 x-asm-traffic-tag Header 进行路由：
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: lua-simple-http
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- lua-simple-http.asmworkshop.io
port:
name: http
number: 80
protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: lua-simple-http-a-vs
spec:
gateways:
- lua-simple-http
hosts:
- lua-simple-http.asmworkshop.io
- http-a
http:
- match:
- headers:
x-asm-traffic-tag:
exact: gray
route:
- destination:
host: http-a
subset: version-gray
- route:
- destination:
host: http-a
subset: version-base
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: lua-simple-http-a-dr
spec:
host: http-a
subsets:
- name: version-gray
labels:
version: gray
- name: version-base
labels:
version: base
测试
# 不带 Authorization 访问服务，访问到 Base 环境
# curl -H "Host: lua-simple-http.asmworkshop.io" 192.168.4.143
{
"name": "http-a",
"uri": "/",
"type": "HTTP",
"ip_addresses": [
"192.168.86.67"
],
"start_time": "2021-10-24T14:39:35.217636",
"end_time": "2021-10-24T14:39:35.217755",
"duration": "118.382µs",
"body": "Web response from http-a-Base",
"code": 200
}
# 带 Authorization (group=admin)访问服务，访问到 Base 环境
# curl -H "Host: lua-simple-http.asmworkshop.io" 192.168.4.143 -H "Authorization: $ADMIN_TOKEN"
{
"name": "http-a",
"uri": "/",
"type": "HTTP",
"ip_addresses": [
"192.168.86.67"
],
"start_time": "2021-10-24T14:39:49.560351",
"end_time": "2021-10-24T14:39:49.560463",
"duration": "111.947µs",
"body": "Web response from http-a-Base",
"code": 200
}
# 带 Authorization (group=user)访问服务，访问到 Gray 环境
# curl -H "Host: lua-simple-http.asmworkshop.io" 192.168.4.143 -H "Authorization: $USER_TOKEN"
{
"name": "http-a",
"uri": "/",
"type": "HTTP",
"ip_addresses": [
"192.168.86.68"
],
"start_time": "2021-10-24T14:39:57.268686",
"end_time": "2021-10-24T14:39:57.268825",
"duration": "139.248µs",
"body": "Web response from http-a-Gray",
"code": 200
}
