问题现象
网站业务架构为:
高防->WAF->slb 7层->ECS
Nginx在ECS上获取真实的客户端IP地址,配置不生效。
测试环境
高防IP为:118.xxx.xxx.204
WAF的Cname为:9qlliqgcxxxxxu0z.aliyunwaf.com 120.xxx.xxx.174
测试的域名为:there.pier39.cn
负载均衡的IP为: 119.xxx.xxx.162
ECS的IP为:120.xxx.xxx.201
[root
@xx
nginx]# uname -a
Linux iZ94ypgtdryZ
3.10
.
0
-
327.22
.
2
.el7.x86_64 #
1
SMP Thu Jun
23
17
:
05
:
11
UTC
2016
x86_64 x86_64 x86_64 GNU/Linux
[root
@iZ94ypgtdryZ
nginx]# lsb_release -a
LSB Version: :core-
4.1
-amd64:core-
4.1
-noarch
Distributor ID: CentOS
Description: CentOS Linux release
7.2
.
1511
(Core)
Release:
7.2
.
1511
Codename: Core
[root
@xx
nginx]#
[root
@iZ94ypgtdryZ
nginx]# nginx -V
nginx version: nginx/
1.10
.
2
built by gcc
4.8
.
5
20150623
(Red Hat
4.8
.
5
-
4
) (GCC)
built with OpenSSL
1.0
.1e-fips
11
Feb
2013
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt=
'-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'
--with-ld-opt=
'-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'
|
配置方式:logformat方法
1、高防、WAF、负载均衡相关的配置在此省略
2、Nginx主配置文件
/etc/nginx/nginx.conf
核心配置内容:
log_format there.pier39.cn
'$remote_addr - $remote_user [$time_local] $request'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"'
;
access_log /var/log/nginx/there.pier39.cn there.pier39.cn;
|
在log_format中加入了*$http_x_forwarded_for*
具体以上内容所在的位置,请特别注意,可以到下方完整的配置中寻找。
整体的配置内容如下:
[root
@iZ94ypgtdryZ
nginx]# cat nginx.conf
# For more information on configuration, see:
# * Official English Documentation: http:
//nginx.org/en/docs/
# * Official Russian Documentation: http:
//nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections
1024
;
}
http {
log_format main
'$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent"'
;
access_log /var/log/nginx/access.log main;
log_format there.pier39.cn
'$remote_addr - $remote_user [$time_local] $request'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"'
;
access_log /var/log/nginx/there.pier39.cn there.pier39.cn;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout
65
;
types_hash_max_size
2048
;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http:
//nginx.org/en/docs/ngx_core_module.html#include
#
for
more information.
include /etc/nginx/conf.d/*.conf;
server {
listen
80
default_server;
listen [::]:
80
default_server;
server_name _;
root /usr/share/nginx/html;
# Load configuration files
for
the
default
server block.
include /etc/nginx/
default
.d/*.conf;
location / {
}
error_page
404
/
404
.html;
location = /40x.html {
}
error_page
500
502
503
504
/50x.html;
location = /50x.html {
}
}
# Settings
for
a TLS enabled server.
#
# server {
# listen
443
ssl http2 default_server;
# listen [::]:
443
ssl http2 default_server;
# server_name _;
# root /usr/share/nginx/html;
#
# ssl_certificate
"/etc/pki/nginx/server.crt"
;
# ssl_certificate_key
"/etc/pki/nginx/private/server.key"
;
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 10m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files
for
the
default
server block.
# include /etc/nginx/
default
.d/*.conf;
#
# location / {
# }
#
# error_page
404
/
404
.html;
# location = /40x.html {
# }
#
# error_page
500
502
503
504
/50x.html;
# location = /50x.html {
# }
# }
}
|
3、VirtualHost配置文件
/etc/nginx/conf.d/there.pier39.cn.conf
server {
listen
80
;
server_name there.pier39.cn;
index index.html index.htm index.php;
root /var/www/html;
}
|
4、配置完成后
A、使用nginx检查配置文件是否正确。
B、重启nginx服务(Restart)
[root
@iZ94ypgtdryZ
nginx]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root
@iZ94ypgtdryZ
nginx]# service nginx restart
Redirecting to /bin/systemctl restart nginx.service
[root
@iZ94ypgtdryZ
nginx]#
|
5、测试
A、未配置前的
[root
@iZ94ypgtdryZ
nginx]# tailf there.pier39.cn | grep -v HEAD
100.97
.
15.216
- - [
22
/Feb/
2017
:
14
:
52
:
30
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.1
- - [
22
/Feb/
2017
:
14
:
52
:
48
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.187
- - [
22
/Feb/
2017
:
14
:
52
:
49
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.59
- - [
22
/Feb/
2017
:
14
:
52
:
49
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.214
- - [
22
/Feb/
2017
:
14
:
52
:
49
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.14
- - [
22
/Feb/
2017
:
14
:
52
:
50
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
|
B、配置后的
100.97
.
15.181
- - [
22
/Feb/
2017
:
15
:
05
:
51
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xxx.xx.201, 120.xx.xx.15, 120.xx.xx.232"
100.97
.
15.6
- - [
22
/Feb/
2017
:
15
:
05
:
53
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xxx.xx.201, 120.xx.xx.15, 120.xx.xx.225"
100.97
.
15.189
- - [
22
/Feb/
2017
:
15
:
05
:
54
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xx.xx.201, 120.xx.xx.15, 120.xx.xx.4"
100.97
.
15.50
- - [
22
/Feb/
2017
:
15
:
05
:
54
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xxx.xxx.201, 120.xx.xx.15, 120.xx.xx.4"
100.97
.
15.11
- - [
22
/Feb/
2017
:
15
:
05
:
54
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xxx..201, 120.xx.xx.15, 120.xx.xx.4"
|
真实的IP地址在日志的最尾部,符合log_format配置的格式
配置方式:realip方法
注:用nginx -V检查是否安装过realip模块,如果没有安装需要安装。
1、Nginx主配置文件
/etc/nginx/nginx.conf
核心内容:
log_format there.pier39.cn
'$remote_addr - $remote_user [$time_local] $request'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" '
;
access_log /var/log/nginx/there.pier39.cn there.pier39.cn;
|
具体以上内容所在的位置,请特别注意,可以到下方完整的配置中寻找。
全部配置文件
[root
@iZ94ypgtdryZ
nginx]# cat nginx.conf
# For more information on configuration, see:
# * Official English Documentation: http:
//nginx.org/en/docs/
# * Official Russian Documentation: http:
//nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections
1024
;
}
http {
log_format main
'$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent"'
;
access_log /var/log/nginx/access.log main;
log_format there.pier39.cn
'$remote_addr - $remote_user [$time_local] $request'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" '
;
access_log /var/log/nginx/there.pier39.cn there.pier39.cn;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout
65
;
types_hash_max_size
2048
;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http:
//nginx.org/en/docs/ngx_core_module.html#include
#
for
more information.
include /etc/nginx/conf.d/*.conf;
server {
listen
80
default_server;
listen [::]:
80
default_server;
server_name _;
root /usr/share/nginx/html;
# Load configuration files
for
the
default
server block.
include /etc/nginx/
default
.d/*.conf;
location / {
}
error_page
404
/
404
.html;
location = /40x.html {
}
error_page
500
502
503
504
/50x.html;
location = /50x.html {
}
}
# Settings
for
a TLS enabled server.
#
# server {
# listen
443
ssl http2 default_server;
# listen [::]:
443
ssl http2 default_server;
# server_name _;
# root /usr/share/nginx/html;
#
# ssl_certificate
"/etc/pki/nginx/server.crt"
;
# ssl_certificate_key
"/etc/pki/nginx/private/server.key"
;
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 10m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files
for
the
default
server block.
# include /etc/nginx/
default
.d/*.conf;
#
# location / {
# }
#
# error_page
404
/
404
.html;
# location = /40x.html {
# }
#
# error_page
500
502
503
504
/50x.html;
# location = /50x.html {
# }
# }
}
|
2、VirtualHost配置文件
在该文件中加入了location的内容,请务必注意;
server {
listen
80
;
server_name there.pier39.cn;
index index.html index.htm index.php;
root /var/www/html;
location / {
set_real_ip_from
0.0
.
0.0
/
0
;
real_ip_header X-Forwarded-For;
real_ip_recursive on;
}
}
|
3、配置完成后
A、使用nginx检查配置文件是否正确。
B、重启nginx服务(Restart)
4、测试
1、配置前
[root
@xxx
nginx]# tailf there.pier39.cn | grep -v HEAD
100.97
.
15.4
- - [
22
/Feb/
2017
:
15
:
54
:
40
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.180
- - [
22
/Feb/
2017
:
15
:
54
:
42
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.210
- - [
22
/Feb/
2017
:
15
:
54
:
45
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
100.97
.
15.2
- - [
22
/Feb/
2017
:
15
:
54
:
45
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
|
2、配置后
[root
@xxx
nginx]# tailf there.pier39.cn | grep -v HEAD
42.xxx.xx
.201
- - [
22
/Feb/
2017
:
16
:
18
:
50
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
42.xx
.xx
.201
- - [
22
/Feb/
2017
:
16
:
18
:
50
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
42.xxx
.xx
.201
- - [
22
/Feb/
2017
:
16
:
18
:
50
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
42.xxx
.xx
.201
- - [
22
/Feb/
2017
:
16
:
18
:
50
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
42.xxx
.xx
.201
- - [
22
/Feb/
2017
:
16
:
18
:
51
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
|
总结
1、这里介绍了2中配置获取xff的方法,2种配置的差别,可以很明显的在最终测试的日志中对比出来。
logformat:
100.97
.
15.11
- - [
22
/Feb/
2017
:
15
:
05
:
54
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xxx.xx.201, 120.xx.xx.15, 120.xx.xx.4"
|
realip:
42.xx.xx
.201
- - [
22
/Feb/
2017
:
16
:
18
:
51
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
|
2种方式都可以获取真实IP地址,但是在日志中显示的方式不一样。
当然,也可以2种方式都配置。方法,自行脑补。
42.xx.xx
.201
- - [
22
/Feb/
2017
:
16
:
22
:
42
+
0800
] GET /ddd HTTP/
1.0404
571
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
"42.xx.xx201, 120.xx.xxx.16, 120.xx.xx.231"
|
2、2种取一种即可,根据实际情况选择。
3、该方法,可用于高防->WAF->slb 7层->ECS 上述链路的使用,当然如果只有负载均衡也是可以的。在链路中再加入CDN也能够正常获取到。此方法的原理为获取通信报文中的X-Forwarded-For字段进行解析
附录
1、X-Forwarded-For简介
X-Forwarded-For:简称XFF头,它代表代表客户端,也就是HTTP的请求端真实的IP,只有在通过了HTTP 代理或者负载均衡服务器时才会添加该项。它不是RFC中定义的标准请求头信息,在squid缓存代理服务器开发文档中可以找到该项的详细介绍。
标准格式如下:X-Forwarded-For: client1, proxy1, proxy2。
2、X-Forwarded-For在http报文中的体现(Web Server上抓包)