Tomcat捕获Nginx反向代理协议IP及端口配置

最近的技术调整，需要获取Nginx端请求方式是http还是https。我们当前的架构属于典型的互联网架构，Nginx+Tomcat+Mysql，启用了反向代理。当前并没有启用全站https，因此客户端发出的请求基于模块的方式使用http或者https协议。关于捕获这个请求的协议，我们需要使用到Nginx ngx_http_proxy_module这个模块中的proxy_set_header指令来进行配置。本文描述通过这个命令来实现捕获请求协议。

一、关于proxy_set_header指令

这个指令的使用上下文是http，server，location，它允许重新定义或附加字段到传递给代理服务器的请求头部 。简单的说，当客户端发出web请求后，这个指令可以把真实的客户端请求的如IP地址，端口号，协议等等追加或修改到代理服务器的请求头部。通过这个设置，我们可以获取最终客户的原始请求信息，而不是代理服务器作为客户端的信息。这个指令的值value可以包含文本，变量，以及它们的组合。当且仅当proxy_set_header 在当前级别上没有定义的指令时，这些指令才从前一级继承 。默认情况下，只有两个字段被重新定义，他们是，proxy_set_header Host $proxy_host; proxy_set_header Connection close。

proxy_set_header指令中几个常用的变量
$remote_addr  ：客户端真实地址，非代理服务器地址
$remote_port  ：客户端真实端口，非代理服务器端口
$proxy_host    ：proxy_pass指令中指定的代理服务器的名称和端口 ;
$proxy_port      ：proxy_pass指令中指定的代理服务器 的端口或协议的默认端口;
$proxy_add_x_forwarded_for  ：带有$remote_addr变量的“X-Forwarded-For”客户机请求头字段，用逗号分隔。
如果客户请求头中不存在“X-Forwarded-For”字段，则$proxy_add_x_forwarded_for变量等于该$remote_addr变量。
它在多层代理时会包含真实客户端及中间每个代理服务器的IP。

二、环境描述

nginx服务器环境

[root@node132 ~]# more /etc/redhat-release 
CentOS release 6.7 (Final)
[root@node132 ~]# nginx -v
nginx version: nginx/1.10.2
[root@node132 ~]# ip addr|grep inet|grep global 
    inet 192.168.1.132/24 brd 192.168.1.255 scope global eth0
    inet 192.168.81.148/24 brd 192.168.81.255 scope global eth1

tomcat服务器环境

[root@datanode1 ~]# more /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)  ##，另外一台tomcat环境与此相同，仅仅是ip地址和主机名不同
[root@datanode1 ~]# /usr/local/tomcat/bin/catalina.sh version
erver version: Apache Tomcat/7.0.69
Server built:  Apr 11 2016 07:57:09 UTC
Server number:  7.0.69.0
root@datanode1 ~]# ip addr|grep inet|grep global
    inet 192.168.81.146/24 brd 192.168.81.255 scope global dynamic eno16777728

三、Nginx配置

nginx配置文件

[root@node132 ~]# more /etc/nginx/conf.d/tomcat.conf 
upstream app {
                server 192.168.81.146:8080;
                server 192.168.81.147:8080;
}

server {
    listen 80;
    server_name node132.ydq.com;
    location / {
        proxy_pass http://app;
    
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-scheme $scheme;
        proxy_set_header x-agent $http_user_agent;
        add_header backendIP $upstream_addr;
    }
}

server {
    listen      443 ssl;
    server_name  node132.ydq.com;
    ssl_certificate      /etc/nginx/conf.d/node132.ydq.com.crt;
    ssl_certificate_key  /etc/nginx/conf.d/node132.ydq.com.key;

    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;

    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

    location / {
        proxy_pass http://app;
      
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-Port $remote_port;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-scheme $scheme;
        proxy_set_header x-agent $http_user_agent;
        add_header backendIP $upstream_addr;
    }
} 

[root@node132 ~]# /etc/init.d/nginx reload
[root@node132 ~]# ss -nltp|grep nginx
LISTEN    0      128  *:80    *:*  users:(("nginx",13703,6),("nginx",13705,6))
LISTEN    0      128  *:443  *:*  users:(("nginx",13703,7),("nginx",13705,7))

四、Tomcat配置

TomcatA配置如下
[root@datanode1 ~]# more /usr/local/tomcat/conf/server.xml ##仅列出修改过的部分如下
<Engine name="Catalina" defaultHost="datanode1.example.com" jvmRoute="TomcatA">
<Host name="datanode1.example.com"  appBase="/website"
            unpackWARs="true" autoDeploy="true">
        <Context path="" docBase="webapps" reloadable="true" /> 
</Host>

TomcatB配置如下
[root@datanode2 ~]# more /usr/local/tomcat/conf/server.xml ##仅列出修改过的部分如下
<Engine name="Catalina" defaultHost="datanode2.example.com" jvmRoute="TomcatB">
<Host name="datanode2.example.com"  appBase="/website"
            unpackWARs="true" autoDeploy="true">
        <Context path="" docBase="webapps" reloadable="true" /> 
</Host>

两台tomcat均创建如下目录，第二台演示略
[root@datanode1 ~]# mkdir -pv /website/webapps
[root@datanode1 ~]# vi /website/webapps/reverse_proxy.jsp 
<%@page contentType="text/html; charset=UTF-8" trimDirectiveWhitespaces="true"%>
<%

String proto=request.getHeader("X-Forwarded-scheme");
String serverName = request.getServerName();
String remoteName = request.getRemoteAddr();
String get_method = request.getMethod().toString();
int serverPort = request.getServerPort();
int clientPort = request.getRemotePort();

String xforwardedip = request.getHeader("X-Forwarded-For");
String xrealip = request.getHeader("X-Real-IP");
String Host = request.getHeader("Host");

String client_ip_port = proto+"://"+xforwardedip+":"+clientPort;
String reverse_proxy_ip_port = proto+"://"+remoteName+":"+serverPort;

String request_url=request.getRequestURL().toString();
String request_uri=request.getRequestURI().toString();
String usr_agent = request.getHeader("x-agent");
%>

<h1>Client Information</h1>
ClientPort:<%=clientPort%><br>
Client IP and Port : <%=client_ip_port%> <br>
Client Request method : <%=get_method%> <br>
Full Request Link : <%=request_url%> <br>
user_agent:<%=usr_agent%> <br>

<h1>Reverse Proxy Server Information</h1>
Server Name : <%=serverName%> <br>
Reverse Hostname : <%=Host%> <br>

Reverse Proxy Server IP and Port : <%=reverse_proxy_ip_port%> <br>
Request URL : <%=reverse_proxy_ip_port%>  <br>
Request URI : <%=request_uri%>  <br>

五、测试

在Windows客户端测试
浏览器http请求：http://node132.ydq.com/reverse_proxy.jsp
Client Information
ClientPort:58906
Client IP and Port : http://192.168.1.242:58906  //正确捕获到了客户端的ip地址及端口
Client Request method : GET
Full Request Link : http://node132.ydq.com/reverse_proxy.jsp  //完整的请求链接
user_agent:Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0

Reverse Proxy Server Information
Server Name : node132.ydq.com
Reverse Hostname : node132.ydq.com
Reverse Proxy Server IP and Port : http://192.168.81.148:80  //服务器端对应的ip地址及端口
Request URL : http://192.168.81.148:80
Request URI : /reverse_proxy.jsp                                                 

浏览器https请求：https://node132.ydq.com/reverse_proxy.jsp
Client Information  ## Author : Leshami
ClientPort:58908   ## Blog : http://blog.csdn.net/leshami
Client IP and Port : https://192.168.1.242:58908
Client Request method : GET
Full Request Link : http://node132.ydq.com/reverse_proxy.jsp
user_agent:Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0

Reverse Proxy Server Information
Server Name : node132.ydq.com
Reverse Hostname : node132.ydq.com
Reverse Proxy Server IP and Port : https://192.168.81.148:80
Request URL : https://192.168.81.148:80  //此处与上一行端口应当为443才对
Request URI : /reverse_proxy.jsp

Google了N多，需要修改tomcat的配置，修改如下(修改后重启tomcat)：
<Valve className="org.apache.catalina.valves.RemoteIpValve"  
remoteIpHeader="X-Forwarded-For"  
requestAttributesEnabled="true"
protocolHeader="X-Forwarded-scheme"  
protocolHeaderHttpsValue="https"/> 

Client Information
ClientPort:41168
Client IP and Port : https://null:41168  //这个地方显示为Null值了
Client Request method : GET
Full Request Link : https://node132.ydq.com/reverse_proxy.jsp
user_agent:Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0

Reverse Proxy Server Information
Server Name : node132.ydq.com
Reverse Hostname : node132.ydq.com
Reverse Proxy Server IP and Port : https://192.168.1.242:443
Request URL : https://192.168.1.242:443  //此行同上行IP地址应该为反向代理服务器IP才对，端口现在OK
Request URI : /reverse_proxy.jsp

上述的这个问题被遗留，暂时没有找到明确的答案。有知道的专家们请指点。

六、更多参考

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header
http://tomcat.apache.org/tomcat-7.0-doc/servletapi/javax/servlet/http/HttpServletRequest.html