Nginx是一个高性能的web服务器，同时也是一个优秀的反向代理服务器，本文利用两台Dell R720 构建一个高可用兼负载均衡的Linux web集群。
原理
通过nginx分别搭建两个web服务器，监听在本地非80端口；
然后利用nginx构建一个包含两个节点的负载均衡池；
最后通过keepalived实现负载均衡池的高可用。两个节点同时运行时，备用节点可以承载一半的前端流量，一个节点宕机后，负载均衡器通过健康检查，将失效节点踢出集群。
环境：CentOS 6.4 x86_64
| 软件 |
版本 |
| nginx |
1.2.7 |
| keepalived |
1.2.7 |
| php-fpm |
5.4.13 |
| 主机名 |
IP地址 |
| web01 |
192.168.122.10 |
| web02 |
192.168.122.20 |
| VIP |
192.168.122.30 |
安装nginx、keepalived和php-fpm等（编译过程略）
| yum install nginx keepalived php-fpm php-gd php-xml php-mysql php-pecl-memcached php-pecl-sphinx watchodg -y chkconfig nginx on chkconfig keepalived on chkconfig php-fpm on chkconfig watchdog on |
配置nginx
nginx主配置文件 /etc/nginx/nginx.conf
| user nginx; worker_processes 2; worker_rlimit_nofile 65535;
error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid;
events { use epoll; worker_connections 2048; }
http { include /etc/nginx/mime.types; default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';
#access_log /var/log/nginx/access.log main;
sendfile on; server_tokens off;
#tcp_nopush on;
keepalive_timeout 65;
gzip on; gzip_static on; gzip_disable "msie6"; gzip_http_version 1.1; gzip_vary on; gzip_comp_level 6; gzip_proxied any; gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x$ gzip_buffers 16 8k;
client_max_body_size 20m; client_body_buffer_size 128k;
proxy_hide_header Vary; proxy_connect_timeout 600; proxy_read_timeout 600; proxy_send_timeout 600; proxy_buffer_size 16k; proxy_buffers 4 64k; proxy_busy_buffers_size 128k; #proxy_temp_file_write_size 128k
proxy_temp_path /dev/shm/tmp; proxy_cache_path /dev/shm/proxycache/ levels=1:2 keys_zone=shmcache:10m inactive=2h max_size=500m;
include /etc/nginx/conf.d/*.conf; } |
|
|
网站主配置文件/etc/nginx/conf.d/default.conf
| upstream webservers { ip_hash ; server 192.168.122.10:88 max_fails=3 fail_timeout=3s weight=2 ; server 192.168.122.20:88 max_fails=3 fail_timeout=3s weight=2 ; }
server { listen 80 ; server_name localhost ; access_log /var/log/nginx/web01.access.log main;
location / { proxy_redirect off; proxy_pass http://webservers ;
proxy_cache shmcache; proxy_cache_valid 200 302 1d; proxy_cache_valid 404 1h; proxy_cache_valid any 10m; proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_next_upstream error timeout invalid_header http_500 http_502 http_504; }
location /status { stub_status on; access_log off; allow 192.168.122.0/24; } }
server { listen 192.168.122.10:88; server_name localhost; root /usr/share/nginx/html; index index.html index.htm index.php;
#charset koi8-r;
location / { if (!-e $request_filename) { rewrite ^/(.*)$ /index.php?q=$1 last; } }
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html;
# proxy the PHP scripts to Apache listening on 127.0.0.1:80 # #location ~ \.php$ { # proxy_pass http://127.0.0.1; #}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 # location ~ \.php$ { #try_files $uri = 404; fastcgi_pass 127.0.0.1:9000; #fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_buffer_size 128k; fastcgi_buffers 256 16k; fastcgi_busy_buffers_size 256k; fastcgi_temp_file_write_size 256k; fastcgi_read_timeout 240; include fastcgi_params; }
if ($fastcgi_script_name ~ \..*\/.*php) { return 403; }
# deny access to hiden file . (filename begin with ".") location ~ /\. { access_log off; log_not_found off; deny all; }
# deny access to bakup file .(any filename end with "~" ) location ~ ~$ { access_log off; log_not_found off; deny all; }
# cache image file location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml|swf)$ { expires 1d; }
# don't log robots and favion location = /robots.txt { access_log off; log_not_found off; } location = /favicon.ico { access_log off; log_not_found off; }
# deny access to .htaccess files, if Apache's document root # concurs with nginx's one # location ~ /\.ht { deny all; } } |
配置php-fpm
php-fpm全称php fastcgi process manager，用于管理php的fastcgi进程，自从php5.3.3之后集成进了php源码中。默认的php-fpm用户是apache，我们需要修改成nginx，代码：
| sed -i 's/apache/nginx/g' /etc/php-fpm.d/www.conf |
配置keepalived
keepalived是一个高可用软件，通过vrrp心跳来检测对方是否存活。
keepalived主配置文件
| global_defs { notification_email { root@localhost } notification_email_from keepalived01@web01.test.org smtp_server 127.0.0.1 smtp_connect_timeout 30 }
vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.122.30 label eth0:1 } } |
配置watchdog
watchdog是个健壮可靠的内核模块，即使在高负载的系统中仍然可以存活。它能够执行检测脚本，定时检查nginx和keepalived进程是否存在，如果不存在可以重新开启进程。
watchdog主配置文件/etc/watchdog.conf
| min-memory = 1 repair-binary = /etc/watchdog.d/repair.sh test-binary = /etc/watchdog.d/test.sh test-timeout = 5 watchdog-device = /dev/watchdog admin = root interval = 10 logtick = 1 realtime = yes priority = 1 pidfile = /var/run/syslogd.pid pidfile = /var/run/nginx.pid pidfile = /var/run/keepalived.pid pidfile = /var/run/php-fpm/php-fpm.pid |
测试脚本，权限751
| #!/usr/bin/env bash #author:purplegrape #desc: shell script to check if keepalived/nginx/php-fpm down.
keepalived_proc=`pgrep keepalived |wc -l` nginx_proc=`pgrep nginx|wc -l` php_fpm_porc=`pgrep php-fpm|wc -l`
if [$keepalived_proc == 0 ];then exit 1 fi
if [$nginx_proc == 0 ];then exit 1 fi
if [$php_fpm_proc == 0 ];then exit 1 fi |
修复脚本,权限751
| #!/usr/bin/env bash #author:purplegrape #desc: shell script to repair keepalived/nginx/php-fpm.
keepalived_proc=`pgrep keepalived |wc -l` nginx_proc=`pgrep nginx|wc -l` php_fpm_proc=`pgrep php-fpm|wc -l`
if [$keepalived_proc == 0 ];then wall “keepalived is dead” /etc/init.d/keepalived restart sleep 30 /etc/init.d/nginx restart fi
if [$nginx_proc == 0 ];then if [$keepalived_proc != 0 ];then wall “nginx is dead” /etc/init.d/nginx restart fi fi
if [ $php_fpm_proc == 0 ]; then wall “php-fpm is dead” /etc/init.d/php-fpm restart fi |
测试
打开浏览器，访问地址http://192.168.122.30
