Oracle CRS 集群资源管理
Oracle Cluster Ready Service（Oracle集群就绪服务）,用于构建Oracle RAC环境，对Oracle RAC下资源管理。但CRS不但可以用于RAC环境，也可以用于通用的集群服务，本文就是通过CRS来管理WEB服务，介绍CRS的集群服务的功能。
在CRS中，所有要管理的服务，可以称之为资源（Resource）
以下为：CRS对资源管理的流程
CRS 管理案例：通过CRS搭建linux 下的WEB集群
系统环境：
操作系统： RH EL5.5
集群软件： Oracle 11g GI
数据库： Oracle 11g
前提已经安装和配置了 Oracle 11g RAC
1、配置 apache
[root@node1 ~]# rpm -qa |grep httpd
[root@node1 ~]# mount /dev/cdrom /media
mount: block device /dev/cdrom is write-protected, mounting read-only
安装Apache：
[root@node1 ~]# yum install -y httpd*
配置linux 下普通用户启动httpd
更改httpd主目录的拥有者
chown -R grid：oinstall/etc/httpd
修改httpd 主配置文件
1)、在oracle主目录建立目录
[oracle@node2 ~]$ mkdir logs run
2)、编辑httpd.conf 文件，修改以下选项
ServerName 192.168.8.20 （192.168.8.20 为web 集群服务的vip 地址）
Listen 192.168.8.20:8088
Use grid
Group oinstall
ErrorLog /home/oracle/logs/error_log
CustomLog /home/oracle/logs/access_log combined
3)、编辑 /etc/httpd/conf.d/ssl.conf
ErrorLog /home/oracle/logs/ssl_error_log
TransferLog /home/oracle/logs/ssl_access_log
CustomLog /home/oracle/logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
#Listen 443;注释此选项，普通用户不能启动1024以下的端口的服务
4)、修改文件访问权限
chmod o+w/var/lock/subsys
-----Grid 用户可以启动/关闭httpd 服务
注意：
[grid@node1 ~]$ /etc/init.d/httpd stop
rm: cannot remove `/home/oracle/run/httpd
.pid': Permission denied
rm: cannot remove `/home/oracle/run/httpd.pid': Permission denied
[grid@node1 ~]$ su -
Password:
修改/etc/init.d/httpd脚本：
[root@node1 ~]# vi /etc/init.d/httpd
#!/bin/bash
#
# httpd Startup script for the Apache HTTP Server
#
# chkconfig: - 85 15
# description: Apache is a World Wide Web server. It is used to serve \
# HTML files and CGI.
# processname: httpd
# config: /etc/httpd/conf/httpd.conf
# config: /etc/sysconfig/httpd
# pidfile: /var/run/httpd.pid
# Source function library.
. /etc/rc.d/init.d/functions
if [ -f /etc/sysconfig/httpd ]; then
. /etc/sysconfig/httpd
fi
# Start httpd in the C locale by default.
HTTPD_LANG=${HTTPD_LANG-"C"}
# This will prevent initlog from swallowing up a pass-phrase prompt if
# mod_ssl needs a pass-phrase from the user.
INITLOG_ARGS=""
# Set HTTPD=/usr/sbin/httpd.worker in /etc/sysconfig/httpd to use a server
# with the thread-based "worker" MPM; BE WARNED that some modules may not
# work correctly with a thread-based MPM; notably PHP will refuse to start.
# Path to the apachectl script, server binary, and short-form for messages.
apachectl=/usr/sbin/apachectl
httpd=${HTTPD-/usr/sbin/httpd}
prog=httpd
#pidfile=${PIDFILE-/var/run/httpd.pid}
pidfile=${PIDFILE-/home/grid/run/httpd.pid} ；修改pidfile 位置
lockfile=${LOCKFILE-/var/lock/subsys/httpd}
RETVAL=0
# check for 1.3 configuration
check13 () {
CONFFILE=/etc/httpd/conf/httpd.conf
Grid 用户启动Apache服务：
[grid @rac2 run]$/etc/init.d/httpd start
[grid @rac2 run]$ ps -ef |grep httpd
grid 4979 271330 05:50 pts/100:00:00 grep httpd
grid 2762810 05:42 ?00:00:00 /usr/sbin/httpd
grid 27641 276280 05:42 ?00:00:00 /usr/sbin/httpd
......
2、配置web_vip (apache 集群服务的vip 地址服务）
[oracle@node1 ~]$ su -
Password:
[grid@node1 profile]$ crs_profile -create web_vip -t application -a /u01/11.2.0/grid/bin/usrvip -o oi=eth0,ov=192.168.8.20,on=255.255.255.0
资源注册：
[grid@node1 profile]$ crs_register web_vip
-----192.168.8.20 用于web 服务 vip 地址
[grid@node1 ~]$ su -
Password:
资源授权：
[root@node1 ~]# crs_setperm web_vip -o root
[root@node1 ~]# crs_setperm web_vip -u user:grid:r-x
[root@node1 ~]#
[root@node1 ~]# crs_start web_vip
Attempting to start `web_vip` on member `node1`
Start of `web_vip` on member `node1` succeeded.
[root@node1 ~]# crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DG1.dg ora....up.type ONLINE ONLINE node1
ora.DG2.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node2
ora....VOTE.dg ora....up.type ONLINE ONLINE node1
ora.RCY1.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.eons ora.eons.type ONLINE ONLINE node1
ora.gsd ora.gsd.type ONLINE ONLINE node1
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsd application ONLINE ONLINE node1
ora.node1.ons application ONLINE ONLINE node1
ora.node1.vip ora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsd application ONLINE ONLINE node2
ora.node2.ons application ONLINE ONLINE node2
ora.node2.vip ora....t1.type ONLINE ONLINE node2
ora.node3.vip ora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.type ONLINE ONLINE node2
ora.ons ora.ons.type ONLINE ONLINE node1
ora.prod.db ora....se.type OFFLINE OFFLINE
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vip ora....ip.type ONLINE ONLINE node2
web_vip application ONLINE ONLINE node1
[root@node1 ~]# ip ad sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 08:00:27:7a:c6:9c brd ff:ff:ff:ff:ff:ff
inet 192.168.8.11/24 brd 192.168.8.255 scope global eth0
inet 192.168.8.13/24 brd 192.168.8.255 scope global secondary eth0:1
inet 192.168.8.20/24 brd 192.168.8.255 scope global secondary eth0:2
inet6 fe80::a00:27ff:fe7a:c69c/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 08:00:27:ee:fd:51 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.1/24 brd 10.10.10.255 scope global eth1
inet6 fe80::a00:27ff:feee:fd51/64 scope link
valid_lft forever preferred_lft forever
4: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
[root@node1 ~]#
------- 切换 web_vip resource 到 node2
[root@node1 ~]# crs_relocate web_vip
Attempting to stop `web_vip` on member `node1`
Stop of `web_vip` on member `node1` succeeded.
Attempting to start `web_vip` on member `node2`
Start of `web_vip` on member `node2` succeeded.
[root@node1 ~]# crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DG1.dg ora....up.type ONLINE ONLINE node1
ora.DG2.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node2
ora....VOTE.dg ora....up.type ONLINE ONLINE node1
ora.RCY1.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.eons ora.eons.type ONLINE ONLINE node1
ora.gsd ora.gsd.type ONLINE ONLINE node1
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsd application ONLINE ONLINE node1
ora.node1.ons application ONLINE ONLINE node1
ora.node1.vip ora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsd application ONLINE ONLINE node2
ora.node2.ons application ONLINE ONLINE node2
ora.node2.vip ora....t1.type ONLINE ONLINE node2
ora.node3.vip ora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.type ONLINE ONLINE node2
ora.ons ora.ons.type ONLINE ONLINE node1
ora.prod.db ora....se.type OFFLINE OFFLINE
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vip ora....ip.type ONLINE ONLINE node2
web_vip application ONLINE ONLINE node2
[root@node1 ~]#
[root@node2 ~]# ip ad sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 08:00:27:70:1d:15 brd ff:ff:ff:ff:ff:ff
inet 192.168.8.12/24 brd 192.168.8.255 scope global eth0
inet 192.168.8.18/24 brd 192.168.8.255 scope global secondary eth0:1
inet 192.168.8.14/24 brd 192.168.8.255 scope global secondary eth0:3
inet 192.168.8.15/24 brd 192.168.8.255 scope global secondary eth0:4
inet 192.168.8.20/24 brd 192.168.8.255 scope global secondary eth0:2
inet6 fe80::a00:27ff:fe70:1d15/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 08:00:27:60:ee:a3 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.2/24 brd 10.10.10.255 scope global eth1
inet6 fe80::a00:27ff:fe60:eea3/64 scope link
valid_lft forever preferred_lft forever
4: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
[root@node2 ~]#
三、配置 rac_web 资源（以grid 用户的身份）
[grid@node1 ~]$ crs_profile -create rac_web -t application -B /etc/init.d/httpd -d RAC_APACHE -a rac_web.src -r web_vip -o ci=5,ft=5
[grid@node1 ~]$
rac_web 资源依赖于 web_vip resource
[grid@node1 public]$ cd /u01/11.2.0/grid/crs/public/
配置管理脚本：
[grid@node1 public]$ cat rac_web.src
#!/bin/ksh -p
# *****************************************************************
# * *
# * Copyright (c) 2002, 2003 Oracle Corporation. *
# * All rights reserved. *
# * *
# * Copyright (c) 1991, 1999, 2002 Digital Equipment *
# * Corporation *
# * *
# * *
# * All Rights Reserved. Unpublished rights reserved under *
# * the copyright laws of the United States. *
# * *
# * The software contained on this media is proprietary to *
# * and embodies the confidential technology of Digital *
# * Equipment Corporation and Oracle Corporation. Possession, *
# * use, duplication or dissemination of the software *
# * and media is authorized only pursuant to a valid written *
# * license from Digital Equipment Corporation and Oracle *
# * Corporation *
# * *
# * RESTRICTED RIGHTS LEGEND Use, duplication, or disclosure *
# * by the U.S. Government is subject to restrictions as set *
# * forth in Subparagraph (c)(1)(ii) of DFARS 252.227-7013, *
# * or in FAR 52.227-19, as applicable. *
# * *
# * *
# *****************************************************************
#
# @(#)$RCSfile: crstmpl.scr $ $Revision: has/crs/template/crstmpl.scr#0 $ (DEC) $Date: 21-feb-2005.14:52:54 $
#
#
#
####################################################################
#
# The following section contains variables that can be set to best
# suit your application.
#
# Please review each variable and set as needed.
#
# Set CAA_SCRIPT_DEBUG when invoking the script from command line
# for testing. This will cause all output events to go to the terminal,
# rather than being sent to EVM.
#
####################################################################
#
# Application name - set this variable to a name that describes this
# (mandatory) application. Enclose the name in double quotes.
# Examples: "apache", "netscape"
SERVICE_NAME="rac_web"
# Associated Processes - the application configured may consist of
# (mandatory single or multiple processes. Specifying the names
# of the processes here allows CAA to monitor that they
# are running and allows CAA to completely clean up when
# stopping the application.
# Ex: "proc1 proc2"
PROBE_PROCS="httpd"
# Application Startup Command - CAA will invoke this command when starting
# (mandatory) the application. Include the command to execute along
# with any flags and arguments needed. Use this
# variable along with START_APPCMD2 (see below) when
# dealing with a simple application start procedure.
#
# If the start procedure is complicated and/or involves
# many commands, you may find it easier to disregard
# this variable and manually code the commands needed
# in the "Start" section of this script (see below).
#
# Another alternative for a complicated start procedure
# is to create a separate script containing those
# commands and specifying that script in this variable.
#
# Ex: "/cludemo/avs/avsetup -s"
#
# NOTE: if not set, you must manually code the commands
# to start the application in the "Start" section of
# this script.
START_APPCMD="/etc/init.d/httpd start"
# Secondary Application Startup Command - used in conjunction with the
# (optional) Application Startup Command just described above. Use
# if desired to implement a two-step startup process for
# your application, if needed.
START_APPCMD2=""
STOP_APPCMD="/etc/init.d/httpd stop"
将配置文件拷贝到其他节点：
[grid@node1 public]$ scp rac_web.src node2:/u01/11.2.0/grid/crs/public/
rac_web.src 100% 13KB 12.8KB/s 00:00
对web 资源注册：
[grid@node1 public]$ crs_register rac_web
[grid@node1 public]$
查看资源状态：
[grid@node1 public]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DG1.dg ora....up.type ONLINE ONLINE node1
ora.DG2.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node2
ora....VOTE.dg ora....up.type ONLINE ONLINE node1
ora.RCY1.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.eons ora.eons.type ONLINE ONLINE node1
ora.gsd ora.gsd.type ONLINE ONLINE node1
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsd application ONLINE ONLINE node1
ora.node1.ons application ONLINE ONLINE node1
ora.node1.vip ora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsd application ONLINE ONLINE node2
ora.node2.ons application ONLINE ONLINE node2
ora.node2.vip ora....t1.type ONLINE ONLINE node2
ora.node3.vip ora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.type ONLINE ONLINE node2
ora.ons ora.ons.type ONLINE ONLINE node1
ora.prod.db ora....se.type OFFLINE OFFLINE
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vip ora....ip.type ONLINE ONLINE node2
rac_web application OFFLINE OFFLINE
web_vip application ONLINE ONLINE node2
[grid@node1 public]$
启动资源：
[grid@node1 conf]$ crs_start -f rac_web
Attempting to stop `web_vip` on member `node2`
Stop of `web_vip` on member `node2` succeeded.
Attempting to start `web_vip` on member `node1`
Start of `web_vip` on member `node1` succeeded.
Attempting to start `rac_web` on member `node1`
Start of `rac_web` on member `node1` succeeded.
[grid@node1 conf]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DG1.dg ora....up.type ONLINE ONLINE node1
ora.DG2.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node2
ora....VOTE.dg ora....up.type ONLINE ONLINE node1
ora.RCY1.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.eons ora.eons.type ONLINE ONLINE node1
ora.gsd ora.gsd.type ONLINE ONLINE node1
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsd application ONLINE ONLINE node1
ora.node1.ons application ONLINE ONLINE node1
ora.node1.vip ora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsd application ONLINE ONLINE node2
ora.node2.ons application ONLINE ONLINE node2
ora.node2.vip ora....t1.type ONLINE ONLINE node2
ora.node3.vip ora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.type ONLINE ONLINE node2
ora.ons ora.ons.type ONLINE ONLINE node1
ora.prod.db ora....se.type OFFLINE OFFLINE
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vip ora....ip.type ONLINE ONLINE node2
rac_web application ONLINE ONLINE node1
web_vip application ONLINE ONLINE node1
[grid@node1 conf]$
资源切换
[oracle@rac2 logs]$ crs_relocate rac_web -c rac1 -f
Attempting to stop `rac_web` on member `rac2`
Stop of `web_webs` on member `rac2` succeeded.
Attempting to stop `web_vip` on member `rac2`
Stop of `web_vip` on member `rac2` succeeded.
Attempting to start `web_vip` on member `rac1`
Start of `web_vip` on member `rac1` succeeded.
Attempting to start `rac_web` on member `rac1`
Start of `web_webs` on member `rac1` succeeded.
[oracle@rac2 logs]$
查看Apache 服务启动信息：
[oracle@rac1 ~]$ netstat -an|grep 80
tcp00 0.0.0.0:6800.0.0.0:*LISTEN
tcp00 192.168.8.20:80880.0.0.0:*LISTEN
四： 验证
通过以下地址访问： http://192.168.8.20:8080 访问，无论web服务切换到哪一个节点，都能访问到相同的网页。
总结：以上测试，是Oracle CRS对资源管理的测试，证明CRS软件不但可以对Oracle的数据库的集群进行管理，同样也可以搭建和配置其他服务的集群环境，将来Oracle可能会把它扩展为一个通用的集群软件！
