Oracle Clusterware提供了丰富的工具，我们把常用的工具按照rac的四层机构做了划分，然后分别介绍：
节点层：
olsnodes
参数-n -i -p
网络层：
oifcfg
四个子命令可以通过oifcfg -help查看下
iflist 显示网口列表
getif 可以获得单个网口信息
setif 配置单个网口
delif 删除网口
集群层：
crstcl 、orccheck 、ocrdump、ocrconfig
应用层：
srvctl 、onsctl、crs_stat----=-------=-----=------=---------=---------=-------=-------=---------=----------=----------=------=---------=--------=---------=--------=--------=---------=--------=-------=-------=-------
通过以上表述，我们能否分清楚crsctl命令，是用于集群层的管理；而srvctl命令，是用于集群层之下的各项服务的管理。如itpub上讨论帖：http://www.itpub.net/thread-1701358-1-1.html
crsctl 应该是cluster control 用的，它管理的是cluster level的内容，如crsd cssd css 等进程和配置的管理
srvctl 是资源的管理，是cluster内资源的管理，比如instance asm listener vip ons gsd service .....
| Oracle的官方解释，这个没什么好纠结的，各司其职，不同的level管理： CRSCTL Overview： CRSCTL is an interface between you and Oracle Clusterware, parsing and calling Oracle Clusterware APIs for Oracle Clusterware objects. Oracle Clusterware 11g release 2 (11.2) introduces cluster-aware commands with which you can perform check, start, and stop operations on the cluster. You can run these commands from any node in the cluster on another node in the cluster, or on all nodes in the cluster, depending on the operation. You can use CRSCTL commands to perform several operations on Oracle Clusterware, such as: Starting and stopping Oracle Clusterware resources Enabling and disabling Oracle Clusterware daemons Checking the health of the cluster Managing resources that represent third-party applications Integrating Intelligent Platform Management Interface (IPMI) with Oracle Clusterware to provide failure isolation support and to ensure cluster integrity Debugging Oracle Clusterware components Overview of SRVCTL Use SRVCTL to manage configuration information. You can use SRVCTL commands to add, remove, start, stop, modify, enable, and disable a number of entities, such as databases, instances, listeners, SCAN listeners, services, grid naming service (GNS), and Oracle ASM. Some SRVCTL operations modify the configuration data stored in the Oracle Cluster Registry (OCR). SRVCTL performs other operations, such as starting and stopping instances, by sending requests to the Oracle Clusterware process (CRSD), which then starts or stops the Oracle Clusterware resources. |
但还有点疑问，如果执行crsctl stop resources ，我发现instance vip 等都可以停掉，所以csrctl是不是也可以管理到“cluster内资源”
答：
额。。。这个应该怎么解释呢：
主机停掉了主机里的进程也会停，这样直观吗？vip 是cluster的资源，cluster 如果被停掉了 cluster里的资源instance ,asm ,vip ....都会停掉
