系统环境:
操作系统:RedHat EL5.5
Cluster: GI 11.2.0.1.0
数据库软件:Oracle 11.2.0.1
故障现象:
添加新的node3的Instance 后,再启动Instance 时,出现以下故障。
[oracle@node3 dbs]$ sqlplus '/as sysdba'
sqlplus '/as sysdba'
SQL*Plus: Release 11.2.0.1.0 Production on Thu Apr 17 15:47:41 2014
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SYS@ prod3>startup nomount;
ORA-15081: failed to submit an I/O operation to a disk
告警日志:
ORACLE_BASE from environment = /u01/app/oracle
Errors in file /u01/app/oracle/diag/rdbms/pmydb/prod3/trace/prod3_rbal_16308.trc:
ORA-15183: ASMLIB initialization error [driver/agent not installed]
WARNING: FAILED to load library: /opt/oracle/extapi/32/asm/orcl/1/libasm.so
Errors in file /u01/app/oracle/diag/rdbms/pmydb/prod3/trace/prod3_rbal_16308.trc:
ORA-15183: ASMLIB initialization error [driver/agent not installed]
SUCCESS: diskgroup DG1 was mounted
USER (ospid: 16204): terminating the instance due to error 15081
NOTE: dependency between database pmydb and diskgroup resource ora.DG1.dg is established
Instance terminated by USER, pid = 16204
查看相关trace 文件:
[root@node3 node3]# tail /u01/app/oracle/diag/rdbms/pmydb/prod3/trace/prod3_rbal_16308.trc
*** 2014-04-17 15:47:55.374
*** SESSION ID:(11.1) 2014-04-17 15:47:55.374
*** CLIENT ID:() 2014-04-17 15:47:55.374
*** SERVICE NAME:() 2014-04-17 15:47:55.374
*** MODULE NAME:() 2014-04-17 15:47:55.374
*** ACTION NAME:() 2014-04-17 15:47:55.374
ERROR: asm_version error. err: driver/agent not installed rc:2
ORA-15183: ASMLIB initialization error [driver/agent not installed]
ORA-15183: ASMLIB initialization error [driver/agent not installed]
已经安装ASM相关的driver:
[root@node3 ~]# rpm -qa |grep asm
oracleasmlib-2.0.4-1.el5
oracleasm-support-2.1.3-1.el5
oracleasm-2.6.18-194.el5-2.0.5-1.el5
[root@node3 ~]# uname -a
Linux node1 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:43 EDT 2010 i686 i686 i386 GNU/Linux
查看Oracle 可执行文件:
[root@node3 asm]# ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle
-r-sr-s--x 1 oracle oinstall 173515905 Apr 17 15:07 /u01/app/oracle/product/11.2.0/db_1/bin/oracle
发现文件的所属的group错误,Oracle 后台进程启动时会执行此文件,而如果属于oinstall组,则无权访问ASM磁盘驱动,从而无法访问ASM disk。
正常属性应该是:
[oracle@node1 ~]$ ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle
-r-sr-s--x 1 oracle asmadmin 173515905 Dec 28 18:17 /u01/app/oracle/product/11.2.0/db_1/bin/oracle
修改属性:
[root@node3 ~]# chown oracle:asmadmin /u01/app/oracle/product/11.2.0/db_1/bin/oracle
[root@node3 asm]# ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle
-r-xr-x--x 1 oracle asmadmin 173515905 Apr 17 15:07 /u01/app/oracle/product/11.2.0/db_1/bin/oracle
[root@node3 asm]# chmod 6551 /u01/app/oracle/product/11.2.0/db_1/bin/oracle
[root@node3 asm]# ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle
-r-sr-s--x 1 oracle asmadmin 173515905 Apr 17 15:07 /u01/app/oracle/product/11.2.0/db_1/bin/oracle
@更改完成后,再启动Instance ,Instance 启动正常!