故障现象：lsnrctl stat后即无任何反应，而SQLPLUS正常：
[oracle@wlserver3 ~]$ lsnrctl stat
LSNRCTL for Linux: Version 10.2.0.1.0 - Production on 13-SEP-2011 23:16:11
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost.localdomain)(PORT=1521)))
尝试ps -ef|grep lsnrctl后kill -9 杀进程，但杀后重启时也碰到同样问题
尝试ps -ef|grep lsnr（注意和上述命令的区别少了ctl）后kill -9 杀进程，后重启成功
[root@wlserver3 admin]# ps -ef|grep lsnr
oracle 3067 16807 0 Sep10 ? 00:00:01 /oracle/product/10.2.0/db_1/bin/tnslsnr LISTENER -inherit
oracle 16807 1 0 Jul08 ? 00:02:07 /oracle/product/10.2.0/db_1/bin/tnslsnr LISTENER -inherit
oracle 18378 18350 0 Sep13 pts/1 00:00:00 lsnrctl stat
oracle 18386 18350 0 Sep13 pts/1 00:00:00 lsnrctl stat
oracle 18423 18350 0 Sep13 pts/1 00:00:00 lsnrctl stat
root 19686 18480 0 01:18 pts/3 00:00:00 grep lsnr
[root@wlserver3 admin]# kill -9 3067
[root@wlserver3 admin]# kill -9 16807
重启成功:
oracle@wlserver3 bdump]$ lsnrctl start
LSNRCTL for Linux: Version 10.2.0.1.0 - Production on 14-SEP-2011 01:21:35
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Starting /oracle/product/10.2.0/db_1/bin/tnslsnr: please wait...
TNSLSNR for Linux: Version 10.2.0.1.0 - Production
System parameter file is /oracle/product/10.2.0/db_1/network/admin/listener.ora
Log messages written to /oracle/product/10.2.0/db_1/network/log/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC0)))
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost.localdomain)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 10.2.0.1.0 - Production
Start Date 14-SEP-2011 01:21:37
Uptime 0 days 0 hr. 0 min. 0 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /oracle/product/10.2.0/db_1/network/admin/listener.ora
Listener Log File /oracle/product/10.2.0/db_1/network/log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC0)))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
[oracle@wlserver3 bdump]$
补充:处理过程中看到过一个10.2.0.1环境下的BUG,也一并修复掉了
10g Listener: High CPU Utilization - Listener May Hang [ID 284602.1]
Symptoms
1) If the listener is hanging, a sqlnet trace (level 16) will show that the last operation was a fork.
[000068 28-SEP-2004 14:16:17:398] sntpcall: entry
[000068 28-SEP-2004 14:16:17:401] sntpcall: detaching from parent with additional fork 68 bytes to NS buffer
2) netstat -na reports numerous (maybe thousands) of sockets associated with port 6100 in a TIME_WAIT status
127.0.0.1.6100 127.0.0.1.55228 49152 0 49152 0 TIME_WAIT
127.0.0.1.6100 127.0.0.1.55229 49152 0 49152 0 TIME_WAIT
127.0.0.1.6100 127.0.0.1.55230 49152 0 49152 0 TIME_WAIT
127.0.0.1.6100 127.0.0.1.55231 49152 0 49152 0 TIME_WAIT
127.0.0.1.6100 127.0.0.1.55232 49152 0 49152 0 TIME_WAIT
3) "sdtprocess" (Solaris), or "top" show that the listener process (tnslsnr) is using a large amount of cpu.
ID Name Owner CPU% RAM Size Started Parent
-----------------------------------------------------------------------------------------------
21145 tnslsnr mseibt 44 9440 22368 13:36:03 1 /u01/app/oracle/product/10.1.0/bin/tnslsnr LISTENER -inherit
4) The 10g(iAS) $ORACLE_HOME/opmn/log on the same system shows the following repeating error in the ons logs:
04/10/01 13:30:43 [4] Local connection 0,127.0.0.1,6100 missing form factor
04/10/01 13:30:43 [4] Local connection 0,127.0.0.1,6100 missing form factor
04/10/01 13:30:43 [4] Local connection 0,127.0.0.1,6100 missing form factor
04/10/01 13:30:43 [4] Local connection 0,127.0.0.1,6100 missing form factor
04/10/01 13:30:43 [4] Local connection 0,127.0.0.1,6100 missing form factor
Changes
- Both 10g(iAS) and Oracle 10g(rdbms) are installed on the same system.
- Oracle 10g (non-RAC) is installed on a system with or without other Oracle installations.
Cause
The Oracle Notification Server (ONS) running in the 10g(iAS) home uses the ports defined in $ORACLE_HOME/opmn/conf/ons.config. The database listener, which is an ONS client, also utilizes the ONS ports identified in its own $ORACLE_HOME/opmn/conf/ons.config. When 10g(iAS) and 10g(rdbms) are installed on the same box, the installer will mistakenly configure identical ONS ports in both homes which creates an operational conflict when both the 10g(iAS) ONS and 10g(rdbms) listener services are running.
In non-RAC environments where Oracle 10g is the only Oracle product on the system, the existence of the ons.config may still cause a hang condition in the listener.
Solution
This issue is corrected in versions 10.2.0.3 and higher.
(or apply backport patch <<4518443>>)
You may also use either of the following 2 workarounds to address this problem:
1) In the 10g(rdbms) home, rename the $ORACLE_HOME/opmn/conf/ons.config file (so that the listener does not find or use it).
For example:
cd $ORACLE_HOME/opmn/conf
mv ons.config ons.config.orig
(then restart the listener)
- or -
2) Explicitly disable the 10g TNS Listener from subscribing to the ONS.
Edit the active 10g listener.ora file and add the parameter below:
SUBSCRIBE_FOR_NODE_DOWN_EVENT_<listener_name>=OFF
where <listener_name> would be replaced with the actual listener name configured in the listener.ora file.
For example, if the listener name is LISTENER (default), the parameter would be:
SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF
(then restart the listener)
References
BUG:3881276 - LISTENER HANGS INTERMITTANTLY AFTER APPLYING 10.1.0.3 PATCHSET
BUG:6017068 - PMON SPINNING IN NETWORKING CODE
本文转自zylhsy 51CTO博客，原文链接：http://blog.51cto.com/yunlongzheng/663511，如需转载请自行联系原作者
