今天早上nagios报警一台线上的tomcat故障,此tomcat我监控的是应用的首页top.jsp,拿IE浏览器登录测试也无法正常跳转到报警的应用,邮件报警信息如下:
***** Nagios *****
Notification Type: PROBLEM
Service: check http sup
Host: B2G server 192.168.15.15
Address: 192.168.15.15
State: CRITICAL
Date/Time: Thu Apr 11 09:38:15 CST 2013
Additional Info:
CRITICAL - Socket timeout after 10 seconds
解决办法:
重启tomcat,重启后提示如下报错信息:
严重: IOException while loading persisted sessions: java.io.EOFException
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2281)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2750)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:780)
at java.io.ObjectInputStream.(ObjectInputStream.java:280)
at org.apache.catalina.util.CustomObjectInputStream.(CustomObjectInputStream.java:58)
at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:362)
at org.apache.catalina.session.StandardManager.load(StandardManager.java:321)
at org.apache.catalina.session.StandardManager.start(StandardManager.java:637)
at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:438)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4358)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:987)
at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:909)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:495)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
2013-4-11 9:44:50 org.apache.catalina.session.StandardManager start
严重: Exception loading sessions from persistent storage
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2281)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2750)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:780)
at java.io.ObjectInputStream.(ObjectInputStream.java:280)
at org.apache.catalina.util.CustomObjectInputStream.(CustomObjectInputStream.java:58)
at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:362)
at org.apache.catalina.session.StandardManager.load(StandardManager.java:321)
at org.apache.catalina.session.StandardManager.start(StandardManager.java:637)
at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:438)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4358)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:987)
at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:909)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:495)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
报错原因:
EOFException表示输入过程中意外地到达文件尾或流尾的信号,导致从 session中获取数据失败。异常是tomcat本身的问题,由于tomcat上次非正常关闭时有一些活动session被持久化(表现为一些临时文 件),在重启时,tomcat尝试去恢复这些session的持久化数据但又读取失败造成的。此异常不影响系统的使用。
解决办法:
删除目录/usr/local/tomcat/work/Catalina/localhost下的所有文件,重启tomcat问题解决!
***** Nagios *****
Notification Type: RECOVERY
Service: check http sup
Host: B2G server 192.168.15.15
Address: 192.168.15.15
State: OK
Date/Time: Thu Apr 11 09:47:05 CST 2013
Additional Info:
HTTP OK: HTTP/1.1 302 Moved Temporarily - 253 bytes in 0.013 second response time