RAC一节点重启时CSSD日志(加黑处是表明踢出)

简介:

[    CSSD]2010-05-26 19:15:07.589 [3153675504] >TRACE:   clssscAllocAsyncMsg: msg (0x2aaab0008f78), asqhd (0x2aaab0008f50), flags 0x081
[    CSSD]2010-05-26 19:15:07.589 [1230670144] >TRACE:   clssnmCheckDskInfo: No wait for node 0, , defined 0, network state 0, disk state 0, disk status 0
[    CSSD]2010-05-26 19:15:07.589 [1230670144] >TRACE:   clssnmCheckDskInfo: No wait for node 1, racdb5, defined 1, network state 3, disk state 3, disk status 3
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmCheckDskInfo: node 1, clustersz 1, leader 1
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmCheckDskInfo: skipping active node 1.
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmEvict: Start
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmEvict: Evicting node 2, racdb6, birth 136, death 137, impendingrcfg 1, stateflags 0x40d
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmSendShutdown: req to node 2, kill time 379123574
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmAllocAsyncMsg: msg (0x2aaaac08d648), asqhd (0x2aaaac08d620), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmDiscHelper: racdb6, node(2) connection failed, con (0x99e6240), probe((nil))
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmDiscHelper: node 2 clean up, con (0x99e6240), init state 5, cur state 5
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 1
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock DG+ASM event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: sending message to client 0x2aaaac065f80 id 0x2aaaac062d30 size 144 seq 36
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssscAllocAsyncMsg: msg (0x2aaab000c778), asqhd (0x2aaab000c750), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmFreeAsyncMsg: freeing msg (0x2aaaac08d648), asqhd (0x2aaaac08d620), sendct 0, flags 0x00000000
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmClusterListener: Entering select blocking
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 1
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock ORA_CLSRD_1_racweb event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: sending message to client 0x2aaaac07ac20 id 0x2aaaac07a920 size 144 seq 29
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssscAllocAsyncMsg: msg (0x2aaab000bb28), asqhd (0x2aaab000bb00), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitOnEvictions: Start
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmClusterListener: Entering select blocking
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitOnEvictions: node 2, racdb6, undead 0
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitThread: thrd(3), timeout(0), wakeonpost(2)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock ORA_CLSRD_1_racweb event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock ORA_CLSRD_2_racweb event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock OSM_ALL event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: No event, groupName(OSM_ALL)  event(2) member 0, events 0x1, state 0x0
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmSetupAckWait: Ack message type (15)
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmDiscCheckin: disconnected connection(0x99e6240)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 1
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmSetupAckWait: node(1) is ACTIVE
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmClusterListener: Entering select blocking
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock DBRACWEB event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmSendUpdate: syncSeqNo(137)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: sending message to client 0x2aaaac0841f0 id 0x2aaaac083ef0 size 144 seq 36
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssscAllocAsyncMsg: msg (0x2aaab0009c58), asqhd (0x2aaab0009c30), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmAllocAsyncMsg: msg (0x2aaaac0a1ee8), asqhd (0x2aaaac0a1ec0), flags 0x083
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmsendmsg: sending msg type 14 to node 1
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmsendmsg: msg queued type 14, qmsgh (0x2aaaac05eaa0) qmsg (0x2aaaac05eab0)
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmFreeAsyncMsg: freeing msg (0x2aaaac0a1ee8), asqhd (0x2aaaac0a1ec0), sendct 0, flags 0x00000002
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 1
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitForAcks: Ack message type(15), ackCount(1)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock DGRACWEB event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmHandleUpdate: src[1] dest[1] dom[0] seq[60] sync[137]
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitThread: thrd(3), timeout(1000), wakeonpost(2)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: sending message to client 0x2aaaac080de0 id 0x2aaaac080ae0 size 144 seq 36
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmUpdateNodeState: node 0, state (0/0) unique (0/0) prevConuni(0) birth (0/0) (old/new)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssscAllocAsyncMsg: msg (0x2aaab000f878), asqhd (0x2aaab000f850), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmDeactivateNode: node 0, state 0

[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmUpdateNodeState: node 1, state (3/3) unique (1274492478/1274492478) prevConuni(0) birth (122/122) (old/new)
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmUpdateNodeState: node 1, racdb5, state set to 3
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmUpdateNodeState: node 2, state (5/0) unique (1274793098/1274793098) prevConuni(1274793098) birth (136/136) (old/new)
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmDeactivateNode: node 2, state 5

[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmDeactivateNode: node 2 (racdb6) left cluster

[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmSendAck: node(1) domain(0) syncSeqNo(137) type(15)
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmAllocAsyncMsg: msg (0x9beab18), asqhd (0x9beaaf0), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmsendmsg: sending msg type 15 to node 1
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmsendmsg: msg queued type 15, qmsgh (0x9be8940) qmsg (0x9be8950)
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmFreeAsyncMsg: freeing msg (0x9beab18), asqhd (0x9beaaf0), sendct 0, flags 0x00000000
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmQueueClientEvent:  Sending Event(1), type 1, incarn 137
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmQueueClientEvent: Node[1] state = 3, birth = 122, unique = 1274492478
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmQueueClientEvent: Node[2] state = 0, birth = 0, unique = 0
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >USER:    clssnmHandleUpdate: SYNC(137) from node(1) completed
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >USER:    clssnmHandleUpdate: NODE 1 (racdb5) IS ACTIVE MEMBER OF CLUSTER
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmHandleUpdate: diskTimeout set to (200000)ms
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: groupName(DGRACWEB) count(2) master(0) event(2), incarn 136, mbrc 2, to member 0, events 0x0, state 0x0
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 1
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmHandleAck: src[1] dest[1] dom[0] seq[60] sync[137] type[15] ackCount(0)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock DAALL_DB event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmClusterListener: Entering select blocking
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: No event, groupName(DAALL_DB)  event(2) member 0, events 0x1, state 0x0
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitForAcks: done, msg type(15)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: No event, groupName(DAALL_DB)  event(2) member 1, events 0x1, state 0x0
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmDoSyncUpdate: Sync 137 complete!
[    CSSD]2010-05-26 19:15:07.590 [1157241152] >TRACE:   clssnmClusterListener: Entering select blocking
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 2
[    CSSD]2010-05-26 19:15:07.590 [1230670144] >TRACE:   clssnmWaitThread: thrd(3), timeout(-1), wakeonpost(0)
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: skipping remote member 3
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmSuspendAllGrocks: Issue grock CRSDMAIN event SUSPEND
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: sending message to client 0x2aaaac05ad70 id 0x2aaaac05aa70 size 144 seq 44
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssscAllocAsyncMsg: msg (0x2aaab0007ba8), asqhd (0x2aaab0007b80), flags 0x081
[    CSSD]2010-05-26 19:15:07.590 [3153675504] >TRACE:   clssgmQueueGrockEvent: groupName(CRSDMAIN) count(2) master(1) event(2), incarn 136, mbrc 2, to member 1, events 0x8, state 0x0










本文转自 jxwpx 51CTO博客,原文链接:http://blog.51cto.com/jxwpx/321615,如需转载请自行联系原作者
相关实践学习
日志服务之使用Nginx模式采集日志
本文介绍如何通过日志服务控制台创建Nginx模式的Logtail配置快速采集Nginx日志并进行多维度分析。
目录
相关文章
|
4月前
|
Kubernetes Shell Linux
linux|shell脚本|有趣的知识---格式化输出日志和脚本调试方法以及kubernetes集群核心服务重启和集群证书备份脚本
linux|shell脚本|有趣的知识---格式化输出日志和脚本调试方法以及kubernetes集群核心服务重启和集群证书备份脚本
64 0
|
1月前
rac 节点驱逐
rac 节点驱逐
10 0
|
5月前
|
canal 负载均衡 关系型数据库
Flink CDC如何获得增量binlog,可能是跟canal一样,伪装成从节点获取日志?
Flink CDC如何获得增量binlog,可能是跟canal一样,伪装成从节点获取日志?
97 1
|
5月前
|
应用服务中间件 Linux
Linux不重启Tomcat清空catalina.out日志的几种方法
Linux不重启Tomcat清空catalina.out日志的几种方法
255 0
|
11月前
|
Oracle 关系型数据库 数据库
RAC中,控制文件的快照文件必须能够被所有节点的数据实例访问到 ORA-00245
RMAN在使用控制文件备份的时候,备份开始点需要最新的检查点信息以及文件头信息。
|
CDN
CDN诊断工具与日志的作用——如何利用诊断工具进行问题分析(查看定位到的是否是cdn节点)
CDN诊断工具与日志的作用——如何利用诊断工具进行问题分析(查看定位到的是否是cdn节点)自制脑图
136 0
CDN诊断工具与日志的作用——如何利用诊断工具进行问题分析(查看定位到的是否是cdn节点)
|
存储 SQL JSON
无需重启应用,动态采集任意点位日志
借助日志治理的现有能力,我们能够在不重启应用的前提下,动态采集任意点位信息,同时由于日志治理在采集信息时会引入链路信息,在分析复杂调用问题时能够起到很好的效果。
无需重启应用,动态采集任意点位日志
|
Oracle 关系型数据库
oracle rac 添加节点常用命令
rac 添加节点常用命令
69 0
|
Arthas 监控 Java
一个迷惑性很高的生产故障-Elasticsearch日志rotate导致节点CPU激增
Elasticsearch CPU很高的场景很常见,优化读写以及扩容即可解决问题。 如果只有一个节点CPU高,那可能的情况就比较多了,节点机器异常?读写不均匀?GC过高?forcemerge? 这里描述一个极具迷惑性的case。
428 0
一个迷惑性很高的生产故障-Elasticsearch日志rotate导致节点CPU激增
|
消息中间件 Kafka Scala
从源码和日志文件结构中分析 Kafka 重启失败事件
上次的 Kafka 重启失败事件,对为什么重启失败的原因似乎并没有解释清楚,那么我就在这里按照我对 Kafka 的认识,从源码和日志文件结构去尝试寻找原因。
207 0
从源码和日志文件结构中分析 Kafka 重启失败事件