一次Oracle DG错误警示（生产库）-阿里云开发者社区

生产库DG ,防火墙限制，导致归档传送失败

主库在北京，备库在云南，主备、库网络通道之间有防火墙，由于防火墙对数据包传输的大小做了限制，导致北京和云南之间的主备库传输归档日志失败。

1、告警日志错误

主库：（北京）

WARN: ARC3: Terminating ARCH (pid 9608) hung on a network operation
WARN: ARC3: Terminating ARCH (pid 21006) hung on a network operation
krsv_proc_kill: Killing 140329466462209 processes (Process by index)
ARC2: Detected ARCH process failure
ARC2: Detected ARCH process failure
ARC2: STARTING ARCH PROCESSES
Wed Nov 06 16:33:49 2013
ARC1 started with pid=72, OS id=21750
ARC1: Archival started

备库：（云南）

Archived Log entry 860 added for thread 1 sequence 34061 ID 0xaa380b96 dest 1:
Wed Nov 06 16:25:04 2013
RFS[38]: Assigned to RFS process 10796
RFS[38]: Identified database type as 'physical standby': Client is ARCH pid 20273
krsv_proc_kill: Killing 1 processes (idle RFS by threadquence)
RFS[38]: Selected log 13 for thread 1 sequence 34063 dbid -1439133290 branch 817469144
Wed Nov 06 16:26:02 2013
RFS[39]: Assigned to RFS process 10798
RFS[39]: Identified database type as 'physical standby': Client is ARCH pid 9618
krsv_proc_kill: Killing 1 processes (idle RFS by threadquence)
Wed Nov 06 16:26:04 2013
Fetching gap sequence in thread 1, gap sequence 33964-33964
Error 12154 received logging on to the standby
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
Errors in file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc:
ORA-12154: TNS:could not resolve the connect identifier specified
Errors in file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc:
ORA-12154: TNS:could not resolve the connect identifier specified
Wed Nov 06 16:26:05 2013
RFS[40]: Assigned to RFS process 10800
RFS[40]: Identified database type as 'physical standby': Client is ARCH pid 21006
RFS[40]: Opened log for thread 1 sequence 33964 dbid -1439133290 branch 817469144
Wed Nov 06 16:26:06 2013
RFS[41]: Assigned to RFS process 10802
RFS[41]: Identified database type as 'physical standby': Client is ARCH pid 9608
RFS[41]: No standby redo logfiles of size 7307 blocks available
RFS[41]: Opened log for thread 1 sequence 34068 dbid -1439133290 branch 817469144
Wed Nov 06 16:30:13 2013
RFS[42]: Assigned to RFS process 10809
RFS[42]: Identified database type as 'physical standby': Client is ARCH pid 21750
krsv_proc_kill: Killing 1 processes (idle RFS by threadquence)
Wed Nov 06 16:30:13 2013
RFS[43]: Assigned to RFS process 10811
RFS[43]: Identified database type as 'physical standby': Client is ARCH pid 17175
krsv_proc_kill: Killing 1 processes (idle RFS by threadquence)
Wed Nov 06 16:30:14 2013
RFS[44]: Assigned to RFS process 10813
RFS[44]: Identified database type as 'physical standby': Client is ARCH pid 21730
krsv_proc_kill: Killing 1 processes (idle RFS by threadquence)
Wed Nov 06 16:30:14 2013
Error 12154 received logging on to the standby
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
Errors in file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc:
ORA-12154: TNS:could not resolve the connect identifier specified
Errors in file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc:
ORA-12154: TNS:could not resolve the connect identifier specified
RFS[42]: Selected log 13 for thread 1 sequence 34063 dbid -1439133290 branch 817469144
RFS[43]: Opened log for thread 1 sequence 33964 dbid -1439133290 branch 817469144
RFS[44]: No standby redo logfiles of size 7307 blocks available
RFS[44]: Opened log for thread 1 sequence 34068 dbid -1439133290 branch 817469144
Wed Nov 06 16:30:48 2013
Archived Log entry 861 added for thread 1 sequence 33964 rlc 817469144 ID 0xaa380b96 dest 3:
Wed Nov 06 16:30:49 2013
Media Recovery Log /arch/PROD/archivelog/1_33964_817469144.dbf
Media Recovery Log /arch/PROD/archivelog/1_33965_817469144.dbf
Wed Nov 06 16:31:00 2013
Media Recovery Waiting for thread 1 sequence 33966
Fetching gap sequence in thread 1, gap sequence 33966-33966
Error 12154 received logging on to the standby
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
Errors in file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc:
ORA-12154: TNS:could not resolve the connect identifier specified
Errors in file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc:
ORA-12154: TNS:could not resolve the connect identifier specified

RFS[78]: Assigned to RFS process 10933
RFS[78]: Identified database type as 'physical standby': Client is ARCH pid 26388
krsv_proc_kill: Killing 1 processes (idle RFS by threadquence)
RFS[78]: Selected log 15 for thread 1 sequence 34077 dbid -1439133290 branch 817469144

备库端：trace文件

Wed Nov 06 17:16:20 2013
WARN: ARC8: Terminating ARCH (pid 27095) hung on a network operation
krsv_proc_kill: Killing 139874199928833 processes (Process by index)
ARC8: Detected ARCH process failure
ARC8: STARTING ARCH PROCESSES
Wed Nov 06 17:16:22 2013
ARC7 started with pid=108, OS id=27818
Wed Nov 06 17:16:23 2013
Reclaiming FAL entry from dead process [pid 27095]
ARC7: Archival started
ARC8: STARTING ARCH PROCESSES COMPLETE
ARC9: Standby redo logfile selected for thread 1 sequence 34077 for destination LOG_ARCHIVE_DEST_3
Wed Nov 06 17:17:23 2013
WARN: ARC8: Terminating ARCH (pid 25138) hung on a network operation
krsv_proc_kill: Killing 139874199928833 processes (Process by index)
ARC8: Detected ARCH process failure
ARC8: STARTING ARCH PROCESSES
Wed Nov 06 17:17:25 2013
ARC4 started with pid=103, OS id=27971
ARC4: Archival started
ARC8: STARTING ARCH PROCESSES COMPLETE
Reclaiming FAL entry from dead process [pid 25138]

Wed Nov 06 16:52:55 2013
Archived Log entry 872 added for thread 1 sequence 33977 ID 0xaa380b96 dest 1:
Media Recovery Log /arch/PROD/archivelog/1_33977_817469144.dbf
Wed Nov 06 16:53:00 2013
Media Recovery Waiting for thread 1 sequence 33978 (in transit)

Trace file /opt/oracle/diag/rdbms/ynstandbyboss/PROD/trace/PROD_pr00_10641.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /opt/oracle/db_1
System name: Linux
Node name: yunnanDB107
Release: 2.6.32-5-amd64
Version: #1 SMP Fri May 10 08:43:19 UTC 2013
Machine: x86_64
Instance name: PROD
Redo thread mounted by this instance: 1
Oracle process number: 31
Unix process pid: 10641, image: oracle@yunnanDB107 (PR00)

*** 2013-11-06 15:16:37.649
*** SESSION ID:(365.1) 2013-11-06 15:16:37.649
*** CLIENT ID:() 2013-11-06 15:16:37.649
*** SERVICE NAME:() 2013-11-06 15:16:37.649
*** MODULE NAME:() 2013-11-06 15:16:37.649
*** ACTION NAME:() 2013-11-06 15:16:37.649

Started Parallel Media Recovery
*** 2013-11-06 15:16:37.660 4132 krsh.c
Managed Standby Recovery not using Real Time Apply

*** 2013-11-06 15:16:37.709
Dumping database incarnation table:
Resetlogs 0 scn and time: 0x0000.000e6c20 06/07/2013 10:45:44
Recovery target incarnation = 2, activation ID = -1439167594
Influx buffer limit = 15778 min(50% x 31556, 100000)
Start recovery at thread 1 ckpt scn 1338098741 logseq 33959 block 2

*** 2013-11-06 15:16:38.118
Media Recovery add redo thread 1
*** 2013-11-06 15:16:38.219 4132 krsh.c
Media Recovery Waiting for thread 1 sequence 33959
*** 2013-11-06 15:16:38.219 4132 krsh.c
Fetching gap sequence in thread 1, gap sequence 33959-33959
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:16:38.220 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:16:38.220 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:16:48.219
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:16:48.220 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:16:48.220 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:16:58.219
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:16:58.220 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:16:58.220 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:17:08.219
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:17:08.220 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:17:08.220 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:17:18.219
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:17:18.220 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:17:18.220 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:17:28.220
-----------------------------------------------------------
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-----------------------------------------------------------
Redo shipping client performing standby login

*** 2013-11-06 15:35:33.232
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:35:33.232 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:35:33.232 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:35:43.232
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:35:43.232 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:35:43.232 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:35:53.232
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:35:53.232 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:35:53.232 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:36:03.232
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:36:03.232 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:36:03.232 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:36:13.232
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:36:13.232 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:36:13.232 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:36:23.232
-----------------------------------------------------------
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-----------------------------------------------------------

*** 2013-11-06 15:45:33.239
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:45:33.240 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:45:33.240 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:45:43.239
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:45:43.240 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:45:43.240 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:45:53.239
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:45:53.240 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:45:53.240 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:46:03.243
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:46:03.244 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:46:03.244 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:46:13.243
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 15:46:13.244 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 15:46:13.244 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 15:46:23.244
-----------------------------------------------------------
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-----------------------------------------------------------

*** 2013-11-06 15:52:28.244
Media Recovery Log /arch/PROD/archivelog/1_33959_817469144.dbf
Log read is SYNCHRONOUS though disk_asynch_io is enabled!

*** 2013-11-06 15:52:30.415
Media Recovery Log /arch/PROD/archivelog/1_33960_817469144.dbf

*** 2013-11-06 15:52:36.796
Media Recovery Log /arch/PROD/archivelog/1_33961_817469144.dbf

*** 2013-11-06 16:01:59.564
Media Recovery Log /arch/PROD/archivelog/1_33962_817469144.dbf

*** 2013-11-06 16:02:02.783
Media Recovery Log /arch/PROD/archivelog/1_33963_817469144.dbf
*** 2013-11-06 16:26:04.571 4132 krsh.c

*** 2013-11-06 16:26:04.571
Fetching gap sequence in thread 1, gap sequence 33964-33964
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:26:04.572 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:26:04.572 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:30:14.588
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:30:14.588 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:30:14.588 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:30:49.587
Media Recovery Log /arch/PROD/archivelog/1_33964_817469144.dbf

*** 2013-11-06 16:30:56.675
Media Recovery Log /arch/PROD/archivelog/1_33965_817469144.dbf

*** 2013-11-06 16:31:00.752
*** 2013-11-06 16:31:00.752 4132 krsh.c
Media Recovery Waiting for thread 1 sequence 33966
*** 2013-11-06 16:31:00.752 4132 krsh.c
Fetching gap sequence in thread 1, gap sequence 33966-33966
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:31:00.753 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:31:00.753 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:31:10.871
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:31:10.872 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:31:10.872 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:31:20.875
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:31:20.876 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:31:20.876 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:31:30.876
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:31:30.876 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:31:30.876 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:31:40.876
Redo shipping client performing standby login
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
OCIServerAttach failed -1
.. Detailed OCI error val is 12154 and errmsg is 'ORA-12154: TNS:could not resolve the connect identifier specified
'
*** 2013-11-06 16:31:40.876 4132 krsh.c
Error 12154 received logging on to the standby
*** 2013-11-06 16:31:40.876 4132 krsh.c
FAL[client, USER]: Error 12154 connecting to MASTERBOSS,STANDBYBOSS for fetching gap sequence
ORA-12154: TNS:could not resolve the connect identifier specified
ORA-12154: TNS:could not resolve the connect identifier specified

*** 2013-11-06 16:31:50.876
-----------------------------------------------------------
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-----------------------------------------------------------

2、主备库信息和listener配置

主库
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_file_name_convert string
db_name string PROD
db_unique_name string MASTERBOSS
global_names boolean FALSE
instance_name string PROD
lock_name_space string
log_file_name_convert string
service_names string MASTERBOSS
北京备库
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_file_name_convert string
db_name string PROD
db_unique_name string STANDBYBOSS
global_names boolean FALSE
instance_name string PROD
lock_name_space string
log_file_name_convert string
service_names string STANDBYBOSS
云南备库
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_file_name_convert string
db_name string PROD
db_unique_name string YNSTANDBYBOSS
global_names boolean FALSE
instance_name string PROD
lock_name_space string
log_file_name_convert string /arch/PROD/onlinelog/, /arch/
PROD/onlinelog/
service_names string YNSTANDBYBOSS

--- 初始化参数文件

主库pfile
PROD.__db_cache_size=536870912
PROD.__java_pool_size=67108864
PROD.__large_pool_size=67108864
PROD.__oracle_base='/opt/oracle'#ORACLE_BASE set from environment
PROD.__pga_aggregate_target=12012486656
PROD.__sga_target=1744830464
PROD.__shared_io_pool_size=0
PROD.__shared_pool_size=805306368
PROD.__streams_pool_size=134217728
*.audit_file_dest='/opt/oracle/admin/PROD/adump'
*.audit_sys_operations=TRUE
*.audit_trail='db'
*.compatible='11.2.0.0.0'
*.control_files='/data/PROD/datafile/control01.ctl','/data/PROD/flash_recovery_area/PROD/control02.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='PROD'
*.db_recovery_file_dest='/data/PROD/flash_recovery_area'
*.db_recovery_file_dest_size=2147483648
*.DB_UNIQUE_NAME='MASTERBOSS'
*.diagnostic_dest='/opt/oracle'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=PRODXDB)'
*.log_archive_dest_1='location=/arch/PROD/archlog mandatory'
*.memory_target=13743895347
*.open_cursors=500
*.pga_aggregate_target=11978932224
*.processes=300
*.remote_login_passwordfile='EXCLUSIVE'
*.sessions=335
*.sga_target=1610612736
*.sql92_security=TRUE
*.undo_tablespace='UNDOTBS1'
北京备库
PROD.__db_cache_size=67108864
PROD.__java_pool_size=67108864
PROD.__large_pool_size=67108864
PROD.__oracle_base='/opt/oracle'#ORACLE_BASE set from environment
PROD.__pga_aggregate_target=6106906624
PROD.__sga_target=805306368
PROD.__shared_io_pool_size=0
PROD.__shared_pool_size=603979776
PROD.__streams_pool_size=0
*.audit_file_dest='/opt/oracle/admin/PROD/adump'
*.audit_sys_operations=TRUE
*.audit_trail='db'
*.compatible='11.2.0.0.0'
*.control_files='/data/PROD/datafile/control01.ctl','/data/PROD/flash_recovery_area/PROD/control02.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='PROD'
*.db_recovery_file_dest='/data/PROD/flash_recovery_area'
*.db_recovery_file_dest_size=2147483648
*.DB_UNIQUE_NAME='STANDBYBOSS'
*.diagnostic_dest='/opt/oracle'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=PRODXDB)'
*.FAL_CLIENT='STANDBYBOSS'
*.FAL_SERVER='MASTERBOSS'
*.LOG_ARCHIVE_CONFIG='DG_CONFIG=(MASTERBOSS,STANDBYBOSS)'
*.LOG_ARCHIVE_DEST_1='LOCATION=/arch/PROD/archivelog/'
*.memory_target=6871947673
*.open_cursors=500
*.pga_aggregate_target=5989466112
*.processes=300
*.remote_login_passwordfile='EXCLUSIVE'
*.sessions=335
*.sga_target=805306368
*.sql92_security=TRUE
*.standby_file_management='AUTO'
*.undo_tablespace='UNDOTBS1'
云南备库
PROD.__db_cache_size=536870912
PROD.__java_pool_size=67108864
PROD.__large_pool_size=67108864
PROD.__oracle_base='/opt/oracle'#ORACLE_BASE set from environment
PROD.__pga_aggregate_target=12012486656
PROD.__sga_target=1744830464
PROD.__shared_io_pool_size=0
PROD.__shared_pool_size=805306368
PROD.__streams_pool_size=134217728
*.audit_file_dest='/opt/oracle/admin/PROD/adump'
*.audit_sys_operations=TRUE
*.audit_trail='db'
*.compatible='11.2.0.0.0'
*.control_files='/data/PROD/datafile/control01.ctl','/data/PROD/flash_recovery_area/PROD/control02.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='PROD'
*.db_recovery_file_dest='/data/PROD/flash_recovery_area'
*.db_recovery_file_dest_size=2147483648
DB_UNIQUE_NAME=YNSTANDBYBOSS
*.diagnostic_dest='/opt/oracle'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=PRODXDB)'
*.memory_target=6871947673
*.open_cursors=500
*.pga_aggregate_target=5989466112
*.processes=300
*.remote_login_passwordfile='EXCLUSIVE'
*.sessions=335
*.sga_target=805306368
*.sql92_security=TRUE
*.undo_tablespace='UNDOTBS1'
log_archive_config='DG_CONFIG=(MASTERBOSS,STANDBYBOSS,YNSTANDBYBOSS)'
LOG_ARCHIVE_DEST_1='LOCATION=/arch/PROD/archivelog/'
LOG_ARCHIVE_DEST_2='SERVICE=MASTERBOSS LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=MASTERBOSS'
LOG_ARCHIVE_DEST_3='SERVICE=STANDBYBOSS LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=STANDBYBOSS'
log_archive_dest_state_1='ENABLE'
log_archive_dest_state_2='ENABLE'
log_archive_dest_state_3='ENABLE'
LOG_FILE_NAME_CONVERT='/arch/PROD/onlinelog/','/arch/PROD/onlinelog/'
FAL_SERVER='MASTERBOSS,STANDBYBOSS'
FAL_CLIENT='YNSTANDBYBOSS'
STANDBY_FILE_MANAGEMENT=AUTO

--- listener 配置

主库listener
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.xxx.107)(PORT = 1521))
)
)
)
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(ORACLE_HOME = /opt/oracle/db_1)
(SID_NAME = PROD)
)
)
北京备库
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.xxx.108)(PORT = 1521))
)
)
)
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(ORACLE_HOME = /opt/oracle/db_1)
(SID_NAME = PROD)
)
)
云南备库
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.xxx.107)(PORT = 1521))
)
)
)
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(ORACLE_HOME = /opt/oracle/db_1)
(SID_NAME = PROD)
)
)

--- tnsnames 配置

tnsnames.ora
# tnsnames.ora Network Configuration File: /opt/oracle/db_1/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.
MASTERBOSS =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.xxx.107)(PORT = 1521))
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = PROD )
)
)
STANDBYBOSS =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.xxx.108)(PORT = 1521))
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = PROD )
)
)
YNSTANDBYBOSS =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.xxx.107)(PORT = 1521))
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = PROD)
)
)

3、主备库 listener 状态信息

主库
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 06-NOV-2013 16:52:49

Copyright (c) 1991, 2009, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.200.107)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 25-AUG-2013 16:44:47
Uptime 73 days 0 hr. 8 min. 1 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /opt/oracle/db_1/network/admin/listener.ora
Listener Log File /opt/oracle/diag/tnslsnr/db07/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.200.107)(PORT=1521)))
Services Summary...
Service "MASTERBOSS" has 1 instance(s).
Instance "PROD", status READY, has 1 handler(s) for this service...
Service "PROD" has 1 instance(s).
Instance "PROD", status UNKNOWN, has 1 handler(s) for this service...
Service "PRODXDB" has 1 instance(s).
Instance "PROD", status READY, has 1 handler(s) for this service...
The command completed successfully

云南备库
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 06-NOV-2013 16:51:44

Copyright (c) 1991, 2009, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.201.107)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 12-OCT-2013 10:03:21
Uptime 25 days 6 hr. 48 min. 23 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /opt/oracle/db_1/network/admin/listener.ora
Listener Log File /opt/oracle/diag/tnslsnr/yunnanDB107/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.201.107)(PORT=1521)))
Services Summary...
Service "PROD" has 1 instance(s).
Instance "PROD", status UNKNOWN, has 1 handler(s) for this service...
Service "YNSTANDBYBOSS" has 1 instance(s).
Instance "PROD", status READY, has 1 handler(s) for this service...
The command completed successfully