一套Linux上的11.1.0.7的physical standby物理备库在使用fast incremental backup进行高于0级的增量备份时会出现ORA-19648错误，其出错记录如下：
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
RMAN>  backup incremental  level  1  database ;
 
Starting backup  at  22-MAR-11
using channel ORA_DISK_1
using channel ORA_DISK_2
channel ORA_DISK_1: starting incremental  level  1 datafile backup  set
channel ORA_DISK_1: specifying datafile(s)  in  backup  set
input datafile file number=00002  name =/standby/oradata/SBDB2/datafile/o1_mf_sysaux_22m6ov92_.dbf
input datafile file number=00003  name =/standby/oradata/SBDB2/datafile/o1_mf_undotbs1_23m6ovap_.dbf
input datafile file number=00006  name =/standby/oradata/SBDB2/datafile/o1_mf_enc_25m6ovba_.dbf
channel ORA_DISK_1: starting piece 1  at  22-MAR-11
channel ORA_DISK_2: starting incremental  level  1 datafile backup  set
channel ORA_DISK_2: specifying datafile(s)  in  backup  set
input datafile file number=00001  name =/standby/oradata/SBDB2/datafile/o1_mf_system_21m6ov92_.dbf
input datafile file number=00005  name =/standby/oradata/SBDB2/datafile/o1_mf_example_24m6ovar_.dbf
input datafile file number=00004  name =/standby/oradata/SBDB2/datafile/o1_mf_users_26m6ovba_.dbf
channel ORA_DISK_2: starting piece 1  at  22-MAR-11
RMAN-03009: failure  of  backup command  on  ORA_DISK_1 channel  at  03/22/2011 20:20:28
ORA-19648: datafile 2: incremental-start SCN equals  checkpoint  SCN
ORA-19640: datafile  checkpoint  is  SCN 1249353  time  03/09/2011 05:50:27
continuing other job steps, job failed will  not  be re-run
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure  of  backup command  on  ORA_DISK_2 channel  at  03/22/2011 20:22:33
ORA-19648: datafile 1: incremental-start SCN equals  checkpoint  SCN
ORA-19640: datafile  checkpoint  is  SCN 1249352  time  03/09/2011 05:50:26
经过分析该ORA-19640->ORA-19648是由11.1.0.7上的Bug引起的，MOS已经确认其为Bug 9288598:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
Affects:
Product (Component)  Oracle Server (Rdbms)
Range  of  versions believed  to  be affected    Versions BELOW 12.1
Versions confirmed  as  being affected
11.1.0.7
Platforms affected   Generic ( all  / most platforms affected)
Fixed:
 
This issue  is  fixed  in
12.1 (Future Release)
11.2.0.2 (Server Patch  Set )
 
An RMAN Backup incremental  level  1 can fail  with  ORA-19648 / ORA-19640 if the standby  database  is  open  read  only  rather than just skipping the datafile.
PROBLEM:
--------
Following error occurring during incremental backup:
 
Starting backup  at  02- DEC -2009 20:37:44
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=154 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Veritas NetBackup  for  Oracle - Release 6.5
(2008052301)
channel ORA_SBT_TAPE_1: starting incremental  level  1 datafile backup  set
channel ORA_SBT_TAPE_1: specifying datafile(s)  in  backup  set
input datafile file number=00002
name =/data/ora_data01/KAHCB4P/KAHCB4P_sysaux_01.dbf
input datafile file number=00001
name =/data/ora_data01/KAHCB4P/KAHCB4P_system_01.dbf
input datafile file number=00003
name =/data/ora_data01/KAHCB4P/KAHCB4P_undo_01.dbf
input datafile file number=00004
name =/data/ora_data01/KAHCB4P/KAHCB4P_users_01.dbf
input datafile file number=00005
name =/data/ora_data01/KAHCB4P/KAHCB4P_tools_01.dbf
input datafile file number=00006
name =/data/ora_data01/KAHCB4P/KAHCB4P_backup_test_01.dbf
channel ORA_SBT_TAPE_1: starting piece 1  at  02- DEC -2009 20:38:01
RMAN-3009: failure  of  backup command  on  ORA_SBT_TAPE_1 channel  at  12/02/2009
20:38:02
ORA-19648: datafile 2: incremental-start SCN equals  checkpoint  SCN
ORA-19640: datafile  checkpoint  is  SCN 3911695  time  11/27/2009 12:36:48
continuing other job steps, job failed will  not  be re-run
channel ORA_SBT_TAPE_1: starting incremental  level  1 datafile backup  set
channel ORA_SBT_TAPE_1: specifying datafile(s)  in  backup  set
including  current  control file  in  backup  set
including  current  SPFILE  in  backup  set
 
DIAGNOSTIC ANALYSIS:
--------------------
Same problem  as  Bug 6903819
Requested backport, but this hasn't fixed the problem.
Confirmed that the patch was installed correctly  and  relinked successfully.
 
RELEASE NOTES:
]]Backup incremental leve 1 fails  with  ORA-19648  and  ORA-19640  when  standby
]] database  opened  for  read  only .
*** 02/14/10 02:43 pm *** ( ADD : Impact/Symptom->FEATURE UNUSABLE )
 
REDISCOVERY INFORMATION:
Backup incremental  level  1 fails  with  ORA-19648  and  the standby  database  is
open  read - only .
WORKAROUND:
None
可以从上述Note中看到该Bug需要到11.2.0.2中才得到fix，那么在11.1.0.7上我们有什么办法能解决或者绕过该问题吗？ 经过测试，我得到2种workaround的方法： 该ORA-19648错误首先可以通过在物理备库(physical standby)上停止介质恢复(MRP)进程的方式来workaround：
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
SQL>   alter  database  recover managed standby  database  cancel;
Database  altered.
 
RMAN> backup incremental  level  1  database ;
 
Starting backup  at  22-MAR-11
using target  database  control file  instead  of  recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=141 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=140 device type=DISK
channel ORA_DISK_1: starting incremental  level  1 datafile backup  set
channel ORA_DISK_1: specifying datafile(s)  in  backup  set
input datafile file number=00002  name =/standby/oradata/SBDB2/datafile/o1_mf_sysaux_22m6ov92_.dbf
input datafile file number=00004  name =/standby/oradata/SBDB2/datafile/o1_mf_users_26m6ovba_.dbf
input datafile file number=00006  name =/standby/oradata/SBDB2/datafile/o1_mf_enc_25m6ovba_.dbf
channel ORA_DISK_1: starting piece 1  at  22-MAR-11
channel ORA_DISK_2: starting incremental  level  1 datafile backup  set
channel ORA_DISK_2: specifying datafile(s)  in  backup  set
input datafile file number=00001  name =/standby/oradata/SBDB2/datafile/o1_mf_system_21m6ov92_.dbf
input datafile file number=00003  name =/standby/oradata/SBDB2/datafile/o1_mf_undotbs1_23m6ovap_.dbf
input datafile file number=00005  name =/standby/oradata/SBDB2/datafile/o1_mf_example_24m6ovar_.dbf
channel ORA_DISK_2: starting piece 1  at  22-MAR-11
channel ORA_DISK_1: finished piece 1  at  22-MAR-11
piece handle=/standby/backup/2km7suui_1_1.bak tag=TAG20110322T212538 comment=NONE
channel ORA_DISK_1: backup  set  complete, elapsed  time : 00:00:33
channel ORA_DISK_2: finished piece 1  at  22-MAR-11
piece handle=/standby/backup/2lm7suv1_1_1.bak tag=TAG20110322T212538 comment=NONE
channel ORA_DISK_2: backup  set  complete, elapsed  time : 00:00:27
Finished backup  at  22-MAR-11
 
Starting Control File  and  SPFILE Autobackup  at  22-MAR-11
piece handle=/standby/backup/c-157018592-20110322-01.bak comment=NONE
Finished Control File  and  SPFILE Autobackup  at  22-MAR-11
该ORA-19648错误也可以通过禁用fast incremental backup的块跟踪(block change tracking)特性来绕过问题，当然在可能的情况下我们更推荐使用上面那种方法，因为毕竟还可以利用到fast incremental backup特性：
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
SQL>  alter  database  recover managed standby  database  using  current  logfile disconnect;
Database  altered.
 
SQL>  select  process,status  from  v$managed_standby;
 
PROCESS   STATUS
--------- ------------
ARCH      CONNECTED
ARCH      CONNECTED
ARCH      CONNECTED
ARCH      CONNECTED
MRP0      APPLYING_LOG
 
SQL>  select  status  from  v$block_change_tracking;
 
STATUS
----------
ENABLED
 
 
SQL>  alter  database  disable block change tracking;
Database  altered.
 
 
RMAN>  backup incremental  level  2  database ;
 
Starting backup  at  22-MAR-11
using channel ORA_DISK_1
using channel ORA_DISK_2
channel ORA_DISK_1: starting incremental  level  2 datafile backup  set
channel ORA_DISK_1: specifying datafile(s)  in  backup  set
input datafile file number=00002  name =/standby/oradata/SBDB2/datafile/o1_mf_sysaux_22m6ov92_.dbf
input datafile file number=00004  name =/standby/oradata/SBDB2/datafile/o1_mf_users_26m6ovba_.dbf
input datafile file number=00006  name =/standby/oradata/SBDB2/datafile/o1_mf_enc_25m6ovba_.dbf
channel ORA_DISK_1: starting piece 1  at  22-MAR-11
channel ORA_DISK_2: starting incremental  level  2 datafile backup  set
channel ORA_DISK_2: specifying datafile(s)  in  backup  set
input datafile file number=00001  name =/standby/oradata/SBDB2/datafile/o1_mf_system_21m6ov92_.dbf
input datafile file number=00003  name =/standby/oradata/SBDB2/datafile/o1_mf_undotbs1_23m6ovap_.dbf
input datafile file number=00005  name =/standby/oradata/SBDB2/datafile/o1_mf_example_24m6ovar_.dbf
channel ORA_DISK_2: starting piece 1  at  22-MAR-11
channel ORA_DISK_1: finished piece 1  at  22-MAR-11
piece handle=/standby/backup/2nm7sv8d_1_1.bak tag=TAG20110322T213052 comment=NONE
channel ORA_DISK_1: backup  set  complete, elapsed  time : 00:00:15
channel ORA_DISK_2: finished piece 1  at  22-MAR-11
piece handle=/standby/backup/2om7sv8s_1_1.bak tag=TAG20110322T213052 comment=NONE
channel ORA_DISK_2: backup  set  complete, elapsed  time : 00:00:15
Finished backup  at  22-MAR-11
 
Starting Control File  and  SPFILE Autobackup  at  22-MAR-11
piece handle=/standby/backup/c-157018592-20110322-02.bak comment=NONE
Finished Control File  and  SPFILE Autobackup  at  22-MAR-11