Oracle RAC集群环境部署
一. 安装前准备
(1) Linux系统版本
SUSE Linux Enterprise Server 11 (x86_64)
(2) Oracle database和Grid安装包
linux.x64_11gR2_database_1of2.zip
linux.x64_11gR2_database_2of2.zip
linux.x64_11gR2_grid.zip
(3) ASMlib安装包
oracleasm-support-2.1.8-1.SLE11.x86_64.rpm
oracleasmlib-2.0.4-1.sle11.x86_64.rpm
oracleasm-2.0.5-7.37.3.x86_64.rpm
oracleasm-kmp-default-2.0.5_3.0.76_0.11-7.37.3.x86_64.rpm
(4) 依赖软件包
| gcc gcc-c++ gcc-32bit glibc-devel glibc-devel-32bit libaio libaio-devel libaio-devel-32bit libstdc++43-devel-32bit libstdc++43-devel sysstat libstdc++-devel libcap1 libcap1-32bit libcap2 libcap2-32bit compat* libgomp unixODBC unixODBC-devel |
(5) 磁盘分区
| 名称 |
分区 |
容量 |
| / |
/dev/sda |
100G |
| Swap |
Swap |
4G |
| CRS1 |
/dev/sdb1 |
2G |
| CRS2 |
/dev/sdb2 |
2G |
| CRS3 |
/dev/sdb3 |
2G |
| DATA1 |
/dev/sdb5 |
100G |
| REC1 |
/dev/sdb6 |
100G |
(6) IP地址划分
| Hostname |
Node name |
IP |
Device |
Type |
|
hmracdb1 |
hmracdb1 |
192.168.6.154 |
eth1 |
Public |
| hmracdb1-vip |
192.168.6.54 |
eth1 |
VIP |
|
| hmracdb1-priv |
10.17.81.154 |
eth0 |
Private |
|
|
hmracdb2 |
hmracdb2 |
192.168.6.155 |
eth1 |
Public |
| hmracdb2-vip |
192.168.6.55 |
eth1 |
VIP |
|
| hmracdb2-priv |
10.17.81.155 |
eth0 |
Private |
|
| Rac-scan |
hmracdb-scan |
192.168.6.66 |
二. 系统环境配置
(1) 主机hosts文件解析配置，两台主机使用相同配置
| # public ip 192.168.6.154 hmracdb1 192.168.6.155 hmracdb2
# private ip 10.17.81.154 hmracdb1-priv 10.17.81.155 hmracdb2-priv
# vip 192.168.6.54 hmracdb1-vip 192.168.6.55 hmracdb2-vip
# scan ip 192.168.6.66 hmracdb-scan |
(2) 创建oracle、grig用户和组
| 1. 创建组和用户 /usr/sbin/groupadd -g 501 oinstall /usr/sbin/groupadd -g 502 dba /usr/sbin/groupadd -g 503 oper /usr/sbin/groupadd -g 504 asmadmin /usr/sbin/groupadd -g 505 asmdba /usr/sbin/groupadd -g 506 asmoper /usr/sbin/useradd -u 501 -g oinstall -Gdba,oper,asmdba,asmadmin oracle -m /usr/sbin/useradd -u 502 -g oinstall -Gdba,asmadmin,asmdba,asmoper,oper grid -m
2. 创建密码 echo oracle | passwd --stdin oracle echo oracle | passwd --stdin grid
# id oracle uid=501(oracle)gid=501(oinstall)groups=502(dba),503(oper),504(asmadmin),505(asmdba),501(oinstall) # id grid |
(3) 创建安装目录
| mkdir -p /u01/app/{grid,oracle} chown -R grid:oinstall /u01/ chown -R grid:oinstall /u01/app/grid/ chown -R oracle:oinstall /u01/app/oracle/ chmod -R 775 /u01/
ls -l /u01/app/ total 8 drwxrwxr-x 2 grid oinstall 4096 Nov 16 19:09 grid drwxrwxr-x 2 oracle oinstall 4096 Nov 16 19:09 oracle |
(4) 配置grid、oracle用户环境变量
| ------------------------------------------------------------------------------------------------------- 1. 配置hmracdb1节点grid用户的profile，ORACLE_SID为+ASM1，hmracdb2节点改为+ASM2 ------------------------------------------------------------------------------------------------------- export ORACLE_SID=+ASM1 export ORACLE_BASE=/u01/app/oracle exportORACLE_HOME=/u01/app/grid/product/11.2.0 export PATH=$PATH:$ORACLE_HOME/bin export TMP=/tmp export TMPDIR=$TMP export NLS_DATE_FORMAT="yyyy-mm-ddHH24:MI:SS" export THREADS_FLAG=native export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib export CVUQDISK GRP=oinstall if [ $USER = "oracle" ] || [$USER = 'grid' ];then if [ $SHELL = "/bin/ksh" ];then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi
------------------------------------------------------------------------------------------------------- 2. 配置hmracdb1节点oracle用户的profile，ORACLE_SID为rac1，hmracdb2节点改为rac2 ------------------------------------------------------------------------------------------------------- export ORACLE_SID=rac1 export ORACLE_BASE=/u01/app/oracle exportORACLE_HOME=$ORACLE_BASE/product/11.2.0 exportPATH=$ORACLE_HOME/bin:/usr/sbin:$PATH export TMP=/tmp export TMPDIR=$TMP export ORACLE_TERM=xterm exportLD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/lib64 exportCLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib export NLS_DATE_FORMAT="yyyy-mm-ddHH24:MI:SS" export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK if [ $USER = "oracle" ] || [$USER = 'grid' ];then if [ $SHELL = "/bin/ksh" ];then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi |
(5) 修改内核参数配置
| 1. 修改/etc/sysctl.conf配置 # vim /etc/sysctl.conf kernel.shmmax = 68719476736 kernel.shmall = 4294967296 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 net.ipv4.ip_local_port_range = 9000 65500 fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.sem = 250 32000 100 128 kernel.shmmni = 4096 # sysctl –p
2. 修改/etc/security/limits.conf # vim /etc/security/limits.conf oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 # vim /etc/pam.d/login session required pam_limits.so |
(5) 配置NTP时间同步
| 1. 同步时间 # sntp -P no -r 10.10.0.2 &&hwclock –w
2. 配置hmracdb1为上层NTP服务器 hmracdb1:~ # vim /etc/ntp.conf #添加以下配置 restrict hmracdb1 mask 255.255.255.0 nomodify notrap noquery server hmracdb1 server 127.127.1.0
3. 配置hmracdb2节点NTP服务 hmracdb2:~ # vim /etc/ntp.conf server hmracdb1
4. 两个节点修改如下配置，并重启ntp服务 # vim /etc/sysconfig/ntp #修改以下配置 NTPD_OPTIONS="-x -g -u ntp:ntp" # service ntp restart |
(6) 配置oracle、grid用户ssh密钥
| su - grid grid@hmracdb1:~> ssh-keygen -t rsa grid@hmracdb2:~> ssh-keygen -t rsa grid@hmracdb1:~> ssh-copy-id -i~/.ssh/id_rsa.pub grid@hmracdb1 grid@hmracdb1:~> ssh-copy-id -i~/.ssh/id_rsa.pub grid@hmracdb2 grid@hmracdb2:~> ssh-copy-id -i~/.ssh/id_rsa.pub grid@hmracdb1 grid@hmracdb2:~> ssh-copy-id -i~/.ssh/id_rsa.pub grid@hmracdb2
su - oracle oracle@hmracdb1:~> ssh-keygen -t rsa oracle@hmracdb2:~> ssh-keygen -t rsa oracle@hmracdb1:~> ssh-copy-id -i~/.ssh/id_rsa.pub oracle@hmracdb1 oracle@hmracdb1:~> ssh-copy-id -i~/.ssh/id_rsa.pub oracle@hmracdb2 oracle@hmracdb2:~> ssh-copy-id -i~/.ssh/id_rsa.pub oracle@hmracdb1 oracle@hmracdb2:~> ssh-copy-id -i ~/.ssh/id_rsa.pub oracle@hmracdb2 |
三. 配置ASM磁盘
(1 )安装ASM软件(两个节点都安装)
| # zypper in -y oracleasmoracleasm-kmp-default # zypper inoracleasmlib-2.0.4-1.sle11.x86_64.rpm # zypper in oracleasm-support-2.1.8-1.SLE11.x86_64.rpm |
(2) 创建ASM分区(只需在节点1上创建)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
hmracdb1:~
# fdisk /dev/sdb
Device contains neither a valid DOSpartition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with diskidentifier 0xd60a0f97.
Changes will remain
in
memory only, untilyou decide to write them.
After that, of course, the previouscontent won't be recoverable.
Warning: invalid flag 0x0000 of partitiontable 4 will be corrected by w(rite)
Command (m
for
help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4, default 1):
Using default value 1
First sector (2048-419430399, default2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G}(2048-419430399, default 419430399): +2G
Command (m
for
help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4, default 2):
Using default value 2
First sector (4196352-419430399, default4196352):
Using default value 4196352
Last sector, +sectors or +size{K,M,G}(4196352-419430399, default 419430399): +2G
Command (m
for
help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4, default 3):
Using default value 3
First sector (8390656-419430399, default8390656):
Using default value 8390656
Last sector, +sectors or +size{K,M,G}(8390656-419430399, default 419430399): +2G
Command (m
for
help): n
Command action
e extended
p primary partition (1-4)
e
Selected partition 4
First sector (12584960-419430399, default12584960):
Using default value 12584960
Last sector, +sectors or +size{K,M,G}(12584960-419430399, default 419430399):
Using default value 419430399
Command (m
for
help): n
First sector (12587008-419430399, default12587008):
Using default value 12587008
Last sector, +sectors or +size{K,M,G}(12587008-419430399, default 419430399): +100G
Command (m
for
help): n
First sector (222304256-419430399,default 222304256):
Using default value 222304256
Last sector, +sectors or +size{K,M,G}(222304256-419430399, default 419430399):
Using default value 419430399
Command (m
for
help): p
Disk
/dev/sdb
: 214.7 GB, 214748364800bytes
255 heads, 63 sectors
/track
, 26108cylinders, total 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical
/physical
): 512 bytes/ 512 bytes
I
/O
size (minimum
/optimal
): 512 bytes
/512
bytes
Disk identifier: 0xd60a0f97
Device Boot Start End Blocks Id System
/dev/sdb1
2048 4196351 2097152 83 Linux
/dev/sdb2
4196352 8390655 2097152 83 Linux
/dev/sdb3
8390656 12584959 2097152 83 Linux
/dev/sdb4
12584960 419430399 203422720 5 Extended
/dev/sdb5
12587008 222302207 104857600 83 Linux
/dev/sdb6
222304256 419430398 98563071+ 83 Linux
Command (m
for
help): w
The partition table has been altered!
Calling ioctl() to re-
read
partition table.
Syncing disks.
在节点2上查看：
hmracdb2:
/usr/local/src
# fdisk -l /dev/sdb
Disk
/dev/sdb
: 214.7 GB, 214748364800bytes
255 heads, 63 sectors
/track
, 26108cylinders, total 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical
/physical
): 512 bytes/ 512 bytes
I
/O
size (minimum
/optimal
): 512 bytes
/512
bytes
Disk identifier: 0xd60a0f97
Device Boot Start End Blocks Id System
/dev/sdb1
2048 4196351 2097152 83 Linux
/dev/sdb2
4196352 8390655 2097152 83 Linux
/dev/sdb3
8390656 12584959 2097152 83 Linux
/dev/sdb4
12584960 419430399 203422720 5 Extended
/dev/sdb5
12587008 222302207 104857600 83 Linux
/dev/sdb6
222304256 419430398 98563071+ 83 Linux
|
(3) 初始化ASM配置(两个节点都执行)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
1.加载asm内核模块
hmracdb1:~
# oracleasminit
Creating
/dev/oracleasm
mount
point:
/dev/oracleasm
Loading module
"oracleasm"
:oracleasm
Configuring
"oracleasm"
to usedevice physical block size
Mounting ASMlib driver filesystem:
/dev/oracleasm
2.初始化配置
hmracdb1:~
# /etc/init.d/oracleasmconfigure
Configuring the Oracle ASM librarydriver.
This will configure the on-bootproperties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions itwill have. The current values
will be shown
in
brackets (
'[]'
). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface[]: grid
Default group to own the driver interface[]: asmadmin
Start Oracle ASM library driver on boot(y
/n
) [n]: y
Scan
for
Oracle ASM disks on boot (y
/n
)[y]: y
Writing Oracle ASM library driverconfiguration:
done
Initializing the Oracle ASMLibdriver:
done
Scanning the system
for
Oracle ASMLib disks:
done
|
(4) 创建ASM盘(只需在节点1上创建)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
|
1. 在节点1上ASM磁盘
hmracdb1:~
# oracleasm listdisks
hmracdb1:~
# oracleasm createdisk CRS1/dev/sdb1
Writing disk header:
done
Instantiating disk:
done
hmracdb1:~
# oracleasm createdisk CRS2/dev/sdb2
Writing disk header:
done
Instantiating disk:
done
hmracdb1:~
# oracleasm createdisk CRS3/dev/sdb3
Writing disk header:
done
Instantiating disk:
done
hmracdb1:~
# oracleasm createdisk DATA1/dev/sdb5
Writing disk header:
done
Instantiating disk:
done
hmracdb1:~
# oracleasm createdisk REC1/dev/sdb6
Writing disk header:
done
Instantiating disk:
done
hmracdb1:~
# oracleasm listdisks
CRS1
CRS2
CRS3
DATA1
REC1
2. 节点2上扫描ASM共享盘
hmracdb2:~
# oracleasm scandisks
Reloading disk partitions:
done
Cleaning any stale ASM disks...
Scanning system
for
ASM disks...
Instantiating disk
"CRS1"
Instantiating disk
"CRS2"
Instantiating disk
"CRS3"
Instantiating disk
"DATA1"
Instantiating disk
"REC1"
hmracdb2:~
# oracleasm listdisks
CRS1
CRS2
CRS3
DATA1
REC1
3. 检查ASM磁盘路径
hmracdb2:~
# oracleasm querydisk/dev/sdb*
Device
"/dev/sdb"
is not markedas an ASM disk
Device
"/dev/sdb1"
is marked anASM disk with the label
"CRS1"
Device
"/dev/sdb2"
is marked anASM disk with the label
"CRS2"
Device
"/dev/sdb3"
is marked anASM disk with the label
"CRS3"
Device
"/dev/sdb4"
is notmarked as an ASM disk
Device
"/dev/sdb5"
is marked anASM disk with the label
"DATA1"
Device
"/dev/sdb6"
is marked anASM disk with the label
"REC1"
hmracdb2:~
# ll /dev/oracleasm/disks/
total 0
brw-rw---- 1 grid asmadmin 8, 17 Nov 2114:19 CRS1
brw-rw---- 1 grid asmadmin 8, 18 Nov 2114:19 CRS2
brw-rw---- 1 grid asmadmin 8, 19 Nov 2114:19 CRS3
brw-rw---- 1 grid asmadmin 8, 21 Nov 2114:19 DATA1
brw-rw---- 1 grid asmadmin 8, 22 Nov 2114:19 REC1
hmracdb2:~
# ll -ltr /dev | grep "8,*17"
brw-rw---- 1 root disk 8, 17 Nov 21 14:19 sdb1
hmracdb2:~
# ll -ltr /dev | grep "8,*18"
brw-rw---- 1 root disk 8, 18 Nov 21 14:19 sdb2
hmracdb2:~
# ll -ltr /dev | grep "8,*19"
brw-rw---- 1 root disk 8, 19 Nov 21 14:19 sdb3
hmracdb2:~
# ll -ltr /dev | grep "8,*21"
brw-rw---- 1 root disk 8, 21 Nov 21 14:19 sdb5
hmracdb2:~
# ll -ltr /dev | grep "8,*22"
brw-rw---- 1 root disk 8, 22 Nov 21 14:19 sdb6
|
(5) 安装oracle依赖软件包
| hmracdb1:~ # for i in "gcc gcc-c++gcc-32bit glibc-devel glibc-devel-32bit libaio libaio-devel libaio-devel-32bitlibstdc++43-devel-32bit libstdc++43-devel sysstat libstdc++-devel libcap1libcap1-32bit libcap2 libcap2-32bit compat* libgomp unixODBC unixODBC-devel";dorpm -q $i;done hmracdb1:~ # zypper in gcc gcc-c++ gcc-32bit glibc-develglibc-devel-32bit libaio libaio-devel libaio-devel-32bitlibstdc++43-devel-32bit libstdc++43-devel sysstat libstdc++-devel libcap1libcap1-32bit libcap2 libcap2-32bit compat* libgomp unixODBC unixODBC-devel |
(6) 安装cvuqdisk软件
| 解压grid软件包，安装grid目录里的cvuqdisk-1.0.7-1.rpm软件 hmracdb1:/usr/local/src # cd grid/rpm/ hmracdb1:/usr/local/src/grid/rpm # ls cvuqdisk-1.0.7-1.rpm hmracdb1:/usr/local/src/grid/rpm # rpm -ivh cvuqdisk-1.0.7-1.rpm 本文转自 HMLinux 51CTO博客，原文链接:http://blog.51cto.com/7424593/1876401 |
