[RAC]虚拟机linux7.0+oracle12cr2RAC+asm安装

1.主机名规划

为漳州等地区用户提供了全套网页设计制作服务，及漳州网站建设行业解决方案。主营业务为网站建设、成都网站设计、漳州网站设计，以传统方式定制建设网站，并提供域名空间备案等一条龙服务，秉承以专业、用心的态度为用户提供真诚的服务。我们深信只要达到每一位用户的要求，就会得到认可，从而选择与我们长期合作。这样，我们也可以走得更远！

[RAC] 虚拟机linux7.0 + oracle12cr2 RAC + asm 安装 vps+3nZEm+sFS5WD4dGxwiwvZtf6bREdK/xxAZgDogXmgGiBOeARA6ZwOS08YsAShUcsbhy8d8Pt2CV2td8W8bG6Qf8HdZh7RPvzWmsAAAAASUVORK5CYII=">

2.OS安装(具体过程略过)

swap规则: 内存<8G则物理内存2倍，>8G <16G 则等于物理内存，>16G物理内存一半

3.OS准备工作

(1).关闭防火墙

systemctl status firewalld 查看防火墙状态
systemctl stop firewalld 关闭防火墙
systemctl disable firewalld 禁用防火墙

(2).关闭selinux

getenforce 查看selinux的模式

修改/etc/selinux/config 文件，SELINUX=enforce –> disable

#setenforce 0 配置生效

(3).设置FTP

查看 getsebool -a |grep ftp

修改权限

setsebool ftpd_use_nfs on

setsebool ftp_home_dir on

setsebool ftpd_full_access on

setsebool ftpd_use_passive_mode on

允许root登录FTP

注释掉ftpusers和user_list 中的root行
重启服务systemctl restart vsftpd.service

(4).配置yum

删除rm /etc/yum.repos.d/packagekit-media.repo
建立 local.repo 文件
[test]
name=test
baseurl="file:///run/media/root/RHEL-7.0 Server.x86_64"
gpgcheck=0
enable=1

# yum clean all
# yum list

(5). 根据官方文档进行patch

The following packages (or later versions) must be installed:

binutils-2.23.52.0.1-12.el7 (x86_64)
compat-libcap1-1.10-3.el7 (x86_64)
compat-libstdc++-33-3.2.3-71.el7 (i686)
compat-libstdc++-33-3.2.3-71.el7 (x86_64)
glibc-2.17-36.el7 (i686)
glibc-2.17-36.el7 (x86_64)
glibc-devel-2.17-36.el7 (i686)
glibc-devel-2.17-36.el7 (x86_64)
ksh
libaio-0.3.109-9.el7 (i686)
libaio-0.3.109-9.el7 (x86_64)
libaio-devel-0.3.109-9.el7 (i686)
libaio-devel-0.3.109-9.el7 (x86_64)
libgcc-4.8.2-3.el7 (i686)
libgcc-4.8.2-3.el7 (x86_64)
libstdc++-4.8.2-3.el7 (i686)
libstdc++-4.8.2-3.el7 (x86_64)
libstdc++-devel-4.8.2-3.el7 (i686)
libstdc++-devel-4.8.2-3.el7 (x86_64)
libxcb-1.9-5.el7 (i686)
libxcb-1.9-5.el7 (x86_64)
libX11-1.6.0-2.1.el7 (i686)
libX11-1.6.0-2.1.el7 (x86_64)
libXau-1.0.8-2.1.el7 (i686)
libXau-1.0.8-2.1.el7 (x86_64)
libXi-1.7.2-1.el7 (i686)
libXi-1.7.2-1.el7 (x86_64)
libXtst-1.2.2-1.el7 (i686)
libXtst-1.2.2-1.el7 (x86_64)
make-3.82-19.el7 (x86_64)
net-tools-2.0-0.17.20131004git.el7 (x86_64) (for Oracle RAC and Oracle Clusterware)
nfs-utils-1.3.0-0.21.el7.x86_64 (for Oracle ACFS)
smartmontools-6.2-4.el7 (x86_64)
sysstat-10.1.5-1.el7 (x86_64)

yum install binutils –y
yum install compat-libcap1.x86_64 –y
yum install glibc.i686 –y
yum install glibc.x86_64 –y
yum install glibc-devel.i686 -y
yum install glibc-devel.x86_64 –y
yum install libaio.i686 –y
yum install libaio.x86_64 -y
yum install libaio-devel.i686 –y
yum install libaio-devel.x86_64 –y
yum install libX11.i686 –y
yum install libX11.x86_64 –y
yum install libXau.i686 -y
yum install libXau.x86_64 –y
yum install libXi.x86_64 –y
yum install libXi.i686 –y
yum install libXtst.i686 –y
yum install libXtst.x86_64 –y
yum install libgcc.i686 –y
yum install libgcc.x86_64 –y
yum install libstdc++.i686 –y
yum install libstdc++.x86_64 –y
yum install libstdc++-devel.i686 –y
yum install libstdc++-devel.x86_64 –y
yum install libxcb.i686 -y
yum install libxcb.x86_64 –y
yum install make.x86_64 –y
yum install net-tools.x86_64 –y
yum install smartmontools.x86_64 –y
yum install sysstat.x86_64 –y
yum install ksh.x86_64 -y

(6).修改hosts文件

#public ip
xxx.xxx.xxx.xxx rac1
xxx.xxx.xxx.xxx rac2

#priv ip
xxx.xxx.xxx.xxx rac1-priv
xxx.xxx.xxx.xxx rac2-priv

#vip ip
xxx.xxx.xxx.xxx rac1-vip
xxx.xxx.xxx.xxx rac2-vip

#scan ip
xxx.xxx.xxx.xxx rac-scan

(7).添加组和用户

groupadd -g 60001 oinstall
groupadd -g 60002 dba
groupadd -g 60003 oper
groupadd -g 60004 backupdba
groupadd -g 60005 dgdba
groupadd -g 60006 kmdba
groupadd -g 60007 asmdba
groupadd -g 60008 asmoper
groupadd -g 60009 asmadmin

useradd -u 61001 -g oinstall -G asmadmin,asmdba,dba,asmoper grid
useradd -u 61002 -g oinstall -G dba,backupdba,oper,dgdba,kmdba,asmdba,asmadmin oracle

(8).创建软件安装目录

mkdir -p /oracle/app/grid
mkdir -p /oracle/app/12.2.0.1/grid
chown -R grid:oinstall /oracle

mkdir -p /oracle/app/oraInventory
chown -R grid:oinstall /oracle/app/oraInventory

mkdir -p /oracle/app/oracle
chown -R oracle:oinstall /oracle/app/oracle
chmod -R 775 /oracle

(9).vi /etc/security/limits.conf

#ORACLE SETTING
grid    soft    nproc   2047
grid    hard    nproc   16384
grid    soft    nofile 1024
grid    hard    nofile 65536
grid    soft    stack   10240
grid    hard    stack   32768
oracle soft    nproc   2047
oracle hard    nproc   16384
oracle soft    nofile 1024
oracle hard    nofile 65536
oracle soft    stack   10240
oracle hard    stack   32768

(10)./etc/security/limits.d/20-nproc.conf

注释* soft nproc 4096
添加* - nproc 16384

(11)./etc/pam.d/login

添加
#ORACLE SETTING
session required pam_limits.so

(12).修改内核参数
#vi /etc/sysctl.conf
#ORACLE SETTING
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 524288
kernel.shmmax = 1610612736
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
kernel.panic_on_oops = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 9000 65500
# sysctl –p 配置生效

参考官方文档

https://docs.oracle.com/database/122/LADBI/minimum-parameter-settings-for-installation.htm#LADBI-GUID-CDEB89D1-4D48-41D9-9AC2-6AD9B0E944E3

kernel	说明
SHMMAX	Linux进程可以分配的单独共享内存段的最大值。一般设置为内存总大小的一半。这个值的设置应该大于SGA_MAX_TARGET或MEMORY_MAX_TARGET的值
SHMMIN	最小的内存segment的大小
shmall	设置共享内存总页数SHMMAX/4K

(13).配置环境变量
grid
PS1="[`whoami`@`hostname`:"'$PWD]$'
export PS1
umask 022
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
export ORACLE_HOSTNAME=rac1
export ORACLE_SID=+ASM1
export ORACLE_TERM=xterm
export ORACLE_BASE=/oracle/app/grid
export ORACLE_HOME=/oracle/app/12.2.0.1/grid
export NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"
PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH
export THREADS_FLAG=native

oracle
PS1="[`whoami`@`hostname`:"'$PWD]$'
export PS1
umask 022
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
export ORACLE_HOSTNAME=rac1
export ORACLE_UNQNAME=racdb12c
export ORACLE_BASE=/oracle/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/12.2.0.1/db_1
export ORACLE_SID=racdb12c1
export ORACLE_TERM=xterm
export NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH
export THREADS_FLAG=native

(14).配置SSH

在主节点上以grid,oracle用户身份生成公钥和私钥
mkdir ~/.ssh
ssh-keygen -t rsa
ssh-keygen -t dsa

在另外的节点上执行相同的操作
在1节点使用oracle(grid)用户执行以下操作
cat ~/.ssh/id_rsa.pub >> ./.ssh/authorized_keys
cat ~/.ssh/id_dsa.pub >> ./.ssh/authorized_keys
ssh rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys

在1,2节点上测试ssh
ssh rac1 date
ssh rac2 date
ssh rac1-priv date
ssh rac2-priv date

4.修改虚拟机配置文件(.vmx),添加以下内容
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
disk.EnableUUID = "TRUE"
scsi1.sharedBus = "VIRTUAL"

5.配置共享存储

Name	Size
dggrid1	40GB
dggrid2	10GB
dgsystem	21GB
dgrecover	22GB
dgdata1	23GB
dgdata2	24GB

(1).编辑/etc/scsi_id.config，添加下面内容
options=--whitelisted --replace-whitespace

(2).获取需要绑定ASM Disk磁盘的uuid
for i in b c d e f g;
do
echo "sd$i" "`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`";
done

sdb 36000c29b0e11cfc09873aa3dc45baf3e
sdc 36000c293313041c3f662d837263cbef3
sdd 36000c2908886e8ba4e0e6b410b456616
sde 36000c29cb127c1af8c28efd8af0d92d4
sdf 36000c29ecd42be581ecb6d8f2b37e2f1
sdg 36000c29e8abe980615022c02a11960d1

(3).编写udev rules 文件
for i in b c d e f g;
do
echo "KERNEL==\"sd*\",BUS==\"scsi\",PROGRAM==\"/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\",RESULT==\"`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\",NAME=\"asm-disk$i\",OWNER=\"grid\",GROUP=\"asmadmin\",MODE=\"0660\""
done

redhat 7
for i in b c d e f g;
do
echo "KERNEL==\"sd*\", ENV{DEVTYPE}==\"disk\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d \$devnode\", RESULT==\"`/usr/lib/udev/scsi_id -g -u /dev/sd$i`\", RUN+=\"/bin/sh -c 'mknod /dev/asmdisk$i b \$major \$minor; chown grid:asmadmin /dev/asmdisk$i; chmod 0660 /dev/asmdisk$i'\""
done

KERNEL=="sd*", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c2952c880a7bd3b532de2be6e7c1", RUN+="/bin/sh -c 'mknod /dev/asm-dggrid1 b $major $minor; chown grid:asmadmin /dev/asm-dggrid1; chmod 0660 /dev/asm-dggrid1'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c29d8069ccccdc4254861e4de50a", RUN+="/bin/sh -c 'mknod /dev/asm-dggrid2 b $major $minor; chown grid:asmadmin /dev/asm-dggrid2; chmod 0660 /dev/asm-dggrid2'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c29228ddf02cfb4e980a4a06fbdc", RUN+="/bin/sh -c 'mknod /dev/asm-dgsystem b $major $minor; chown grid:asmadmin /dev/asm-dgsystem; chmod 0660 /dev/asm-dgsystem'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c29f3195a284faae34072f347a93", RUN+="/bin/sh -c 'mknod /dev/asm-dgrecover b $major $minor; chown grid:asmadmin /dev/asm-dgrecover; chmod 0660 /dev/asm-dgrecover'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c298e69bb4ea5bf85d9b9b49fd02", RUN+="/bin/sh -c 'mknod /dev/asm-dgdata1 b $major $minor; chown grid:asmadmin /dev/asm-dgdata1; chmod 0660 /dev/asm-dgdata1'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c2918d25c19177895d053c751663", RUN+="/bin/sh -c 'mknod /dev/asm-dgdata2 b $major $minor; chown grid:asmadmin /dev/asm-dgdata2; chmod 0660 /dev/asm-dgdata2'"

# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
#/sbin/udevadm trigger --type=devices --action=change

(4).重启udev设备
/sbin/udevadm control –reload-rules
/sbin/start_udev #redhat7 整合到systemctl restart systemd-udev-trigger.service 中
ll /dev/asm*

6.安装VNC
#yum install libvncserver.x86_64 –y
#yum install tigervnc.x86_64 –y
#yum install tigervnc-server.x86_64 –y

# vncserver 开启服务并设置密码

7.安装cvuqdisk包并验证(所有节点都装)
# cd /tmp/wwj/GI/cv/rpm
# export CVUQDISK_GRP=oinstall
# rpm -ivh cvuqdisk-1.0.10-1.rpm

./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup –verbose
./runcluvfy.sh stage –post hwos –n rac1,rac2 –verbose

8.安装GI

su – grid
cd $ORACLE_HOME
unzip -q download_location/grid.zip

执行安装脚本
gridSetup.sh

[RAC] 虚拟机linux7.0 + oracle12cr2 RAC + asm 安装

安装后检查

[root@rac1 ~]# crsctl status res -t
--------------------------------------------------------------------------------
Name           Target State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE ONLINE       rac1                     STABLE
               ONLINE ONLINE       rac2                     STABLE
ora.DGGRID1.dg
               ONLINE ONLINE       rac1                     STABLE
               ONLINE ONLINE       rac2                     STABLE
ora.LISTENER.lsnr
               ONLINE ONLINE       rac1                     STABLE
               ONLINE ONLINE       rac2                     STABLE
ora.net1.network
               ONLINE ONLINE       rac1                     STABLE
               ONLINE ONLINE       rac2                     STABLE
ora.ons
               ONLINE ONLINE       rac1                     STABLE
               ONLINE ONLINE       rac2                     STABLE
ora.proxy_advm
               OFFLINE OFFLINE      rac1                     STABLE
               OFFLINE OFFLINE      rac2                     STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE ONLINE       rac1                     STABLE
ora.MGMTLSNR
      1        OFFLINE OFFLINE                               STABLE
ora.asm
      1        ONLINE ONLINE       rac1                     Started,STABLE
      2        ONLINE ONLINE       rac2                     Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE ONLINE       rac1                     STABLE
ora.qosmserver
      1        ONLINE ONLINE       rac1                     STABLE
ora.rac1.vip
      1        ONLINE ONLINE       rac1                     STABLE
ora.rac2.vip
      1        ONLINE ONLINE       rac2                     STABLE
ora.scan1.vip
      1        ONLINE ONLINE       rac1                     STABLE

[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     409568
         Used space (kbytes)      :       1848
         Available space (kbytes) :     407720
         ID                       : 1005996313
         Device/File Name         :   +DGGRID1
                                    Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@rac1 ~]# crsctl query css votedisk
## STATE    File Universal Id                File Name Disk group
-- -----    -----------------                --------- ---------
1. ONLINE   90f6afc163534f18bf96e54a9bf239db (/dev/asm-grid1) [DGGRID1]
Located 1 voting disk(s).

9.创建ASM磁盘组

su – grid
$asmca

[RAC] 虚拟机linux7.0 + oracle12cr2 RAC + asm 安装