How to install Oracle RAC 11g on Linux

11.2.0.2  you can use  <= linux 6.3

VMWare settings

Create a VM with below configurations show below picture

Ram                 2GB

Processors       1          (if you are using dual core)

Hard Disk         35GB    (one disk is default, after install Linux we will add remaining disks)

Network adapter 2         (one card is default, after install Linux we will add remaining)

Prepare your host for cluster installation
-------------------------------------------
1.  Varify the os and kernel version
    visit the official documents at tahit.oracle.com
2.  Ensure all mandatory packages are installed
    rpm -q libaio-devel
3.  Ensure necessary users/ groups are created
    SOD
    Grid software : grid/oinstall (asmadmin, asmoper)
    Oracle software : oracle/oinstall (dba, oper)
4.  Ensure the mechines are time-synched. like NTP setup
    date && ssh rac2 date
    Or Ctssd will take care of it.
 (ctssd status ACTIVE OR OBSERVER)
5.  Establish SSH user equivalence
6.  Install ASMLib software on your host or rpms for ASM on every node.
     /etc/init.d/oracleasm configure -i
        ------------------- : grid
        ------------------- : asmadmin
        ------------------- : y
    Check /etc/sysconfig/oracleasm  file
7.  Provision of ASM disks
8.  Ensure the OS kernel parameters
    /etc/sysctl.conf
9.  Ensure the OS limits are configured.
    /etc/security/limits.conf
10. Ensure the mount points have suitable ownership and permissions.
11. Setup /etc/resolv.conf for DNS resolution
          /etc/hosts
         Check $ nslookup xxxxx
12.  Extract Oracle GI software
     Run cluster verification utility
     Invoke installer from one node
     Apply latest PSU bundle before running the root.sh script

-----------------------------------------------------------------------------------
 

Shared storage must be configured as ASM Disks.

1. Install linux in vmware

while installing change hostname

change these settings also (if you are using linux 6.3)

2. install all rpms

os related rpms form linux disk

asm related rpms from asm disk

grid realated rpms from grid disk

COPY ALL RPM FILES FROM LINUX CD
-------------------------------------------------------

copy all rpm files to local disk in folder "repo"

$cp /oracle/server/*.rpm   /u02/repo

CREATE A REPOSITORY FOR RPM

---------------------------------------------------

#cd /U02

#createrepo  -v repo

3.  change all shell parameters for linux

4.  change limits for oracle

Manual Setup

install required rpm:

---------------------

Goto ”repo” folder

#cd  /u02/repo

rpm -Uvh binutils-2.*

rpm -Uvh compat-libstdc++-33*

rpm -Uvh compat-libstdc++-33*.i386.rpm

rpm -Uvh elfutils-libelf*

rpm -Uvh gcc-4.*

rpm -Uvh gcc-c++-4.*

rpm -Uvh glibc-2.*

rpm -Uvh glibc-common-2.*

rpm -Uvh glibc-devel-2.*

rpm -Uvh glibc-headers-2.*

rpm -Uvh ksh*

rpm -Uvh libaio-0.*

rpm -Uvh libaio-devel-0.*

rpm -Uvh libgomp-4.*

rpm -Uvh libgcc-4.*

rpm -Uvh libstdc++-4.*

rpm -Uvh libstdc++-devel-4.*

rpm -Uvh make-3.*

rpm -Uvh sysstat-7.*

rpm -Uvh unixODBC-2.*

rpm -Uvh unixODBC-devel-2.*

rpm -Uvh numactl-devel-*

Oracle recommend the following minimum parameter settings.

#vi     /etc/sysctl.conf

fs.suid_dumpable = 1

fs.aio-max-nr = 1048576

fs.file-max = 6815744

kernel.shmall = 268435456

kernel.shmmax = 536870912

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default=262144

net.core.rmem_max=4194304

net.core.wmem_default=262144

net.core.wmem_max=1048586

Run the following command to change the current kernel parameters.

#/sbin/sysctl   -p

#vi    /etc/security/limits.conf

# refer orabug15971421 for more info.
oracle   soft   nproc    16384
# oracle-rdbms-server-11gR2-preinstall setting for nproc hard limit is 16384
oracle   hard   nproc    16384
# oracle-rdbms-server-11gR2-preinstall setting for stack soft limit is 10240KB
oracle   soft   stack    10240
# oracle-rdbms-server-11gR2-preinstall setting for stack hard limit is 32768KB
oracle   hard   stack    32768
# oracle-rdbms-server-11gR2-preinstall setting for memlock hard limit is maximum of {128GB (x86_64) / 3GB (x86) or 90 % of RAM}
oracle   hard   memlock    3145728
# oracle-rdbms-server-11gR2-preinstall setting for memlock soft limit is maximum of {128GB (x86_64) / 3GB (x86) or 90% of RAM}
oracle   soft   memlock    3145728

grid hard nproc 16384
grid soft nproc 2047
grid hard nofile 65536

5. Create the new groups and users.

#groupadd oinstall

#groupadd dba
#groupadd osoper

#groupadd asmadmin
#groupadd asmdba
#groupadd asmoper

#useradd -g oinstall –G dba, osoper  oracle
# useradd -g oinstall -G asmadmin, asmdba, asmoper  grid

#passwd oracle

#passwd grid

OR

 you can skip above 2,3,4 and 5, if you use below commands (need internet connection)

# yum install oracle-rdbms-server-11gR2-preinstall

# yum install oracleasm*

From ISO disk 

# yum install cuvqdisk-*.rpm

6. Set secure Linux  by editing

#vi   /etc/selinux/config

SELINUX=disable

[root@ora12a ~]# service iptables stop

iptables: Flushing firewall rules:                                  [  OK  ]
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Unloading modules:                                      [  OK  ]

[root@ora12a ~]# chkconfig iptables off

7. Create the directories in which the Oracle software will be installed.

#mkdir -p /u01/app/oracle/product/11.2.0/db_1

mkdir -p /u01/app/11.2.0/grid

#chown -R grid:oinstall /u01

#chmod -R 775 /u01

#chmod -R 777 /u02

8. /etc/init.d/oracleasm configure
---------------------------------: oracle
---------------------------------:oinstall
---------------------------------: y
---------------------------------: y

8. install vmbox-configuramtion settings

     change time synch

9. change /etc/hosts

Change the /etc/hosts entries as below

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1               rac1.oracl.com rac1 localhost.localdomain localhost

::1             localhost6.localdomain6 localhost6

#Public

192.168.1.101           rac1.oracl.com             rac1

192.168.1.102           rac2.oracl.com             rac2

#Private

10.10.1.1               rac1-priv.oracl.com        rac1-priv

10.10.1.2               rac2-priv.oracl.com        rac2-priv

#Virtual

192.168.1.201           rac1-vip.oracl.com         rac1-vip

192.168.1.202           rac2-vip.oracl.com         rac2-vip

#SCAN

192.168.1.51            rac-scan.oracl.com           rac-scan

192.168.1.52            rac-scan.oracl.com           rac-scan

192.168.1.53            rac-scan.oracl.com           rac-scan8.

10. disable ntpd

# service ntpd stop

Shutting down ntpd: [ OK ]

# chkconfig ntpd off

# mv /etc/ntp.conf /etc/ntp.conf.orig

11. shut it down and clone vm.

12.  Add one or three 1GB hard-disks to any one VM machine as independent and allocate storage.

add the code to share disks in VM ware for both Machines.

disk.locking = "FALSE"

diskLib.dataCacheMaxSize = "0"

diskLib.dataCacheMaxReadAheadSize = "0"

diskLib.dataCacheMinReadAheadSize = "0"

diskLib.dataCachePageSize = "4096"

diskLib.maxUnsyncedWrites = "0"

scsi1.present = "TRUE"

scsi1.sharedBus = "VIRTUAL"

scsi1.virtualDev = "lsilogic"

scsi1:1.present = "TRUE"

scsi1:1.fileName = "D:\SharedDisks\Shared1.vmdk"

scsi1:1.writeThrough = "TRUE"

scsi1:1.mode = "independent-persistent"

scsi1:2.present = "TRUE"

scsi1:2.fileName = "D:\SharedDisks\Shared2.vmdk"

scsi1:2.writeThrough = "TRUE"

scsi1:2.mode = "independent-persistent"

scsi1:3.present = "TRUE"

scsi1:3.fileName = "D:\SharedDisks\Shared3.vmdk"

scsi1:3.writeThrough = "TRUE"

scsi1:3.mode = "independent-persistent"

scsi1:4.present = "TRUE"

scsi1:4.fileName = "D:\SharedDisks\Shared4.vmdk"

scsi1:4.writeThrough = "TRUE"

scsi1:4.mode = "independent-persistent"

scsi1:5.present = "TRUE"

scsi1:5.fileName = "D:\SharedDisks\Shared5.vmdk"

scsi1:5.writeThrough = "TRUE"

scsi1:5.mode = "independent-persistent"

13. Start rac1 VM Machine  and configure ASM disks as below

# fdisk -l

# fdisk /dev/sdf

type n

type 1

enter 

enter 

type w

# /etc/init.d/oracleasm createdisk VOTE1  /dev/sdf1

-------------------------------------------------------------: [OK]

14.  Start second VM Machine and rename it's hostname as rac2

#vi /etc/sysconfig/network

hostname= rac2.oracle.com

:wq

15. Shutdown  the machine.

16.  add 2 nic cards, 

Start the VM ware 

change their ip addresses using neat command 

or  Administraton--> network configuration

as shown in /etc/hosts

Restart the Mechine.

17. copy Grid and Database Software in /u02 and change the permissions on both nodes

# chmod -R 775 /u02/grid
# chown -R grid:oinstall /u02/grid

18.  install Grid Software 

# su - grid

$ cd /u02/grid

. runInstaller

while setup it will configure ssh setup will be done 

After install grid will be installed.

to diconfigure rac

/u01/app/11.2/0/grid/crs/install/rootcrs.pl -deconfig -force -lastnode

Oracle Grid Infrastructure Groups and Users Example

• An Oracle central inventory group, or oraInventory group (oinstall), whose members that have this group as their primary group. Members of this group are granted the OINSTALL system privileges, which grants permissions to write to the oraInventory directory, and other associated install binary privileges.
• An OSASM group (asmadmin), associated with Oracle Grid Infrastructure during installation, whose members are granted the SYSASM privileges to administer Oracle ASM.
• An OSDBA for ASM group (asmdba), associated with Oracle Grid Infrastructure storage during installation. Its members include grid and any database installation owners, such as oracle1 and oracle2, who are granted access to Oracle ASM. Any additional installation owners that use Oracle ASM for storage must also be made members of this group.
• An OSOPER for ASM group for Oracle ASM (asmoper), associated with Oracle Grid Infrastructure during installation. Members of asmoper group are granted limited Oracle ASM administrator privileges, including the permissions to start and stop the Oracle ASM instance.
• An Oracle Grid Infrastructure installation owner (grid), with the oraInventory group (oinstall) as its primary group, and with the OSASM (asmadmin) group and the OSDBA for ASM (asmdba) group as secondary groups.
• /u01/app/oraInventory. The central inventory of Oracle installations on the cluster. This path remains owned by grid:oinstall, to enable other Oracle software owners to write to the central inventory.
• An OFA-compliant mount point /u01 owned by grid:oinstall before installation, so that Oracle Universal Installer can write to that path.
• An Oracle base for the grid installation owner /u01/app/grid owned by grid:oinstall with 775 permissions, and changed during the installation process to 755 permissions.
• A Grid home /u01/app/12.1.0/grid owned by grid:oinstall with 775 (drwxdrwxr-x) permissions. These permissions are required for installation, and are changed during the installation process to root:oinstall with 755 permissions (drwxr-xr-x).

6.1.11.2 Oracle Database DB1 Groups and Users Example

• An Oracle Database software owner (oracle1), which owns the Oracle Database binaries for DB1. The oracle1 user has the oraInventory group as its primary group, and the OSDBA group for its database (dba1) and the OSDBA for ASM group for Oracle Grid Infrastructure (asmdba) as secondary groups. In addition the oracle1 user is a member of asmoper, granting that user privileges to start up and shut down Oracle ASM.
• An OSDBA group (dba1). During installation, you identify the group dba1 as the OSDBA group for the database installed by the user oracle1. Members of dba1 are granted the SYSDBA privileges for the Oracle Database DB1. Users who connect as SYSDBA are identified as user SYS on DB1.
• An OSBACKUPDBA group (backupdba1). During installation, you identify the group backupdba1 as the OSDBA group for the database installed by the user oracle1. Members of backupdba1 are granted the SYSBACKUP privileges for the database installed by the useroracle1 to back up the database.
• An OSDGDBA group (dgdba1). During installation, you identify the group dgdba1 as the OSDGDBA group for the database installed by the useroracle1. Members of dgdba1 are granted the SYSDG privileges to administer Oracle Data Guard for the database installed by the useroracle1.
• An OSKMDBA group (kmdba1). During installation, you identify the group kmdba1 as the OSKMDBA group for the database installed by the useroracle1. Members of kmdba1 are granted the SYSKM privileges to administer encryption keys for the database installed by the user oracle1.
• An OSOPER group (oper1). During installation, you identify the group oper1 as the OSOPER group for the database installed by the useroracle1. Members of oper1 are granted the SYSOPER privileges (a limited set of the SYSDBA privileges), including the right to start up and shut down the DB1 database. Users who connect as OSOPER privileges are identified as user PUBLIC on DB1.
• An Oracle base /u01/app/oracle1 owned by oracle1:oinstall with 775 permissions. The user oracle1 has permissions to install software in this directory, but in no other directory in the /u01/app path.

6.1.11.3 Oracle Database DB2 Groups and Users Example

• An Oracle Database software owner (oracle2), which owns the Oracle Database binaries for DB2. The oracle2 user has the oraInventory group as its primary group, and the OSDBA group for its database (dba2) and the OSDBA for ASM group for Oracle Grid Infrastructure (asmdba) as secondary groups. However, the oracle2 user is not a member of the asmoper group, so oracle2 cannot shut down or start up Oracle ASM.
• An OSDBA group (dba2). During installation, you identify the group dba2 as the OSDBA group for the database installed by the user oracle2. Members of dba2 are granted the SYSDBA privileges for the Oracle Database DB2. Users who connect as SYSDBA are identified as user SYS on DB2.
• An OSBACKUPDBA group (backupdba2). During installation, you identify the group backupdba2 as the OSDBA group for the database installed by the user oracle2. Members of backupdba2 are granted the SYSBACKUP privileges for the database installed by the useroracle2 to back up the database.
• An OSDGDBA group (dgdba2). During installation, you identify the group dgdba2 as the OSDGDBA group for the database installed by the useroracle2. Members of dgdba2 are granted the SYSDG privileges to administer Oracle Data Guard for the database installed by the useroracle2.
• An OSKMDBA group (kmdba2). During installation, you identify the group kmdba2 as the OSKMDBA group for the database installed by the useroracle2. Members of kmdba2 are granted the SYSKM privileges to administer encryption keys for the database installed by the user oracle2.
• An OSOPER group (oper2). During installation, you identify the group oper2 as the OSOPER group for the database installed by the useroracle2. Members of oper2 are granted the SYSOPER privileges (a limited set of the SYSDBA privileges), including the right to start up and shut down the DB2 database. Users who connect as OSOPER privileges are identified as user PUBLIC on DB2.
• An Oracle base /u01/app/oracle2 owned by oracle1:oinstall with 775 permissions. The user oracle2 has permissions to install software in this directory, but in no other directory in the /u01/app path.

# useradd -u 54322 -g oinstall -G asmadmin,asmdba grid
# useradd -u 54321 -g oinstall -G dba1,backupdba1,dgdba1,kmdba1,asmdba,asmoper oracle1
# useradd -u 54331 -g oinstall -G dba2,backupdba2,dgdba2,kmdba2,asmdba oracle2