Omarsoft For IT Solutions (Java Codes ,C#.NET Codes , ASP.NET Codes ,VB.NET Codes ,Oracle Database Administration, Real Application Cluster , Remote Support, Cloud Services , Networks ,Virtualization....
  • الأنظــمــة المكتبية        Windows App Programming
  • أنظــمـةالويــب        Web based systems programming
  • تطبيقات الهواتف الذكية     smartphones programming
  • إدارة قواعــــــد البيــــــــــــــــانات        Database Administration
  • إدارة الشبكـــــــــــــــــــــــــــــــــات        Networks Administration
  • إدارة الســـيــرفرات (ويب - محلية)  Servers Administration
  • إدارة مخـــــــــــــــــازن البيــــــــــــانات     Storage Administration
  •             N Computing & 2X Application services

    Social Icons

Loading...

Oracle 11g RAC Installation on Linux

Oracle 11g RAC Installation on Linux



Prepare all nodes for Cluster Software Installation:
Change /etc/hosts file to contain name of all nodes or DNS should be able to provide service name if you are not using SCAN (For 11gR2 release only).
Configure SSH
On both server run /usr/bin/ssh-keygen -t rsa from oracle account.
When prompted press enter (for now.)
all key files are located in ~/.ssh directory
1. id_rsa - is private key do not distribute.
2. id_rsa.pub is public key. (Copy this content to another server in authorzed_keys file.)
Similarly generate dsa key also and place in all nodes authorized_keys file.
Once copy is complete verify ssh is working from both server using
From node1 run ssh node2 date
From node2 run ssh node1 date

Verify that ntp server is runnin. Also make sure that both machine shows same time using following command:
date ;ssh anothernodename date
both should print same time.
If not use ntpdate command (check linux link).

Use RAID Software to create proper 3 LUNs  (check linux link for more detail.)

Configure public and private networks using /usr/sbin/system-config-network for network adapters.
Verify those adapter are up and running using ifconfig command.
if particular adapter is not up then command ifup to restart those adapter and make sure it shows correct IP addresses and those IPs are pingable.
ifup eth1
For private networks try to use network adapter that uses Interconnect/RDMA technology.

Make sure that firewall is not filtering UDP packet on private network (Because Oracle is using UDP packet for private network.)
 verify that using /etc/rc.d/init.d/iptables status command.
The following command display current iptables settings.
/sbin/iptables -L
Make backup of iptable files before you make any changes.
p>To take backup use /etc/init.d/iptables save command
then copy /etc/sysconfig/iptables BACKUPFILENAME
If you have Oracle Enterprise Linux installed then you can use Oracle public yum server to update:
To configure you can goto /etc/yum.repos.d and try following command
public-yum-el5.repo, public-yum-el4.report or public-yum-ovm2.repo depending on your configuration and set enabled=1 accordingly.
Once updated correctly you can see list of packages by issuying
yum list

If vncserver is installed on server and would like to use it then you have to start vncserver if it is not running.
/usr/bin/vncserver
first time it will ask you for password.
To enable desktop on vnc server change file /root/.vnc/xstartup.
#Uncomment the following two lines for normal desktop :
unset SESSION_MANAGER
exec /etc/X11/xinit/xinitrc

To generate kickstart file you can always use kickstart program.
system-config-kickstart
To add/remote program on Linux with gui you can use pirut program.
pirut
You can copy all installation files on drive using following command.
mkdir /mnt/iso
mount -p loop disk1.iso /mnt/iso
copy all files to one location for example /root/binaries/linux/RPMS/
unmount /mnt/iso
Repeat above instruction for all CDs and copy it in same directory.
or use DVD drive to mount DVD media
cd /mnt
mkdir cdrom
mount /dev/cdrom /mnt/cdrom

Once files are copied you can change directory to repodata under RPMS and run following command (optional)
createrepo -g comps-rhel5-server-core.xml .
Edit yum config file from /etc/yum.repos.d and add [oel5] tag and its content in file.

[oel5]
name=Enterprise Linux 5
baseurl=file:///root/binary/linux/RPMS
enabled=1
gpgcheck=0
 
Then run yum clean all and run pirut that should give you original installation screen.

Verify partition using cat /proc/partition command.

For EMC Fibre Channel storage make sure that  EMC Navisphere agent is installed on each node. And on storage make sure that Navisphere software assign the correct storage group to each node. 
If you have SCSI storage then those devices display as sdb, sdc and so on. The LUNs on Fibre Channel storage or SCSI enclosure should also be seen as SCSI devices.
With one RAID container it should show as sda and if you have two Logical Disk Groups then those will be display as sdb and sdc. If you have Fibre Channel Controller that should show as sda and RAID group on on storage will show as emcpowera and emcpowerb as pseudo devices.
Make sure that all nodes on cluster is able to see same number of LUNs. (Duplicate mount of same lun is also not allowed so if you use multiple device path to access same storage Oracle will error out.)
If you dont see any external storage devices for a Fibre Channel system thenstop services by following commands:
service naviagent stop
service PowerPath stop
Then sync HBA driver by reloading it using
rmmod qla2300
modprobe qla2300
Above example is for QLogic HBAs
For Emulex HBA
rmmod lpfcdd
modprobe lpfcdd
Then restart services using
service PowerPath start
service naviagent start
For SCSI enclosure reboot all nodes.
For iSCSI storage use following command to stop start services
service iscsi stop
service iscsi start

Before start installation set host equivalence for oracle user from all nodes otherwise it will give an error PRVF-4007. 
exec /usr/bin/ssh-agent $SHELL
/usr/bin/ssh-add

Create directory strcture for OFA on both nodes.
mkdir -p /u01/app/oracle
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle

To install Oracle ASM libraries for Linux, make sure that you are using version that match with your kernel, use uname -r command to find your kernel information. Install the following three packages using root user on all nodes.
  • oracleasm-support - contains binary files (/usr/sbin/*) for supporting oracleasm
  • oracleasm modules - which is kernel loadable module
  • oracleasmlib - Library files needed to support oracleasm
rpm -ivh oracleasm-support-2.1.3-1.el5.i386.rpm
rpm -ivh oracleasm-2.6.18-194.el5-2.0.5-1.el5.i686.rpm
rpm -ivh oracleasmlib-2.0.4-1.el5.i386.rpm
Display configure oracle asm using
/usr/sbin/oracleasm configure
Configure oracle asm using oracleasm -i parameter
/usr/sbin/oracleasm configure -i
default user : oracle
default group : oinstall
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y] : y
This load oracle ASMLib driver filesystem, creates /etc/sysconfig/oracleasm file and creates /dev/oracleasm mount point.
If there is any problem loading module then check /var/log/oracleasm log file to find out more to resolve issue.
/usr/sbin/oracleasm init to load driver.
You should see message Mounting ASMLib driver filesystem: /dev/oracleasm
In case if it fails then it will log messages like Unable to load modules "oracleasm" then run up2date -i oracleasm-`uname -r`

To add package using rpm use
rpm -ivh packagename.rpm
and to remove package use
rpm -e packagename.rpm
In case if you want to find out which packages are loaded use rpm -qa |grep -in 'oracle'

Prepare ASM volumes
Use fdisk /dev/sde to partition the volume.
create new partition using n command when ask for partittion type select primary.
select first partition by selecting 1 and then create from 1st Cylinder by typing First Cylinder value : 1
Last Cylinder or +size or +sizeM or +sizeK (xxx) :512
similarly create another partition of same size but partition number 2. (from 512 to 1024)
Once all partition created run
partprobe 
By using partprobe command if you are using SCSI LUN then ownership will change for kernel 2.6 and above
Then assign those partition to raw device using raw command
raw /dev/raw/raw1 /dev/sdc1
raw /dev/raw/raw2 /dev/sdd2
Perform similar action on other nodes based on their mount point(sometime it may be different because their number of disk may be different.) so finally map it on /dev/raw/rawN correctly so from all node it can access.
Also add entry in /etc/sysconfig/rawdevices
/dev/raw/raw1 /dev/sdc1
/dev/raw/raw2 /dev/sdd2
So next time it boot it will autogenerate those devices.
If you have kernel 2.6 then create file /etc/udev/rules.d/99-raw.rules (for Oracle RHE5, OEL5, and SLES9)
For example:
KERNEL=="raw[1-2]*", OWNER="oracle",GROUP="oinstall", MODE="640"
First line will take raw1 and raw2 and change ownership to oracle:oinstall and set mode to 640.
now add those disk in asm by giving the following command (You may do this for 11gR2 because OCR and Voting Disk can be placed on ASM).
/usr/sbin/oracleasm createdisk DG_CRS1 /dev/raw/raw1
To display list use
/usr/sbin/oracleasm listdisks
/usr/sbin/oracleasm configure/init/exit/scandisk/status/listdisks/querydisk/createdisk/deletedisk/renamedisk/update-driver are options
In case if you have already existing asm disk then (in case of backup restored) use /usr/sbin/oracleasm scandisk (usually from other nodes.)


Try following command to start Oracle Universal Installer.
xhost +
./runInstaller &
On Welcom click next
Specify Inventory directory and credentials
Enter Full path of the inventory directory : /u01/app/oraInventory
Specify Operating System group name :  oinstall
Click next
You will see Specify Home Details
Destination
Name: OraCrs11g_Home
Path : /u01/crs
Click next
Product-specific Prerequisite Checks will verify all setup. if anything failed then correct that and come back on this screen.
Click next
Specify Cluster Configuration
Cluster Name : clustername_cluster
Cluster Nodes : node01.domain.com
Private Nodes : node01-priv.domain.com
Virtual Host Name : node01-vip.domain.com
Click on Add to add more nodes . for example this is two cluster nodes then

Cluster Nodes : node02.domain.com
Private Nodes : node02-priv.domain.com
Virtual Host Name : node02-vip.domain.com
Then click OK
Click Next.
Specify Network Interface Usage
Make sure that at least one adapter is public and another adapter is set to private.
If you have network adapter that are not in use or belongs to another purpose then select do not use by clicking on edit.
If Interface_type is wrong for given adapter then change it by clicking on Edit and select from Interface Type ( Public / Private / Do Not Use).
Then click Next
OCR Config : /dev/raw/raw1
Voting Disk Location : External Redundancy : /dev/raw/raw2 (Usually these should be on different set of disks)
Next
Make sure that both nodes under Remote Nodes are checked before resuming further.
Next.
Provide OCR and CRS drive path /dev/raw/raw1 and /dev/raw/raw2 and click Next.
After installation it will ask you to run script file /u01/app/crs/root.sh
Oracle recommend that you install OCR, OCR Mirror and Voting Disks on separate set of disks. It will impact your performance if these devices are slow.
In case if you get Shared Librarires /u01/app/crs/lib/libnnz11.so: cannot restore segment prot after reloc: Permission denied 
Failed to upgrade Oracle Clusterware Registry configuration 
error then run the following command (Usually CentOS):
getenforce
This will print as Enforcing.
After this set to permission
setenforce 0
Run getenforce and it should display Permissive and /etc/sysconfig/selinux and change SELINUX=disabled or permissive.
If vip or any node address is set wrong then you can correct it.
Make sure everything shutdown except crs stack
srvctl stop database -d dbname
srvctl stop nodeapps -n node1
srvctl stop nodeapps -n node2
To display command run as oracle user :
oifcfg getif
oifcfg delif -global eth0
oifcfg setif -global eth0/10.0.0.1:public
To modify VIP address run as root
/u01/app/crs/bin/srvctl modify nodeapp -n node1 -A 10.0.0.101/255.255.255.0/eth0
Then shutdown crs using crsctl stop crs command and restart all nodes.

Create data disk if you have not create earlier (The following example create DATA1DG1 disk group on Coraid storage).
/etc/init.d/oracleasm createdisk DATA1DG1 /dev/etherd/e0.2p3
Change directory to oracle binaries directory cd binaries/database
./runInstaller &
Welcome screen click Next
Select Custom click Next
Select Oracle Base /u01/app/asm
Name OraDb11g_home1
Path /u01/app/asm/product/11gr1
Click next
Specify Hardware Cluster Installation Mode
click all nodes then next
 On product-specifc Perequisite Checks click Next if pass.
On Available product Components screen please select the components you have license for then click Next.
Select Privileged Operating System Groups
Select database administrator (OSDBA) Group : dba
Select Databae Operator (OSOPER) Group : dba
Select ASM administrator (OSASM) Group : dba
Click Next
On Create Database screen select Configure Automatic Storage Management (ASM) then click Next.
On summary page make sure that all Remote Nodes are checked then click Install
When binaries are installed on Configuraton Assistant page You will configure Listener using Oracle Net Configuration Assistant
Type LISTENER name as LISTENER clic Next
On next screen select No, I do not want to configure additional naming methods.
Database Configuration Assistant, Step 1 of 3 type SYS password and Create server parameter file (SPFILE) then clic Next
ASM Creation - It will create ASM Instance.
Run /u01/app/asm/product/11gr1/root.sh on both nodes
Click Exit on End of Installation.

ORACLE PRVF-5436 : The NTP daemon running on one or more nodes lacks he slewing option "-x"
To resolve this editing /etc/sysconfig/ntpd to
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"
NTPDATE_OPTIONS=""
add
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"

Sana'a Yemen - 50th st.

+967 738166685

omar.soft2000@gmail.com

للاتصال بنا CONTACT US

الاسم Name

بريد إلكتروني Email *

رسالة Message *

2015-2023 © All Rights Reserved Omarsoft
Back To Top