How to Configure Oracle RAC Backup Disk Storage

Information about Configuring Oracle RAC Backup Disk Storage

This document describes how EMC SRV storage is configured as DISK Backup storage on all RAC Nodes.

Document Creator Modified

Todd Walters 3/22/2010

Production Overview

• Oracle RAC Production Linux Servers have a /export/home/oracle/backup filesytem that is composed of 4 500gb SATAII drives on the SRV. They are attached as /dev/emcpowerl and the LUN is only attached to csrv0dbx03.  
• csrv0dbx03 then exports with NFS the /export/home/oracle/backup filesystem to the other two RAC Nodes, csrv0dbx01 and csrv0dbx02./dev/emcpowerl is EMC Pseudo device for LUN 237.
• LUN 237 is only bound to EMC Storage Group dbx03
• Raid Group 237 consistes of 4 500gb SATA II drives. The fifth SATA II is Hot Spare
• /dev/emcpowerl is mounted to /export/home/oracle/backup on csrv0dbx03 and is in the /etc/fstab file.
• /export/home/oracle/backup is exported to csrv0dbx01 and csrv0dbx02 through the /etc/exports file
• NFS Service on csrv0dbx03 is set to startup automatically at boot time via chkconfig.
• /export/home/oracle/backup is owned by oracle:dba

Dev/Test/Srvdbox Overview

• Dev/Test: Created 50gb vmdk file on local ESXi datastore esxihost-vmfs02/oracle/ [nfsback.vmdk,nfsback-flat.vmdk]
• Add disk to last node of Cluster and place disk on scsi1 and last target = 1:15
• Mounted /dev/sdl1 on node 3 to /export/home/oracle/backup for each environment.
• Exported to other nodes in respective env. (dev/test/srvdbox)
• Srvdbox:  Oracle 20gb  Backup Disk = mounted on csrv0dbox92 at /export/home/oracle/backup.
• Srvdbox disk resides on ESXi Test host at /vmfs/volumes/lesxit01-vmfs02/oracle/srvdbox. Attached to csrv0dbox9 at SCSI 1:15 and /dev/sdl1  and backup Exported to csrv0dbox1 via NFS and mounted to /export/home/oracle/backup

Backup Schedule

• dbx03 - full backup on Sunday, 1PM; Incremental on Wednesday, 2AM (on hold now)
• dbx01 - full backup on Sunday, 2PM
• dbx02 - full backup on Sunday, 11am 

How to Configure Backup Disk and Mounting them on all RAC Nodes

1. Login to EMC Navisphere at 10.67.2.91
2. Create Raid Group #237 consisting of four remaining 500gb SATA drives as RAID5
3. Right click on newly created Raid Group and choose Add to Storage Group
4. Add dbx03 storage group to Raid Group 237
5. Click 'Update Host Information'  and you should now see csrv0dbx03 show up in RAID Group 237
6. Choose File > Logout  to exit EMC Navisphere Console.
7. ssh to csrv0dbx03
8. su -  to switch to root user
9. cd /opt/cci/sbin and run lun_scan.sh all to scan for new LUNS
10. start and stop Navisphere Agent and PowerPath
• /etc/init.d/naviagent stop
• /etc/init.d/PowerPath stop
• /etc/init.d/naviagent start
• /etc/init.d/PowerPath start
11. Run powermt to determine pseudo device name for LUN 237
• powermt display dev=all | more
• look for LUN 237 it should be assigned /dev/emcpowerl
• [root@csrv0dbx03 ~]# powermt display dev=all | more
  Pseudo name=emcpowerl
  CLARiiON ID=APM00072801596 [dbx03-sg]
  Logical device ID=60060160F5701D000A55BDF01B84DD11 [LUN 237]
12. Make new filesystem on new device
•   mkfs -t ext3 /dev/emcpower1
13. Make directory in /export/home/oracle
• mkdir /export/home/oracle/backup
14. Change ownership of /export/home/oracle/backup
• chown -R oracle:dba /export/home/oracle/backup
15. Mount new filesystem
• mount /dev/emcpowerl /export/home/oracle/backup
16. Verify filesystem mounted
• df -h/dev/emcpowerl        1.4T  102M  1.3T   1% /export/home/oracle/backup
17. Add filesystem to /etc/fstab to automatically mount after reboot
• vi /etc/fstab and add the following line:
  /dev/emcpowerl  /export/home/oracle/backup      nfs     defaults 0 0
18. Umount backup and remount to verify /etc/fstab file is  configured properly
• umount /export/home/oracle/backup
• mount  :to view what is currently mounted: should NOT see /export/home/oracle/backup mounted.
• mount -a :to reread /etc/fstab
• mount   :to view again and should see /export/home/oracle/backup mounted
19. Configure /etc/exports to export /export/home/oracle/backup only to the other two RAC Nodes
• vi /etc/exports to look like this:
• [root@csrv0dbx03 ~]# cat /etc/exports
  /export/home/oracle/backup      csrv0dbx01.mysite.com(rw,sync,no_root_squash) csrv0dbx02.mysite.com(rw,sync,no_root_squash)
20. Check to see if NFS is setup to start at boot time. If not, configure it to start at boot.
• [root@csrv0dbx03 ~]# chkconfig --list nfs
• chkconfig --level 345 on nfs  :to setup NFS to startup at boot.
• [root@csrv0dbx03 ~]# chkconfig --list nfs : again to verify setting is set to ON
21. Start NFS daemon
• service nfs start
22. Verify NFS is being exported only to the two RAC Nodes

• [root@csrv0dbx03 ~]# exportfs/export/home/oracle/backup
                  csrv0dbx01.mysite.com
  /export/home/oracle/backup
                  csrv0dbx02.mysite.com
• Now that /export/home/oracle/backup is being exported, it's time to mount them on the remote RAC Nodes
• ssh to csrv0dbx01
• vi /etc/fstab file and add the following line:

• csrv0dbx03:/export/home/oracle/backup   /export/home/oracle/backup nfs  rw,bg,hard,nfsvers=3,tcp,rsize=32768,wsize=32768,timeo=300,actimeo=0 0 0
• Make the backup directory and change ownership.
• mkdir /export/home/oracle/backup
• chown -R oracle:dba /export/home/oracle/backup
• Run mount command to verify /etc/fstab file is configured properly.
• type mount and you should not see csrv0dbx03 in the list
• type mount -a to reread the /etc/fstab file
• type mount again and you should now see csrv0dbx03:/export/home/oracle/backup in the list
• Repeat Steps 24-27 for node 2 csrv0dbx02
• Let DBA know that /export/home/oracle/backup location is now configured.

home | oracle | templates | about