mirror 10gR2 OCR and voting disks

During the ClusterWare installation phase, there is an option to choose mirroring of OCR and Voting Disks. As we did not mirror them, we did it. Here is the instruction for doing it.


VOTING DISKS
1. Login as root on pind22
2. . /home/oracle/.profile_crs ## source env
3. crsctl query css votedisk ##should return /dev/raw/raw6
4. take a backup: dd if=/dev/raw/raw6 of=/data/oradata/votingdisk.linn.bk
5. ls -al /dev/raw/* ##check pending disk device
6. add /dev/raw/raw7 and /dev/raw/raw8 as mirrors of voting disks online.
crsctl add css votedisk /dev/raw/raw7
crsctl add css votedisk /dev/raw/raw8
7. if adding online failed
crsctl stop crs ###on pind21/22, very important, need to shutdown crs on all cluster nodes
crsctl add css votedisk /dev/raw/raw7 -force
crsctl add css votedisk /dev/raw/raw8 –force
crsctl start crs ###on pind21/22
8. crsctl query css votedisk ## should return /dev/raw/raw6, /dev/raw/raw7, /dev/raw/raw8

OCR
1. login as root on pind22
2. . /home/oracle/.profile_crs ## source env
3. ocrcheck ## should return /dev/raw/raw5
4. ls –al /dev/raw/* ## /dev/raw/raw9 is pending disk device
5. ocrconfig –export /data/oradata/ocr.linn.dmp –s online ## for backup
6. ocrconfig –replace ocrmirror /dev/raw/raw9 ## add ocr mirror
7. ocrcheck ## should return /dev/raw/raw5 and /dev/raw/raw9 as ocr disk device


Output log:

[root@pind22 ~]# cd /home/oracle/
[root@pind22 oracle]# . .profile_crs
crs.root.pind22>crsctl query css votedisk
0. 0 /dev/raw/raw6

located 1 votedisk(s).
crs.root.pind22>dd if=/dev/raw/raw6 of=/data/oradata/votingdisk.linn.bk
2064384+0 records in
2064384+0 records out
1056964608 bytes (1.1 GB) copied, 399.387 seconds, 2.6 MB/s
crs.root.pind22>ls -al /dev/raw/*
crw-r----- 1 root oinstall 162, 5 Aug 13 10:45 /dev/raw/raw5
crw-rw-rw- 1 oracle oinstall 162, 6 Aug 24 13:18 /dev/raw/raw6
crw-rw-rw- 1 oracle oinstall 162, 7 Aug 13 10:45 /dev/raw/raw7
crw-rw-rw- 1 oracle oinstall 162, 8 Aug 13 10:45 /dev/raw/raw8
crw-r----- 1 root oinstall 162, 9 Aug 13 10:45 /dev/raw/raw9
crs.root.pind22>crsctl add css votedisk /dev/raw/raw7
Cluster is not in a ready state for online disk addition
crs.root.pind22>crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
crs.root.pind22>crsctl add css votedisk /dev/raw/raw7
Cluster is not in a ready state for online disk addition
crs.root.pind22>crsctl add css votedisk /dev/raw/raw7 -force
Now formatting voting disk: /dev/raw/raw7
successful addition of votedisk /dev/raw/raw7.
crs.root.pind22>crsctl add css votedisk /dev/raw/raw8 -force
Now formatting voting disk: /dev/raw/raw8
successful addition of votedisk /dev/raw/raw8.
crs.root.pind22>crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly
crs.root.pind22>crsctl query css votedisk
0. 0 /dev/raw/raw6
1. 0 /dev/raw/raw7
2. 0 /dev/raw/raw8

located 3 votedisk(s).
crs.root.pind22>

crs.root.pind22>ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 1031920
Used space (kbytes) : 3320
Available space (kbytes) : 1028600
ID : 330979008
Device/File Name : /dev/raw/raw5
Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

crs.root.pind22>ls -al /dev/raw/*
crw-r----- 1 root oinstall 162, 5 Aug 24 13:24 /dev/raw/raw5
crw-rw-rw- 1 oracle oinstall 162, 6 Aug 24 13:25 /dev/raw/raw6
crw-rw-rw- 1 oracle oinstall 162, 7 Aug 24 13:25 /dev/raw/raw7
crw-rw-rw- 1 oracle oinstall 162, 8 Aug 24 13:25 /dev/raw/raw8
crw-r----- 1 root oinstall 162, 9 Aug 13 10:45 /dev/raw/raw9
crs.root.pind22>ocrconfig -export /data/oradata/ocr.linn.dmp -s online
crs.root.pind22>ocrconfig -replace ocrmirror /dev/raw/raw9
crs.root.pind22>ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 1031920
Used space (kbytes) : 3320
Available space (kbytes) : 1028600
ID : 330979008
Device/File Name : /dev/raw/raw5
Device/File integrity check succeeded
Device/File Name : /dev/raw/raw9
Device/File integrity check succeeded

Cluster registry integrity check succeeded

crs.root.pind22>


Due to bug 3972986 (see ref 2), Cluster is not in a ready state for online disk addition. Need to shutdown all databases and ASM, and then shutdown CRS.



REF:

1. OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE), including moving from RAW Devices to Block Devices. [ID 428681.1]
2. OCR Corruption after Adding/Removing voting disk to a cluster when CRS stack is running [ID 390880.1]