RAC在OCR磁盘故障且有备份时的恢复操作

问题描写：

CRS启动失败，并且RAC进程全部退出。

查看messages日志：
... ...
Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.
Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.
Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.
Apr 10 15:59:31 node74 su: (to oracle) root on none
Apr 10 15:59:31 node74 su: (to oracle) root on none
Apr 10 15:59:31 node74 su: (to oracle) root on none
Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.
Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.
Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.
Apr 10 16:00:26 node74 su: (to oracle) root on /dev/pts/0
Apr 10 16:00:31 node74 su: (to oracle) root on none
Apr 10 16:00:31 node74 su: (to oracle) root on none
Apr 10 16:00:31 node74 su: (to oracle) root on none
Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.
Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.
Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.
... ...

取其中1个文件下来，/tmp/crsctl.7174：

Oracle Cluster Registry initialization failed with invalid format: PROC⑵2: The OCR backend has an invalid format

怀疑是OCR破坏。运行crsctl check boot命令，与上面的日志报错是1致的：

oracle@node74:~> crsctl check boot
Oracle Cluster Registry initialization failed with invalid format: PROC⑵2: The OCR backend has an invalid format

问题处理：

查看是不是有OCR自动备份，很不幸，没有结果输出：

oracle@node74:~> ocrconfig -showbackup
---空

幸亏之前做OCR迁移实验时，有做过OCR的手动备份。

node74:/home/database/oracle/oracrs/bin # ll /home/oracle/ocr.exp
-rw-r--r-- 1 root root 87246 Dec 29 16:45 /home/oracle/ocr.exp

导入之前备份的OCR：

node74:/home/database/oracle/oracrs/bin # ./ocrconfig -import /home/oracle/ocr.exp

检查OCR：

node74:/home/database/oracle/oracrs/bin # ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :     975584
         Used space (kbytes)      :       3844
         Available space (kbytes) :     971740
         ID                       :  811637621
         Device/File Name         : /dev/raw/raw1
                                    Device/File integrity check succeeded

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

---修复成功

启动CRS：         

node74:/home/database/oracle/oracrs/bin # /etc/init.d/init.crs start

所有RAC进程启动成功：

node74:~ # su - oracle
oracle@node74:~> crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....SM1.asm application    ONLINE    ONLINE    node74      
ora....74.lsnr application    ONLINE    ONLINE    node74      
ora.node74.gsd application    ONLINE    ONLINE    node74      
ora.node74.ons application    ONLINE    ONLINE    node74      
ora.node74.vip application    ONLINE    ONLINE    node74      
ora....SM2.asm application    ONLINE    ONLINE    node76      
ora....76.lsnr application    ONLINE    ONLINE    node76      
ora.node76.gsd application    ONLINE    ONLINE    node76      
ora.node76.ons application    ONLINE    ONLINE    node76      
ora.node76.vip application    ONLINE    ONLINE    node76      
ora.orcl.db    application    ONLINE    ONLINE    node74      
ora....l1.inst application    ONLINE    ONLINE    node74      

ora....l2.inst application    ONLINE    ONLINE    node76   

--问题解决