5 Installing Oracle Database 10g with Oracle Real Application Clusters

This chapter describes phase two of the installation procedures for installing Oracle Database 10g with Oracle Real Application Clusters (RAC). This chapter also describes some of the Oracle Universal Installer (OUI) features. This chapter contains the following topics:

Verifying System Readiness for installing Oracle Database with CVU
Selecting a Configuration Option
Installation of Oracle Database 10g with RAC Using Oracle Universal Installer
De-Installing Oracle Real Application Clusters Software

5.1 Verifying System Readiness for installing Oracle Database with CVU

To help verify that your system is prepared to install Oracle Database with RAC successfully, enter a Cluster Verification Utility (CVU) command using the following command syntax:

/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage -pre dbinst -n node_list [-r {10gR1|10gR2}] [-osdba osdba_group][-verbose]

In the preceding syntax example:

The variable mountpoint is the mountpoint of the installation media.
The variable node_list is the list of nodes in your cluster, separated by commas.
The optional flag -r takes the options 10gR1 (test for an Oracle Database 10g Release 1 Oracle Cluster Ready Services installation) or 10gR2 (test for an Oracle Database 10g Release 2 Oracle Clusterware installation); if the -r flag is missing, by default, the command tests for Oracle Clusterware for Oracle Database 10g Release 2 (10.2) installation.
The variable osdba_group is the name of your OSDBA group, which by default is dba.

For example, to perform a pre-installation check for Oracle Database 10g Release 2 (10.2) with RAC installation on a two-node cluster with the mountpoint /dev/dvdrom, with nodes node1 and node2, and with the OSDBA group dba, enter the following command:

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh stage -pre dbinst -n node1,node2 -verbose

Oracle recommends that you select the option -verbose to receive progress updates as the CVU performs its system checks. The -verbose option provides detailed test reporting, which you can use to identify the cause of any checks that fail.

If the cluster verification check fails, then review and correct the relevant system configuration steps, and run the test again. Use the system configuration checks described in "Troubleshooting Installation Setup for HP Tru64" (which follows this section) to assist you.

5.1.1 Troubleshooting Installation Setup for HP Tru64

If you run the CVU and your system fails system configuration checks, then review the the CVU report, and use the output to resolve failed configuration checks. Refer to the following list to assist you with responding to failed checks.

User Equivalence Check Failed

Cause: Failure to establish user equivalency across all nodes.

Action: Check user equivalence on each node for the oracle account.

Check for user equivalence for the oracle account by performing a remote login (rlogin) to each node in the cluster. For example:

# su - oracle
$ rlogin node_name

If you are prompted for a password, then you have not given the oracle account the same attributes on all nodes.

Check permissions on each node to ensure that the oracle account can write to the new mount points. For example:

# su - oracle
$ touch /u01/test
$ ls -l /u01/test
-rw-rw-r-- 1 oracle dba 0 Aug 15 09:36 /u01/test

As the oracle user, use SSH2 to connect from each node to itself, and from each node to each other node. For example, to check a two-node cluster with nodes named node1 and node2:

[oracle@node1 oracle]:$ ssh2 node1
[oracle@node1 oracle]:$ ssh2 node2
[oracle@node2 oracle]:$ ssh2 node2
[oracle@node2 oracle]:$ ssh2 node1
[oracle@node1 oracle]:$

If you are prompted for a password when logging in as the oracle user from one node to another in the cluster, then improper SSH configuration is a probable cause. Check to ensure that you have turned off X11 forwarding, and that the ~/ .ssh/id_rsa and ~/.ssh/dsa.pub files for all nodes are listed in the ~/.ssh/authorized_keys file that you create during pre-installation.

Node Reachability Check Failed

Cause: Failure of one or more nodes to be properly connected for communication.

Action: Possible causes of this message include the following:

Improper network configuration.
The node running CVU is unable to connect to one or more nodes in the cluster.

Use the following command to check your current configuration on each node:

ifconfig -a

Node Connectivity Check Failed: Cause: One or more of the cluster nodes is not able to be connected from all nodes in the cluster.; Action: Check for firewalls preventing the nodes from communicating on their private network interfaces.

User Existence Check Failed or User-Group Relationship Check Failed: Cause: The administrative privileges for users and groups required for installation are missing or incorrect.; Action: Use the id command on each node to confirm that the oracle account is created with the correct group membership.

See Also:
"Creating Required Operating System Groups and User" in Chapter 2, "Pre-Installation Tasks"

Shared Storage Check Failed: Cause: Shared storage access from each node in the cluster is not confirmed.; Action: If you are using shared disk storage, then as root, use the command fdisk -l to confirm the reachability of shared storage.

System Requirements Check Failed: Cause: Insufficient system resources, missing software packages, or other operating system or hardware problem.; Action: If you did not run the CVU command with the -verbose option, then run the command again using -verbose, and review the report to determine which system requirement failed. Correct the problem.

See Also:
Chapter 2, "Pre-Installation Tasks" contains instructions for completing any system requirement configuration that CVU lists as incomplete

5.2 Selecting a Configuration Option

This section describes OUI features that you should understand before beginning phase two of the RAC installation process.

See Also:

Appendix D, " Converting to Oracle Real Application Clusters from Single-Instance Oracle Databases" for information about how to convert single-instance databases to RAC

On the Select Configuration page, you are given the following options:

Create a database: Provides you with the option to create a database using a preconfigured database template designed for particular system load demands, as described in the section "Database Configuration Type Descriptions". If you select the Automatic Storage Management option for storage, then an ASM instance is installed as part of the database installation. The Oracle home is used for the ASM home when you install the database using this method. If you plan to use the ASM instance for multiple database homes, you should select the Configure Automatic Storage Management option to create a separate ASM home.
Configure Automatic Storage Management: Installs an ASM instance only. You are prompted for the ASM SYS password.
Install database software only: Installs Oracle Database software; you must complete database configuration later.

If you are installing Oracle Database software, then Oracle recommends that you use one of the preconfigured database options, or select the Advanced option on the Select Configuration page, and configure a custom starter database.

If you want to configure your environment and create your database manually, then select Do not create a database on the Select Configuration page, and refer to the manual database creation procedures posted at the following Web site:

http://otn.oracle.com

5.2.1 Automatic Storage Management Configuration

If you want to use Automatic Storage Management, then select Configure Automatic Storage Management (ASM), and provide information as prompted. Click Help if you need assistance.

5.2.2 Database Configuration Type Descriptions

When you run OUI, you can select the General Purpose, Transaction Processing , Data Warehouse, or Advanced database configuration type.

For the first three configuration types, you can complete additional procedures that are described later in this chapter. If you select Advanced configuration, then you can use Database Configuration Assistant (DBCA) to create the database as described in Chapter 6. Oracle recommends that you use DBCA to create your database.

5.2.2.1 General Purpose, Transaction Processing, and Data Warehouse Configuration Types

The General Purpose, Transaction Processing, and Data Warehouse configuration types use preconfigured database templates.

During installation, if you select one of the preconfigured database templates, then OUI starts Oracle Network Configuration Assistant (NETCA) and DBCA, and installs the preconfigured database without further input. During database installation, OUI displays a progress indicator.

DBCA processing for these three configuration types creates a starter database, and configures the Oracle network services. If you choose raw devices on the Specify Database File Storage Option page, then DBCA verifies that you configured the raw devices for each tablespace.

If you select Advanced configuration, then you must enter specific information as described in the next section.

5.2.2.2 Using the Advanced Configuration Type

If you select the Advanced configuration type, then OUI runs DBCA, which displays General Purpose, Transaction Processing, Data Warehouse, and a fourth configuration type, Custom Database.

The first three templates are customizable versions of the preconfigured database types. The Custom Database type creates a database without using preconfigured options.

The following section provides more detail about OUI and DBCA processing when creating a RAC database.

5.2.3 Actions of OUI, DBCA, and Other Assistants During Installation

After installation, OUI starts the NETCA. After the NETCA completes its processing, OUI runs DBCA to create your database using Optimal Flexible Architecture (OFA). This means that DBCA creates your database files, including the default server parameter file (SPFILE), using standard file naming and file placement practices. The primary phases of DBCA processing are the following:

Verify that you correctly configured the shared disks for each tablespace if you are using raw storage.
Create the database.
Configure the Oracle network services.
Start the listeners and database instances.

You can also use DBCA in standalone mode to create a database.

See Also:

Oracle Database Net Services Administrator's Guide if you experience problems (for example, with the listener configuration), and for further information about Lightweight Directory Access Protocol (LDAP) support

You can use your Oracle9i database language and territory definition files with Oracle Database 10g Release 2 (10.2) that you are about to install. To enable these functions, you must run OUI from a command line, as described in step 0 in "Installation of Oracle Database 10g with RAC Using Oracle Universal Installer", and use the following statement to set the b_cr9idata variable to true:

runInstaller oracle.rsf.nlsrtl_rsf:b_cr9idata=true

The remainder of this chapter explains how to use OUI to install Oracle Database 10g with Oracle Real Application Clusters (RAC).

5.3 Installation of Oracle Database 10g with RAC Using Oracle Universal Installer

Perform the following procedures to install the Oracle Database 10g software with RAC:

Start the runInstaller command from the DB directory of the Oracle Database 10g Release 2 (10.2) installation media. When OUI displays the Welcome page, click Next.
Provide information or run scripts as root when prompted by OUI. If you need assistance during installation, click Help. If you encounter problems during installation, then examine the OUI actions recorded in the installation log file. The log file is located in the Oracle Inventory directory (typically, oinstall), in the following location:
```
OraInventory/logs/installActionsdate_time.log
```
Note:
The Oracle home name and path that you provide during database installation must be different from the home that you used during the Oracle Clusterware installation in phase one. You must not install Oracle Database 10g with RAC software into the same home in which you installed the Oracle Clusterware software.

The following is a list of additional information to note about installation:
- If you are not using the ASM library driver (ASMLIB), and you select Automatic Storage Management (ASM) during installation, then ASM default discovery finds all disks that ASMLIB marks as ASM disks.
- If you are not using ASMLIB, and you select ASM during installation, then ASM default discovery finds all disks marked /dev/raw/* for which the oracle account has read/write permission. You can change the disk discovery string during the install if the disks you want to use for ASM are located elsewhere.
  
  If you are installing RAC from the Standard Edition, then you must use ASM for your database storage.
  
  On the Select Database Management Option page, if you have already completed the Grid Control Management Agent installation, then you can select either Grid or Local Database control. Otherwise, only Local Database control for database management is supported for RAC. When you use the local Database Control, you can choose the E-mail option and enter the outgoing SMTP server name and E-mail address.
  
  If you perform an installation that does not include Oracle Enterprise Manager (for example, a custom software install without Enterprise Manager, an installation with no Enterprise Manager configuration, or a database creation with your own scripts), then you can configure Enterprise Manager later with OUI, DBCA, or the Enterprise Manager Configuration Assistant (EMCA) utility.
  
  See Also:
  Enterprise Manager Grid Control Installation and Basic Configuration for details about installing Grid Control with OUI, and Enterprise Manager Advanced Configuration Guide for details about installing Database Control with DBCA and EMCA

When you have completed the second and final phase of the installation, proceed to Chapter 7, "Oracle Real Application Clusters Post-Installation Procedures" to perform the post-installation tasks.

Caution:

After installation is completed and you have created the database, if you decide that you want to install additional Oracle Database 10g products in the 10g Release 2 (10.2) database, then you must stop all processes running in the Oracle home before you attempt to install the additional products. For the Oracle Universal Installer to relink certain executables and libraries, all database processes must be down. Refer to Appendix F, "How to Stop Processes in an Existing Oracle Real Application Clusters Database, and How to Perform Oracle Clusterware Rolling Upgrades" for additional information.

5.4 De-Installing Oracle Real Application Clusters Software

If you need to de-install Oracle Real Application Clusters (RAC) software, then you must run OUI to de-install the software on the same node from which you performed the installation, and you must de-install Oracle Database software first before de-installing Oracle Clusterware software.

Perform the following procedures to de-install Oracle Database 10g RAC and Oracle Clusterware software, as described in the following sections:

De-Installing Oracle Database 10g RAC Software and ASM
De-Installing Oracle Clusterware

See Also:

Oracle Database Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide for more information about using RAC scalability features of adding and deleting nodes and instances from RAC databases, and for information about viewing OCR content

5.4.1 De-Installing Oracle Database 10g RAC Software and ASM

This section describes the procedure to de-install the Oracle Database 10g RAC software and Automatic Storage Management (ASM) software. Before you perform these steps, Oracle recommends that you make a backup of any databases that run from the Oracle home you are about to delete.

Examine the oratab file to identify the instance dependencies on this Oracle home.
Start DBCA, select Oracle Real Application Clusters Database, select Delete a database, and select the database that you want to delete. Repeat this step to delete all databases.
If you do not use ASM, then proceed to step 4.

If the database is in the Oracle home from which the ASM database runs, then ensure that there are no other database dependencies on these group of ASM instances, and then remove the ASM configuration by logging on as the oracle user and completing the following steps:
1. Connect to the ASM instance and run the following command to determine database instances using this ASM instance:
```
SQL> select INSTANCE_NAME from GV$ASM_CLIENT;
```
  Note:
  This command lists only database instances that are running. It is possible that other instances are associated with the ASM instance, but they are not currently running. If you removed a database from this Oracle home but the output from the command shows that this ASM instance is supporting a database instance in another Oracle home, then do not remove the ASM instance or the Oracle home.
  If your RAC database uses a different Oracle home from the Oracle home that your ASM environment uses, and you want to delete the RAC database, then perform step 5 in the RAC database home.
2. For each instance listed in the output of the statement you run in step a, stop the respective databases.
3. Oracle recommends that you back up the database files for all the databases that are currently using this ASM instance.
4. Using your connection to the ASM instance, run the following command:
```
SQL> select * from V$ASM_DISKGROUP;
```
5. For each disk group listed in the output of the statement you run in step d, run the following command:
```
SQL> drop diskgroup diskgroup_name including contents;
```
  The variable diskgroup_name is the name of the disk group that you want to delete.
6. Shut down ASM on all RAC nodes, and verify that all ASM instances are stopped.
7. If you are de-installing from a system with ASMLIB, then run the following procedure. First, run this command to list the disks that you need to delete:
```
oracleasm listdisks
```
  Second, run the following command to delete every disk listed by the previous command:
```
oracleasm deletedisks
```
  Third, re-run the oracleasm listdisks command to confirm that the disks are all deleted. Repeat this command on each of the RAC cluster nodes to confirm that the disks have been deleted from all the nodes.
  
  Fourth, as root, run the following commands on all nodes of your RAC cluster:
```
/etc/init.d/oracleasm stop/etc/init.d/oracleasm disable
```
8. Deconfigure ASM, using either DBCA in silent mode, or performing deconfiguration manually.
  
  To deconfigure ASM manually, complete steps i through l.
  
  To deconfigure ASM using DBCA, run DBCA in silent mode to deconfigure ASM. To run DBCA in silent mode, navigate to the directory $ORACLE_HOME/bin and use the following command syntax:
```
dbca -silent -deleteASM -nodelist node1,node2,...
```
  for the variables node1, node2, and so on, provide a list of all the nodes where ASM is configured. When DBCA completes de-installation, proceed to step 4.
9. To remove the ASM entry from the OCR, run the following command for all nodes on which this Oracle home exists:
```
srvctl remove asm -n nodename
```
  where nodename is the name of a node from which you want to remove the ASM instance.
10. If you are using a shared cluster file system for your Oracle home, then run the following commands on the local node:
```
rm -f $ORACLE_HOME/dbs/*ASM*
rm -r $ORACLE_BASE/admin/+ASM
```
  You may need to remove subordinate files or directories before these commands complete successfully.
11. If you are not using a shared cluster file system for your Oracle home, then run the commands from the previous step, step j, on each node on which the Oracle home exists.
12. Remove oratab entries for the deleted Oracle home databases.
If there are no outstanding dependencies, then change directory to the $ORACLE_HOME/bin directory, and use the following NETCA command syntax to remove the listener and its Oracle Clusterware resources.
```
$ ./netca /deinst /nodeinfo node1,node2,...
```
In the preceding syntax example, replace the variables node1, node2, and so on, with all the member nodes of the RAC database.

Note:
You cannot perform a RAC installation from the same OUI session in which you perform a RAC de-installation. In other words, if you de-install RAC with OUI and want to perform another RAC installation, then you must start a new OUI session.
Start OUI, and remove any existing Oracle Database 10g with RAC software by selecting Deinstall Products, and selecting the Oracle home that you want to remove. If any of these databases is an ASM database, then de-install the Oracle home with ASM where you want to de-install the Oracle Database software, and then de-install the Oracle Database. Removing ASM does not remove the ASM data or disk groups.

5.4.2 De-Installing Oracle Clusterware

De-install each Oracle Database 10g RAC home by running the procedure in the previous section, "De-Installing Oracle Database 10g RAC Software and ASM". Then complete the de-installation by removing the Oracle Clusterware software using the following procedures:

Run the command CRS_home/install/rootdelete.sh to disable the Oracle Clusterware applications that are running on the cluster node. The rootdelete.sh script requires three arguments. If you are running this command on a remote node of the cluster, then use remote as the first argument, otherwise use local as the first argument. If the ocr.loc file is on a shared file system, then use sharedvar. Otherwise use nosharedvar as the second argument. If the Oracle Clusterware home is on a shared file system, then use sharedhome. Otherwise, use nosharedhome as the third argument. Repeat this step on each node of the cluster from which you want to de-install Oracle Clusterware.

Note:
A node is a local node if you plan to run step 2 and step 3 on that node. Removing shared OCR contents and the Oracle Clusterware home is done on the local node after you complete removal of other nodes in the cluster.
Run the script CRS_home/install/rootdeinstall.sh on a local node to remove the OCR. You only need to run this script once.
Start OUI. In the Welcome page, click Deinstall Products to display the list of installed products. Select the Oracle Clusterware home to de-install.