Release 1 (9.0.1) for UNIX Systems: AIX-Based Systems, Compaq Tru64 UNIX, HP 9000 Series HP-UX, Linux Intel and Sun SPARC Solaris
Part Number A90347-02
 Contents |
 Index |
| Oracle9i Administrator's Reference Release 1 (9.0.1) for UNIX Systems: AIX-Based Systems, Compaq Tru64 UNIX, HP 9000 Series HP-UX, Linux Intel and Sun SPARC Solaris Part Number A90347-02 |
|
Oracle Cluster Management Software (OCMS) is available with Oracle9i on Linux systems. This appendix contains the following sections:
OCMS is included with the Oracle9i Enterprise Edition for Linux. It provides cluster membership services, a global view of clusters, node monitoring, and cluster reconfiguration. It is included as a part of Oracle9i Real Application Clusters on Linux and is installed automatically when you choose Oracle9i Real Application Clusters. OCMS consists of the following components:
Figure E-1 shows how the Watchdog daemon provides services to the Cluster Manager and to the Node Monitor. The Node Monitor passes node-level cluster information to the Cluster Manager. The Cluster Manager passes instance-level cluster information to the Oracle instance.
The Watchdog daemon (watchdogd) uses the standard Linux Watchdog timer to monitor selected system resources to prevent database corruption.
The Watchdog daemon monitors the Node Monitor and the Cluster Manager and passes notifications to the Watchdog timer at defined intervals. The behavior of the Watchdog timer is partially controlled by the CONFIG_WATCHDOG_NOWAYOUT kernel configuration parameter.
Oracle9i Real Application Clusters requires that you set the value of the CONFIG_WATCHDOG_NOWAYOUT configuration parameter to Y (disable watchdog shutdown on close). When the Watchdog timer detects an Oracle instance or service failure, it resets the server to avoid possible corruption of the database. If the value of the CONFIG_WATCHDOG_NOWAYOUT parameter is N and a failure is detected, the Watchdog timer does not reset the server.
Oracle9i Real Application Clusters uses the software implementation of the Watchdog timer provided by the Linux kernel.
The Node Monitor (oranm) maintains a consistent view of the cluster, and reports the status of the nodes in the cluster to the Cluster Manager. The Node Monitor uses a heartbeat mechanism. During normal operations, if the heartbeat mechanism fails the Node Monitor uses a quorum partition on the shared disk to distinguish between a node failure and a network failure.
The Node Monitors on all nodes in a cluster send heartbeat messages to each other. Each node maintains a database containing status information on the other nodes. The Node Monitors in a cluster mark a node inactive if the node fails to send a heartbeat message within a defined time interval.
The heartbeat message from the Node Monitor on a remote server can fail for the following reasons:
From each cluster node, the Node Monitor periodically updates the designated block on the quorum partition. Other nodes check the timestamp for each updated block. If the heartbeat is dead and the block timestamp is current, the network has failed.
If a node in a cluster stops sending heartbeat messages but continues writing to the shared raw partition, the Node Monitors on other nodes recognize that a network failure occurred. The Node Monitor reconfigures the cluster to terminate the isolated nodes, ensuring that the remaining nodes in the reconfigured cluster continue to function properly.
Abnormally heavy I/O loads can slow down the transmission of heartbeat messages and might indicate a node failure. The Node Monitor works with the Watchdog daemon to stop the node with the abnormally heavy load.
The Cluster Manager (oracm) maintains the process-level cluster status. The Cluster Manager accepts registration of Oracle instances to the cluster and provides a consistent view of Oracle instances. The Cluster Manager also propagates status information to all the Oracle instances, enabling communication among instances.
If the LMON process or another Oracle process that can write to the shared disk quits abnormally, the Cluster Manager daemon on the node detects it and requests the Watchdog daemon to stop the node completely. This stops the node from issuing physical I/O to the shared disk before Cluster Manager daemons on the other nodes report the cluster reconfiguration to Oracle instances on the nodes. This action prevents database corruption.
The following sections describe how to start OCMS:
To start the Watchdog daemon, enter the following commands:
$ su root # cd $ORALE_HOME/oracm/bin # watchdogd -g dba
The default location of the Watchdog log file is $ORACLE_HOME/oracm/log/wdd.log.
The Watchdog daemon does not have configuration files. Table E-1 describes the arguments that you can use when starting the Watchdog daemon.
To configure the Node Monitor, create the nmcfg.ora file in the $ORACLE_HOME/oracm/admin directory on each node and set the following parameters:
/etc/hosts file before installing Oracle9i. For example, enter the following where node1, node2, node3, and node4 are the host names of the nodes in the cluster:
DefinedNodes=node1 node2 node3 node4
/dev/raw1, enter the following:
CmDiskFile=/dev/raw1
/etc/hosts file before installing Oracle9i. For example, enter the following where node1 is the host name used for internode communication:
CmHostName=node1
$ORACLE_HOME/oracm/admin directory.
Table E-2 lists all of the configurable Node Monitor parameters in the nmcfg.ora file.
| Parameter | Valid Values | Default Value | Description |
|---|---|---|---|
|
AutoJoin |
0 or 1 |
0 |
If this parameter is set to 1, the Node Monitor joins the cluster when the Node Monitor starts. The default action is that the Node Monitor joins the cluster when the Cluster Manager requests to join the cluster. |
|
CmDiskFile |
Up to 256 characters |
No default value. You must set the value explicitly. |
Pathname of the quorum partition. |
|
CmHostName |
Up to 256 characters |
Host name of the local node. |
Store the local host name for private networking. Define the name in the |
|
CmServiceName |
Up to 256 characters |
CmSrvr |
Service name to be used for communication among the Node Monitors. If the Node Monitor cannot find the service name in the |
|
CmServicePort |
1 to 65535 |
60001 |
Port number to be used for communication among Node Monitors when the CmService Name parameter cannot designate the port number. |
|
DefinedNodes |
Up to 4096 characters |
No default value. You must set the value explicitly. |
List of host names, separated by spaces, of all the nodes in the cluster. |
|
MissCount |
2 to 1000 |
3 |
When the Node Monitor finds that a node failed to send a heartbeat message within the time specified by adding the value of the MissCount parameter and the value of the PollInterval parameter, the Node Monitor defines the node as dead. |
|
PollInterval |
10 to 180000 milliseconds |
1000 |
Sends heartbeat messages at this interval. |
|
WatchdogMarginWait |
See "Configuring Timing for Cluster Reconfiguration" .
|
70000 |
Specifies the delay between a node failure and the commencement of Oracle9i Real Application Clusters cluster reconfiguration. |
To start the Node Monitor:
$ ps -elf | grep watchdogd
root user, start the Node Monitor as a background process. Redirect the output to a log file (although output is not normally expected).
The following example shows how to start a Node Monitor service:
$ su root # cd $ORACLE_HOME/oracm/bin # oranm </dev/null >$ORACLE_HOME/oracm/log/nm.out 2>&1 &
In the preceding example, all of the output messages and error messages are written to the $ORACLE_HOME/oracm/log/nm.out file.
The oranm process spawns multiple threads. You can list all the threads by entering the ps -elf command.
Table E-3 describes the arguments of the oranm executable.
Perform the following steps to start the Cluster Manager:
nmcg.ora file is listed in the /etc/hosts file.
root user, start the oracm process as a background process. Redirect any output to a log file. For example, enter the following:
$ su root # cd $ORACLE_HOME/oracm/bin # oracm </dev/null >$ORACLE_HOME/oracm/log/cm.out 2>&1 &
In the preceding example, all of the output messages and error messages are written to the $ORACLE_HOME/oracm/log/cm.out file.
The oracm process spawns multiple threads. To list all the threads, enter the ps -elf command.
Table E-4 describes the arguments of the oracm executable.
To avoid database corruption when a node fails, there is a delay before the Oracle9i Real Application Clusters reconfiguration commences. Without this delay, simultaneous access of the same data block by the failed node and the node performing the recovery can cause database corruption. The length of the delay is defined by WatchdogMarginWait parameter. By default, the time between when the failure is detected and the start of the cluster reconfiguration is 70 seconds.
The value off the WatchdogMarginWait parameter must be greater than the value of the Watchdog daemon -m argument plus the value of the soft_margin parameter.
|
See Also:
See "Starting the Watchdog Daemon" for information on the Watchdog daemon |
If you decrease the value of the WatchdogMarginWait parameter, you must ensure that the sum of the value of the Watchdog daemon -m argument value and the value of the soft_margin parameter are less than the value of the WatchdogMarginWait parameter.
For example, if you decrease the value of the WatchdogMarginWait parameter to 65000 ms, set the value of the soft_margin parameter to 50000 ms and the value of the Watchdog daemon -m argument to 10000 ms.
To avoid database corruptions, reduce the value of the soft_margin parameter before you reduce the value of the WatchdogMarginWait parameter:
softdog module from a system startup file, reduce the value of the soft_margin parameter as follows:
softdog module from a system startup file, reduce the value of the soft_margin parameter as follows:
$ORACLE_HOME/oracm/admin/nmcfg.ora file. For example, enter the following line:
WatchdogMarginWait=64000
watchdogd, oranm, and oracm, and the Oracle instance.
This section describes how to disable a system reset caused by a node failure. You can also use this procedure for testing or debugging.
By default, the Watchdog daemon starts with an option of -l 1 and the oracm process starts with an option of /a:1. With these default values, the unexpected termination of the LMON process, oranm, oracm, and watchdogd causes a system reset. Also, in the current version, when the watchdogd daemon is running with an option of -l 1, the only way to stop oracm, oranm, and watchdogd is to reboot the system. Therefore, if you run OCMS to perform testing or debugging, Oracle Corporation recommends using the -l 0 and -d /dev/null options of the watchdogd daemon and the /a:0 option of the oracm command.
This sections describes restrictions that apply when you use the following Oracle9i Real Application Clusters features with OCMS:
Lamport System Change Number (SCN) generation improves the performance of transactions. You can enable or disable the Lamport SCN generation. A delay occurs between the time that the Oracle instance commits an update on a node and the time the instance reflects upon a query on other nodes.
To enable or disable the Lamport SCN generation, set the MAX_COMMIT_PROPAGATION_DELAY initialization parameter, using Table E-5 to determine the appropriate value. The default value is 90000.
| Value | Delay (seconds) | Lamport Clock |
|---|---|---|
|
0 |
0 |
No |
|
1 to 299 |
Value divided by 100 |
No |
|
300 to 700 |
3 |
No |
|
Greater than 700 |
6 |
Yes (default) |
In the preceding table, the Value column lists values of the MAX_COMMIT_PROPAGATION_DELAY initialization parameter. The Delay column lists the maximum delay between a commit occurrence on an instance and the time a commit becomes valid on all other instances. The Lamport Clock column indicates whether the Lamport SCN generation is enabled.
To enable database recovery, set the THREAD initialization parameter to a value other than 0 when you use the database in ARCHIVELOG mode. Otherwise, database recovery is not possible.
If you use Shared Server with Oracle9i Real Application Clusters, the value of the MAX_SERVERS initialization parameter must be equal to or greater than the value of the TRANSACTIONS parameter or the Oracle instance deadlocks.
|
 Copyright © 2001 Oracle Corporation. All Rights Reserved. |
|