8
Management Pack for Oracle Applications Event Tests

The Oracle Applications Advanced Event Tests include a library of applications-specific event tests which are provided for lights-out event monitoring and problem detection of the Applications subsystem.

Note: To successfully use the event tests, the node credentials on Windows NT must be for a user with the "logon as batch" privilege.

The Application subsystem specific events notify administrators of fault, performance, and space problems. The event tests according to category are:

Note: The Fault and Performance event tests are specific to the version of the Agent. The Space event test is independent of the version of the Agent. To determine the version of the Agent, go to the Enterprise Manager Console and find the node on which the concurrent manager is running. Right mouse click on the node target and select "Properties". The event tests for release 8.1.6 and earlier targets are included for backward compatibility purposes. It is highly encouraged that you upgrade to the new event tests because the old event tests may not be supported in future releases of the Management Pack for Oracle Applications.

Summary of Applications Event Tests

The following tables list the Applications event tests. The full descriptions of the individual event tests follow the tables. The event tests are in alphabetical order.

Table 8-1 Applications Fault Management Event Tests

Event Test	Description
Concurrent Manager UpDown	This event test monitors the state of the Internal Concurrent Manager (ICM). If the ICM goes down, an alert is generated.
CRM Waiting on a Lock	This event test checks whether the Conflict Resolution Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning is issued.
ICM Waiting on a Lock	This event test checks whether the Internal Concurrent Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning alert is issued.
Request Error Rate	This event test monitors the error rate of concurrent requests. When the error rate reaches the threshold parameters set by the user, a warning or critical alert is generated.
Request Warning Rate	This event test monitors the warning rate of concurrent requests. When the warning rate reaches the threshold parameters set by the user, a warning or critical alert is generated.
Unresponsive Concurrent Manager	This event test checks the responsiveness of the concurrent managers. If any of your concurrent managers are down at the beginning of the test and are still down after the number of seconds you entered as a parameter, they are considered unresponsive. In other words, this event test tells you if the concurrent manager is having trouble coming back up or can not be brought back online, giving you the opportunity to resolve the problem. The difference between the Unresponsive Concurrent Manager event and the Concurrent Manager UpDown event is that the Concurrent Manager UpDown event notifies you when the internal concurrent manager has gone down or comes back up. The Unresponsive Concurrent Manager event test notifies you if any of the other concurrent managers are down over the time period you specify.

Table 8-2 Applications Performance Management Event Tests

Event Test	Description
Inactive Request Pending	This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in an inactive state, an alert is generated.
Pending Concurrent Request Backlog	This event test monitors for concurrent requests that have been in the pending state for a time period exceeding the threshold, and triggers an alert if they exceed the 'total concurrent requests' threshold.
Request Pending Time	This event test monitors for requests that have been in the pending state for a time period exceeding the threshold set by the user.
Runalone Request Submitted	This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in a run alone state, a warning is generated.

Table 8-3 Applications Space Management Event Tests

Event Test	Description
Concurrent Manager Disk Free	This event test monitors the growth rates of the log, output, and other partitions for the concurrent requests. A check box is provided to select any or all of these partitions, or any other partition on the concurrent processing server node.

Table 8-4 Applications Fault Management Event Tests for V8.1.6 and Earlier Agents

Event Test	Description
Concurrent Manager UpDown	This event test monitors the state of the Internal Concurrent Manager (ICM). If the ICM goes down, an alert is generated.
CRM Waiting on a Lock	This event test checks whether the Conflict Resolution Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning is issued.
ICM Waiting on a Lock	This event test checks whether the Internal Concurrent Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning alert is issued.
Request Error Rate	This event test monitors the error rate of concurrent requests. When the error rate reaches the threshold parameters set by the user, a warning or critical alert is generated.
Request Warning Rate	This event test monitors the warning rate of concurrent requests. When the warning rate reaches the threshold parameters set by the user, a warning or critical alert is generated.

Table 8-5 Performance Management Event Tests for V8.1.6 and Earlier Agents

Event Test	Description
Inactive Request Pending	This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in an inactive state, an alert is generated.
Pending Concurrent Request Backlog	This event test monitors for concurrent requests that have been in the pending state for a time period exceeding the threshold, and triggers an alert if they exceed the 'total concurrent requests' threshold.
Request Pending Time	This event test monitors for requests that have been in the pending state for a time period exceeding the threshold set by the user.
Runalone Request Submitted	This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in a runalone request, a warning is generated.

The Oracle Applications event library also includes node events. See Chapter 3, "Compaq Tru64 Event Tests", Chapter 4, "HP-UX Event Tests", Chapter 5, "IBM AIX Event Tests", Chapter 6, "Solaris Event Tests", and Chapter 7, "Windows NT Event Tests" for listings of the node-specific event tests.

For more information on submitting or scheduling events, see the Oracle Enterprise Manager Administrator's Guide. Pay special attention to the event frequency value, which can be set on the General tab of the Create Event panel. This value determines the frequency of polling for event occurrences. The default value is every 60 seconds, but for many application management events, you may wish to choose larger values, for better performance and a lower overall impact on system resources.

Note: To expand the Intelligent Agent messages related to the Management Pack for Oracle Applications, at the system prompt from the Agent's Oracle home type: oerr <facility> <message number> where <facility> is smamp. For example: oerr smamp 6001 Cause: Failed to open a cursor. Action: Verify that the maximum cursors limit has not been exceeded. Before running any program from the command line (UNIX or Windows NT), verify that all the correct values are present in the necessary environment variables, for example, ORACLE_HOME and PATH.

Descriptions of Applications Event Tests

The Applications event tests for release 9i are listed in alphabetical order.

Concurrent Manager Disk Free

This event test monitors the growth rates of the log, output, and other partitions for the concurrent requests. A check box is provided to select any or all of these partitions, or any other partition on the concurrent processing server node.

Note: This event test requires the Application environment (APPLSYS.env) file which is located in the Application code base (APPL_TOP).

Parameters

1. Critical threshold for log: Free space threshold in percentage. Default is 30%.

2. Warning threshold for log: Free space threshold in percentage. Default is 50%.

3. Critical threshold for output: Free space threshold in percentage. Default is 30%.

4. Warning threshold for output: Free space threshold in percentage. Default is 50%.

5. Name of partition. (* is not a supported name.)

6. Critical threshold for other partitions: Free space threshold in percentage. Default is 30%.

7. Warning threshold for other partitions: Free space threshold in percentage. Default is 50%.

Output

Partition name and space available in percentage on the partition.

Default Frequency

60 seconds.

User Action

Purge the log or output files.

Note: You can configure the Run OS Command job task as a fixit job to remove the log/output files from the Oracle Applications APPL_TOP. See the Oracle Enterprise Manager Administrator's Guide for more information on configuring job tasks.

Concurrent Manager UpDown

This event test monitors the state of the Internal Concurrent Manager (ICM). If the ICM goes down, an alert is generated.

Parameters

None.

Output

None.

Default Frequency

60 seconds.

User Action

Submit a standalone concurrent manager startup job when the event issues an alert. The startup concurrent manager job could also be configured as a fix-it job to automatically restart the ICM when it goes down.

CRM Waiting on a Lock

This event test checks whether the Conflict Resolution Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning is issued.

Parameters

1. Critical threshold: Lock wait time (in minutes) for a critical alert. Default is 3 minutes.

2. Warning threshold: Lock wait time (in minutes) for a warning alert. Default is 2 minutes.

   Note: Threshold values must be between 2 and 32 minutes.

Output

• Session identification number.

• Operating system process identification number.

• Lock mode.

• Machine name.

• Terminal name for the locking session.

Default Frequency

60 seconds.

User Action

Locate the session that is blocking the Conflict Resolution Manager and determine if the session can be deleted.

Note: A fixit job could be registered to automatically delete the blocking session when the event is triggered.

ICM Waiting on a Lock

This event test checks whether the Internal Concurrent Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning alert is issued.

Parameters

1. Critical threshold: Lock wait time (in minutes) for a critical alert. Default is 3 minutes.

2. Warning Threshold: Lock wait time (in minutes) for a warning alert. Default is 2 minutes.

   Note: Threshold values must be between 2 and 32 minutes.

Output

• Session identification number.

• Operating system process identification number.

• Lock mode.

• Machine name.

• Terminal name for the locking session.

Default Frequency

60 seconds.

User Action

Locate the session that is blocking the Internal Concurrent Manager and determine if the session can be deleted.

Note: A fixit job could be registered to automatically delete the blocking session when the event is triggered.

Inactive Request Pending

This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in an inactive state, an alert is generated.

Parameters

1. Monitor Field: Specify or exclude certain concurrent programs from monitoring. Choose one of the following: All Concurrent Programs, All Except These Concurrent Programs, or Only These Specific Concurrent Programs. The default is to monitor all concurrent programs.

2. List Field: Type the application name and concurrent program name for which the event will filter or exclude from monitoring. These names can be found in the Oracle Applications Manager. The default is empty and disabled.

Output

• The number of inactive requests in the concurrent manager(s) queue.

• Table containing the concurrent programs that violated the event test. The columns in the table are: Request ID, Requested By, Application Name, Concurrent Program Name, and Hours Runalone.

Default Frequency

60 seconds.

User Action

Locate the inactive request and remove it from the queue.

Pending Concurrent Request Backlog

This event test monitors for concurrent requests that have been in the pending state for a time period exceeding the threshold, and triggers an alert if they exceed the 'total concurrent requests' threshold.

Parameters

1. Monitor Field: Specify or exclude certain concurrent programs from monitoring. Choose one of the following: All Concurrent Programs, All Except These Concurrent Programs, or Only These Specific Concurrent Programs. The default is to monitor all concurrent programs.

2. List Field: Type the application name and concurrent program name for which the event will filter or exclude from monitoring. These names can be found in the Oracle Applications Manager. The default is empty and disabled.

3. Threshold for total number of requests pending: Number of requests pending in the queue. Default is 1.

4. Time Threshold (in minutes) for including requests in the pending queue backlog. Default is 1 minute.

Output

• Number of requests currently pending.

• Table containing the concurrent programs that violated the event test. The columns in the table are: Request ID, Requested By, Application Name, Concurrent Program Name, and Hours Runalone.

• Length of time the requests has been pending in the queue.

Default Frequency

60 seconds.

User Action

Increase the number of concurrent processes to handle the load.

Request Error Rate

This event test monitors the error rate of concurrent requests. When the error rate reaches the threshold parameters set by the user, a warning or critical alert is generated.

Parameters

1. Monitor Field: Specify or exclude certain concurrent programs from monitoring. Choose one of the following: All Concurrent Programs, All Except These Concurrent Programs, or Only These Specific Concurrent Programs. The default is to monitor all concurrent programs.

2. List Field: Type the application name and concurrent program name for which the event will filter or exclude from monitoring. These names can be found in the Oracle Applications Manager. The default is empty and disabled.

3. Critical threshold: Percentage of requests marked with errors for critical alert. Default is 10%.

4. Warning threshold: Percentage of requests marked with errors for warning alert. Default is 5%.

Output

Percentage of requests with errors.

Default Frequency

60 seconds.

User Action

Review the Request Log files in the APPL_TOP/log directory to determine which requests need to be corrected.

Request Pending Time

This event test monitors for requests that have been in the pending state for a time period exceeding the threshold set by the user.

Parameters

1. Monitor Field: Specify or exclude certain concurrent programs from monitoring. Choose one of the following: All Concurrent Programs, All Except These Concurrent Programs, or Only These Specific Concurrent Programs. The default is to monitor all concurrent programs.

2. List Field: Type the application name and concurrent program name for which the event will filter or exclude from monitoring. These names can be found in the Oracle Applications Manager. The default is empty and disabled.

3. Hours pending: Number of hours the request has been pending. Default is 1 hour.

4. Minutes pending: Number of minutes the request has been pending. Default is 30 minutes.

Output

• Number of hours or minutes that the request has been pending.

• Table containing the concurrent programs that violated the event test. The columns in the table are: Request ID, Requested By, Application Name, Concurrent Program Name, and Hours Runalone.

Default Frequency

60 seconds.

User Action

Check to see if the queue is being serviced actively. Increase the processes for the queue manager to balance the load, if necessary.

Request Warning Rate

This event test monitors the warning rate of concurrent requests. When the warning rate reaches the threshold parameters set by the user, a warning or critical alert is generated.

Parameters

1. Monitor Field: Specify or exclude certain concurrent programs from monitoring. Choose one of the following: All Concurrent Programs, All Except These Concurrent Programs, or Only These Specific Concurrent Programs. The default is to monitor all concurrent programs.

2. List Field: Type the application name and concurrent program name for which the event will filter or exclude from monitoring. These names can be found in the Oracle Applications Manager. The default is empty and disabled.

3. Critical threshold: Percentage of requests marked for critical alert. Default is 10%.

4. Warning threshold: Percentage of requests marked for warning alert. Default is 5%.

Output

Percentage of requests with warnings.

Default Frequency

60 seconds.

User Action

Review the Request Log files in the APPL_TOP/log directory to determine which requests need to be corrected.

Run Alone Request Submitted

This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in a run alone state, a warning is generated.

Parameters

1. Monitor Field: Specify or exclude certain concurrent programs from monitoring. Choose one of the following: All Concurrent Programs, All Except These Concurrent Programs, or Only These Specific Concurrent Programs. The default is to monitor all concurrent programs.

2. List Field: Type the application name and concurrent program name for which the event will filter or exclude from monitoring. These names can be found in the Oracle Applications Manager. The default is empty and disabled.

Output

• The number of run alone requests in the concurrent manager(s) queue.

• Table containing the concurrent programs that violated the event test. The columns in the table are: Request ID, Requested By, Application Name, Concurrent Program Name, and Hours Runalone.

Default Frequency

60 seconds.

User Action

Determine whether or not the run alone request submitted is necessary. If it is not, the request should be deleted so that other requests can be processed.

Unresponsive Concurrent Manager

This event test checks the responsiveness of the concurrent managers. If any of your concurrent managers are down at the beginning of the test and are still down after the number of seconds you entered as a parameter, they are considered unresponsive.

In other words, this event test tells you if the concurrent manager is having trouble coming back up or can not be brought back online, giving you the opportunity to resolve the problem.

The difference between the Unresponsive Concurrent Manager event and the Concurrent Manager UpDown event is that the Concurrent Manager UpDown event notifies you when the internal concurrent manager has gone down or comes back up. The Unresponsive Concurrent Manager event test notifies you if any of the other concurrent managers are down over the time period you specify.

Parameters

Unresponsive Seconds: Type the number of seconds for the time period. This value should be roughly equivalent to the time taken for 2 PMON (Process Monitor) cycles. Calculate this value by multiplying the PMON value by the SLEEP value. The default number of iterations for PMON is 20 and the default number of sleep seconds is 60. Therefore the default is 2400 seconds (2*20*60).

Descriptions of Applications Event Tests for V8.1.6 and Earlier Agents

The Applications event tests for v8.1.6 and earlier agents are listed in alphabetical order.

Concurrent Manager UpDown (for V8.1.6 and Earlier Agents)

This event test monitors the state of the Internal Concurrent Manager (ICM). If the ICM goes down, an alert is generated.

Parameters

None.

Output

None.

Default Frequency

60 seconds.

User Action

Submit a standalone concurrent manager startup job when the event issues an alert. The startup concurrent manager job could also be configured as a fix-it job to automatically restart the ICM when it goes down.

CRM Waiting on a Lock (for V8.1.6 and Earlier Agents)

This event test checks whether the Conflict Resolution Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning is issued.

Parameters

1. Critical threshold: Lock wait time (in minutes) for a critical alert. Default is 3 minutes.

2. Warning threshold: Lock wait time (in minutes) for a warning alert. Default is 2 minutes.

   Note: Threshold values must be between 2 and 32 minutes.

Output

• Session identification number.

• Operating system process identification number.

• Lock mode.

• Machine name.

• Terminal name for the locking session.

Default Frequency

60 seconds.

User Action

Locate the session that is blocking the Conflict Resolution Manager and determine if the session can be deleted.

Note: A fixit job could be registered to automatically delete the blocking session when the event is triggered.

ICM Waiting on a Lock (for V8.1.6 and Earlier Agents)

This event test checks whether the Internal Concurrent Manager is waiting to get a lock. If the lock wait time reaches a specified amount of time, a warning alert is issued.

Parameters

1. Critical threshold: Lock wait time (in minutes) for a critical alert. Default is 3 minutes.

2. Warning Threshold: Lock wait time (in minutes) for a warning alert. Default is 2 minutes.

   Note: Threshold values must be between 2 and 32 minutes.

Output

• Session identification number.

• Operating system process identification number.

• Lock mode.

• Machine name.

• Terminal name for the locking session.

Default Frequency

60 seconds.

User Action

Locate the session that is blocking the Internal Concurrent Manager and determine if the session can be deleted.

Note: A fixit job could be registered to automatically delete the blocking session when the event is triggered.

Inactive Request Pending (for V8.1.6 and Earlier Agents)

This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in an inactive state, an alert is generated.

Parameters

None.

Output

The number of inactive requests in the concurrent manager(s) queue.

Default Frequency

60 seconds.

User Action

Locate the inactive request and remove it from the queue.

Pending Concurrent Request Backlog (for V8.1.6 and Earlier Agents)

This event test monitors for concurrent requests that have been in the pending state for a time period exceeding the threshold, and triggers an alert if they exceed the 'total concurrent requests' threshold.

Parameters

1. Threshold for total number of requests pending: Number of requests pending in the queue. Default is 1.

2. Time Threshold (in minutes) for including requests in the pending queue backlog. Default is 1 minute.

Output

• Number of requests currently pending.

• Length of time the requests has been pending in the queue.

Default Frequency

60 seconds.

User Action

Increase the number of concurrent processes to handle the load.

Request Error Rate (for V8.1.6 and Earlier Agents)

This event test monitors the error rate of concurrent requests. When the error rate reaches the threshold parameters set by the user, a warning or critical alert is generated.

Parameters

1. Critical threshold: Percentage of requests marked with errors for a critical alert. Default is 10%.

2. Warning threshold: Percentage of requests marked with errors for a warning alert. Default is 5%.

Output

Percentage of requests with errors.

Default Frequency

60 seconds.

User Action

Review the Request Log files in the APPL_TOP/log directory to determine which requests need to be corrected.

Request Pending Time (for V8.1.6 and Earlier Agents)

This event test monitors for requests that have been in the pending state for a time period exceeding the threshold set by the user.

Parameters

1. Hours pending: Number of hours the request has been pending. Default is 1 hour.

2. Minutes pending: Number of minutes the request has been pending. Default is 30 minutes.

Output

Number of hours or minutes that the request has been pending.

Default Frequency

60 seconds.

User Action

Check to see if the queue is being serviced actively. Increase the processes for the concurrent manager to balance the load, if necessary.

Request Warning Rate (for V8.1.6 and Earlier Agents)

This event test monitors the warning rate of concurrent requests. When the warning rate reaches the threshold parameters set by the user, a warning or critical alert is generated.

Parameters

1. Critical threshold: Percentage of requests marked for critical alert. Default is 10%.

2. Warning threshold: Percentage of requests marked for warning alert. Default is 5%.

Output

Percentage of requests with warnings.

Default Frequency

60 seconds.

User Action

Review the Request Log files in the APPL_TOP/log directory to determine which requests need to be corrected.

Run Alone Request Submitted (for V8.1.6 and Earlier Agents)

This event test monitors the state of the requests submitted to the Concurrent Manager(s). If any of the requests is in a runalone request, a warning is generated.

Parameters

None.

Output

The number of run alone requests in the concurrent manager(s) queue.

Default Frequency

60 seconds.

User Action

Determine whether or not the runalone request submitted is necessary. If it is not, the request should be deleted so that other requests can be processed.