Concurrent Manager

The concurrent managers serve important administrative functions. The concurrent managers ensure that the applications are not overwhelmed with requests, and the second areas of functions are the management of batch processing and report generation.
In The Oracle e-Business suite has three important Concurrent Managers:
 
Internal Concurrent Manager: The ICM (Internal Concurrent Manager) controls all of the other concurrent managers.

Standard Manager: The Standard Manager accepts any and all requests.  It has no predefined specialization rules and is active all the time.  It is not recommended to add specialization rules to the standard manager as it can cause problems.

Conflict Resolution Manager: The Conflict Resolution Manager resolves conflicts, such as request incompatibilities.

How to check concurrent manager running or not?
Log in to applications as System Administrator responsibility and navigate to the concurrent > manager > administration page and under the processes column if the target and actual column equal the same number (above 0) this means the managers are up and running.

How to Start / Stop Concurrent manager ?
If concurrent managers down, Use this syntax to start the concurrent managers from the command line:
adcmctl.sh start apps/<psswd>  
You stop the concurrent managers from the command line:
adcmctl.sh stop apps/<psswd>
Location where you find adcmctl.sh scripts is $COMMN_TOP/admin/scripts/$CONTEXT_NAME/

Where the location of Concurrent Managers Log files?
The concurrent manager log files can be located in one of the following places:
1. If the environment variable $APPLCSF is set, the default location is $APPLCSF/$APPLLOG
2. If the environment variable $APPLCSF is not set, the logs go to $FND_TOP/$APPLLOG
NOTE: The default name of the concurrent manager log files is std.mgr.  You can change these by setting the parameter logfile=<name>.
Manager Log Files:
Standard manager log:                   w<XXXXXX>.mgr
Transaction manager log:                t<XXXXXX>.mgr
Conflict Resolution manager log:        c<XXXXXX>.mgr

Where: <XXXXXX> is the concurrent process id of the manager

How to Tune the Concurrent Manager?  
Tuning the Internal Concurrent Manager (ICM)
The ICM performance is affected by the three important Oracle parameters PMON cycle, queue size, and sleep time.

PMON cycle - This is the number of sleep cycles that the ICM waits between the times it checks for concurrent managers failures, which defaults to 20. You should change the PMON cycle to a number lower than 20 if your concurrent managers are having problems with abnormal terminations.

Queue Size - The queue size is the number of PMON cycles that the ICM waits between checking for disabled or new concurrent managers. The default for queue size of 1 PMON cycle should be used.

Sleep Time - The sleep time parameter indicates the seconds that the ICM should wait between checking for requests that are waiting to run. The default sleep time is 60, but you can lower this number if you see you have a lot of request waiting (Pending/Normal). However, reducing this number to a very low value many cause excessive cpu utilization.

Purging Concurrent Requests
When the space in FND_CONCURRENT_PROCESSES and FND_CONCURRENT_REQUESTS exceed 50K, you can start to experience serious performance problems within your Oracle Applications. When you experience these space problems, a specific request called "Purge Concurrent Requests And/Or Manager Data" should be scheduled to run on a regular basis. This request can be configured to purge the request data from the FND tables as well as the log files and output files on accumulate on disk.

Adjusting the Concurrent Manager Cache Size
Concurrent manager performance can also be enhanced by increasing the manager cache size to be at lease twice the number of target processes. The cache size specifies the number of requests that will be cached each time the concurrent manager reads from the FND_CONCURRENT_REQUESTS table. Increasing the cache size will boost the throughput of the managers by attempting to avoid sleep time.  

Analyzing Oracle Apps Dictionary Tables for High Performance
It is also very important to run the request Gather Table Statistics on these tables:

·      FND_CONCURRENT_PROCESSES

·         FND_CONCURRENT_PROGRAMS

·        FND_CONCURRENT_REQUESTS

·        FND_CONCURRENT_QUEUES.

Run the request "Analyze All Index Column Statistics" on the indexes of these tables. Since the APPLSYS user is the owner of these tables, so you can also just run the request Analyse Schema Statistics for APPLSYS.

Monitoring Pending Requests in the Concurrent Managers
You may find that requests are stacking up in the concurrent managers with a status of "pending". This can be caused by any of these conditions:
  1. The concurrent managers were brought down will a request was running.
2. The database was shutdown before shutting down the concurrent managers.
3. There is a shortage of RAM memory or CPU resources.
When you get a backlog of pending requests, you can first allocate more processes to the manager that is having the problem in order to allow most of the requests to process, and then make a list of the requests that will not complete so they can be resubmitted, and cancel them.
To allocate more processes to a manager, log in as a user with the System Administrator responsibility. Navigate to Concurrent -> Manager -> Define. Increase the number in the Processes column. Also, you may not need all the concurrent managers that Oracle supplies with an Oracle Applications install, so you can save resources by identifying the unneeded managers and disabling them.

 

 

However, you can still have problems. If the request remains in a phase of RUNNING and a status of TERMINATING after allocating more processes to the manager, then shutdown the concurrent managers, kill any processes from the operating system that won't terminate, and execute the following sqlplus statement as the APPLSYS user to reset the managers in the FND_CONCURRENT_REQUESTS table:
update fnd_concurrent_requests
set status_code='X', phase_code='C'
where status_code='T';

How to find Concurrent Manager Node in Multi-Node environment ?
Method 1: SQLPLUS> select node_name from apps.fnd_nodes where support_cp='Y';
Method 2: Execute "ps -ef | grep FNDLIBR" under application User on all nodes where FND (Foundation) LIBR (Library) is executable.

Can I delete a concurrent manager?
No. You can only disable/Enable the manager by checking the 'Enabled' checkbox

How do I enable/disable the Conflict Resolution Manager?
Use the system profile option 'Concurrent: Use ICM'. Setting this to 'No' (which is the default) allows the CRM to be started. Setting it to 'Yes' causes the CRM to be shutdown and the Internal Manager (ICM) will take over the conflict resolution duties. If the CRM will not start (it is started automatically by the ICM), check this profile option. Note that using the ICM to resolve conflicts is not recommended. The CRM's sole purpose is to resolve conflicts, while the ICM has other functions to perform as well. Only set this option to 'YES' if you have a good reason to do so.

How do I clean out the Concurrent Manager tables?
Run the script, cmclean.sql, article Note 134007.1

How do we process more concurrent requests concurrently?
Login as a Sysdba and navigate ->Concurrent -> Manager -> Define and Query for the relevant concurrent manager), done in two steps: (i) Increase the Number of Target processes for the manager (ii) Change the cache size of the concurrent manager.

How to Handle hang Concurrent Request or How to Kill Concurrent Request session ?
Step 1 Mapping Concurrent request Id to Sid and serai# by using following query
          select v.request_id, s.sid,s.serial#,s.event,p.spid
          from apps.fnd_conc_requests_form_v v,v$process p,v$session s
          where v.request_id = ‘<enter request id>'
          and v.oracle_process_id = p.spid(+)
          and p.addr = s.paddr(+)
Step 2 Execute following command and supply output from query 1
SQL> alter system kill session 'sid, serial#'

How to retrieve Summary of concurrent Jobs/status/Count in Last 1 hour?
Use following query:
select
fcpt.USER_CONCURRENT_PROGRAM_NAME,
DECODE(fcr.phase_code,
‘C’, ‘Completed’,
‘I’, ‘Inactive’,
‘P’, ‘Pending’,
‘R’, ‘Running’,
fcr.phase_code
) PHASE ,
DECODE(fcr.status_code,
‘A’, ‘Waiting’,
‘B’, ‘Resuming’,
‘C’, ‘Normal’,
‘D’, ‘Cancelled’,
‘E’, ‘Errored’,
‘F’, ‘Scheduled’,
‘G’, ‘Warning’,
‘H’, ‘On Hold’,
‘I’, ‘Normal’,
‘M’, ‘No Manager’,
‘Q’, ‘Standby’,
‘R’, ‘Normal’,
‘S’, ‘Suspended’,
‘T’, ‘Terminating’,
‘U’, ‘Disabled’,
‘W’, ‘Paused’,
‘X’, ‘Terminated’,
‘Z’, ‘Waiting’,
fcr.status_code
) STATUS,
count(*)
from apps.fnd_concurrent_programs_tl fcpt,apps.FND_CONCURRENT_REQUESTs fcr
where fcpt.CONCURRENT_PROGRAM_ID=fcr.CONCURRENT_PROGRAM_ID
and fcpt.language = USERENV(’Lang’)
and fcr.ACTUAL_START_DATE > sysdate - 1/24
group by fcpt.USER_CONCURRENT_PROGRAM_NAME,fcr.phase_code,fcr.status_code
/

Problems and Solutions:
Problems: I hit the Restart button to start the Standard manager, but it still did not start? Solution: The ICM will start Standard Manager in the next process monitor session or the next time the ICM starts. Use Activate to start a manager immediately. When a manager is deactivated manually, the ICM will not restart it, you will need to set it to Restart, or activate it manually.
Problem:  When you try to submit a request like Active users or Active responsibilities, request gets submitted. When we view the requests, you find that it is inactive / nomanager. Within 12 to 15 seconds, you refresh-it gets completed.
Solution: when this occurs where a request goes "inactive/no manager" and is then processed a short time later, the solution is to either increase the cache size for your Standard manger, or increase the actual number of Standard manager processes.
HELPFUL DIAGNOSTIC SCRIPTS
The following SQL scripts located under $FND_TOP/sql are useful when diagnosing concurrent manager problems:
afimchk.sql       Tells the status of the ICM and PMON method
afcmstat.sql      Lists active manager processes
afrqrun.sql        Lists all the running, waiting and terminating requests
afrqwait.sql       Lists requests that are constrained and waiting for the ICM to release them
afrqscm.sql       Prints log file name of managers that can run a given request.  It can be    used to check for possible errors when a request stays in pending status.  It requires a request id value.
afcmcreq.sql     Prints the log file name of the manager that processed the request
afrqstat.sql       Summary of completed concurrent requests grouped by completion status and execution type.  It requires number of days prior to today on which to report       parameter.
afimlock.sql       Lists locks that the ICM is waiting to get

afcmrrq.sql       Lists managers that currently are running a request  

1. What is difference between Single Node and Multi Node installation?
Single Node - All components of Apps 11i installed on single machine. Means Database, Forms, Reports and Web Server, CM installed on Single machine.
Multi Node – All component of Apps 11i installed on more the one-machine. Means Database, Forms, Reports and Web Server, CM installed on more then one machine.

 2. What is difference between Single user and multi user installation? 
Single User - Both Database and Application tier are owned by Single Operating System Account
Multi User - 11i Database Tier is owned by one O.S. User where as Application Tier is owned by different user.
Important: In both cases Apps is installed as user ROOT

3. What is difference between express install or advanced install?
Advanced Install – When we choose advanced installation option, Option define O.S user, Mount Point etc.

Express Install – When we choose Express Installation option, installer installed Apps 11i on default location.

4. What is difference between PROD database and VISION database install type?
PROD Database is installed with Apps without data.
VISION Database is installed with Apps with dummy data.

5. What are various components installed after 11.5.10 (11i) install?
9iAS (1.0.2.2.2) web server, Developer 6i Forms & Reports, Discoverer, Jinitiator

6. What is O.S. level software requirement for installing Apps?
ar, ld, make & X Display server for all unix machines (linux, solaris, IBM, HP Unix )
with following additional as per O.S.
Linux- gcc, g++, ksh
HP-Unix- cc, acc
IBM AIX - cc, linkx1C
For Windows you need
Microsoft C++, MKS Toolkit, GNU Make

7. What is minimum approx. disk requirement for 11.5.10?
For 11.5.10
Application Tier File System - 26 GB
Database Tier (Fresh install) - 31 GB
Database Tier (Vision install) - 65 GB

8. What is staging area?
Staging Area is special directory structure where you make dump 11i installation software so that you don’t have to insert CD’s during install and Installer will automatically pick these disks.

9. How you set up staging area ?
Coming Soon.