Sunday, November 25, 2012

Ulimit OS Command


The ulimit command sets or reports user process resource limits, as defined in the /etc/security/limits file. This file contains these default limits:
fsize = 2097151
core = 2097151
cpu = -1
data = 262144
rss = 65536
stack = 65536
nofiles = 2000
threads = -1
nproc = -1
These values are used as default settings when a new user is added to the system.Limits are categorized as either soft or hard. With the ulimit command, you can change your soft limits, up to the maximum set by the hard limits. You must have root user authority to change resource hard limits.
Many systems do not contain one or more of these limits. The limit for a specified resource is set when the Limit parameter is specified. The value of the Limit parameter can be a number in the unit specified with each resource, or the value unlimited. To set the specific ulimit to unlimited, use the word unlimited
Note: Setting the default limits in the /etc/security/limits file sets system wide limits, not just limits taken on by a user when that user is created.
The current resource limit is printed when you omit the Limit parameter. The soft limit is printed unless you specify the -H flag. When you specify more than one resource, the limit name and unit is printed before the value. If no option is given, the -f flag is assumed.
Since the ulimit command affects the current shell environment, it is provided as a shell regular built-in command. If this command is called in a separate command execution environment, it does not affect the file size limit of the caller's environment. This would be the case in the following examples:
nohup ulimit -f 10000
env ulimit 10000
Once a hard limit has been decreased by a process, it cannot be increased without root privilege, even to revert to the original limit.
For more information about user and system resource limits, refer to the getrlimit, setrlimit, or vlimit subroutine in AIX® Version 6.1 Technical Reference: Base Operating System and Extensions, Volume 1.

Flags

Item Description
-a Lists all of the current resource limits.
-c Specifies the size of core dumps, in number of 512-byte blocks.
-d Specifies the size of the data area, in number of K bytes.
-f Sets the file size limit in blocks when the Limit parameter is used, or reports the file size limit if no parameter is specified. The -f flag is the default.
-H Specifies that the hard limit for the given resource is set. If you have root user authority, you can increase the hard limit. Anyone can decrease it.
-m Specifies the size of physical memory, in number of K bytes.
-n Specifies the limit on the number of file descriptors a process may have.
-r Specifies the limit on the number of threads a process can have.
-s Specifies the stack size, in number of K bytes.
-S Specifies that the soft limit for the given resource is set. A soft limit can be increased up to the value of the hard limit. If neither the -H nor -S flags are specified, the limit applies to both.
-t Specifies the number of seconds to be used by each process.
-u Specifies the limit on the number of a process a user can create.

Example

ulimit -a     (Lists all of the current resource limits.)
ulimit -n 65536   (Specifies the limit on the number of file descriptors a process may have.)

Sunday, November 18, 2012

EM Incident: Critical:New: - Row(1): Supplied date is in the future : now =

Here the below alerts/Incidents will be coming to the EM Users once after the DST Time Change.

Ex:
Host=HOST.domain.com
Target type=Database Instance
Target name=ABCDB
Incident creation time=Nov 15, 2012 5:21:01 AM PST
Last updated time=Nov 15, 2012 7:21:05 PM PST
Message=
Row(1): Supplied date is in the future : now = Thu Nov 15 19:21:01 PST 2012 supplied value = Thu Nov 15 20:20:25 PST 2012Severity=Critical
Incident ID=41705
Event count=1
Incident Status=New
Escalation level=2
Priority=None
Incident owner=
Incident Acknowledged By Owner=No
Categories=Error
Rule Name=Rule set for non production group targets,Rule to notify team about metric evaluation error
Rule Owner=SYSMAN
Update Details:
Escalation level set to 2 by rule (Name = Rule set for non production group targets,
Rule to notify team about metric evaluation error; Owner = SYSMAN).


Solution:

Oracle says this is a known behavior in instances that cant handle DST change automatically. Please do the below steps on all agents that see this problem 
./emctl stop agent
./emctl clearstate agent
./emctl start agent
./emctl upload agent

This should help you to get rid of the problem

perl: warning: Falling back to the standard locale ("C") while running emctl status agent

Here Perl executable are throwing warning alerts as below

[oracle@<HOST>:/u01/app/oracle/linux/<HOST>/agent12c/agent_inst/bin]$./emctl status agentperl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "en_US.UTF-8",
        LC_ALL = "ISO-8859-1",
        LANG = "ISO-8859-1"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 12c Cloud Control 12.1.0.1.0
Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 12.1.0.1.0
OMS Version       : 12.1.0.1.0
Protocol Version  : 12.1.0.1.0
Agent Home        : /u01/app/oracle/linux/<HOST>/agent12c/agent_inst
Agent Binaries    : /u01/app/oracle/linux/<HOST>/agent12c/core/12.1.0.1.0
Agent Process ID  : 5203
Parent Process ID : 5151
Agent URL         :
https://<HOST>.domain.com:3872/emd/main/
Repository URL    : https://<OMS_HOST>.domain.com:4900/empbs/upload
Started at        : 2012-11-17 19:39:34
Started by user   : oracle
Last Reload       : (none)
Last successful upload                       : 2012-11-17 19:40:33
Last attempted upload                        : 2012-11-17 19:40:33
Total Megabytes of XML files uploaded so far : 0.05
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 85.29%
Collection Status                            : Collections enabled
Last attempted heartbeat to OMS              : 2012-11-17 19:40:06
Last successful heartbeat to OMS             : 2012-11-17 19:40:06

---------------------------------------------------------------
Agent is Running and Ready


Solution:

Now check for the bash_profile and see for the entry
vi .bash_profile
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8


If the above are not set, please set it and run the bash_profile and re-check the status of the EM Agent.
It should hlep to resolve the perl issue.

[oracle@<HOST>:~]$. .bash_profile

[oracle@<HOST>:~]$ cd /u01/app/oracle/linux/<HOST>/agent12c/agent_inst/bin

[oracle@<HOST>:/u01/app/oracle/linux/<HOST>/agent12c/agent_inst/bin]$./emctl status agent
Oracle Enterprise Manager 12c Cloud Control 12.1.0.1.0
Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 12.1.0.1.0
OMS Version       : 12.1.0.1.0
Protocol Version  : 12.1.0.1.0
Agent Home        : /u01/app/oracle/linux/<HOST>/agent12c/agent_inst
Agent Binaries    : /u01/app/oracle/linux/<HOST>/agent12c/core/12.1.0.1.0
Agent Process ID  : 5203
Parent Process ID : 5151
Agent URL         :
https://<HOST>.domain.com:3872/emd/main/
Repository URL    : https://<OMS_HOST>.domain.com:4900/empbs/upload
Started at        : 2012-11-17 19:39:34
Started by user   : oracle
Last Reload       : (none)
Last successful upload                       : 2012-11-17 19:42:37
Last attempted upload                        : 2012-11-17 19:42:37
Total Megabytes of XML files uploaded so far : 0.05
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 85.29%
Collection Status                            : Collections enabled
Last attempted heartbeat to OMS              : 2012-11-17 19:42:06
Last successful heartbeat to OMS             : 2012-11-17 19:42:06

---------------------------------------------------------------
Agent is Running and Ready


Friday, November 16, 2012

EM Incident: Warning:New: - Agent resident memory utilization in KB is 251,848

If an EM Agent consuming more memory on the server, please do the below work around to resolve the issue.


1. ./emctl stop agent

Ensure no EMAgent process is running with the following commands:

<AGENT_HOME>/bin>./emctl status agent


$ps -ef | grep TMMain

$ps -ef | grep emagent
If you see any process, please kill them.

2. Take a Backup and edit the .../agent_inst/sysman/config/emd.properties file:

from:
agentJavaDefines=-Xmx128M -XX:MaxPermSize=96M

to:
agentJavaDefines=-Xmx512M -XX:MaxPermSize=96M

and save the file.

3. Start the Agent and check the status:

<AGENT_HOME>/bin>./emctl start agent
<AGENT_HOME>/bin>./emctl status agent
 


It should Fix the issue. 

If still does not resolve the issue please change the below Open Files parameter value.

-- Check the Open files value

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 774874
max locked memory       (kbytes, -l) 74136141
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 131072
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

 

--To set the Open files value
$ ulimit -n 65536

-- Recheck the Open files value
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 774874
max locked memory       (kbytes, -l) 74136141
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65536
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 131072
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited



Friday, November 9, 2012

ORA-31693 ORA-29913 ORA-31617 Unable To Open Dump File for Write On RAC

While exporting few tables from 11g RAC environment hit the below Issue and workarounds to resolve the issue.

ORA-31693: Table data object "AABCVSUPPORT"."SOAPV_CHANNEL_INV_ESS_WW" failed to load/unload and is being skipped due to error:
ORA-31617: unable to open dump file "/u01/app/oracle/product/
11.2.0.3/dbhome_1/rdbms/log/ABCVSUPPORT_new_03.DMP" for write
ORA-19505: failed to identify file "/u01/app/oracle/product/
11.2.0.3/dbhome_1/rdbms/log/ABCVSUPPORT_new_03.DMP"
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
. . exported "ABCVSUPPORT"."SAOPV_INTRANSIT_EBS_WW"        1.548 GB 8647683 rows



Here is the command used to export the tables.

expdp system/<pwd> DIRECTORY=DATA_PUMP_DIR DUMPFILE=ABCVSUPPORT_new_%U.DMP LOGFILE=ABCVSUPPORT_new.LOG tables=ABCVSUPPORT.SOAPV_INTRANSIT_EBS_WW, ABCVSUPPORT.SOAPV_SHIPMENTS_EBS_WW, ABCVSUPPORT.SOAPV_SELLTHRU_EBS_WW, ABCVSUPPORT.SOAPV_CHANNEL_INV_EBS_WW, PARALLEL=5 filesize=4G
When the job scheduler starts parallel jobs, one or more processes may be executed on another node in the cluster and may not have access to the target directory. Make sure in a RAC environment the target directory is on SHARE storage (ASM/ACFS is very good for this purpose).

Solution

1. Please try to change the value of PARALLEL clause. (Experiment with value of parallel and see if issue is resolved by changing parallel value).
SUGGESTION: You can try from 2-8.
2. Check if  the storage is shared among all instances at operating system end. See if you can list the files from that mount point on all the instances in RAC (Real Application Cluster) as oracle user as well as at operating system end.
3. Check if  the directory is created on shared storage or is it local to the instance on non shared storage.
4. Please try DataPump export with different schema than you are doing it now for and see the problem is peculiar to specific SCHEMA.
5. Check if DBA_REGISTRY is clean (All components are VALID and showing same version).
Correct the components if they are INVALID:

connect / as sysdba
SELECT COMP_NAME , VERSION , STATUS FROM DBA_REGISTRY;
6. Please check the mount point you are using. Check if  it is NFS or something else. Check if there are some issues with it.
7. Please check if  there were any changes done recently at Oracle/Operating System end.
8. Please check if there is an disk space issue at the time of error.
9. Also check if this behavior is node specific or not. See if you can try DataPump by creating another directory on another node.
10. Check if  you  have created any service which is node specific.
11. Error ORA-31617 usually means that the operating system is not able to write to the destination. The error is returned to oracle by the operating system, so please check at operating system end that the file write operation from the local node to the particular mount point is fine.
12. Check if there is any I/O (Input/Output) error reported in operating system logs.
13. Please check the DIRECTORY definition and also the environment details. See if directory is created on shared server and the write permission is given to the user oracle as well as at operating system end.


Note: But in my case 1st point itself helped me after removing the parallel caluse from the export command it worked fine.